Dr. Boardwell: Development of an AI Interventional Radiology Expert for Real-Time Conversational Oral Boards Simulation and Case-Based Resident Training

Monday, March 31, 2025

3:00 PM – 3:09 PM CT

Location: Music City Center, 205 A

Presenting Author(s)

Kevin F. Seals, MD (he/him/his)

Interventional Radiologist
Cedars-Sinai Medical Center, United States

Author/Co-author(s)

DR

David Reilly, MD

Fellow, Interventional Radiology
University of California Los Angeles, United States
GL

Gabriel Lipshutz, MD

Faculty Interventional Radiologist
Cedars-Sinai Medical Center, United States
Jonathan Steinberger, MD

Vascular and Interventional Radiologist
Cedars-Sinai Medical Center Imaging Group, United States
Nathaniel Mizraki, MD

Radiology Resident
Cedars-Sinai Medical Center, United States
Justin McWilliams, MD, FSIR

Professor of Interventional Radiology
David Geffen School of Medicine at UCLA, United States

Purpose: Our objective was to create an AI-powered conversational tool that simulates an oral boards examiner for interventional radiology, providing real-time verbal interaction. This allows trainees to more effectively prepare for the boards by closely approximating the actual exam experience. Additionally, we aimed to create an innovative learning platform with broader applications for interventional radiology education.

Materials and Methods: Ten de-identified cases covering commonly tested interventional radiology topics were integrated into an AI-based examiner powered by the OpenAI GPT-4 model via API. The tool was implemented on a website using Next.js. Real-time verbal interaction between the AI examiner and the user was facilitated through Deepgram's text-to-speech and speech-to-text APIs. GPT-4 analyzed residents' responses and generated detailed feedback summaries. Fourteen interventional radiology residents participated in a virtual oral exam consisting of 10 cases and evaluated the experience using a five-point Likert scale, assessing factors such as ease of use, accuracy, educational value, and realism.

Results:
The AI-based virtual examiner was successfully developed and tested by 14 interventional radiology residents. It effectively presented cases and asked targeted questions, closely simulating the oral examination experience. Examinees responded using natural speech, resulting in dynamic conversational interactions with relevant follow-up questions and redirection. After each exam, the system provided a detailed feedback summary, highlighting areas for improvement and assigning a numerical score.

The scores for each of the 7 question of the post-test survey were then averaged, and are presented below (Maximum score = 5):

Ease of use: 4.8
Fidelity (correctly identifying user input): 4.1
Accuracy (correctly identifies right and wrong answers): 4.3
Flow (simulates real back-and-forth communication): 4.1
Flexibility (redirects wrong answers and seeks clarification for partially correct answers): 4.6
Educational value: 4.9
Overall rating of educational tool: 4.6

Conclusion: Dr. Boardwell, an AI-powered virtual examiner for the interventional radiology oral boards, demonstrated high accuracy, flexibility, and educational value. It provides real-time training and assessment, offering residents an interactive experience that closely mirrors the actual exam. Additionally, Dr. Boardwell can be used as a broader educational tool, helping residents enhance their clinical skills and decision-making through practical, case-based learning.