PRP046: CASFM: Medical Student Clinical Skills Assessment Utilizing an AI-powered Software Tool
Frances Tepolt, MD; Kevin Peterson, MD, MPH
Context: The ability to apply medical knowledge and clinical skills in the diagnosis of common medical conditions is an essential skill of primary care physicians. Opportunities for students to practice medical history taking and physical exams in the clinical setting, however, are limited. Simulated environments using actors as patients have been developed to enhance student experiences, but current novel coronavirus disease of 2019 (COVID-19) pandemic restrictions have severely limited access to simulated experiences. The integration of Artificial Intelligence has the potential to support simulated clinical experiences and enhance the development of medical education modules designed to teach and evaluate student performance. Objective: To evaluate the feasibility and baseline parameters for an AI-supported evaluation of medical student performance on the Objective Structured Clinical Examination (OSCE). Study Design: Initial feasibility study. Setting: Planned implementation in April 2020 was postponed due to the pandemic. The research strategy is being modified to determine whether the OSCE can be delivered to students using video-conferencing tools. Live OSCE simulated experiences are planned to resume October 2020. Population Studied: 200 second-year medical students. Intervention/Instrument: Course instructors prepared eight cases for the OSCE to provide simulated clinical experiences for second-year students utilizing actor patients. Simulated patients evaluate student performance supported by a second actor viewing the interaction live on video. The student writes a SOAP (subjective, objective, assessment, and plan) note and develops a differential diagnosis following the encounter. An online tool is being introduced for two cases using an AI-enhanced interface to simplify data entry, generate a SOAP note, and provide an immediate assessment of performance. Outcome Measures: Feasibility assessment will support the development of training material, identify potential barriers, and provide initial evaluation of the time it takes to complete the exam and variation in student responses. Results: Feasibility assessment will provide baseline performance score variation for the development of a clinical trial comparing computer-assisted scoring with usual scoring methods. SOAP notes created during the pilot will also improve the AI engine and provide Natural Language Processing training for subsequent implementations.