Artificial Intelligence in Medical Education

The grant will leverage computer vision and AI methods to provide automated assessments of surgical skill in open procedures for surgical residents. Automated video analytics will be captured during incision closures from traditional room video and egocentric recording from smart glasses. The model’s outputs will then be available on a resident-facing web platform designed to support feedback and longitudinal skill tracking. The overall goal is to refine AI-based assessment for open procedures to augment feedback from educators and encourage targeted practice for surgical trainees.

PI: Gabriel Brat, MD, Assistant Professor of Surgery, Beth Israel Deaconess Medical Center, Assistant Professor of Biomedical Informatics, Harvard Medical School

Effective patient-physician communication is crucial for high-quality healthcare, but challenging to teach and assess consistently. Traditional methods often lack the ability to provide timely, objective feedback, leading to gaps in learners' communication skills. To address this, we are developing Communication Compass, an AI-powered system designed to offer personalized, near real-time feedback on learners' communication skills by analyzing ambient audio from patient-physician interactions. This system will use advanced AI to assess communication across various dimensions and provide structured, rigorously-validated feedback with real examples. We will test this system’s impact on both educational outcomes and patient care through a randomized trial. Ultimately, Communication Compass aims to create an ethical, scalable framework for improving communication skills in medical education, benefiting learners and patients alike.

PIs: Yuliya Yoncheva, PhD, Research Assistant Professor, Institute for Innovations in Medical Education; Jesse Burk-Rafel, MD, MRes, Director of Research, Institute for Innovations in Medical Education

Large language models (LLMs) like ChatGPT are one of the fastest adopted medical technologies in history, now routinely appearing in both clinics and medical classrooms alike. The stakes are high – multiple studies now show that high-performing artificial intelligence technologies don’t always improve doctor performance. In order to fully recognize the health benefits of these powerful AI systems, medical educators will need to figure out how to better train doctors to work with AI. The ARiSE Research Network, based at Beth Israel Deaconess Medical Center / Harvard Medical School and Stanford University, will use gold standard social science methodologies to understand how doctors are currently using AI in their clinical care. They will then run national randomized trials to determine which educational strategies truly boost doctor-AI teamwork in important clinical tasks like diagnosing illnesses, explaining care plans to patients and coordinating treatment. The curricula and assessments from this project will be released openly so medical schools and hospitals around the world can train future doctors to use AI confidently while keeping patients safe.

PIs: Jonathan H. Chen, MD, PhD, Assistant Professor & Director for Medical Education in Artificial Intelligence, Stanford Center for Biomedical Informatics Research; Adam Rodman, MD, MPH, FACP, Hospitalist, Beth Israel Deaconess Medical Center; Assistant Professor, Harvard Medical School