The following represents the foreword to the August 2023 supplement of Academic Medicine focused on Ensuring Fairness in Medical Education Assessment, the topic of our 2022 Macy Foundation Conference. Authored by Dowin Boatright, MD, MBA, Louito Edje, MD, MHPE, Larry D. Gruppen, PhD, Karen E. Hauer, MD, PhD, Holly J. Humphrey, MD, and Kayla Marcotte, MS, this foreword introduces the consensus recommendations emanating from conference, as well as the commissioned papers and case studies that provided background and context for discussion during the conference. These many papers are now available in their entirety online ahead of print; you can find links to each of these papers and case studies at the end of this post. While we acknowledge that fairness in assessment is inherently complex, we also believe it serves as a critical area of focus for medical educators in the quest to best serve both learners and patients.
We have reached an inflection point in medical education’s commitment to diversity, equity, and inclusion (DEI). The challenge we face—and will continue to face for the foreseeable future—is to keep moving forward despite the impediments, including a health care system and workforce depleted by the pandemic and by ongoing political, social, economic, legislative, and legal pressures that make it difficult to act. But in difficult times such as these, looking ahead and creating a shared vision for the future can sustain us. We believe that a crucial step toward improving health and health care for everyone is prioritizing equity in medical education assessment systems. We know that achieving DEI in our teaching and training institutions is necessary to build the health care workforce needed for the future, but it can’t be done without confronting and eliminating harmful bias in learner assessment. This is a challenging and complex problem, but we believe that honoring our commitments to our learners and patients will move us forward. This supplemental issue of Academic Medicine features a set of recommendations and 8 supporting articles that can help guide the way.
We are heartened by the indicators suggesting that medical education is making advances in the direction of DEI. According to the Association of American Medical Colleges (AAMC), the majority of medical schools are working to integrate DEI across many levels of their institutions, including the adoption of holistic review processes in admissions (which may be complicated by the impending U.S. Supreme Court decision regarding race-conscious admissions). And AAMC’s most recent medical school enrollment figures indicate that the numbers of Black and Hispanic matriculants are on the rise, as are the numbers of women and matriculants from lower socioeconomic backgrounds. Another indicator: the nascent movement among some prominent medical schools—following the lead of elite law schools—to stop participating in U.S. News & World Report’s (USNWR) annual numerical rankings of institutions over concerns that they reinforce biased practices and inhibit diversity while also failing to accurately reflect the quality of the education being provided.
Dr. Katrina Armstrong, dean of the Vagelos College of Physicians and Surgeons at Columbia University, explained her school’s decision: U.S. News’ medical school rankings “perpetuate a narrow and elitist perspective on medical education … rather than [reflecting] a school’s success in educating a diverse and well-trained cohort of doctors able to change medicine for the better and meet society’s needs. Their focus on standardized test scores comes at a time when it is widely understood that prioritizing these scores rewards well-resourced applicants without regard for selecting the individuals who can best serve the future needs of a diverse and changing world.”
Dr. Armstrong was explaining a difficult decision, but her words reflect a much broader reality: Beyond the over-reliance on standardized tests that advantage more affluent learners, a growing body of research indicates that harmful bias plagues assessment for everyone, particularly learners from groups historically underrepresented in medicine. The literature documents population group differences in standardized examinations, clinical assessments, grading, and academic awards between medical learners who are under-represented and those who are well represented in medicine, differences that virtually always favor the latter group. Studies also suggest that minor differences in assessment outcomes can have a powerful impact on learners’ residency and career opportunities. When disparities related to assessment limit opportunities for physicians from groups historically marginalized in medicine, they may, in turn, impede efforts to enhance physician workforce diversity, which is associated with better patient-centered care, health care access, and patient outcomes.
The literature supporting these and related findings is described in the 9 articles featured in this supplemental journal issue. Eight of the 9 articles explore the impacts of harmful bias in medical education assessment and suggest steps to help ensure that all learners are assessed fairly. The articles were commissioned by the Josiah Macy Jr. Foundation as part of its November 2022 conference, Ensuring Fairness in Medical Education Assessment, which brought together 50 medical education leaders, administrators, faculty, and learners with expertise or significant experience in areas related to DEI and/or learner assessment to develop consensus recommendations around creating equity in assessment. The set of 5 consensus recommendations and supporting action steps—the focus of the remaining article in this issue, titled “Conference Recommendations Report”—were developed by the conferees based on their discussions of the 8 commissioned articles.
The topic of fairness in medical education assessment was chosen for the November 2022 conference as a follow-up to a previous Macy Foundation conference, which focused more broadly on addressing harmful bias and eliminating discrimination in health professions learning environments. During the earlier conference, it became clear that learner assessment in medical education requires immediate and deliberate action if medical educators are to create and sustain learning environments that make everyone who works and learns within them feel like they belong.
During the Macy Foundation conference on ensuring fairness in medical education assessment, the conferees developed a shared vision for moving toward an optimal learner assessment system: “Medical education communities must create, implement, and continuously improve programs of learner assessment that confront the historic legacies of racism and oppression; advance diversity, equity, and justice within a culture of compassion, respect, and dignity; and equip learners and faculty to provide equitable, high-quality, patient-centered health care.” Within this broader vision, the conferees described what an equitable assessment system looks like.
Specifically, such a system prioritizes assessment for learning in combination with assessment of learning to promote growth mindsets in learners. It is evidence-based and uses criterion-based rather than normative frameworks. It uses low-stakes, formative assessment approaches that minimize performative mindsets in learners, including frequent feedback and coaching, and it provides robust supplemental resources to support learners facing educational challenges. It also trains learners in how the assessment system works and offers them transparent information, including how assessment data are used and shared and how fairness and equity are monitored and ensured.
Aware, however, that this optimal assessment system can’t be designed, implemented, or continuously improved upon unless it is situated in a safe and sustaining culture, climate, and learning environment, the conferees also made recommendations and suggested action steps that create supportive scaffolding. These include recommendations to align assessment with core values, such as improving patient care and prioritizing DEI; create and provide faculty development programming in designing, implementing, and continuously improving equitable assessment; develop mechanisms for and remove barriers to co-creation with learners; and more.
As medical educators, we are committed to building a physician workforce that is well-prepared to improve the health of all patients—and we know that a requirement for building that workforce is enhancing its diversity through equity, inclusion, and belonging. Currently, harmful bias permeates learner assessment in medical education. It exacerbates disparities among learners, fosters competition rather than collaboration and teamwork, and fails to recognize many of the positive attributes that learners from diverse backgrounds bring to their medical school experience and future practice. If we are to develop and sustain a physician workforce that can eradicate health disparities and lead an equitable health system, we must do what we can now to address these deficiencies and optimize assessment.
- “My assessments are biased!” Measurement and sociocultural approaches to achieve fairness in assessment in medical education
- The long shadow: A historical perspective on racism in medical education
- Strategies to counteract impact of harmful bias in selection of medical residents
- The urgency of now: Rethinking and improving assessment practices in medical education programs
- Reconstructing Neurath’s ship: A case study in re-evaluating equity in a program of assessment
- Strategies for advancing equity in frontline clinical assessment
- Understanding and addressing bias in grading: progress at Washington University School of Medicine
- Student and Teacher Perspectives on Equity in Clinical Feedback: A Qualitative Study Using a Critical Race Theory Lens
Adapted with permission from:
Boatright, Dowin MD, MBA, MHS; Edje, Louito MD, MHPE, FAAFP; Gruppen, Larry D. PhD; Hauer, Karen E. MD, PhD; Humphrey, Holly J. MD; Marcotte, Kayla MS. Ensuring Fairness in Medical Education Assessment. Academic Medicine ():10.1097/ACM.0000000000005244, April 18, 2023. | DOI: 10.1097/ACM.0000000000005244