A Prospective Multicenter Study Evaluating Endoscopy Competence Among Gastroenterology Trainees in the Era of the Next Accreditation System Purpose: The Next Accreditation System requires training programs to demonstrate competence among trainees. Within gastroenterology (GI), there are limited data describing learning curves and structured assessment of competence in esophagogastroduodenoscopy (EGD) and colonoscopy. In this study, the authors aimed to demonstrate the feasibility of a centralized feedback system to assess endoscopy learning curves among GI trainees in EGD and colonoscopy. Method: During academic year 2016–2017 the authors performed a prospective multicenter cohort study, inviting participants from multiple GI training programs. Trainee technical and cognitive skills were assessed using a validated competence assessment tool. An integrated, comprehensive data collection and reporting system was created to apply cumulative sum analysis to generate learning curves that were shared with program directors and trainees on a quarterly basis. Results: Out of 183 fellowships invited, 129 trainees from 12 GI fellowships participated, with an overall trainee participation rate of 72.1% (93/129); the highest participation level was among first-year trainees (90.9%, 80/88) and lowest among third-year trainees (51.2%, 27/53). In all, 1,385 EGDs and 1,293 colonoscopies were assessed. On aggregate learning curve analysis, third-year trainees achieved competence in overall technical and cognitive skills, while first- and second-year trainees demonstrated the need for ongoing supervision and training in the majority of technical and cognitive skills. Conclusions: This study demonstrated the feasibility of using a centralized feedback system for the evaluation and documentation of trainee performance in EGD and colonoscopy. Furthermore, third-year trainees achieved competence in both endoscopic procedures, validating the effectiveness of current training programs. The authors have informed the journal that they agree that Samuel Han and Joshua C. Obuch have completed the intellectual and other work typical of the first author. Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A712. Funding/Support: This study was funded by the American College of Gastroenterology Clinical Research Award (S. Wani). Ethical approval: Each study site received institutional review board approval and informed consent was obtained from each participant (ClinicalTrials.gov: NCT02891304). Other disclosures: None reported. Previous presentations: This project was presented in part at the American College of Gastroenterology Annual Scientific Meeting, October 9, 2018, in Philadelphia, Pennsylvania. Correspondence should be addressed to Sachin Wani, Division of Gastroenterology and Hepatology, University of Colorado Anschutz Medical Campus, Mail Stop F735, 1635 Aurora Court, Rm 2.031, Aurora, CO 80045; telephone: (720) 848-2786; email: Sachin.Wani@ucdenver.edu. © 2019 by the Association of American Medical Colleges

Teaching Systems Improvement to Early Medical Students: Strategies and Lessons Learned Purpose: Despite increasing emphasis in medical school education on quality and systems improvement, many medical schools lack sufficient faculty with expertise to teach systems improvement. Using the pedagogical content knowledge framework, this study explores how faculty engage students in systems improvement work and faculty perceptions of the outcomes for the health system and students. Method: In May-June 2017, the authors interviewed 12 of 13 invited faculty with experience in teaching and engaging first-year medical students in systems improvement work, the course of students’ systems improvement work over time, the impact of students’ projects on health systems, and students’ learning and attitudes about systems improvement. The authors conducted qualitative analysis iteratively with data collection to sufficiency. Results: Six emergent themes characterized faculty’s approach to guiding students in systems improvement work: faculty-student relationship, faculty role, student role, faculty-student shared responsibility for projects, faculty and student content knowledge, and project outcomes. The faculty-student relationship was foundational for successful systems improvement work. Faculty roles included project selection, project management, and health systems interactions. Students engaged in systems improvement as their faculty leveraged their knowledge and skills and created meaningful student roles. Faculty and students shared responsibility and colearned systems improvement content knowledge. Faculty defined successful outcomes as students’ learning about the systems improvement process and interprofessional collaboration. Conclusions: Findings highlight the critical importance of pedagogical content knowledge to engage early learners in systems improvement work, understand their learning interests and needs, and manage their projects longitudinally. The authors have informed the journal that they agree that both Monica W. Harbell and Descartes Li completed the intellectual and other work typical of the first author. Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A710. Acknowledgments: The authors thank Anna Chang, MD, clinical microsystems clerkship (CMC) director, and the University of California, San Francisco CMC team, for their support and feedback in conducting this research project, and Victoria Ruddick for her assistance in designing Figure 1. Funding/Support: None reported. Other disclosures: None reported. Ethical approval: This study was approved as exempt by the University of California, San Francisco Institutional Review Board on April 15, 2017 (Study Number 17-21762). Correspondence should be addressed to Descartes Li, Department of Psychiatry, University of California, San Francisco, 401 Parnassus Ave., Room 263, San Francisco, CA 94143; telephone: (415) 476-7448; email: Descartes.Li@ucsf.edu. © 2019 by the Association of American Medical Colleges

The Delphi Method No abstract available

Examinee Cohort Size and Item Analysis Guidelines for Health Professions Education Programs: A Monte Carlo Simulation Study Purpose: Using item analyses is an important quality monitoring strategy for written exams. Authors urge caution as statistics may be unstable with small cohorts, making application of guidelines potentially detrimental. Given the small cohorts common in health professions education, this study’s aim was to determine the impact of cohort size on outcomes arising from the application of item analysis guidelines. Method: The authors performed a Monte Carlo simulation study in fall 2015 to examine the impact of applying two commonly used item analysis guidelines on the proportion of items removed and overall exam reliability as a function of cohort size. Three variables were manipulated: Cohort size (6 levels), exam length (6 levels), and exam difficulty (3 levels). Study parameters were decided based on data provided by several Canadian medical schools. Results: The analyses showed an increase in proportion of items removed with decreases in exam difficulty and decreases in cohort size. There was no effect of exam length on this outcome. Exam length had a greater impact on exam reliability than did cohort size after applying item analysis guidelines. That is, exam reliability decreased more with shorter exams than with smaller cohorts. Conclusions: Although program directors and assessment creators have little control over their cohort sizes, they can control the length of their exams. Creating longer exams makes it possible to remove items without as much negative impact on the exam’s reliability relative to shorter exams, thereby reducing the negative impact of small cohorts when applying item removal guidelines. Acknowledgments: The authors would like to thank Brian Hodges for helping them to shape the ideas around issues raised in this manuscript, Valerie Dory for providing information regarding ‘real’ item properties used to simulate the data, and Jing Xiao for assistance with the figures in this manuscript. Funding/Support: The Paul Grand’Maison - Société des médecins de l’Université de Sherbrooke Research Chair in medical education and the SSHRC Insight Grant 435-2014-2159 were used to provide a stipend for André-Sébastien Aubin during his postdoctoral fellowship supervised by Christina St-Onge and Meredith Young. Other disclosures: None reported. Ethical approval: Reported as not applicable. Previous presentation: St-Onge C, Young M, Aubin A-S, Hodges B, Eva K. Limits of guidelines: A Monte Carlo Study exploring the application of item analysis guidelines to small cohorts. Canadian Conference on Medical Education, May 1, 2017, Winnipeg, Manitoba, Canada. Correspondence should be addressed to Christina St-Onge, Centre de pédagogie des sciences de la santé, Faculté de médecine et des sciences de la santé, 3001, 12e avenue Nord, Sherbrooke, Québec, Canada, J1H 5N4; telephone: (819) 821-8000, ext. 75047; email: Christina.St-Onge@USherbrooke.ca. © 2019 by the Association of American Medical Colleges

The AAMC Standardized Video Interview and the Electronic Standardized Letter of Evaluation in Emergency Medicine: A Comparison of Performance Characteristics Purpose: To compare the performance characteristics of the Electronic Standardized Letter of Evaluation (eSLOE), a widely used structured assessment of emergency medicine (EM) residency applicants, and the AAMC Standardized Video Interview (SVI), a new tool designed by the Association of American Medical Colleges to assess interpersonal and communication skills and professionalism knowledge. Method: The authors matched EM residency applicants with valid SVI total scores and completed eSLOEs in the 2018 Match application cycle. They examined correlations and group differences for both tools, United States Medical Licensing Examination (USMLE) Step exam scores, and honor society memberships. Results: The matched sample included 2,884 applicants. SVI score and eSLOE global assessment ratings demonstrated small positive correlations approaching r = 0.20. eSLOE ratings had higher correlations with measures of academic ability (USMLE scores, academic honor society membership) than did SVI scores. Group differences were minimal for the SVI, with scores slightly favoring women (d = -.21) and U.S.-MD applicants (d = .23–.42). Group differences in eSLOE ratings were small, favoring women over men (approaching d = -0.20) and white applicants over black applicants (approaching d = 0.40). Conclusions: Small positive correlations between SVI total score and eSLOE global assessment ratings, alongside varying correlations with academic ability indicators, suggest these are complementary tools. Findings suggest the eSLOE is subject to similar sources and degrees of bias as other common assessments; these group differences were not observed with the SVI. Further examination of both tools is necessary to understand their ability to predict clinical performance. Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A713. Acknowledgments: The authors wish to thank the Council of Residency Directors in Emergency Medicine (CORD-EM) Board of Directors for facilitating access to the Electronic Standardized Letter of Evaluation (eSLOE) database as well as DeAnna McNett for her contributions to data preparation and analysis and Michele Byers for administrative support of the project. They also wish to thank Keith Dowd for helping clean and link data files, Laura Fletcher for standardizing tables, figures and references, and Renee Overton for her feedback on early versions of the manuscript. Funding/Support: This project was supported by the Association of American Medical Colleges (AAMC) and CORD-EM (eSLOE database support) as a part of their routine operating budgets. Other disclosures: B. Naemi and D. Dunleavy are employees of the AAMC. Ethical approval: IRB review was obtained through American Institute for Research (FWA00001666), which approved AAMC Standardized Video Interview research for this study on September 30, 2017 as a component of a larger SVI master research program. Previous presentations: A subset of these data were reported at the following meetings: 2019 CORD-EM Academic Assembly, April 1, 2019, Seattle, Washington; 22nd Annual Western Regional Meeting of Society of Academic Emergency Medicine, March 22, 2019, Napa, California. Data: The eSLOE data were securely sent to the AAMC from CORD-EM on March 29, 2018. The SVI, demographic, and outcome data were obtained from the AAMC Data Warehouse on May 3, 2018. The manuscript was reviewed by CORD-EM, AAMC, and National Board of Medical Examiners leadership and was approved for publication by all parties. Correspondence should be addressed to Bobby Naemi, Association of American Medical Colleges, 655 K St., NW, Suite 100, Washington, DC 20001; e-mail: bnaemi@aamc.org. © 2019 by the Association of American Medical Colleges

Securing A Network for a Research-Intensive, Referral Academic Medical Center: University of Kentucky HealthCare as a Case Study Over the last fifteen years, UK HealthCare, the clinical enterprise of the University of Kentucky, has undertaken three clinical strategic plans in order to secure its position as a research-intensive, referral academic medical center. The first plan, titled Securing the Traditional Marketplace (2005–2010), focused on building advanced subspecialty programs on campus while pursuing partnerships with providers in UK HealthCare’s traditional marketplace, eastern Kentucky. The second plan, Expanding the Footprint (2010–2015), recognized that UK HealthCare needed to cover a population base of 5 to 10 million people to support its subspecialty programs. These two strategic plans were successful and achieved four outcomes: a doubling of annual discharges, a dramatic increase of transfers/external referrals, a significant increase in the case mix index, and impressive growth in subspecialty programs. The third clinical strategic plan, Preparing for Change (2015–2020), has expanded UK HealthCare’s gains in the face of rapidly changing reimbursement systems and delivery models. The pillars of this plan are responding to consumerism, strengthening hallmark programs through service lines, “hardwiring” relationships with partnering organizations including establishing the Kentucky Health Collaborative, and building infrastructure to deal with risk-based reimbursement. UK HealthCare is trying to spearhead a rational system of care for Kentucky rather than a system that rations care. Halfway through the third clinical strategic plan, UK HealthCare has seen increased discharges, transfers, and clinical expansion in its hallmark programs, building evidence that well-thought business practices can lead to improved public policy. Acknowledgments: The authors wish to thank two individuals who have contributed to each of the successive UK HealthCare planning cycles, Murray Clark and Joe Claypool. Funding/Support: None reported. Other disclosures: None reported. Ethical approval: Reported as not applicable. Correspondence should be addressed to Robert L. Edwards, Office of the Executive Vice President for Health Affairs, University of Kentucky, 900 South Limestone, 317 Wethington Building, Lexington, KY 40536-0200; telephone: (859) 323-3193, e-mail: rob.edwards@uky.edu; Twitter: @UK_HealthCare, @RobEdwardsUKY. © 2019 by the Association of American Medical Colleges

What Do I Do When Something Goes Wrong? Teaching Medical Students to Identify, Understand, and Engage in Reporting Medical Errors Problem: Identifying and processing medical errors are overlooked components of undergraduate medical education. Organizations and leaders advocate teaching medical students about patient safety and medical error, yet few feasible examples demonstrate how this teaching should occur. To provide students with familiarity in identifying, reporting, and analyzing medical errors, the authors developed the interactive patient safety reporting curriculum (PSRC), requiring clinical students to engage intellectually and emotionally with personally experienced events in which the safety of one of their patients was compromised. Approach: In 2015, the authors incorporated the PSRC into the third-year internal medicine clerkship. Students completed a structured written report, analyzing a patient-safety incident they experienced. The report focused on severity of outcome, root cause(s) analysis, system-based prevention, and personal reflection. The report was bookended by two interactive, case-based sessions led by faculty with expertise in patient safety, quality improvement, and medical errors. Outcomes: Students accurately analyzed the severity of the outcome, and their reports directly led to two formal root cause analyses and four system-based improvements. Next Steps: The time- and resource-efficient PSRC allows students to apply patient safety knowledge to a medical error they experienced in a way that can directly affect care delivery. This model—interactive learning sessions coupled with engaging in a personally experienced case—can be implemented in various of settings. Educators seeking to use student-experienced events for learning should not discount the emotional effects of those events on medical students. Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A709. Dedication: The authors dedicate this report to their friend, colleague, and coauthor Alan West, PhD, who passed away after the report was completed. They will miss his talents, humor, and deep pursuit of knowledge and truth. Acknowledgments: The authors wish to thank Bradley Vince Watts, MD, and Julia Neily, RN, MS, MPH, patient safety experts, for their analysis of the severity of medical errors described in the students’ reports. The authors wish to thank Spencer James, MD, for his assistance with the statistical analyses presented in this report and Susan D. Furste, RDN, for her grammatical expertise. Finally, the authors wish to thank Jeffrey Bell, MD, for his early work on the patient safety reporting curriculum. Funding/Support: This project was funded through a Geisel School of Medicine at Dartmouth Department of Medicine Advisory Council for Education (DOM-ACE) research grant and with the resources from the White River Junction VA Medical Center in Vermont. Other disclosures: None reported. Ethical approval: Ethics approval by a full Internal Review Board was waived by the Dartmouth College Internal Review Board, the Committee for the Protection of Human Subjects, on October 18, 2016, CPHS# 28762. Disclaimer: The work expressed in this report is that of the authors and does not necessarily reflect the views of the Department of Veterans Affairs or other agencies of the U.S. government. Previous presentations: Poster, “What do I do when something goes wrong?: Teaching Medicine Clerks to Identify, Understand, and Engage in Ameliorating Medical Errors on the Geisel Inpatient Medicine Clerkship”, at the Academic Internal Medicine Week 2017 of the Alliance for Academic Internal Medicine, March 19-22, 2017, in Baltimore, Maryland. Data: The authors’ dataset is available upon request. Please contact the corresponding author. Correspondence should be addressed to Hilary F. Ryder, One Medical Center Drive, Lebanon, NH 03756; telephone: (603) 650-8380; email: hilary.ryder@dartmouth.edu or hilary.f.ryder@hitchcock.org; Twitter: @hilaryfryder. Written work prepared by employees of the Federal Government as part of their official duties is, under the U.S. Copyright Act, a “work of the United States Government” for which copyright protection under Title 17 of the United States Code is not available. As such, copyright does not extend to the contributions of employees of the Federal Government. © 2019 by the Association of American Medical Colleges

Beyond Better, Safer, Cheaper Health Care: What’s Ultimately at Stake? Health care reform continues to be controversial and divisive and take its toll on physicians, patients, and national unity. An emphasis on efficiency and profit that depersonalizes human interactions hampers building physician–patient relationships grounded in compassion and trust. The authors argue that health care reform will be more effective if it is grounded in and anchored by a physician–patient relationship that is relationally transformative rather than transactionally commercial. This health care paradigm shift, from the “transactional-getting” to the “relational-giving,” must be physician-led. The authors propose three next steps. First, establishing discourse “ensembles” will foster conversations where new ideas can emerge and percolate and where participants can renew their collective stand for the humanitarian side of the healing relationship. Second, ensemble unity and effectiveness will be enhanced by the so-called “inward journey of leadership.” Without that journey, we cannot fully connect with the suffering of others, and we lack the wisdom and will to tackle our health care challenges. Third, to begin the process of solidifying this humanistic foundation, transformative leadership becomes essential. In contrast to transactional leadership, which motivates physicians by seducing their self-interests, transformative (relational) leadership connects physicians with their deeply-held values that embody what it means to be a physician and what it means to be human. A shared, collective view of what’s at stake if we settle for purely transactional medicine would help create the necessary physician alignment and commitment to reposition medicine as a profession that values service above reward. Funding/Support: None reported. Other disclosures: Wiley Souba teaches an Association of American Medical Colleges-sponsored course titled “Being a Resilient Leader” and an Association for Physician Leadership-sponsored program titled “Creating a Culture of Well-being and Wholeness.” Ethical approval: Not applicable. Correspondence should be addressed to Wiley Souba, 1 Rope Ferry Road, Hanover, NH 03755; telephone: (603) 650-1200; email: Chip.souba@dartmouth.edu © 2019 by the Association of American Medical Colleges

Achieving Gender and Social Equality: More Than Gender Parity is Needed In this Perspective, the authors review Association of American Medical Colleges data on gender parity and intersectionality, consider the literature on gender parity in academic medicine and the underlying gender norms that explain these statistics, and offer recommendations for moving past indicators of parity to achieve gender and social equality. Improvements in gender parity among medical school graduates have not translated to gender parity among practicing physicians or medical school faculty, particularly for racial/ethnic minorities. Further, gender parity does not correspond to gender equality, such that gender-based disparities in salaries and advancement persist. In addition, social norms related to traditional gender role expectations reinforce existing biases and lead to sexual harassment and the discrimination of women in the workplace. Building on their analysis of existing data and the literature, the authors offer concrete recommendations to achieve gender equality in academic medicine that not only improve parity but also support policies and practices to address the norms that further bias and discrimination. These recommendations include the collection, monitoring, and open reporting of data on salaries as well as on sex and race/ethnicity; stronger policies related to family leave and sexual discrimination and harassment; and accountability structures to ensure that policies are enforced. While these efforts alone cannot eliminate gender inequalities, academic medicine should be at the forefront of creating a climate in medicine that is supportive of gender equality as part of their larger goal of promoting social equality. Funding/Support: None reported. Other disclosures: None reported. Ethical approval: Reported as not applicable. Data: The data reported in this article are from Association of American Medical Colleges and are publicly available. Correspondence should be addressed to Anita Raj, Department of Medicine, University of California, San Diego School of Medicine, 9500 Gilman Drive, La Jolla, CA 92093; telephone: (858) 822-0229; email: anitaraj@ucsd.edu; Twitter: @AnitaRajUCSD. © 2019 by the Association of American Medical Colleges

Patient Shadowing: A Useful Research Method, Teaching Tool, and Approach to Student Professional Development for Premedical Undergraduates Problem: Questions have been raised about whether undergraduate institutions are effectively preparing premedical students in the sociobehavioral and cognitive reasoning content found on the revised Medical College Admission Test, providing opportunities to understand and apply these sociobehavioral and cognitive reasoning concepts in real-world scenarios, and offering career exploration opportunities. Approach: The Research in Physician-Patient Interactions course is a 15-week course designed for premedical students and taught through the collaboration of an emergency medicine physician and applied medical anthropologist. As of January 2016, the course is offered each spring at the University of South Florida, Tampa, Florida. The course provides opportunities for patient and physician shadowing within the anthropological methodological framework of participant observation. Other qualitative research methods are also taught, and students complete a group patient experience quality improvement project. Outcomes: Thematic analysis of students’ field notes and reflection essays and follow-up communications with course alumni revealed three salient themes regarding the utility of patient shadowing as a research method that provides unique types of qualitative data, as a teaching tool for premedical students to understand the perspectives of patients, and as an approach to developing the professional skills necessary in health care, such as effective communication styles, establishment of rapport, and empathy. Next Steps: Similar courses should be offered at other universities to premedical students. While it appears that patient shadowing experiences have a great impact during premedical education, there may also be value in integrating a similar experience into medical school and residency training. Acknowledgments: The authors would like to acknowledge Tampa General Hospital, the University of South Florida (USF) Honors College, the USF Department of Anthropology, and Charles Adams, PhD, dean of the College of Arts and Sciences, USF. Funding/Support: Funding for this course was provided by the USF Honors College. Other disclosures: None reported. Ethical approval: The USF institutional review board reviewed this study and determined that it is not human subjects research and so is exempt. Previous presentations: Data from this manuscript were presented at the Association of American Medical Colleges Western Group on Educational Affairs (WGEA) Regional Meeting in Denver, Colorado, March 24–27, 2018. Correspondence should be addressed to Jason W. Wilson, 1 Tampa General Cir, Tampa, FL 33606; telephone: (813) 843-2110; email: tampaERdoc@gmail.com. © 2019 by the Association of American Medical Colleges