Moderated by Carrie Elzie

PRESENTATION 1 - Diversity Among the Iamse Medical Educator Fellowship Program Graduates    
Khalil Eldeeb    
School of Osteopathic Medicine, Campbell University

PURPOSE The International Association of Medical Science Educators (IAMSE) launched The Medical Educator Fellowship Program in 2010 to develop medical education scholars and leaders. The program operates currently in three phases: (a) completion of the AMEE Essential Skills in Medical Education course, (b) participation in 12 hours of related and preapproved faculty development workshops, and (c) presentation of a mentored capstone research project. However, there is limited data about the program outcomes. In this study, we examined the diversity among the program graduates.

METHODS This study reviewed the profiles of the IAMSE Fellows between 2010-2021. We utilized data published on the IAMSE website, Medical Sciences Educator Journal, and other public websites to gather demographic, geographic distribution, terminal degree, and specialty information of the IAMSE Fellows.

RESULTS During the period 2010-2021, the IAMSE fellowship program inducted 40 Fellows, of which (62.5% n=25) were females. Terminal degrees distribution among the Fellows was Ph.D. (55 %), M.D. (25%), and M.D./Ph.D. (15%), M.S. (2.5%), and EdD (2.5%). Geographically, the Fellows were in North America (72.5%), Asia (10%), South America (7.5%), Europe (5%), and Australia (5%). Most of the Fellows (62.5%) work in the U.S.A., mainly in the northeastern and southeastern states. The Fellow's affiliations included allopathic medical schools (77.5%), osteopathic medical schools (15%), and physical therapy (2.5%), arts and sciences (2.5%), and chiropractic (2.5%). The Fellow's scientific specialties included basic sciences (72.5%) and clinical sciences (27.5%). Thirty-seven fellowship capstone projects were identified and focused on teaching methods and strategies (46%), assessment (38%), and curriculum design (16%).

CONCLUSION The IAMSE fellowship program graduated a diverse group of health sciences educators and scholars. Future studies will assess the impact of the fellowship on the Fellow's scholarship, teaching, academic promotion, and leadership activities.

PRESENTATION 2 - Student Learning From Multiple Rounds of Self- and Peer- Evaluation Decreases Initial Bias Seen Due to Gender    
Simran Singh  
Western Michigan University Homer Stryker MD School of Medicine

PURPOSE Self- and peer-evaluation are commonly used to teach professionalism in medical education. While self- and peer-evaluation can improve professionalism, feedback is influenced by various demographic factors including gender, race, and similarity between the rater and the ratee. Most studies correlate data from a single round of self- or peer-evaluation with a particular outcome or demographic factor. In this study we tested the hypothesis that students learn from the evaluations provided by their peers to improve the accuracy of their self- and peer-ratings in subsequent rounds of evaluation minimizing the bias that may initially exist based on demographic factors.

METHODS Self- and peer-evaluation was completed by 86 M1 allopathic medical students. Students worked in teams of six during Team Based Learning activities. Students evaluated eight skills involved in teamwork for themselves and each of their peers, from which a mean teamwork score was calculated. The difference between student self- and team-scores was determined and compared across gender, age, highest degree, gap year duration, and under-represented in medicine status.

RESULTS After the first-round of peer-evaluation, gender was significantly associated with the difference in self- and group-teamwork scores. More females underrated themselves (27.5% female vs 8.9% males), while more males overrated themselves (20% males vs 7.5% females) compared to their group teamwork rating (p=0.007).  After the second-round of peer-evaluation a greater proportion of students were within one standard deviation of their peer's ratings (65% females, 71.1 % males after round-one, versus 80% females and 82.6% males after round-two), eliminating the difference initially seen by gender (p=0.344). 

CONCLUSION This suggests that students learned from the feedback provided by their peers to more accurately evaluate themselves and their peers; minimizing the gender bias seen in initial peer evaluation. This minimizing of bias further increases the value of peer-evaluation for learning professionalism in medicine.

PRESENTATION 3 - Evaluating Interprofessional Education Training Program for Oral Health Champions in Six States    
Robin Harvan    
Massachusetts College of Pharmacy and Health Sciences

PURPOSE The National Center for Integration of Primary Care and Oral Health (CIPCOH) is committed to evolving the oral health curriculum in every health profession school/program in the U.S. (medical, osteopathic, physician assistant, nurse practitioner, midwifery, obstetrical, internal medicine, family medicine, pediatrics, med-peds, geriatrics). We developed, offered and evaluated a pilot of our 100 Million Mouths (100MM) Project because we feel that if we can create an Oral Health Education Champion in every state, then we can support that Champion in reaching out to health profession schools/programs in their state and engaging them to improve their oral health teaching. As those students and residents graduate and care for patients, they will improve the oral health of millions of patients.

METHODS In 2021, six oral health champions were selected by the CIPCOH representing states assessed with greatest needs and potential benefits for preparing champions of Primary Care Education Enhancement of Oral Health Integration in educational training and collaborative practice. These states included Alabama, Delaware, Hawaii, Iowa, Missouri, and Tennessee. An Evaluation Framework designed according to systems change theories guided formative and summative evaluation activities.  Inputs, processes, outputs and outcomes performance indicators were identified related to the pre-training, intra-training and post-training phases of training.

RESULTS Trainees were very appreciative of the opportunity to be selected as their State representative OH Champion. Trainees generally appreciated the data-driven identification of their states as target for recruitment; the process of notification of selection and induction into the training program; and that their anticipated expectations and aspirations would be considered and met. Program participation and retention were exceptional.

CONCLUSION Trainees would overwhelming recommend this program to future state OH Champions. Overall, trainees would recommend that OH Champion Collaborative Teams for each state be considered in the future to strengthen interprofessional integration and state-wide reach.

PRESENTATION 4 - A Practical and Efficient Faculty Review Process for Improving Multiple Choice Question Metrics    
Monzurul Roni    
University of Illinois College of Medicine Peoria 

PURPOSE Despite their curricular importance, a large proportion of multiple-choice questions (MCQs) are flawed and/or fail to assess the intended knowledge domain or competency. Although faculty peer or committee review has been demonstrated to improve the quality of MCQ exams, such review processes are typically resource-intensive and time-consuming.  We report here a practical and efficient peer review process that consistently improves MCQ exam quality.

METHODS A small peer review committee was formed to review the final exams of six independent block courses taken by first- and second-year students at the University of Illinois College of Medicine. The committee consisted of four to five faculty with experience in item writing. Based on post-exam item metrics, MCQs that had both a discrimination index > 0.2 as well as difficulty level > 0.91 were selected for review. For every exam, the reviewers independently evaluated each item over a two-week period and provided evaluative comments through a common secured document. The committee convened a single virtual meeting to arrive at a consensus on item flaws while providing suggestions for improvement. The final comments were shared with the block directors who then communicated the changes and corrections with the original item authors.

RESULTS Based on the above metrics, 15-20 percent of MCQs per block exam were peer reviewed. Each reviewer took approximately 10 min to review each question and committee meetings were about 90 minutes in length. Upon exam re-administration, this process resulted in a significant increase (p>0.05) in the item discrimination of reviewed questions in all six block exams.

CONCLUSION Our results support the utility of a peer review process in MCQ exam quality and may offer medical educators a more practical and efficient process, requiring limited faculty time and administrative resources, for improving the quality of in-house MCQ exams.