Moderated by Kim Dahlman

PRESENTATION 1 - Teaching Innovative, Human-centered Design Skills as a Project Development Method Creates a Community of Innovators    
Chris Decker    
Medical College of Wisconsin

PURPOSE: We teach innovative, human-centered design skills through our incubator seed grant program. Participants develop their projects using our experiential-learning pedagogy that teaches them to think like innovators. Our intention is to train participants to think like innovators so they will innovate throughout their professional careers and across our university. We set out to understand if projects were continuing after completing the year-long incubator program, and if participants were utilizing the innovative skills they learned in work beyond their project.

METHODS: Participants learn and apply 10 innovation skillsets to develop their project. We assess these skillsets on a 6-point rubric and map participants' progress along a learning progression. After training, teams implement and continually iterate their project with the goal of ending the academic year with a program that is viable to be scaled and adopted into the university. In 2021, we surveyed past team leaders to learn if they were using the skillsets they learned in the program in other parts of their work.

RESULTS: Of the 35 project teams in the program over the past 3 years, 33 teams completed the program. Today, 22 of those projects continue to be implemented within our university (63%). 19 of the 33 team leaders responded to our survey (59%) where all 19 reported they are applying the innovative skills learned in the program in other parts of their work. 100% report using the skills monthly, and 47% of those participants more than once per week.

CONCLUSION: We have demonstrated that applying innovative, human-centered design is a successful approach to developing projects at our university, evidenced by the 63% continuation rate. We also demonstrated that the innovative skillsets are being transferred to work outside of the program. This suggests participants find value in the skills and therefore apply them in other contexts.
 

PRESENTATION 2 - Use of a Novel Virtual Reality and Tablet App to Support Learning of Anatomic and Physiologic Changes During Pregnancy    
Roberto Galvez    
University of Illinois Urbana Champaign 

PURPOSE Traditional approaches to learning human reproductive anatomy and physiology do not convey the dynamic nature of pregnancy. To address this limitation, we developed an interactive reproductive software application for use across virtual reality (VR) or smartphone and tablet devices and integrated its use within a pre-clinical obstetrics course. The following describes its utility in the course while investigating the efficacy and acceptance of VR as a learning tool for pre-clinical medical students.

METHODS The Carle Illinois College of Medicine built upon existing software called "Road to Birth"(RtB), originally developed at The University of Newcastle. Students were assessed pre-course on their visuospatial ability and surveyed on their technological familiarity. RtB was integrated into the course as a lecture preparatory tool, problem-based-learning session resource, method for assessment, and as a supplemental resource. Students also engaged in weekly surveys, end of course focus group discussions, and NBME-style quizzes.

RESULTS Nineteen students participated in the study. Students performed high on visuospatial ability but reported no VR recreational use. Post-survey responses demonstrated that 63% of the students felt the app increased their knowledge; however, only 32% felt the app was a good use of their time. Average weekly in-app quiz performance was 83%, comparable to NBME-style quiz performance of 79%. Students reported perceiving VR as inefficient; however, actual time to complete the in-app quiz did not drastically differ (tablet: 13.9±5.3min, VR: 12.4±4.9min). This perceived inefficiency warrants further exploration of how these technologies can be more effectively and intentionally used in medical education.

CONCLUSIONS Our findings demonstrate that technologically advanced educational materials are generally supported by medical students and can be successfully integrated as a multi-modal curricular tool. However, educators should consider the time required to use such tools and student prior perceptions as these factors heavily influence student preferences and use patterns.
 

PRESENTATION 3 - Student-led Curricular Development in the Biomedical Science Master's Program Using Virtual Dissection    
David Wang    
College of Osteopathic Medicine, Marian University

PURPOSE Cadaveric dissection is an important tool within graduate medical education (GME), but not always possible to implement. When cadaveric dissection cannot be conducted, virtual dissection may be used to supplement anatomical education. We developed student-designed curricular activities to incorporate the Anatomage Table into GME. The goal of this project is to determine whether integration of this tool improves student learning outcomes in Marian University's (MU) Biomedical Science Master's (BMS) program.

METHODS Our method is notable and novel; former BMS Master's students created activities which were designed to align with lecture content and used for the pilot study presented here. These students-turned-teachers offered unique perspectives and approaches to material.  Participation in this study was voluntary. Participants (n=24) and non-participants (n=29) were both exposed to the Anatomage Table during formal anatomy lectures; however, participants received additional, directed time with the table. Unpaired t-tests were used to compare exam performances between groups, and surveys were distributed intermittently to measure changes in students' perceptions of virtual dissection technology and its use in supplementing traditional anatomy education.

RESULTS Students who participated in the Anatomage Table activities consistently scored higher, on average, on all exams, exam question categories supplemented by Anatomage activities, and within the course. Notably, significantly higher scores (p<0.05) among participants were observed for several exam question categories related to the activities. Additionally, qualitative data from surveys before and after students' experience with the Anatomage Table shows an increase in confidence, comfort, and motivation for learning using virtual technology.

CONCLUSIONS This research aimed to determine whether implementation of a virtual dissection tool improved student learning and exam scores. Ongoing quantitative and qualitative analyses suggest that use of the Anatomage Table, paired with former student-created activities, can benefit student learning and consequent course performance.
 

PRESENTATION 4 - Can Artificial Intelligence Address the Burden Associated With Scoring Narrative Assessments?    
Denise Kay    
University of Central Florida College of Medicine

PURPOSE A long-standing problem for educators has been determining how to create meaningful assessments that can be efficiently and reliably scored within a reasonable time frame. Medical educators must assess students' clinical skills and reasoning which, if done right, requires time and labor-intensive performance-based assessments, such as Objective Standardized Clinical Examination (OSCEs). OSCEs are composed of multiple encounters with standardized patients. After each encounter, students complete a post-encounter note (PEN). Scoring PENs is time and labor-intensive. This project developed and tested an artificial intelligence model to automatically score OSCE PENs.

METHODS The AI model was developed in two steps. 1) We used medical textbooks to pretrain the model to understand the specialized key medical terms needed to score OSCE PENs. 2) The pretrained model was then fine-tuned using an 80-20 testing experiment wherein PENs from previously graded, five station OSCEs was split, with 80% for additional model training (n=3335) and 20% for model testing (n=660).

RESULTS According to the results on the testing dataset (n=660), the current model grades students' examinations with a 6.23% error (i.e, 0.24 Mean Squared Error). Moreover, the results show that the model outperforms state-of-art models that are pretrained on clinical notes.

CONCLUSION The performance of the current AI model suggests a computer can be trained to score PENs and can provide a solution for labor-intensive OSCE grading. The next step is to continue training the current AI model using all required medical textbooks at this medical school (n=310). We also plan to pilot the further refined model in action with ungraded and graded PENs in future examinations. With increased exposure to examination data and medical terminology, we hypothesize that the accuracy of scoring should increase by a larger margin.