Moderated by Kerrie Lashley
Session Coordinator: Dologar Baatar

Presentation 1 - Combining Computer and Learning Sciences: Using Artificial Intelligence to Systematically Assess OSCE PENs  
Denise Kay    
University of Central Florida College of Medicine

Purpose
The Observed Structured Clinical Examination (OSCE) plays a critical role in assessing medical students' clinical skills. Scoring OSCE exams is resource intensive, especially the Patient Encounter Notes (PENs) often part of the student exam. Using artificial intelligence (AI) that can provide automatic, trustworthy PEN note scoring can free up faculty time, providing more opportunities for hands-on training with students. To alleviate the expensive process of requiring careful grading by faculty for each examination, we developed an AI model that is now increasingly accurate in automatically grade unseen examinations.

Methods
The deep learning AI model was developed in two phases. In phase 1, we tested the model's performance based on training with 5, seen OSCE stations (trained on 80% of PEN notes and faculty scores and independently scoring the remaining 20%). Now, in phase 2, we tested the models' performance on 5, completely unseen stations presented with different grading criteria. This is a more challenging task for the model given that, without any prior knowledge about the new stations, the model's accuracy can be compromised.

Results
Without prior training on OSCE stations or the grading criteria, the model scored new, previously unseen PEN notes and/or scoring rubrics with 78.2% accuracy (phase 2), an accuracy level of only 4.42% less than when the model scored PEN notes and rubrics it was familiar with via training on 80% of the PEN notes and scoring the remaining unseen 20% (phase 1).

Conclusion
This study shows that our model is able to automatically grade unlabeled and unseen OSCE stations with different scoring criteria. In phase 3, we plan to compare the AI scoring of unseen OSCE stations with two or more human raters to establish trust in the AI model competence.

Presentation 2 - Concept Mapping in Team-based Learning (TBL) to Assist Medical Students in Integrating Complex Topics
Kyeorda Kemp    
Oakland University William Beaumont School of Medicine

Purpose
Novice medical students are not always able to connect concepts from different disciplines. The concept mapping methodology allows for a visual representation of information and is a highly effective learning strategy that allows learners to make connections between complex topics. This study aimed to investigate the effectiveness of using concept mapping to integrate medical school topics during a TBL activity held during the first semester of medical school. 

Methods
First-year medical students (n=125) engaged in a TBL to reinforce immunology and microbiology concepts taught in a basic foundations course. After completing Readiness Assurance Tests, student groups were instructed to create concept maps to illustrate how the loss of CD4+ T-cells due to HIV infection leads to an inability to clear opportunistic fungal infections as an application exercise. A class consensus concept map was created that could be used for future reference. A seven-question survey was distributed to assess student attitudes regarding this learning method.

Results
Thirty-one students responded to the survey. Overall, 77% of students were satisfied with how the TBL improved their understanding of immunology, and 63% felt their understanding of microbiology was enhanced. 74% of the respondents felt the experience allowed them to identify weaknesses in immunology, while 62% thought they could identify weaknesses in microbiology. Paradoxically, 45% agreed that concept mapping was suitable for a TBL. Free responses from the students indicate that they prefer an MCQ-based TBL over a concept mapping session TBL.

Conclusions
While concept mapping in a TBL format allowed students to identify weaknesses and clarify concepts, students disliked the format and preferred a standard MCQ-based format.

Presentation 3 - Facilitating the Use Evidence-based Learning Strategies via a Platform-based Model of Curricular Delivery  
Rasha Noureddine    
University of Central Florida

Purpose
Evidence-based learning strategies such as Interleaving and Spaced retrieval have been clearly validated as highly effective ways of augmenting the acquisition of knowledge, yet they are rarely implemented as part of a curricular structure in a defined way. This is likely because courses are generally taught by different subject matter experts making spacing and interleaving of content often difficult to achieve and monitor. Internet-based learning platforms can help solve this problem by facilitating the use of these strategies and tracking compliance.

Methods
The use of an Internet-based learning platform was used to deliver didactic content to students at a government-chartered private medical school established to increase capacity in an underserved part of Jamaica. To optimize the learning potential of the students, many of whom lacked well developed study skills, the curricular content was delivered via an Internet-based learning platform which was designed to facilitate the use of evidence-based learning strategies. All didactic videos were followed by recall questions which, after being answered, were entered into an algorithm-driven spaced repetition deck. Pop-up summary notes encouraged free recall and customized assignments allowed for the interleaving of concepts. Compliance and performance could be tracked which allowed academic coaches assigned to each student to help them refine their study strategies, and teachers to adjust the focus of their teaching sessions.

Results
Using a platform-based approach to implementing and tracking the use of evidence-based learning strategies helped overcome the hesitation to use these tools and thus leverage the benefits of using strategies associated with "desirable difficulty".

Conclusion
Using Internet-based platforms to integrate and leverage the use of evidence-based learning strategies provided significant benefit to the students in this school and should be considered for use in a wide array of settings due to its ease of use. 

Presentation 4 - Use of a Modular Mastery-directed Curricular Structure Designed to Accommodate a Diverse Group of Medical Students    
Peter Horneffer    
All American Institute of Medical Sciences

Purpose
As efforts to create a more diverse and inclusive physician workforce increase, students from less advantaged socioeconomic backgrounds often have less well developed academic skills and may need additional time to assimilate the large amount of new material introduced in medical school curricula. Students who ultimately acquire the study skills can be adversely affected when they are held back from their cohorts in traditional lock-step model of academic progression. Internet-based learning platforms now have the ability to facilitate differential progression through a set of curricular requirements.

Methods
A medical school established to increase the supply of physicians in the underserved country of Jamaica recently transitioned to a modular mastery-directed approach to delivering its medical school curriculum. All curricular content was assigned to students through an Internet-based Learning platform. The platform tracked progression and compliance as well as facilitated the use of evidence-based learning strategies to augment understanding and retention of concepts. Students typically started the program taking a course load of 4 courses/ term but were advised to reduce their course loads to just two or three courses/ term if their academic performance was marginal. They could subsequently take additional courses as their performance improved. Their progression and studies were guided by academic coaches.

Results
While there was initial resistance to being decelerated, students quickly realized the advantages of being able to spend more time on a given course and devoid of the pressure to maintain an excessive workload, they could focus on improving their study skills and comprehension of the material. Initial results show that grades improved when course loads were limited.

Conclusion
Variable progression through a medical school curriculum facilitated by Internet-based platforms provides a student-centric approach to the educational process allowing a more diverse group of students to be educated as physicians.