Presented By: Erica Sutton, Morehouse School of Medicine
Co-Authors: Courtney Cross, OnlineMedEd

Purpose
While the use of third-party resources has become more prevalent, research on the effects of these resources is minimal. Evaluating their impact is challenging. To gain a deeper understanding of how third-party resources are being integrated and their impacts, it is crucial to continuously evaluate their usage and adoption, and to implement designed interventions if needed. 

Methods 
Morehouse School of Medicine partnered with an online medical education learning platform to provide comprehensive supplemental learning material to 3rd year MD students. The resource was evaluated using the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) program evaluation framework. In this study, we identified measurements for each dimension to comprehensively evaluate the implementation of the learning platform using quantitative data collected by the school and from the platform and qualitative data collected from two learner focus groups. 

Results 
Of the 85 students enrolled, 70 (81%) logged in and completed content. Students with a delayed rotation schedule or participating in the Fee Assistance Program were more likely to use the resource. 100% of students completed video content, while 60% completed reading content. A significantly smaller percentage of students utilized board-style questions or flashcards (p<0.05). Qualitative data showed preferred learning styles and lack of awareness of other learning modalities primarily underlie adoption. Students used the content primarily to prepare for shelf exams. They identified the need to engage with the content to truly learn, usually through notetaking and review. High resource usage was associated with higher Step 2 scores, while low usage was associated with delaying taking the Step 2 exam (p<0.05). 

Conclusion 
Preliminary data using the RE-AIM framework and qualitative feedback indicate positive utilization and outcomes from implementing an online medical education learning platform. Next steps include integration of the platform into the curriculum and implementing strategies to improve Reach, Adoption, and Implementation.