Grace Park, University of Illinois Urbana-Champaign
Chester Brown, University of Illinois Urbana-Champaign
Youngjin Cho, Geisinger Commonwealth School of Medicine
Purpose
Incorporation of foundational science probes to existing or novel cases require a wide array of expertise. The use of large language models (LLM) or generative artificial intelligence (AI) may provide an ability to augment the ability of medical educators to introduce a balance of foundational science topics to new cases in the undergraduate medical education curriculum.
Methods
An LLM (ChatGPT) was used to develop a Foundational Sciences Integration Advisor, a 7-step chatbot aimed at integrating unbiased and patient-centered questions into an existing or new case. A medical educator is asked to 1) provide learning objectives to a case; 2) input the case; 3) generate foundational science questions; 4) answer each probe with scientific references; 5) tag each question with the appropriate foundational science discipline and case learning objective; 6) integrate foundational science content into the revised case; and 7) provide a final review for bias and patient-centered language. We evaluated the feasibility of a chatbot to aid in the integration of foundational science probes in 5 different cases and provide an overview of coverage of different foundational science topics and performance.
Results
We evaluated the chatbot’s performance by providing 56 learning objectives covering the musculoskeletal system, across five cases designed for a first-year undergraduate medical course. Overall, probes were primarily focused on physiology (52%), pathology (32%), anatomy (28%), and histology (18%). When the chatbot was asked to provide specific scientific references, a 30% hallucination rate was observed, which limits the chatbot’s ability to aid the medical educator in guiding learners to appropriate resources without further vetting.
Conclusion
The use of generative AI tools as part of a foundational sciences integration advisor can augment the capacity of medical educators to provide a good balance of foundational sciences topics in new or existing cases.