Presented By: Gabriel Yapuncich, Duke University School of Medicine
Co-Authors: Jennifer Carbrey, Duke University School of Medicine
Matthew Velkey, Duke University School of Medicine

Purpose 
Generative artificial intelligence programs, specifically large language models (LLMs), have the potential to be powerful tools in educational settings. This study utilizes LLMs to summarize didactic lectures in the biomedical science curriculum at the Duke University School of Medicine. As succinct descriptions of session activity, the summaries may provide multiple benefits for students and educators, including aiding student review, integrating with curriculum management processes, and modeling appropriate use of generative LLMs. 

Methods 
Session transcripts of 42 physiology lectures were input into the GPT-3.5 LLM with the prompt to summarize the text in 5-6 sentences. After minor editing of the output, the summaries were posted as supporting information on the session Canvas webpage. The utility of the summaries was evaluated through 1) narrative comments were solicited from students during course feedback sessions and 2) specific questions on the end-of-course survey. The utility for curriculum management was evaluated by directly comparing educator-provided objectives with the content of the summary. 

Results 
Students had a positive response to the LLM-generated summaries but expressed concerns about the reliability of generative AI to capture key points (i.e., to accurately summarize the learning objectives). However, direct comparisons between the educator-provided objectives and summaries show very strong correspondences, with the LLM-generated summaries generally providing more detailed objectives. 

Conclusion 
This study demonstrates a straightforward but effective method to leverage the potential of LLMs in biomedical science education. The LLM-generated summaries created from session transcripts serve as easily interpretable signposts of session activity for students. They also facilitate curriculum management, providing information on session content and goals/objectives that can be readily incorporated into a curriculum database. However, areas of improvement remain, particularly clearly demonstrating the process and reliability of generative AI to all members of the learning community.