Name
Evaluating the Accuracy of Course-Restricted and Internet-Based Chatbots for Animal Physiology Instruction at the Graduate and Professional Levels
Date & Time
Wednesday, June 24, 2026, 11:00 AM - 11:15 AM
Amy T. Desaulniers
Description

Chatbots are conversational artificial intelligence (AI) tools that can support student learning through practice and formative feedback. Recent meta-analyses report positive learning and engagement outcomes with chatbot integration, particularly in STEM disciplines. However, chatbot accuracy remains a major barrier to implementation, especially in conceptually demanding courses. Animal Physiology I is the first course in a year-long foundational sequence for Animal Science graduate students and first-year veterinary students. Students often perceive physiology as challenging due to its reliance on integrative, mechanistic reasoning across dynamic organ systems. Therefore, the objective of this pilot study was to qualitatively evaluate the accuracy of two chatbots with identical instructions and prompts but different knowledge sources (course-restricted or internet-based). A commercially available large language model–based generative AI system (ChatGPT, v5.2) was used. The course-restricted chatbot used a 25-page course note set, whereas the internet-based chatbot relied solely on web search capabilities. Both bots were instructed to function as course-aligned chatbots for graduate- and professional-level Animal Physiology, providing accurate, mechanistic explanations while avoiding unsupported claims. In general, both bots answered questions with a high degree of accuracy and provided appropriate answer explanations. Although subtle inaccuracies were observed from both bots, the course-restricted bot displayed greater accuracy on muddier topic areas. The course-restricted bot also showed tighter alignment with course materials and assessment structures, producing more concise, on-topic responses. Notably, the course-restricted bot paused due to ethical concerns when asked to complete an assignment, unlike the internet bot. Both bots struggled with interpreting tables and complex figures. Despite specific instructions to provide citations, neither bot included in-text references unless prompted. In summary, these findings suggest that chatbots may serve as effective supplemental tools for animal physiology instruction and that course-restricted chatbots may offer advantages in accuracy, instructional alignment, and response length compared with internet-based counterparts.

Location Name
Dunn Jr Conference Room
Full Address
The Mill at Mississippi State University
600 Russell Street
Starkville, MS 39759
United States
Session Type
Oral Presentation
Presentation Topic(s)
Practice of Teaching
Presentation Track(s)
Morning
Schedule Block
Block 1
Authors

Amy T Desaulniers, University of Nebraska Renee M McFee, University of Nebraska