Purpose
Medical students increasingly use AI chatbots for learning support, yet
these tools frequently generate inaccurate information. Meanwhile, faculty
availability for clarification remains limited outside business hours. We
deployed an AI teaching assistant that provides 24/7 support while
constraining responses to instructor-curated course materials, reducing
hallucinations while maintaining pedagogical utility.
Methods
We developed "NeuroBot TA" for a second-year Neuroscience &
Neurology course across two cohorts (n=190). The system embedded course
documents into a searchable vector database; when students ask questions, it
retrieves relevant content and generates responses grounded in course
materials rather than general internet data from pretraining. Students
accessed NeuroBot TA 24/7 through the learning management system. We
evaluated adoption through usage analytics and end-of-course surveys with
thematic coding.
Results
Students initiated 360 conversations generating 2,946 messages. Usage
surged during pre-exam periods (p<0.001) with substantial after-hours
utilization. Students primarily sought clarification on foundational content
and course logistics. Qualitative feedback revealed students valued 24/7
availability and appreciated that source-grounded responses increased trust.
However, "limited scope" emerged as a concern, where students noted
frustration when the system couldn't answer questions beyond course
materials.
Conclusion
The pre-exam usage surge and after-hours utilization demonstrate that
students leverage AI support at high-pressure times and when faculty are
least available. Student appreciation for source-grounding suggests this
approach addresses legitimate concerns about AI accuracy. However, scope
limitations that build trust also constrain utility, a fundamental tradeoff
students need to learn to navigate. For implementation, we recommend
including course logistics documents, deploying early in the curriculum, and
clearly communicating system boundaries.