PURPOSE
While generative AI tools show promise for personalized learning, limited
research examines their effectiveness when deployed without comprehensive
implementation frameworks. This study evaluated Google NotebookLM, configured
as a Socratic tutor, on biochemistry student performance when provided
without training, scaffolding, or assessment alignment.
METHODS
In Fall 2025, 63 students in a biochemistry course received voluntary
access to a faculty-curated NotebookLM notebook containing course materials
and Socratic tutoring guidelines. Students self-selected into AI users (n=43,
68.3%) or control (n=20). Primary outcome was Exam 2 to Exam 3 improvement,
compared against a Fall 2024 historical cohort (n=54) without AI access.
Statistical analyses included independent samples t-tests and exploratory
subgroup analyses by baseline performance (Exam 1 scores: Struggling 0.29).
CONCLUSIONS
Providing AI tutoring without implementation support produced no measurable
learning benefit. However, high voluntary adoption (68.3%), particularly
among at-risk students (86.7%), demonstrates student demand. Modest benefits
among motivated at-risk and metacognitively sophisticated strong students
suggest potential when proper support structures exist. Effective educational
AI integration requires comprehensive pedagogical frameworks rather than
standalone technology provision.