Number
812
Name
Integrating generative AI into student-written formative assessments: A pilot initiative to enhance question quality and volume
Date & Time
Sunday, June 7, 2026, 5:30 PM - 7:00 PM
Location Name
Oglethorpe Ballroom
Speakers
Authors
Ruth Liepold, University of Cincinnati College of Medicine
Megha Mohanakrishnan, University of Cincinnati College of Medicine
Maia Zafirova, University of Cincinnati College of Medicine
Aaron Kakazu, University of Cincinnati College of Medicine
Brandon Jacome-Mendez, University of Cincinnati College of Medicine
Sidney Gossard, University of Cincinnati College of Medicine
Mary Hall, University of Cincinnati College of Medicine
Bhavana Pavuluri, University of Cincinnati College of Medicine
Michael Dempsey, University of Cincinnati College of Medicine
Bode Wamsley, University of Cincinnati College of Medicine
Kristin Duma, University of Cincinnati College of Medicine
Cameron Bennett, University of Cincinnati College of Medicine
Heather Christensen, University of Cincinnati College of Medicine
Presentation Topic(s)
Technology and Innovation
Description
PURPOSE
Formative assessments are critical preparatory tools for summative
examinations. The Student-Run Committee on Practice Exams (SCOPE) was
established at the University of Cincinnati College of Medicine (UCCOM) in
2024 to produce formative questions to prepare for in-house summative exams.
Initially, SCOPE participants wrote questions from scratch, followed by a
student peer-review process. SCOPE began considering generative AI in 2025
following a faculty-led pilot.
METHODS
Students completed anonymous surveys (26 M1s, 29 M2s; 14.4% and 16.1%
response rates, respectively) assessing perceptions of SCOPE and faculty
AI-assisted exams. The primary objective of AI integration is to address the
supply/demand challenge, given that only 46% of students previously felt
sufficient questions were available. SCOPE’s strategy for AI integration
involves leveraging generative AI for initial question writing, which will
then be followed by rigorous student peer review to ensure accuracy and
relevance.
RESULTS
Most students agreed that the original student-written SCOPE questions
reflected the style (87%) and difficulty (83%) of summative exams. However,
when evaluating course director-generated AI-assisted exams against the SCOPE
exams, 70% of M1 and 64% of M2 students found the AI-assisted exams inferior.
Despite this, 78% of M1 and 72% of M2 respondents supported integrating
generative AI into the SCOPE workflow, followed by peer review, to expand the
question bank. These findings support the proposed hybrid methodology to
address student demand for a greater volume of high-quality formative
questions.
CONCLUSION
Students value the authenticity of peer-written assessments and endorse a
hybrid approach. The study's results justify SCOPE’s strategy to leverage
generative AI for question expansion while maintaining quality through
rigorous student peer review to meet the significant student demand. This
methodology merges human contextual insight with AI efficiency and provides a
replicable model for other medical education institutions seeking to
efficiently increase the volume of authentic, high-quality formative
assessments.
Formative assessments are critical preparatory tools for summative
examinations. The Student-Run Committee on Practice Exams (SCOPE) was
established at the University of Cincinnati College of Medicine (UCCOM) in
2024 to produce formative questions to prepare for in-house summative exams.
Initially, SCOPE participants wrote questions from scratch, followed by a
student peer-review process. SCOPE began considering generative AI in 2025
following a faculty-led pilot.
METHODS
Students completed anonymous surveys (26 M1s, 29 M2s; 14.4% and 16.1%
response rates, respectively) assessing perceptions of SCOPE and faculty
AI-assisted exams. The primary objective of AI integration is to address the
supply/demand challenge, given that only 46% of students previously felt
sufficient questions were available. SCOPE’s strategy for AI integration
involves leveraging generative AI for initial question writing, which will
then be followed by rigorous student peer review to ensure accuracy and
relevance.
RESULTS
Most students agreed that the original student-written SCOPE questions
reflected the style (87%) and difficulty (83%) of summative exams. However,
when evaluating course director-generated AI-assisted exams against the SCOPE
exams, 70% of M1 and 64% of M2 students found the AI-assisted exams inferior.
Despite this, 78% of M1 and 72% of M2 respondents supported integrating
generative AI into the SCOPE workflow, followed by peer review, to expand the
question bank. These findings support the proposed hybrid methodology to
address student demand for a greater volume of high-quality formative
questions.
CONCLUSION
Students value the authenticity of peer-written assessments and endorse a
hybrid approach. The study's results justify SCOPE’s strategy to leverage
generative AI for question expansion while maintaining quality through
rigorous student peer review to meet the significant student demand. This
methodology merges human contextual insight with AI efficiency and provides a
replicable model for other medical education institutions seeking to
efficiently increase the volume of authentic, high-quality formative
assessments.
Presentation Tag(s)
Student Presentation