Purpose
The aim of this study was to create an ethically designed chatbot that generates high-quality, USMLE Step 1-style multiple-choice questions (MCQs) aligned with the gross anatomy, radiology, and ultrasound curriculum at Case Western Reserve University School of Medicine (CWRU SOM), providing access to formative assessments aligned with the local curriculum that also help prepare students for Step 1. This ensures equal access to formative assessments, promoting fairness and reducing financial barriers for use of MCQs to support study.

Methods
A chatbot powered by ChatGPT 4o was designed to create clinical vignette-based MCQs, styled like USMLE board exam questions, based on weekly curriculum content in gross anatomy, radiology, and ultrasound. It uses only open-access resources and integrates open-access images through an API, ensuring comprehensive question sets. Faculty members review and revise the questions to ensure alignment with learning objectives, accuracy, and usability for students. A qualitative content analysis was conducted to evaluate and categorize the edits provided by faculty members. This evaluation included one-on-one interviews with faculty to assess their views on the revision process and the chatbot’s utility. Further, we are analyzing student performance data to determine learning gains.

Results
Preliminary results indicate that the chatbot effectively generates curriculum-aligned MCQs. Faculty revisions have improved question quality by refining content depth, terminology, and alignment with the curriculum. Preliminary student performance data suggest these questions support effective learning.

Conclusions
While the chatbot successfully produces curriculum-specific MCQs, its application is currently limited to select content areas. Faculty involvement is essential for maintaining quality, and further development is needed to expand its scope across other subjects. This chatbot offers a scalable solution to address financial inequities, providing free and accessible formative assessments. With continued development, it has the potential to be applied across additional areas of the medical education curriculum.