The USMLE Step exams are designed to assess students' critical analysis skills through questions that integrate clinical and basic science knowledge. To prepare effectively, students need to practice with multiple-choice questions (MCQs) that match the quality and complexity of USMLE items. However, many faculty face challenges in creating such high-quality assessments due to a lack of specialized training, the time-intensive nature of writing questions, and the demanding review process.

Generative AI offers a solution by assisting faculty in efficiently crafting USMLE-style MCQs. This workshop will introduce strategies for leveraging AI as a co-pilot in the question development process, enabling faculty to produce high-quality formative and summative assessment items that enhance student preparation for the USMLE Step exams.
Participants will review the purpose of the NBME licensing exam, identify the features of high-quality MCQs, and explore methods for transforming lecture, session, and clinical topics into content maps that guide question development. The session will then delve into principles of creating effective AI prompts and provide hands-on practice in using AI to develop MCQs.
The workshop integrates a combination of didactic presentations, think-pair-share activities, and small group work, culminating in a group discussion to consolidate learning. By the end of the session, participants will be equipped to use AI in producing USMLE-aligned assessments, ultimately improving student performance on the exam.

Agenda & Methods

1. Introduction - Purpose of NBME licensing exam and features of a high quality NBME USMLE Step MCQ. (10 minutes) (Method: didactic)
2. Identifying features of a high-quality NBME USMLE Step item. (10 minutes) (Method: Think-pair-share)
3. Lecture / session content mapping. (20 minutes) (Methods: small group activity)
4. Principles of developing AI prompts. (10 minutes) (Method: didactic)
5. Applying AI prompts to develop MCQ items. (25 minutes) (Method: small group activity)
6. Summary & attendee reflections (15 minutes) (Method: group discussion)