Number
101
Name
Adapting the CBSE Performance Report to Improve Step 1 Preparation: Balancing User Needs and Validity Considerations
Date & Time
Sunday, June 7, 2026, 5:30 PM - 7:00 PM
Location Name
Oglethorpe Ballroom
Speakers
Authors
Nicholas Cowling, NBME
Francis O'Donnell, NBME
Presentation Topic(s)
Assessment
Description
PURPOSE
The Comprehensive Basic Science Exam (CBSE) assesses students’ readiness
for the United States Medical Licensing Exam® Step 1. Students receive an
Examinee Performance Report with results and score interpretation guidance.
Although the report was informed by user needs and psychometric best
practices, recent feedback indicated a need for changes. This presentation
details the process of adapting this report following a user-centered design
and validity framework for better Step 1 preparation.
METHODS
Initial feedback was collected through focus groups with learning specialists
and used to create two mockup alternative reports. Additional feedback on the
current report and alternatives will be gathered from students and learning
specialists through a feedback session and usability survey. Measures include
the ability to locate information on the reports, ability to identify
potential study areas and additional resources, and Likert-scale ratings of
the clarity and usefulness of the reports.
Report elements will be redesigned using this feedback, with an additional
feedback session and usability survey before publication and use.
RESULTS
Initial results indicate challenges with the current Examinee Performance
Report that could impact students’ effectiveness preparing for Step 1. These
challenges include difficulty in understanding their current probability of
passing Step 1, how to improve performance, and finding specific information
in the report. Redesigned elements under consideration include alternative
probability displays, a streamlined layout, and custom text suggesting areas
for study.
CONCLUSIONS
Passing Step 1 is a critical milestone for medical students. Students
should understand their current status and how to reduce potential
performance gaps. While the CBSE provides robust data, the current report may
limit its usefulness. Feedback is most effective when it is clearly
understood and actionable. Improvements to the Examinee Performance Report
will have a positive impact on students’ ability to prepare for Step 1.
The Comprehensive Basic Science Exam (CBSE) assesses students’ readiness
for the United States Medical Licensing Exam® Step 1. Students receive an
Examinee Performance Report with results and score interpretation guidance.
Although the report was informed by user needs and psychometric best
practices, recent feedback indicated a need for changes. This presentation
details the process of adapting this report following a user-centered design
and validity framework for better Step 1 preparation.
METHODS
Initial feedback was collected through focus groups with learning specialists
and used to create two mockup alternative reports. Additional feedback on the
current report and alternatives will be gathered from students and learning
specialists through a feedback session and usability survey. Measures include
the ability to locate information on the reports, ability to identify
potential study areas and additional resources, and Likert-scale ratings of
the clarity and usefulness of the reports.
Report elements will be redesigned using this feedback, with an additional
feedback session and usability survey before publication and use.
RESULTS
Initial results indicate challenges with the current Examinee Performance
Report that could impact students’ effectiveness preparing for Step 1. These
challenges include difficulty in understanding their current probability of
passing Step 1, how to improve performance, and finding specific information
in the report. Redesigned elements under consideration include alternative
probability displays, a streamlined layout, and custom text suggesting areas
for study.
CONCLUSIONS
Passing Step 1 is a critical milestone for medical students. Students
should understand their current status and how to reduce potential
performance gaps. While the CBSE provides robust data, the current report may
limit its usefulness. Feedback is most effective when it is clearly
understood and actionable. Improvements to the Examinee Performance Report
will have a positive impact on students’ ability to prepare for Step 1.