PURPOSE
Our institution uses the NBME CAS, Basic Science, Clinical Science, and
Comprehensive Subject Exams. These exams provide data that offer insights
into a learner’s performance and help identify knowledge and educational
gaps. However, the results from these assessments are returned in multiple
formats and separate files, even for the same exam. Thus, it can be
challenging to aggregate the data for review.
METHODS
We developed an app that extracts data from the various NBME files and
converts them into “Content Area Performance” (CAP) reports using Excel pivot
tables. These CAP reports are provided individually to students. We provide
aggregated reports across students for single and multiple exams in a course
for course directors and across multiple courses for the curriculum
committee. Further, we generate CAP reports that combine multiple exams to
allow comparison between different groups of students across academic years.
RESULTS
Our initial attempts to manually generate CAP reports were time-consuming,
taking many hours and were prone to data entry errors. Using the NBME Data
Converter App, any combination of reports can be generated in a few minutes.
This app has enabled us to be more effective in exam quality review and in
identifying knowledge and concept gaps within courses and across the
curriculum. Further, it provides a valuable tool to develop individualized
learning plans for students.
CONCLUSIONS
This app provides an efficient method to aggregate exam performance data
and identify learning gaps for individual students and cohorts. Students
benefit through the development of individualized learning plans. Our courses
benefit from data for continuous quality improvement. Our curriculum
committee benefits from data that enables them to review content placement
and student competence in that content.