PURPOSE
Assessment structure shapes how medical students study, influencing both
knowledge acquisition and the development of skills. In our school, we
introduced the Diagnostic Reasoning Exam (DxR) that is based on a multi-part
single case with progressive disclosure and open-ended questions. The exam is
taken at the end of organ system courses, together with a customized
assessment from the National Board of Medical Examiners (NBME).
METHODS
At the end of courses including DxR and NBME exams (n=15, 4 cohorts), we
calculated average performance and Pearson correlation. We collected
students’ opinions on study habits differences and faculty’s agreement with
DxR assessing published diagnostic reasoning criteria. The item-level content
validity index (CVI) was calculated based on content expert responses.
Chi-square test was used to compare reported resources and strategies used to
study for DxR vs. MCQ exams
Results
Average student performance on DxR exams was the same as MCQ exams (84%).
Pearson correlation values for performance on the two exam formats ranged
between 0.131 and 0.478, with an average of 0.343 for all exams. Significant
correlations were found for 11 of 14 exam pairs. CVI indicated acceptable
overall validity (S-CVI/Ave = 0.92; S-CVI/UA = 0.81). The overall inter-rater
agreement, as measured by Fleiss’ ?, was found to be 0.41. Based on student
feedback, NBME preparation emphasized board-oriented materials, particularly
question banks and review texts (p < 0.001, Cramér’s V = 0.26). DxR exam
preparation favored institution resources, including lectures, small-group
cases, clinical skills content, and course readings. Strategies differed
significantly (p < 0.001, Cramér’s V = 0.19): NBME preparation emphasized
individual study, while DxR exam involved more self-created cases, content
reorganization, and group study.
Conclusion
Incorporating complementary formats may align curricular objectives with
both licensing preparation and diagnostic reasoning skill development.