Name
Application of Yes-No Angoff Standard Setting for Examinations
Date & Time
Friday, October 25, 2024, 10:15 AM - 10:29 AM
Description

Purpose
Students in educational degree programs must demonstrate sufficient mastery of knowledge to pass summative assessments to advance further in training toward a health science career. Setting an examination passing standard is crucial to provide a means for classifying students as sufficiently competent to move forward and obtain a degree. We investigated the outcome of academic success in passing a pre-clinical curriculum using standard setting cut-off scores derived from the Yes-No Angoff method which yields passing standards set by faculty experts assessing levels of student performance. 

Methods
We analyzed exam performance for three student cohorts across four pre-clinical semesters with application of the Yes-No Angoff procedure.  Cut-off scores using an Angoff faculty-derived standard were compared to a historic single cutoff score, and to lowest performing students.  We compared these cut-off scores to pre-matriculation admissions qualifying examination scores predicting academic success.   We applied binary logistic regression and receiver-operator characteristic (ROC) analyses to inform passing standards that predicted academic success.

Results
We found the Yes-No Angoff yielded the best predictor for academic success in the first year of the curriculum.  Our school’s historical 65% passing standard was the best predictor for the second year of the curriculum.  ROC analysis showed excellent or outstanding prediction results for 11 of 12 semesters studied.

Conclusion
The Yes-No Angoff standard setting method yielded cut-off pass scores comparable to the school’s historic pass score, and provided acceptable to outstanding ROC classification results for predicting academic success aligned to pre-matriculation qualifying examination results for admission to medical school.

Edward Klatt