Name
EPAs in a Surgical Training Rotation: A Reliability Analysis
Date & Time
Tuesday, June 17, 2025, 10:47 AM - 11:12 AM
Authors

Emilio Violato, Northern Alberta Institute of Technology
Efrem Violato, Northern Alberta Institute of Technology
Claudio Violato, University of Minnesota Medical School

Presentation Topic(s)
Assessment
Description

Purpose
To describe the full implementation of the 13 Core Entrustable Professional Activities (EPAs) as a core component of the surgery clerkship program of assessment in undergraduate medical education, and report psychometric properties of the assessment system.

Method
A total of 159 (91 women - 57.0%; 68 men - 43.0%) third year medical students with a mean age = 24.1 years (SD = 2.67) at matriculation, participated. Students were in a required surgery clerkship. They requested EPA assessments by assessors (n=206, mean number of assessments = 13.6, SD = 4.8; residents, faculty members) while working on a patient case together. Student ratings were depicted as curves describing their performance over time; regression models were employed to fit the curves.

Results
There were 2,801 EPA-based assessments with a mean number of 17.5 assessments per student. Reliability analysis using generalizability theory showed that Ep2 = 0.71 was achieved with four assessors on four occasions. Growth curves of the EPA ratings follow the predicted negative exponential learning theory, with the slope of the growth curves showing considerable variation by EPA. The time to attain entrustability differed by EPA.

Conclusions
The results of the present study are in concordance with the negative exponential learning theory and suggest that EPA ratings provide reliable and dependable data to make entrustment decisions about surgical students’ performance. The results of this analysis of more than 2,800 assessments in the clinical immersion program are consistent with long-standing learning theory and provide reliability and validity evidence for the EPA program of assessment. The EPA program has great promise as a standardized learning and assessment system during the clinical immersion of medical education.

Presentation Tag(s)
International Presenter