Name
Using Learning Analytics to Identify Examples of Productive Failure in One Residency Program
Date & Time
Sunday, June 15, 2025, 4:00 PM - 4:15 PM
Authors

Kimberly Papp, University of Alberta
Shelley Ross, University of Alberta

Presentation Topic(s)
Assessment
Description

Purpose
Productive failure is effective for learning and occurs when learning spaces are safe for learners to make mistakes, identify their learning gaps, and make a learning path moving forward. Despite calls to support productive failure in competency-based medical education, minimal research has been published exploring this concept in postgraduate medical education. In this study, we examined narrative feedback on workplace-based assessment forms called FieldNotes to identify patterns in feedback about performance gaps in resident education.

Methods
We examined eight years (2016 to 2024) of de-identified FieldNotes at one mid-sized Canadian Family Medicine Residency Program using learning analytics through secondary data analysis. FieldNotes contain a narrative summary of feedback shared with a resident after direct observation as well as an indication of competence (Progress Level). We examined FieldNotes that received the Progress Level of “Stop, Important Correction” (“Stop”).

Results
Examination of 47,516 FieldNotes found that 2% were designated as “Stop”. Of those, 36% were entered by the residents themselves; of these, 76% were entered by residents in rural placements. Over 28% of learners receive at least one “Stop” FieldNote in any given year, and 8.9% of observers provide at least one “Stop” in any given year. Thematic analysis of the feedback found that both “hard skills” (Medical Expert, Physical Exam, Procedures) and “soft skills” (Communication, Clinic Management, Professionalism) were represented in a balanced way. The most common “Stop” feedback topics for each theme above, respectively, were identifying a sick patient based on vital signs, neurological exam, delivering babies, documentation of a management plan, time efficiency, and completion of assigned tasks.

Conclusions
Examination of “Stop” FieldNotes revealed that residents in rural placements in this program demonstrate a growth mindset and embrace productive failure, as shown by their willingness to identify and document their learning gaps.