Presented By: Sally Binks, University of Toronto
Co-Authors: Ryan Brydges, University of Toronto
Jaimie Coleman, University of Toronto
Vyshnave Jeyabalan, University of Toronto
Kulamakan Kulasegaram, University of Toronto
Nicole Woods, University of Toronto

Purpose
Multiple-choice questions (MCQs) should be optimized to prepare learners for future learning. One way is if MCQs promote learners' engagement in distinctive processing, whereby learners recruit prior conceptual knowledge to differentiate confusable categories. This is an example of a beneficial response process - an essential component of validity for formative tests. We report response process validity evidence that purposefully designed MCQs can elicit distinctive processing. 

Methods 
We conducted a comparative think-aloud study with nursing and medical students. Participants were assigned to a 'competitive' or non-competitive version of a 19-question A-type MCQ test. Items assessed basic science relevant to clinical skills in critical care medicine. All items were developed using the key feature identification approach. The distinctive processing aligned 'competitive' version of the test had 3 response options that were developed from the key features. The non-competitive MCQ version had distractors that were not key features. Think-alouds were recorded and then coded by two raters to identify the response processes used by participants to select answers for each version of the test. We compared frequency of response processes between the two test groups. 

Results 
Sixteen participants completed the study (8 in each group). We identified 10 categories of response processes: Four categories entailed distinctive processing of the response options; 4 categories entailed non -specific reasoning processes; the other 2 categories included guessing and retrieval of prior knowledge without reviewing response options. Participants who took the competitive version of the test used distinctive processing significantly more often than those who took the non-competitve version [?2 (df=1,) = 17.9, n=192, p <0.001]. 

Conclusion 
MCQ tests designed to have competitive response options induce cognitive processing that may optimally prepare learners for future learning. This response process validity is the first step in designing formative testing to promote clinical reasoning.