In this highly interactive workshop, the audience will participate in hands-on exercises that will enhance their skills in creating, reviewing and refining effective assessments using USMLE-style multiple choice questions that assess higher order cognition. The activities will be organized to demonstrate a systematic process for “just-in-time” faculty development so that participants can adopt/adapt the approach at their home institution to build capacity for enhancing their own assessment programs.

 

Following a brief didactic highlighting guiding principles for programmatic assessment and best practices in question-writing, participants will work in small groups as “mock assessment teams” to model processes that promote peer mentoring,  mutual accountability, and the creation of a community of practice focused on assessment. Faculty facilitators will serve as coaches for each small group throughout the workshop. The first set of exercises will provide the opportunity for the teams to collaborate to review and refine USMLE-style questions that integrate basic and clinical science concepts and that challenge students to integrate and synthesize knowledge in order to demonstrate learning at higher levels of Bloom’s taxonomy. In a facilitated report out after this exercise, the large group will construct guiding principles to avoid common pitfalls in question writing to share with colleagues at their home institution.  Participants will also have time to craft questions that they can take home to use in student assessment.

 

After another brief didactic reviewing key data analytics to use in assessing the performance of a test item and of examinations, the assessment teams will have the opportunity to review sample item analytics and psychometrics for a mock exam and engage in discussion about how this information should be used to make final scoring decisions, to address challenges from learners about individual items and to identify curriculum gaps or learning needs of the students. In a report out after this exercise, the groups will compare the decisions made during the mock exercise to promote consensus building across groups.  The large group will then brainstorm how best to present the information presented in this workshop to individual faculty, course leaders and/or the educational administrative leaders at their institutions. 

 

At the end of the workshop, participants will have the opportunity to create an individual action plan that builds on what they have learned from the workshop to improve and augment the assessment programs at their home institutions.