Moderated by Belinda Carrion
Session Coordinator: Sateesh Babu Arja

Presentation 1 - Systematically Integrating Validity Evidence to Support Clinical Science Domain Composite Scores    
Brock Mutcheson    
Virginia Tech Carilion School of Medicine

Purpose
Assessment performance scores are essential to both low and high-stakes medical education decision-making at several organizational levels. Modern validity frameworks provide foundational tools for systematically integrating quantitative and qualitative evidence to support the intended uses of scores. We present a comprehensive validity argument for the intended organizational uses of Phase-1 Clinical Science Domain scores and demonstrate the utility of modern validity frameworks.

Methods
First, we present a literature review of historical and modern approaches to measurement validity and describe how they relate to reliability theory and critical efforts towards assessment equity. Next, we describe Virginia Tech Carilion School of Medicine (VTCSOM) phase-1 clinical science program and assessment instruments, linking current learning objectives to construct maps, items, scores, and measurement models. We use three years of multi-dimensional assessments to demonstrate how we integrate data and related assumptions to support inferences in the progression from a single observation to a final decision. Data sources include a rigorously analyzed multiple-choice exam, assessments of interview skills, physical skills, written presentation skills, clinical reasoning, and communication/interpersonal skills.

Results
The result of this analysis is the integration of qualitative and quantitative data into a comprehensive validity argument for clinical science comprehensive domain scores for their intended uses. We organize evidence following Kane's inferences of scoring, generalization, extrapolation, and implications.

Conclusion
Validity is fundamentally about integrating information to create scores that align with their intended interpretations and uses. One advantage of Kane's Framework is that it provides an infrastructure to apply to individual quantitative assessment instruments, qualitative assessment tools, and assessment programs.

Presentation 2 - Use of Board-Style, TrueLearn Quizzes as Formative Assessments in Medical School Pathology Curriculum    
Cathryn Caudill    
Kentucky College of Osteopathic Medicine/University of Pikeville

Purpose
Based on our experience with faculty-authored formative assessments that improved student performance on pathology course examination items, we began recommending and/or assigning board-style practice questions from the commercial test bank, TrueLearn©. We explored student preferences for and perceptions about these two types of assessments for developing content understanding and mastery. We also examined the influences of these assessments on student communication with faculty and examination preparation.

Methods
We administered a voluntary and anonymous SurveyMonkey© survey to students enrolled in our pathology course. Survey questions asked whether students preferred faculty-authored and/or TrueLearn© assessments for various types of learning approaches and skills related to course examination and board preparation, and how they engaged with the assessments and course faculty. A subset of survey questions explored student perceptions about TrueLearn© assessments and their influence on motivation and study habits. We also examined TrueLearn© assessment data and reviewed faculty records of student office visits and email correspondence.

Results
The student response rate to our survey was 77%. Students believed both faculty-authored and TrueLearn© assessments were valuable for supporting various skills in concept development and mastery. Students were more likely to communicate with faculty about TrueLearn© assessments, and reported a slight preference in their use for board preparation, although they were less likely to utilize these questions when recommended as an ungraded activity versus graded assignment. Students reported increased workload and stress with graded TrueLearn© assignments, but also reported increased confidence, earlier engagement, and review of material for board preparation.

Conclusion
Students appreciate a mix of formative assessment strategies. Adding TrueLearn© to the pathology curriculum is a novel approach that engages students with board-style questions early in their learning process, and results in increased student engagement with faculty for question-answering strategies.

Presentation 3 - Characteristics of Meaningful Feedback During Clerkships: A Student Perspective    
Laura Bauler    
Western Michigan University Homer Styrker M.D. School of Medicine

Purpose
Receipt and delivery of feedback is a major component of medical education. Individualized, constructive feedback takes time, effort, and skill to deliver but also requires the recipient to be willing to receive that feedback. Feedback is a major component of clerkship assessment and has long been a challenge in medical education. Few studies have explored what makes feedback meaningful for students, thus this study aims to identify the characteristics of useful feedback from the student perspective.

Methods
A qualitative thematic analysis was completed on survey feedback collected from medical students following completion of their core clerkships. Data was collected from 152 students from the 2021 and 2022 graduating classes at Western Michigan University Homer Stryker MD School of Medicine. Students were asked to describe the experiences that helped students give and receive feedback, and the most meaningful feedback they received. Using an inductive approach, free text responses were coded independently by two authors, themes were identified by all authors in an iterative process, and similar codes were grouped.

Results
Three themes were identified from the experiences that helped students give and receive feedback including: 1) the importance of practicing these skills in low stakes peer-based environments, 2) repetition of feedback, and 3) scheduled formal one-on-one feedback with preceptors during clerkships. Four themes emerged from the thematic analysis of the most meaningful feedback received including: 1)the importance of specific, constructive, actionable feedback, 2) the student-perceived authenticity of the feedback, 3) the format and delivery of the feedback, and 4) the specific feedback of "be confident and speak up".

Conclusion
Feedback is an essential part of growing and learning, but delivery/receipt of meaningful feedback is challenging. Careful design of the ways in which feedback is delivered to students can improve their perception of how significantly that feedback promotes their career growth.

Presentation 4 - Neurodiverse Learners in the Health Sciences: The Benefits of Universal Design for Learning    
Sara Keeth    
Lecturio

Purpose
Neurodiverse learners in medical, nursing, and other health education may hesitate to self-identify due to lingering stigma around neurodiverse diagnoses such as attention deficit disorder and autism spectrum disorders. Although the percentage of students who self-identify as neurodiverse in health professions education is low, the actual percentage of neurodiverse learners in medical and nursing school is estimated to be higher. Educators in the health professions can help neurodiverse students succeed without requiring those students to self-identify or "out" themselves by using Universal Design for Learning (UDL) principles for their courses. In particular, online elements delivered via learning platforms offer opportunities for neurodiverse student success through UDL. An added benefit of UDL means that all students benefit from more accessible and more easily navigable courses.

Methods
A review of neurodiversity in health professions Brief discussion of challenges and benefits of neurodiversity Practical application for UDL in courses Practical application for UDL in online elements delivered via learning platforms Description of how UDL benefits both neurodiverse and neurotypical students.

Results
Using Universal Design for Learning in health professions education can reduce unnecessary barriers to learning for all students.

Conclusion
Small changes in course design using the principles of Universal Design for Learning can benefit all students in health professions education but may have particular benefits for neurodiverse students.