Introduction/Background

Outreach and Pathway programs (OPPs) to promote entry into STEMM have existed for decades. As OPP organizers, we expect that program participation leads to entry into and persistence in STEM-associated majors. We seek to improve this gap among our OPP participants by 1) incorporating and promoting activities that cultivate pre-matriculation competencies and learning skills and 2) developing assessments for measuring the specific learning skills and competencies mentioned above. Despite these efforts, the number of URM and low-income students matriculating into medical school remains disproportionately low. Pathway programs are evaluated by perception-based assessments, college matriculation rates, and the grade point averages of participants. Therefore, we created a way to assess the direct implications of outreach programs on participants’ learning skills and competencies. To that effect, we developed a situational judgment case scenario that measures the application of participants’ self-directed learning abilities upon experience with OPPs.

Relevance to ASPBP

Our session is useful for members of the ASPBP organization who are actively involved in developing outreach and pathway programs for students at various levels e.g. high-school students, undergraduates, etc.

This roundtable discussion offers a unique opportunity for ASPBP members to:

a) Learn strategies to evaluate their participants' learning competencies after administering the outreach program

b) Address and overcome challenges and generate ideas to assess their program's impact on participants' immediate learning skills and competencies.

c) Explore situational judgement case-based assessment method that goes beyond perception-based assessment of OPPs.

Importance of the Topic or Problem

With OPPs we focus on the long-term impact of STEMM admission and retention. However, it is very important to assess what skills and competencies program participants acquire through these programs as these are the skills that eventually affect STEMM persistence and retention. 

Rationale for the Roundtable/Problem-Solving Session

Perception-based assessments are one tool for measuring student learning skills and competencies, but situational judgment cases allow us to see how students apply their skills. This enhances our ability to assess students.

Challenges/Concerns/Gaps in the Field Being Addressed

There is a lack of literature assessing what students learn from participation in outreach and pathway programs. This work will improve out abilities to assess whether our programs are promoting learning skills and pre-matriculation competencies needed for success in STEMM careers.

Learning Objectives (limit 3-5)

• Identify a learning theory to be assessed among students
• Learn how to construct or utilize a previously published theoretical framework
• Formulate a situational case that mimics the application of real-life learning competencies
• Map the situational case to a predetermined framework
• Identify mechanisms to validate situational judgment cases

Discussion Triggers

• How and why do you validate an instrument?
• How do we assess pre-matriculation competencies and learning skills?
•  How do you construct or identify a theoretical framework?
• What are the learning skills we want students to have when they matriculate?