PURPOSE
Generative AI tools promise to enhance assessment validity, ensure
equitable measurement of competencies, and support scalability while reducing
faculty burden. However, they frequently generate questions misaligned with
course-specific requirements, Bloom’s Taxonomy cognitive levels, and clinical
demands, thereby necessitating substantial manual editing. The challenge is
to create question sets that accurately reflect authentic practice and adhere
to pedagogical standards. We developed and evaluated a structured AI-assisted
framework to address these gaps by producing exam questions tailored to
students’ knowledge and learning objectives.
METHODS
A multi-step AI-assisted question-generation pipeline was developed for
second-year nursing pharmacology students. The AI was trained on course
materials, assessment items, and National Board of Medical Examiners (NBME)
guidelines to ensure alignment with learning objectives, cognitive levels,
and clinical expectations. The process included: (1) generating
multiple-choice, open-ended, and case-based questions incorporating clinical
reasoning; (2) faculty review to evaluate conceptual clarity, accuracy, and
pedagogical suitability, along with clinician validation for authenticity;
(3) iterative refinement through manual and AI-prompting revisions; and (4)
categorizing items by cognitive complexity and clinical relevance. This
framework was prepared for integration into an online self-assessment
platform to support formative learning.
RESULTS
The AI methodology yielded items closely aligned with the targeted
cognitive levels, incorporated realistic clinical elements, and encompassed
students’ experiential scope. Evaluations by faculty and clinicians
underscored challenges concerning strict content coherence and the balance
between clinical richness and cognitive load. The process resulted in a
validated question bank prepared for pilot testing within digital formative
assessments, with subsequent analysis of student engagement and performance
scheduled after deployment.
CONCLUSION
AI-generated assessments demonstrate significant potential to enhance
validity, relevance, and efficiency in preclinical pharmacology evaluation.
This organized, faculty-supervised framework offers a scalable model
adaptable across various healthcare disciplines, thereby promoting
competency-based education, alleviating faculty workload, and improving
assessment quality and design.