Purpose
Explore using AI for curriculum mapping to determine if it can accurately tag keywords to learning objectives. Medical schools are tasked with quickly identifying specific curricular topics for LCME accreditation and national surveys. It’s imperative to have keywords tagged to the curriculum with ease of access. Given thousands of learning objectives in medical education, this is a time-consuming task. Can AI replace manual tagging of keywords to learning objectives?

Methods
Curating a list of 101 potential keywords and provided this list to Claude AI, with instructions to tag ? 6 keywords to learning objectives in two courses. We asked faculty, without knowledge that AI completed the tagging, to review the tagging and provide corrections. Two different faculty reviewed identical ~200 keyword-tagged learning objectives for each course. We tallied instances where both reviewers agreed with AI, both reviewers made different changes, and both reviewers made the same changes.

Results
Reviewers agreed with 79% and 66% of AI tagging of basic science learning objectives in course 1 and 2, respectively. In contrast, reviewers agreed with only 22% and 1% of AI tagging of non-basic science-related learning objectives in course 1 and 2, respectively. Of 117 learning objectives in course 1 where reviewers didn’t agree with tagging, reviewers made the same changes for only 10 objectives. In course 2, for 197 learning objectives where reviewers didn’t agree with the tagging, reviewers made the same changes for only 45 objectives.

Conclusion
AI is more accurate in tagging keywords related to basic science compared to non-basic science learning objectives. Faculty also did not make consistent changes, so this raises the question: what is accurate tagging? Both manual and AI keyword tagging appear highly variable. Perhaps with more training, AI will be more accurate in tagging keywords. We plan to see if adding definitions to the keywords will improve AI tagging accuracy.