Presented By: Robert Treat, Medical College of Wisconsin
Co-Authors: Gauri Agarwal, University of Miami Miller School of Medicine

Purpose 
Artificial intelligence (AI)-related technology deployment has begun in medical education, but it is often implemented in a fragmented and siloed approach. Medical faculty and students may prioritize the importance of these implementations differently, and this should be analyzed to assure a consistent set of outcomes. AI-technologies such as natural language processing (NLP) using generative pre-trained transformers (GPT) can assist in the construction of self-reported surveys, while the data can be analyzed with machine learning approaches such as factor and regression analysis. The study's goal is to validate GPT-augmented faculty and student surveys on the importance of AI-related technologies in medical education. 

Methods 
AI-based QuillBot abstracts and extracts key findings from scholarly literature and two medical education conference panel sessions. Utilizing this software, we first explored potential constructor variables for medical school and faculty self-report questionnaires. GPT 3.5 provided individual survey items from the QuillBot summaries, which were content validated by the authors to create a 20-item survey (scale:1=not important/5=extremely important) with one overall item (10-point scale) analyzed with SPSS 28.0 with t-test, Cohen's d effect size, linear regression and factor analysis (with varimax rotation). The study is IRB approved. 

Results 
Forty participants reported significantly (p=0.043, Cohens d=0.33) higher faculty (7.8(±1.6)) than student (7.3(±1.6)) overall importance scores. Factor analysis (KMO=0.70, Bartlett's sphericity: chi-square=429, p=0.001) yielded a six-factor solution (student learning/communication/medical data/patient issues/clinical encounter/education resources) from 20 items (alpha=0.90) with communication (beta=0.51, p=0.001) and medical data (beta=0.49, p=0.001) being significant predictors of overall importance in regression analysis (R²=0.70, p=0.001). 

Conclusion 
A reliable AI in medical education survey was created from QuillBot summaries of peer-reviewed conference panel sessions and GPT. There was a higher level of AI importance reported by faculty over students. The survey was validated with an internal structure of six factors, two of which predicted the overall importance of AI in medical education.