Loading page…
The study measured correct answers, not the safety of explanations. Its findings reflect tests conducted in 2025, not current model performance.
Listen to this story
The leaderboard barely separated the systems: just 1.8 percentage points divided the highest and lowest scores, while performance varied more sharply by task. Image questions yielded 69.2% accuracy versus 91.5% on text-only questions, but the visual sample was only 13 items and mostly antimicrobial structures. The test used model versions available in May–July 2025; it graded answer correctness, not explanation completeness or clinical safety, so it does not establish readiness for clinical use.
ChatGPT-4o-mini led at 91.2%, followed by Gemini 2.5 Flash at 90.7% and ChatGPT-4.1 at 90.3%.
Accuracy fell significantly from 92.0% on remembering and understanding questions to 85.6% on evaluating and creating questions.
Researchers tested 227 questions from one required second-year infectious-diseases course at the University of the Pacific School of Pharmacy.
High pharmacy exam scores came with a teaching caveat: the AI systems’ explanations were not separately graded for completeness or clinical safety. A study published November 7, 2026, in Scientific Reports found five models answered 90.2% of infectious-disease questions correctly overall, but performed worse on images and questions requiring higher-order thinking.
Researchers at the University of the Pacific School of Pharmacy used 227 questions from one required second-year infectious-diseases course. The sample covered five subjects in its accelerated Doctor of Pharmacy program. Data were collected from May through July 2025, so the results do not describe later model updates.
Each system received identical prompts asking for an answer and explanations of why its choice was correct and the alternatives were wrong. Researchers scored answers against instructor-provided keys. Two investigators independently reviewed questions any model missed and agreed on all classifications.
Only 1.8 percentage points separated the highest and lowest overall scores. News-Medical’s account identifies the tested versions and results:
Researchers also grouped questions by the thinking they required. Accuracy fell from 92.0% for remembering and understanding to 85.6% for evaluating and creating, a statistically significant decline. Medicinal chemistry and microbiology also scored significantly below therapeutics, the study of treatment.
Across five models, 45 of 65 answers were correct.
Across five models, 979 of 1,070 answers were correct.
The visual result needs a narrow reading. Only 13 questions contained images, and eight showed antimicrobial chemical structures. That small, chemistry-heavy sample limits broader conclusions about visual interpretation.
Similar totals did not mean the systems got the same questions right. Agreement across model responses was moderate overall. The authors suggested that comparing responses could help students practice critical evaluation, while advising them to verify AI-generated information against reliable references.
Those are educational recommendations, not demonstrated learning gains. The study measured model answers rather than improvement in students’ understanding. The authors supported supplementary use with faculty guidance and cautioned that exam accuracy does not establish the reliability or safety of clinical recommendations.
Loading discussion...
Join the conversation
Explain how much the reasoning behind an answer should matter.
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.