Turn on cookies to sign in
Signing in saves your progress to your Curie account. We can only do that with cookies on — turn them on to continue.
All five major AI platforms tested in rheumatology clinical scenarios made errors — some significant. A new study found that while OpenEvidence led in guideline accuracy and lowest hallucination rates, no platform was error-free. Researchers are urging clinicians to stay in the driver's seat when using AI for medical decision-making.
AI is everywhere in medicine, but a new study presented at ACR Convergence 2026 is a timely reminder that "fluent is not the same as correct." Researchers tested five AI platforms — ChatGPT (GPT-5.5), Gemini, Claude, Perplexity, and OpenEvidence — across 60 rheumatology clinical scenarios, evaluating them on guideline concordance, hallucination rates, and educational quality.
OpenEvidence came out on top for accuracy and fewest hallucinations, while ChatGPT impressed on completeness, educational value, and speed. But every single platform produced clinically meaningful errors, including incorrect dosing, outdated treatment choices, and inappropriate screening recommendations.
By the Numbers:
Why it matters: As AI tools become routine in clinical settings, this study underscores that no platform can replace clinician judgment. Errors in dosing or treatment selection can directly harm patients — making human oversight not just recommended, but essential.
