Curie Brief
Turn on cookies to sign in
Signing in saves your progress to your Curie account. We can only do that with cookies on — turn them on to continue.
News
Not all AI chatbots are created equal when it comes to COPD education. A blinded study presented at the CHEST Annual Meeting found ChatGPT-4 outperformed ChatGPT-5, Gemini, and OpenEvidence on adequacy, empathy, readability, and GOLD guideline alignment. Clinicians are urged to validate AI-generated content before sharing it with patients.
As patients increasingly turn to AI chatbots for health information, a new study is putting those tools to the test — and the results are a mixed bag. Researchers conducted a blinded, cross-sectional evaluation of four AI models (ChatGPT-4, ChatGPT-5, Gemini, and OpenEvidence) on COPD patient education topics drawn from the 2025 GOLD Pocket Guide. Two board-certified pulmonologists independently scored responses across metrics including adequacy, completeness, readability, empathy, safety, and guideline concordance.
ChatGPT-4 came out on top with the highest overall composite score, followed by ChatGPT-5, Gemini, and OpenEvidence — in that order. Notably, the newer GPT-5 didn't clearly surpass its predecessor. Gemini and OpenEvidence lagged significantly on guideline specificity and clarity around exacerbation safety instructions — arguably the most critical content for patients managing COPD at home.
Key Takeaways:
Why it matters: Clinicians are increasingly using AI to draft patient education materials, but this study underscores that these tools aren't interchangeable. Weak guidance on exacerbation warning signs could delay care-seeking — making expert validation of AI-generated content a clinical imperative before it reaches patients.
