Curie Brief
Turn on cookies to sign in
Signing in saves your progress to your Curie account. We can only do that with cookies on — turn them on to continue.
News
AI in medicine has a reproducibility problem. A new Dana-Farber/MIT study reviewed ~4,000 clinical prediction model studies and found that only 12% shared their underlying code—and even then, it was rarely reproducible. Without verifiable code, clinicians and regulators can't reliably assess whether these AI models are safe for real-world patient care.
A new study published in Nature Medicine by researchers at Dana-Farber Cancer Institute and MIT is raising red flags about the reliability of AI-driven clinical prediction models. After reviewing approximately 4,000 published studies using an AI-assisted scoping approach, the team found that code was shared in just 12% of studies—and even when it was shared, it was rarely reproducible. Code-sharing rates also varied significantly by journal and country.
This matters because reproducibility is the backbone of trustworthy science. Without access to working, verifiable code, independent researchers can't confirm published results—leaving clinicians, regulators, and health systems in the dark about whether a given AI model is actually reliable enough to use with patients.
By the Numbers
Why it matters: As AI tools increasingly enter clinical workflows, the inability to independently verify prediction models is a serious patient safety concern. The authors argue that stronger code-sharing standards in clinical AI research are essential to support real-world deployment and build trust across healthcare systems.
