Turn on cookies to sign in
Signing in saves your progress to your Curie account. We can only do that with cookies on — turn them on to continue.
Generative AI is transforming clinical practice, but it comes with real pitfalls. Experts at Spain's SEMERGEN congress flagged AI "sycophancy," hallucinated citations, and patient privacy risks as top concerns. Physicians are urged to build AI literacy, use structured prompts, and establish clear workflows — because knowing how to manage AI well is becoming an essential clinical skill.
Generative AI is rapidly embedding itself into clinical practice — helping with diagnoses, ECG interpretation, vaccination scheduling, and cutting administrative burden. But at Spain's 48th SEMERGEN National Congress, experts sounded a clear warning: these tools are powerful, but far from foolproof.
One of the biggest concerns raised? AI sycophancy — the tendency of models to agree with users rather than push back, which can quietly reinforce incorrect clinical assumptions. Add to that the risk of hallucinations (think: confidently fabricated research citations) and a "black box" problem where it's impossible to trace how an algorithm reached its conclusion, and the case for critical AI literacy becomes hard to ignore.
Privacy is another flashpoint. Experts raised concerns about patients voluntarily sharing full medical records with consumer AI tools outside any regulated healthcare framework, and about vulnerable adolescents interacting with chatbots without appropriate safeguards.
Key Takeaways:
Why it matters: AI won't replace physicians — but physicians who use AI responsibly and skillfully may have a meaningful edge. Building sound AI habits is fast becoming as essential as any clinical skill.
