AI reduces clinical risk only when it is governed

, Digital health

ECRI put AI at the top of its health technology hazards list for 2025. A few months later the Joint Commission and CHAI released their first guidance on responsible AI use in hospitals. Both point at the same gap: deployment is running ahead of governance.

That gap is where clinical risk will play out over the next few years.

With Giancarlo Stoppani I published an analysis in Ethica societas of AI as a component of clinical governance: where the evidence shows it reduces risk (early warning of deterioration, discharge documentation, safety report analysis, adverse drug events, antibiotic decisions in sepsis) and under which conditions.

The thesis is simple. AI does not reduce clinical risk on its own. It reduces risk when it is validated locally, integrated into workflows, monitored in real-world use and placed under explicit human accountability. Left alone, it becomes a new source of error.

For hospitals we propose an architecture of seven rules, from a single register of every AI system in use to a surveillance dashboard with seven indicators (utilization, override rate, observed accuracy, equity, performance drift, incidents, operational benefit), reviewed quarterly by the clinical risk committee.