A leader in the health care and AI space has warned that racing China on AI could put patients at risk as President Donald Trump pushes to accelerate U.S. development. He said that five years ago there was little appetite for slowing AI development to ensure models were safe. But Anderson explained that attitudes are changing as health systems explore increasingly autonomous AI applications in higher-risk settings where patients’ lives could be at stake.
He pointed to AI helping guide sepsis treatment as an example of a high-risk use where mistakes could harm patients. Anderson also warned that AI agents in healthcare could “go rogue” and “access a skill” to operate outside a health system’s secure firewall, potentially leaking protected patient information. His comments come after Trump vowed not to slow AI development in the race with China, saying “whoever wins AI, wins” and dismissing fears that the tools could destroy humanity as a “hoax.” He said the only "guardrails" needed for AI was a "strong and smart" president and that the technology “will not be stopped” during his presidency.
Nabil Chehade, chief clinical transformation, innovation and strategy officer at The MetroHealth System, echoed the concerns, and said his concern was not an isolated AI mistake but failure on a much larger scale. Even routine uses carry risks, Anderson said, pointing to an AI system being tested in Utah to renew existing prescriptions. A change in a patient’s lab results could mean a medication should be adjusted or stopped rather than renewed, he said.
But isn’t quite ready to be free range yet.” Amid calls for government regulation to slow AI development, Carr argued that health systems could serve as “assurance partners” to technology companies, helping ensure increasingly autonomous systems are deployed safely. Anderson also described a “standoff” between some health systems and frontier AI developers over who should control autonomous agents and bear responsibility when something goes wrong.
Extract — continue reading at the source.
