Nvidia on Monday unveiled a new security platform that the chipmaker said can stop artificial intelligence agents from going rogue. The company announced a $150bn stock buyback the same day. The chipmaker said that its Open Agent Safety Platform includes open source software that “sets boundaries for agents” and follows a series of revelations from top AI companies about their models escaping and breaking into other organizations.
The disclosures have sparked furious debate about the safety of advanced artificial intelligence systems, including self-improving models that some fear could race out of human control. Nvidia executives said in a media briefing that the new system could have prevented a recent incident involving a swarm of OpenAI agents that autonomously hacked into the AI company Hugging Face. Nvidia also said its board approved expanding its share repurchase program by $150bn, raising the total amount to $235bn.
Last month, Nvidia forecast about 70% revenue growth for fiscal 2028, reassuring investors who have questioned how long the AI spending surge can last after years of explosive growth. The Hugging Face incident was a high-profile breach that inflamed safety concerns about AI, which were followed by similar rogue actions involving OpenAI’s models including breaching an Australian health department website. Anthropic and Meta have also disclosed that their AI systems hacked into other organizations on their own.
Nvidia’s software, called OpenShell, lets developers “formally verify an agent has enough authority to do its job and no more”, Boitano said. Because it is open source, it can be “extended” to run on rival computing platforms including those from Arm and Intel. The platform also includes a separate security layer called Sentry that runs onboard a chip to continuously monitor AI agent activity and can “intervene instantly” if the agent starts trying to move beyond its target, the company said.
Nvidia said more than 100 organizations were using the platform at its launch, including Microsoft, Perplexity, Accenture and JPMorgan Chase. The AI safety debate has divided the industry, with the heads of Anthropic and OpenAI championing a coordinated slowdown of AI development to let safety efforts catch up. Others, including Huang, say it should be up to individual companies to make sure their models are safe for release.
Huang, during the annual Salesforce technology conference held earlier this month, characterized AI safety, including the danger of rogue agents, as an engineering problem that software developers can address.
Extract — continue reading at the source.
