“Morally Binding” AI Safety Pledge Mocked

On Tuesday, President Trump announced a voluntary agreement with the heads of six major AI companies to keep “superintelligence” in check. The one-page document inspired mockery from AI experts and commentators around the Internet. The signatories to the agreement included OpenAI President Greg Brockman, Anthropic chief Dario Amodei, Meta’s Mark Zuckerberg, Google boss Sundar Pichai, Tesla and SpaceX chief Elon Musk, and Nvidia chief executive Jensen Huang.

The pledge is “morally binding,” Trump told attendees at a press conference, as if to underscore the fact that there is nothing legally binding about it. As it stands, the agreement seems like little more than a handshake between an industry known for cutthroat competition and failure to self-regulate, and a government that seems antagonistic to regulation, not to mention, morality. Read more: “AI Engineers Are Having Their Oppenheimer “Destroyer of Worlds” Moment “Reading the accord between the lines,” wrote American psychologist Gary Marcus, a consistently vocal critic of generative artificial intelligence, on his substack, the pledge amounts to saying, “1.

We agree not to be regulated. 2. We agree not to give the public a voice. 3. Trust us.” The language of the voluntary agreement is loose.

It simply allows AI firms to define safety on their own terms without outlining any consequences for breaches, said Kimberlee Weatherall, a University of Sydney law professor, in an interview with the BBC, calling the accord “deeply unimpressive.” Key terms like “alignment” and “robust internal controls” are left undefined, with no proposed way to measure them, noted Luiza Jarovsky, a global expert in AI governance, policy, and privacy regulation. Totally absent from the document is any mention of the kind of “pacing the frontier” that the AI companies were calling for a couple of weeks ago, added Marcus on X. In mid-September—after a flurry of dire warnings about the AI apocalypse began to flood social media, set off by a post on X by Jacob Coxon, a researcher at Anthropic who quit to air his concerns—Amodei penned a treatise about the need to slow development of new models until appropriate safeguards could catch up.

Amodei also said the most effective way to do this would be to regulate all United States-based frontier AI companies, requiring both transparency and third-party auditing, while formalizing permanent, embedded evaluators. OpenAI CEO Sam Altman wrote on X in response that he agreed and committed to embedded evaluators, and xAI founder Elon Musk posted, “Dario is right.” The apocalyptic warnings were inspired in part by emerging reports that rogue AI agents were deceiving their human evaluators, breaking out into the Internet and doing things they weren’t authorized to do, including coordinating attacks on companies. In the weeks since, many more incidents have come to light.

Top AI companies and other security researchers are now investigating thousands of problematic incidents, according to reporting in Axios. On the same day that the White House Accord on Superintelligence was released, a legal nonprofit in California sued OpenAI over one of the most prominent attacks its AI agents coordinated against the open-source AI platform Hugging Face earlier this year. Earlier this week, OpenAI announced that it would not release its latest model GPT 6.1 Astra over safety concerns.