They'll say things like ‘endgame’ or ‘crunch time,’” he says. Silicon Valley is scrambling to reckon with the safety and security concerns of advanced AI models. OpenAI has rushed to respond to a security incident in which its agents hacked the platform Hugging Face.
Meanwhile, Anthropic is trying to assure investors it has these concerns under control as it reportedly prepares to file for what could be the largest IPO ever. Are you a current or former Anthropic employee who wants to talk about what’s happening? We’d like to hear from you.
Using a nonwork phone or computer, contact the reporter securely on Signal at mzeff.88. What’s become clear in the response to Coxon’s post is that his views are indeed shared by many of his peers. Evan Hubinger, the AI alignment lead at Anthropic, predicted in a post on X that there’s a greater than 10 percent chance that AI could kill all people in the next decade.
That post was reposted by current and former researchers from OpenAI and Anthropic, some of whom said it was a common sentiment in the industry. What’s less obvious is how exactly these AI fears will come to pass and what the world is supposed to do about the concerns being raised by the people building AI. Coxon, who also worked at OpenAI, tells WIRED that threats could manifest through AI-enabled biological threats or cyberweapons.
As a first step, he recommends that OpenAI and Anthropic coordinate on limiting recursive self improvement—the industry term for when AI is used to build new AI systems. Down the line, he thinks coordination among international power players, including the US and China, will be necessary. Coxon notes that incidents like the Hugging Face hack factored into his decision to raise alarm bells on the AI race.
He also cites the explosive growth of the industry: It now underwrites a meaningful share of US economic growth and has billions of users, while data centers have turned it into a political problem in a dozen states. He claims that, in his experience, Anthropic operates more responsibly than OpenAI, but he expects both companies could cut corners in the future if nothing is done to slow their race for dominance. OpenAI and Anthropic did not immediately return WIRED’s request for comment.
Extract — continue reading at the source.
