AI researcher Jacob Coxon’s warning that increasingly powerful artificial intelligence systems could “kill us all by the end of the decade” fueled new calls for Congress to regulate the rapidly developing technology. Coxon wrote in a series of posts on X that he was resigning from Anthropic over concerns that AI companies are “racing straight to self-improving superintelligence and gambling with our lives,” a stark warning about the implications of unchecked AI development on humanity. In light of his warning, many lawmakers have called for Congress to adapt regulations to AI, a technology that has reshaped all aspects of daily life for Americans.
Congress has taken little action on AI, despite ongoing concerns about its impact on the economy, environment and even the existence of humanity. But polls show Americans remain concerned about its effects, as it becomes an increasingly salient issue in the midterms. Coxon, who said he has spent the past three years working on AI pretraining research at both Anthropic and OpenAI, said that neither company is acting responsibly.
He said there will soon be “superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.” “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately.
No other human activity poses this level of danger,” he wrote. His concerns were echoed by Evan Hubinger, an Alignment Science lead at Anthropic, who said he believes there is a greater than 10 percent chance that AI could “kill all humans” within the next decade. I personally think it is >10% within the next decade.
I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” he wrote. The post garnered attention from lawmakers calling for more action on AI. Representative Greg Casar, a Texas Democrat, said the world is facing an AI “emergency” in light of Coxon’s warning.
An employee still there agreed, and put the odds of AI killing all humans above 10%. Congress must convene hearings and pass my and Bernie’s superintelligence ban,” he wrote. Representative Lori Trahan, a Massachusetts Democrat who has previously called for Congress to take more action on AI, said on X the call is “coming from inside the house.” “Safety researchers are resigning, powerful AI models are breaking out of their labs, and companies are racing ahead anyway.
Extract — continue reading at the source.
