Recently, Jacob Coxon, a researcher at Anthropic, quit his job over concerns that AI was on a collision course with humanity. A flurry of headlines put the spotlight on the current crisis: unregulated competition between technology companies and between the US and China is putting our global infrastructure at risk at the very least, and threatening human existence at worst. One’s mind fills with pictures of robot armies or systematic blackmail.
In the spirit of never letting a good crisis go to waste, an important opportunity will come with the summit this week between the US president, Donald Trump, and China’s president, Xi Jinping. It is powerful, helping us as individuals to plan our travel, book our gym classes and order takeaway food, while at the institutional level it is boosting the efficiency of businesses and bureaucracies. In academia it is providing stunning breakthroughs in mathematics that have eluded the best theoretical mathematicians for decades.
In the sense that AI is widely available and can do a lot of good, it is like medicines, electricity and road vehicles. But unlike these technologies, we have not yet worked out how to ensure that the benefits of AI outweigh the risks. In contemplating the way forward, it is worth thinking about how much time we have, or don’t have, to get it right.
Technology development used to be slow. The first use of stone tools began 3.4m years ago and barely changed for the next 2m years. Computer chips have been doubling their capacity every two years from 1965 to the present.
That is fast, but slow compared with AI development. According to the METR research institute, generative AI capability doubled every seven months from 2020 to 2024. Since then, generative AI capability has been doubling every four months, and with the recent release of ChatGPT 6-Astra and Claude Fable 5.1, the doubling rate is most likely even faster.
Just a few days after Coxon’s resignation, Anthropic’s CEO, Dario Amodei, expressed the same concerns, and called for a slowdown of AI development. He was quickly backed by OpenAI’s CEO, Sam Altman, and xAI’s CEO, Elon Musk. In defence of unregulated AI development, Trump dismissed their concerns, saying that statements to the effect that AI might destroy or massively harm humanity are a hoax and that calls to slow down AI development are a plot that will benefit China.
Extract — continue reading at the source.
