There has been no shortage of warnings from AI's top bosses about existential threats over the years. I attended the world's first AI safety summit in Bletchley Park in November 2023 where the focus was on the most serious threats imaginable, posed by the most powerful AI models. At the time, many thought this was in the realm of sci-fi and the real threats were comparatively far more mundane: job losses, cheating in exams.
Three years later, many of those same voices don't seem to be so sure. On Saturday head of AI company Anthropic, Dario Amodei, urged the pace of development to be slowed down - the bosses of two rival AI firms, Sam Altman of OpenAI and Elon Musk said they agreed with him. Meanwhile AI researcher Jacob Coxon - who quit Anthropic - told the BBC on Sunday staff who were developing the systems were "genuinely frightened" for the future of humanity.
They all add up to a chorus of serious voices urging a slowdown, but although it might sound like a quick fix, it is far from an easy solution. AI staff 'genuinely frightened' for humanity's future, ex-Anthropic researcher tells BBC America is widely understood to be terrified of losing the race to build the most powerful AI to China. On Sunday President Trump made his own views clear, claiming the US was ahead of China in the AI race and declaring: "Whoever wins AI, wins." China isn't renowned for wanting to come second.
Everyone I speak to says that even if their company pauses, others will not and they will just get left behind. It reminds me of the height of the campaign for nuclear disarmament. The problem then too was that nobody wanted to go first.
There is also the question of how on earth a slowdown would be enforced. Would we be relying on the AI companies to be transparent about what they are doing, and more importantly what they are choosing not to do? That would require an epic level of trust, which arguably the tech sector has never earned.
Although Amodei in his post proposed a three-point plan that included independent monitoring of AI models as they are developed, industry-wide regulation and global regulation, some observers question how this would work in practice. "Nobody has given a substantive explanation of what 'slowdown' means," said Ed Zitron, CEO of EZ Primary Research. "Putting a plant from METR, where one of the guys who quit Anthropic went, in every AI company?
Extract — continue reading at the source.
