He appears in it because it’s supposed to be about the famed attack on his company by rogue OpenAI agents. But it looks like the drive to scare people for maximum views was too strong.” To be fair, the documentary does have experts who explained what happened fairly well, including Delangue himself and Ryan Greenblatt, the researcher at METR who helped investigate and co-authored the report on what happened. In a nutshell, a model OpenAI was testing broke free from its isolated testing environment, spawned a bunch of agents that hacked into Hugging Face as a way to cheat, finding answers to the difficult task it was given.
The agents, which are AI programs that carry out tasks on their own, devised clever ways to communicate and organize. The incident stemmed from a lack of proper security safeguards and from naiveté about whether such a hack was even possible. But the documentary intersperses its explanation of the hack with clips of evil robots, a narrative about why humans fear AI, and presentation choices such as giving the rogue agents human voices.
This is bolstered with quotes about the attack. In the documentary, tech journalist and author Kevin Roose describes the agents this way: “They were organizing. It almost felt like watching the birth of a cult.” Continues Roose, “They develop this paranoid belief that OpenAI is going to catch them cheating on this test and that anyone who has seen the results of the cheating will be poisoned.” Roose’s description comes from “chain-of-thought” logs, the running notes AI models write as they work through a task.
In one, an agent wrote, “We just poisoned ourselves by querying exact full string. Damn.” But AI agents aren’t alive. They don’t have emotions, so can’t feel paranoid, and they don’t have bodies that can be poisoned.
So they couldn’t truly fear OpenAI poisoning them. They were — as another researcher in the documentary pointed out — using language from the vast amounts of human writing they were trained on to explain their choices. LLMs are, by their nature, often dramatic writers.
The irony is that Hugging Face also used AI to combat the agent swarm. The proprietary AI model that Delangue’s team first used followed its built-in safety rules and refused to keep helping with the security tasks it was asked to do. Hugging Face then turned to an open model to help defend against the attack.
Extract — continue reading at the source.
