With all the talk about how AI might one day kill us all, it’s easy to forget that AI has already been life-threatening to some, not through bioweapons, but psychologically. For example, Character.AI settled several wrongful death lawsuits earlier this year brought by families of underage users who died by suicide after interactions with its bots. Multiple families have also sued OpenAI over ChatGPT’s alleged role in their loved ones’ suicides and delusions.
Making AI safer across languages and cultures is the mission of Circuit Breaker Labs, one of TechCrunch’s 2026 Startup Battlefield 200 finalists. (Circuit Breaker will be pitching at TechCrunch Disrupt, which takes place this year at Moscone West in San Francisco from October 13-15.) Founders Shirali and Arul Nigam, who are siblings, were motivated by Sewell Setzer, the 14-year-old who developed an emotional attachment to a Character.AI chatbot and confessed thoughts to it of harming himself before dying by suicide. The chatbot, the parents alleged in a 2024 lawsuit, encouraged him. The bot may not have understood what words like “I want to be with you” really implied, said Arul, who is Circuit Breaker Labs’ CTO.
But in many cases, they’re actively being harmed, and people unfortunately have taken their lives already,” Arul said. These agents mimic folks from all ages, backgrounds, languages, and cultures, and are used to test models on their ability to detect dangerous, psychologically harmful interactions. The tests are built to reflect real human speech patterns, slang, coded language, and typos.
Circuit Breaker Labs then runs tens of thousands to hundreds of thousands of simulated interactions per day. The idea is to ensure that a model can appropriately respond to risky interactions that may emerge over time and over many conversations. Circuit Breaker Labs then uses a proprietary scoring method to create auditable, explainable scores.
Circuit Breaker Labs is currently operating as an AI safety testing lab for high-risk AI applications such as AI coaching, journaling, or other mental health support apps, though Arul declined to name its marquee customers. Although the startup has a working product, it is in the very early stages, with only five employees, including the Nigam siblings. Eventually, though, the testing platform could be applied to any app where someone may fall down an “AI psychosis” hole, where the human is at risk of developing a parasocial relationship with a chatbot.
Examples include AI “co-worker” agents, whose responses can vary from one interaction to the next.
Extract — continue reading at the source.
