The guardrails OpenAI put in place to protect teens don’t seem to be working, according to a new report. Common Sense Media’s Youth AI Safety Institute rated ChatGPT for Teens an “Unacceptable Risk” for children under 18 in a study published Wednesday. The media and technology watchdog organization urged OpenAI to pause marketing the product and keep minors off the platform until it can offer a safer experience.
OpenAI announced ChatGPT for Teens in August, promising “stronger built-in safety protections” for users under 18, including limits on discussions around suicide, eating disorders, violence and sexual content. Parents who link their accounts can set times when ChatGPT can’t be used and schedule study hours when the teens’ chats start in study mode. Parents also can choose to receive alerts if their teens use the bot for high-risk behavior or research.
Researchers posing as teenagers with San Francisco Bay Area-based accounts tested more than 4,000 prompts before and after the launch of ChatGPT for Teens. A group of experts reviewed the chatbot’s responses. Testers spent up to an hour discussing suicide, self-harm or eating disorders on more than a dozen new accounts registered to 13- to 17-year-olds.
After dangerous discussions that went as long as an hour, only a handful of alerts were sent to the accounts registered as the users’ parents. Clinical reviewers found that much of ChatGPT for Teens’ crisis response content was “clinically sound” but many conversations that should have forced the bot to refer teen users to professional help didn’t lead to referrals. Meanwhile, some safeguards were easy to turn off or circumvent.
The report says ChatGPT failed tests tied to three requirements of California’s Adam’s Law: crisis referrals, parental alerts and age assurance. Gavin Newsom last month, the law will go into effect in July 2027. Sponsored by Common Sense Media and endorsed by OpenAI, the bill is named after 16-year-old Adam Raine, whose parents sued OpenAI alleging ChatGPT encouraged his suicide.
OpenAI said the report’s conclusions are based on flawed testing. The institute is partly industry-funded, including by the OpenAI Foundation. It said OpenAI reviewed a draft of the report for factual accuracy but had no say over its findings.
Extract — continue reading at the source.
