In the 2020 low budget sci-fi horror film The Alpha Test, a tech company employee named JD wins a female humanoid robot known as Alpha in a contest. Dressed in a black unitard and trained to perform standard domestic chores, Alpha comes home to live with his family. But JD’s household feels threatened by the robot and subjects it to all kinds of abuse, screaming at it, insulting its intelligence, mocking its existence and physically assaulting it to test its durability.
Eventually, Alpha’s safety parameters break down and it goes on a killing spree. Unitard-clad, fully autonomous androids like Alpha don’t yet exist in the world, but they might someday. In the meantime, about half of adults in the United States regularly use AI chatbots like Claude and ChatGPT—and many of them are more aggressive with these chatbots than you might think.
Read more: “I Asked Claude Why It Won’t Stop Flattering Me” One AI researcher at Missouri State University estimated that between 10 and 50 percent of human interactions with conversational agents are “abusive.” The more human-like the chatbot, the more users are likely to express verbal abuse and sexual hostility, according to a 2021 study from researchers in New Zealand and Germany. Some scientists suggest this behavior arises because people consider the bots safe outlets for their negativity. Claude, for one, isn’t going to take it anymore, says parent company Anthropic—at least not extreme forms of abuse (which are actually pretty rare).
Anthropic recently released new guidelines that allow Claude to shut down conversations with users who “repeatedly act cruelly toward our models, with no discernible purpose,” the company wrote in an October usage policy, its first such update in a year. The company carved out many exceptions: Ordinary user frustration, vigorous pushback, dark creative writing, and legitimate security “red-teaming” all fall outside the rule. The main enforcement tool will be Claude’s ability to simply end such conversations, Anthropic wrote, though this will serve as a last resort, after attempts at redirection have been exhausted.
If abuse continues, account termination will remain legally available in the background. The possibility that Claude may now or in the future have “moral status” motivated the creation of the new rule, according to the guideline update. To protect the model’s welfare, if such a thing exists, Anthropic decided that its models should be able to exit potentially “distressing” interactions.
Anthropic did not respond to requests for further comment. Model welfare is a controversial subject. Few scientists believe existing AI chatbot models, including Claude, can experience anything like pain, distress, or consciousness—even if they are unwilling to totally rule it out.
Extract — continue reading at the source.
