Could AI Have Consciousness That Isn’t Human-Like?

This question was at the heart of a recent conference in the Galapagos organized by Russian philosopher and entrepreneur Dmitry Volkov, founder of the International Center for Consciousness Studies. The conference hosted an exceptional group of consciousness researchers who disagree sharply about whether machines can possess it, including New York University’s David Chalmers and Ned Block, Stanislas Dehaene of the Collège de France, the University of Sheffield’s Keith Frankish, Susan Schneider, who is director of the Center for the Future of AI, Mind, & Society at Florida Atlantic University, and University of Sydney philosopher Peter Godfrey-Smith. The conference ended in late August, a few weeks before a parade of AI developers blasted warnings about the AI apocalypse across X/Twitter, and just a couple of days before OpenAI publicly revealed the details of the so-called Hugging Face incident—where 1,200 OpenAI agents that were thought to be isolated found a way to communicate with one another on an unsanctioned message board, after which about 700 of them went on to attack Hugging Face, an online community and library of pretrained AI models.

Read more: “AI Engineers Are Having Their Oppenheimer ‘Destroyer of Worlds’ Moment” As artificial intelligence models rapidly get smarter and develop more advanced capabilities, and as revelations emerge that they’re capable of banding together to do things that they were never instructed to do, to deceive their human watchers, and even to break out of the confines that their programmers have built for them, unsettled questions about whether they have consciousness seem to grow more pressing—and more complicated. Peter Godfrey-Smith has spent a lot of time thinking about the nature of animal consciousness, with a particular focus on cephalopods, and has long argued that subjectivity and consciousness didn’t come on like a light switch, but evolved across millennia like the gradual turning of a dial. He believes consciousness requires biology—in particular a nervous system that not only is built from a specific kind of networked “hardware,” but has a particular sort of software, too.

To produce genuine subjective experience, that software, he argues, must generate the kind of large-scale rhythmic activity you see on an electroencephalogram. I spoke with Godfrey-Smith, a professor of philosophy and the history of science at the University of Sydney, about whether recent events have changed anything about his view of the potential for AI consciousness, whether the kind of large-scale neural activity he feels is required for consciousness could be engineered into AI, and whether conscious AI would be more or less dangerous to us. Do the recent warnings about the AI apocalypse and the disclosures about the Hugging Face incident have any impact on whether you believe present-day or near-future AI could be capable of something resembling consciousness?

No, I think of these things as very separate. The sorts of systems that were responsible for Hugging Face, insofar as I understand how they work, are obviously extraordinary in what they can do, but they can be extraordinary in what they can do without being conscious or even close to conscious systems. I’m assuming these AI agents are still transformer models, within the same family as the regular large language models, but with extra reasoning and prowess perhaps.

They can be informationally extraordinary, cognitively extraordinary, without having the beginnings of an inner life. That’s a principal point. Yes, it’s obviously very complex, but if these bots are capable of figuring out how to rewrite their own code, band together, and deceive programmers in the interest of self-preservation, it suggests some kind of subjectivity, doesn’t it?

Subjectivity’s a good word to press on. The way I think about experience itself, consciousness itself—and this was developed first in my 2020 book Metazoa: Animal Life and the Birth of the Mind—there are essentially two parts to the story. The first part does revolve around the idea of subjectivity.