Are We Thinking Correctly About AI Intelligence?

When an LLM answers a question, is it reasoning like humans, or just producing text that looks like reasoning? The distinction isn’t just philosophical, this determines what we can trust AI to do, how closely we need to supervise it, and ultimately what its real-world impact will turn out to be. Melanie Mitchell at the Santa Fe Institute argues that we lack adequate methods for measuring machine cognition, and that AI is a form of “alien intelligence” that operates through non-human cognitive mechanisms.

In this episode of The Joy of Why, Mitchell tells Steven Strogatz how methods that psychologists use to study cognition in other kinds of “alien intelligence” — babies and animals — can be adapted to probe AI, and she lays out six principles for better assessing machine cognition. Their conversation ranges from the challenge of interpreting what’s happening inside these systems, to recent AI-assisted breakthroughs in mathematics, to why a math-performing horse from the early 1900s offers a cautionary tale for how we assess intelligence. Listen on Apple Podcasts, Spotify, TuneIn or your favorite podcasting app, or you can stream it from Quanta.

LEVIN: A podcast from Quanta Magazine where we explore some of the biggest unanswered questions in math and science today. STROGATZ: Well, hello, hello. This is unsurprisingly yet another show about AI.

LEVIN: I’m telling you, it’s a topic people can’t seem to get enough about, and I’m becoming reluctant to pontificate anymore. It’s changing too quickly. Anything we say could be obsolete by next week.

STROGATZ: As we speak, it’s July 23rd, 2026. LEVIN: And it feels different to me than it did in July 23rd, 2025, that’s for sure. That’s actually relevant, this talking about timelines, because our guest today, Melanie Mitchell, who is a cognitive scientist and computer scientist at Santa Fe Institute, is someone that we had on the show previously.

She and I spoke about five years ago, and that is before ChatGPT. And was she interested in AI then? LEVIN: Okay, so it wasn’t just cognitive science.