A new AI tool can guess what you’re looking at just by analyzing your brain scans—and recreate that image with remarkable precision. It can go the other way too, and predict a person’s brain activity based on what they’re looking at. In the image above, for example, the left-hand image of each pair is what the user actually saw—and its right-hand counterpart is what the model recreated based on the brain scan.
Michal Irani, who developed the tool with her colleagues at the Weizmann Institute of Science in Rehovot, Israel, hopes her “mindreading” tool will ultimately reveal more about how the brain works, and could perhaps be used to help locked-in people communicate, or allow scientists to recreate the content of dreams. Judy Illes, a neuroethicist and professor of neurology at the University of British Columbia in Canada, who was not involved in the research, describes the work as “magnificent.” “The idea [of using this approach] to help people with neurologic conditions … therapeutically is tremendously exciting,” she says. But other scientists warn that a similar approach could be used to reveal the inner thoughts and mental imagery of people, potentially without their consent.
The first attempts produced images that were blurry and hard to make sense of. Advances in technology—both in the fMRI scans themselves and the tools used to make sense of the results—have led to improvements over the years. Irani and her colleagues started by analyzing publicly available brain scan data.
Other researchers had already collected scans from volunteers who were shown hundreds of images while they lay in fMRI scanners. fMRI uses a giant magnet to track the flow of oxygenated blood through the brain. Brain areas that “light up” on fMRI scans are thought to be those that are particularly active at any given moment. They’re not especially specific—in typical fMRI scanners, each highlighted “voxel” of activity covers around three cubic millimiters, containing around 16,000 neurons.
But Irani and her colleagues used newer datasets collected using scanners with a higher resolution—with each voxel covering around one cubic millimeter of neurons, she says. Those datasets showed what the brain activity of volunteers looked like when they viewed various images. Other teams have done this, too, and several other tools have been used to recreate images based on brain scan data.
But they’re not good enough, says Irani. Say a person saw a banana. These models can generate an image of a banana, but it would look different, she says.
Extract — continue reading at the source.
