A brain implant has enabled people with paralysis to speak while gesturing, such as nodding or waving. These sorts of exchanges, done via an avatar, could help them communicate with more nuance. Experimental brain implants have boosted the ability to either speak or move among people with paralysis, but attempts to do both simultaneously haven’t been very successful.
To achieve this, implants need to cover a large part of the sensorimotor cortex, a brain region involved in speech and movement, says Brosler. Now, Brosler and her colleagues have tested an iPhone-sized implant in this part of the brain in two people with paralysis. The first participant, who the researchers call Bravo-1r, became paralysed after a stroke, while the second, Bravo-6, had amyotrophic lateral sclerosis, the most common form of motor neuron disease.
Both participants had lost almost all movement in their limbs. They also couldn’t speak and could only make unintelligible sounds. First, the team recorded the participants’ brain activity while they repeatedly attempted to say 10 phrases – such as “hello”, “nice to meet you” and “how’s it going?” – or perform 10 gestures, like waving, clapping and shaking their head.
In some cases, the participants attempted to do these separately, while in others they tried both simultaneously. The team then used each of the participants’ brain recordings to train customised machine learning models to predict their intended phrases and gestures. To put the models to the test, the participants attempted to repeat various pairs of phrases and gestures that were displayed on a screen, which the models weren’t aware of.
Their predictions were used to control the movement of an avatar resembling the participants, while the speech predictions flashed up on the screen. The team chose to use avatars, rather than attempting for the individuals themselves to speak and gesture, because this offered a relatively simple way to prove that both actions could be simultaneously decoded from brain activity, says Brosler. Keith Thomas, a man in his 40s with no sensation or movement in his hands, is able to feel and move objects by controlling another person’s hand via a brain implant.
The technique might one day even allow us to experience another person’s body over long distances. Bravo-1r’s intended gestures and speech were predicted with 88 and 84 per cent accuracy, respectively, while Bravo-6’s were predicted with 66 and 70 per cent accuracy. But it’s based on a very limited set of phrases and gestures, and these still need improving, he says.
Extract — continue reading at the source.
