With brain scanners and AI, US researchers have been able to at least roughly capture certain types of thoughts in volunteer test subjects. For example, a decoder they developed was able to roughly reproduce what was going through the minds of the participants in certain experimental situations using an imaging process.
As the team now reports in the journal “Nature Neuroscience”, the technology cannot be used to secretly read out thoughts. The brain-computer interface, which does not require surgery, could help people who have lost their ability to speak, for example as a result of a stroke. However, experts are skeptical.
Brain-computer interfaces (BCI) are based on the principle of reading human thoughts through technical circuits, processing them and translating them into movement or speech. In this way, paralyzed people could use mind control to control an exoskeleton, or people with locked-in syndrome could communicate with their outside world. However, many of the corresponding systems currently being researched require the surgical implantation of electrodes.
In the new approach, a computer forms words and sentences based on brain activity. The researchers trained this speech decoder by having three test subjects hear stories for 16 hours while they lay in a functional magnetic resonance imaging (fMRI) scanner. Such an fMRT can be used to visualize blood flow changes in areas of the brain that indicate the activity of the neurons.
In the next step, the subjects heard new stories while their brain activity was further examined. The previously trained speech decoder was now able to create sequences of words from the fMRT data, which the researchers said reproduced the content of what was heard largely correctly. The system did not translate the information recorded in the fMRI into individual words. Rather, it used the connections recognized in the training and artificial intelligence (AI) to assign the measured brain activities to the most likely phrases heard in new stories.
Rainer Goebel, Head of the Department of Cognitive Neurosciences at Maastricht University in the Netherlands, explains this approach in an independent classification: “A central idea of the work was to use an AI language model to count the number of possible phrases that can be combined with consistent with a pattern of brain activity.”
In a press conference on the study, co-author Jerry Tang illustrated the results of the tests: The decoder reproduced the sentence “I don’t have my driver’s license yet” as “She hasn’t even started learning to drive”. According to Tang, the example illustrates major difficulties of the model with pronouns. “But we don’t yet know why that is.”
Overall, the decoder is successful in selecting phrases in new, untrained stories that have words in the original text or at least have a similar meaning, according to Goebel. “But there were also quite a lot of errors, which is very bad for a fully-fledged brain-computer interface, since for critical applications – for example communication with locked-in patients – it is particularly important not to generate false statements.” Even more errors were generated when the subjects were asked to independently imagine a story or watch a short animated silent film and the decoder was asked to play back events in it.
For Goebel, the results of the system presented are too bad overall: “fMRI-based BCIs will probably remain limited to research work with a few test persons in the future – as in this study.”
Christoph Reichert from the Leibniz Institute for Neurobiology is also skeptical: “If you look at the examples of the presented and reconstructed text, it quickly becomes clear that this technology is still a long way from reliably generating an imaginary text from brain data.” Nevertheless the study hints at what could be possible if measurement techniques improved.
There are also ethical concerns: Depending on future developments, measures to protect intellectual privacy could be necessary, the authors themselves write. However, tests with the decoder showed that the rehearsers had to actively participate in both the training and the subsequent application. “If they counted in their heads, named animals or thought of another story while decoding, the process was sabotaged,” describes Jerry Tang. The decoder also performed poorly when the model had been trained with another person.
