It's frustrating to have a clear mental image of something but not be able to exactly get it across in words or a drawing. Now, a team of neuroscientists from the University of Toronto Scarborough has developed a way to digitally recreate exactly the image someone is thinking about, by scanning their brain.
So-called mind-reading technology is getting eerily accurate. Along with allowing people to control prosthetics with their thoughts, these systems have quickly advanced from picking out what number you're thinking of to decoding more complex concepts. It's all happening so fast that some researchers have proposed new human rights regarding how the brain can be read or manipulated.
The new study was designed to see whether specific images could be plucked out of a person's mind. To test out the idea, the team hooked people up to electroencephalography (EEG) equipment and then showed them pictures of faces on a computer screen. The EEG system recorded their brain waves, and after running the data through machine learning algorithms the system was able to digitally recreate the face that the test subject had just seen.
"When we see something, our brain creates a mental percept, which is essentially a mental impression of that thing," says Dan Nemrodov, co-author of the study. "We were able to capture this percept using EEG to get a direct illustration of what's happening in the brain during this process."
Brain-reading studies generally involve one of two methods – EEG and functional magnetic resonance imaging (fMRI). The former measures the electrical activity in the brain through a cap full of electrodes, while fMRI uses a magnetic field to monitor blood flow in different parts of the brain. Both have their advantages and disadvantages, but EEG is more commonly used, less expensive, and can record faster changes.
"fMRI captures activity at the time scale of seconds, but EEG captures activity at the millisecond scale," says Nemrodov. "So we can see with very fine detail how the percept of a face develops in our brain using EEG."
That high time accuracy allowed the team to determine that it only takes about 170 milliseconds for the human brain to create a decent mental picture of a face it's looking at.
In future, the team wants to expand the technique to be able to recreate objects other than faces, and do so over longer periods of time, allowing virtual reconstruction of images that a person remembers seeing more than a few seconds ago.
"It could provide a means of communication for people who are unable to verbally communicate," says Adrian Nestor, co-author of the study. "Not only could it produce a neural-based reconstruction of what a person is perceiving, but also of what they remember and imagine, of what they want to express. It could also have forensic uses for law enforcement in gathering eyewitness information on potential suspects rather than relying on verbal descriptions provided to a sketch artist."
The research was published in the journal eNeuro. The team demonstrates the technique in the video below.
Source: University of Toronto Scarborough
They didn't say what the EEG image recovery algorithms were trained on - if it was subject-specific, i.e. if they were trained on only a few faces and that particular subject's EEG images of them, you'd expect a much higher degree of success than if the training was more generic, i.e. on a wide variety of faces and many subject's EEG images of them.
MAD ~ Mind Assisted Drawing
Imagine the family of an accident victim who's body is breathing and pumping blood, but has not given any other sign of higher cerebral activity. If they could know conclusively that their loved one is truly gone and only the shell remains it would offer some closure. On the other hand, if the patient is conscious and aware of their surroundings and simply unable to physically respond to external stimuli, it would allow them to communicate with medical personnel and family.
As with any other tool there is a potential for misuse, but the potential for good is considerable.