Elon Musk's superhuman vision system looks terrible in simulations

Elon Musk's ambitious eye implants that "ultimately may exceed normal human vision" are unlikely to reach this lofty benchmark, according to scientists who have used 'virtual patients' to expose the limitations of this innovative technology. Because even the greatest engineering is no match for human neurophysiology in restoring sight.

Back in March, Musk used his X platform to announce the progress that was being made in developing Blindsight, a cortical implant that aims to not just restore vision but provide 'high-resolution' sight.

"Blindsight is the next product after Telepathy," he posted. "I should mention that the Blindsight implant is already working in monkeys. Resolution will be low at first, like early Nintendo graphics, but ultimately may exceed normal human vision. (Also, no monkey has died or been seriously injured by a Neuralink device!)"

Musk has said Blindsight will enable people who have no eyesight, or who have lost their eyes, to be able to 'see' by targeting how the brain processes optical information. To do this, millions of minute electrodes would be implanted in the visual cortex, the area at the rear of the brain that is largely responsible for processing and interpreting visual information from the eyes.

But University of Washington (UW) researchers say there are fundamental flaws in the design of cortical implants like Blindsight, which underestimate the complexities of human eye-brain communication, and they've used detailed computational modeling simulations – which they call virtual patients – to demonstrate that implants are unlikely to ever "exceed" normal vision.

And it all comes down to the limitations of these electrodes and their ability to stimulate the nerve cells (neurons) needed to recreate vision artificially. It's a complex process that relies on the creation of a multitude of intricate neural codes that are needed to properly get visual information properly processed by the brain.

“Even to get to typical human vision, you would not only have to align an electrode to each cell in the visual cortex, but you’d also have to stimulate it with the appropriate code,” said the study's lead author Ione Fine, a professor of psychology at UW. “That is incredibly complicated because each individual cell has its own code. You can’t stimulate 44,000 cells in a blind person and say, ‘Draw what you see when I stimulate this cell.’ It would literally take years to map out every single cell.”

On paper, 45,000 electrodes can be somewhat clumsily equated to 45,000 pixels, like that of a TV or computer screen – so the more electrodes, the sharper and more detailed the image is meant to be. However, the researchers say each neuron relays information about a small "receptive field", and these receptive fields overlap with other neurons and their receptive fields. So that single small spot of light coming into the eye actually stimulates a host of interconnected neurons that help process the information. As such, there's a huge challenge in getting any number of electrodes to recreate the work of thousands of neurons involved in this complex function.

“Engineers often think of electrodes as producing pixels,” Fine said, “but that is simply not how biology works. We hope that our simulations based on a simple model of the visual system can give insight into how these implants are going to perform. These simulations are very different from the intuition an engineer might have if they are thinking in terms of a pixel on a computer screen.”

To demonstrate this, the researchers designed a range of simulations, including a movie of a cat as seen at 45,000 pixels compared to what it would look like for a patient with 45,000 electrodes in their visual cortex. The implant models were designed by using data gathered from existing studies on cortical implants much like the one Blindsight has in mind. While the electrodes were able to interpret some sort of visual picture, the cat was extremely blurry and difficult to recognize beyond shape.

In these videos, the researchers simulated two different arrays of electrode configurations to demonstrate how the cat video would be perceived by someone with cortical implants.

The researchers noted that while this would be an improvement for someone with no sight at all, they caution that this approach to restoring vision may never reach the kind of benchmark Musk believes it will.

“Somebody might one day have a conceptual breakthrough that gives us that Rosetta Stone,” Fine said. “It’s also possible that there can be some plasticity where people can learn to make better use of an incorrect code. But my own research and that of others shows that there’s currently no evidence that people have massive abilities to adapt to an incorrect code.”

Without being able to replicate the neural codes required, no amount of engineering will elevate this technology to anything even close to adequate human sight. And the researchers believe that this is something that should be front of mind when assessing the feasibility of biotech like Blindsight.

“Many people become blind late in life,” Fine said. “When you’re 70 years old, learning the new skills required to thrive as a blind individual is very difficult. There are high rates of depression. There can be desperation to regain sight. Blindness doesn’t make people vulnerable, but becoming blind late in life can make some people vulnerable. So, when Elon Musk says things like, ‘This is going to better than human vision,’ that is a dangerous thing to say.”

The research was published in the journal Nature.

Source: University of Washington