A new 3D camera inspired by jumping spiders' eyes may well be found in the next generation of battery-operated wearable tech, assistive devices, robots and drones. The technology utilizes “less energy than a nightlight.”
Named the SpiderCam, the prototype from Northwestern University replicates the manner in which the spiders' eyes allow the creatures to gauge distances before making a surprise sideways jump at some unsuspecting victim minding its own business.
“Jumping spiders jump to catch prey, to avoid predators and to get around, and that requires excellent vision,” says Northwestern’s Asst. Prof. Emma Alexander, the study’s corresponding author and expert in bio-inspired computer vision.
Most standard 3D cameras capture images from various viewpoints to estimate depth, either by comparative analysis of each image or through projecting and measuring light. These methods demand significant computing power, expensive hardware and considerable energy consumption.
Human eyes are limited to one retina each, whereas jumping spiders have multiple layers of retinas in each eye that are able to separately focus at slightly different distances – one layer might view an object in focus while another layer sees the same object, but slightly out of focus.
“They see multiple levels of focus at all times,” Alexander says. “So, they are always collecting pairs of images. Then, their brains could compare these differences in sharpness to judge distance.”
“But their brains are very small – the size of a poppy seed – so they have to compute these distances in a highly efficient way. We wanted to understand whether we could borrow some of the same principles to create an extremely energy-efficient depth sensor that could be used in resource-constrained situations where users don’t have unlimited access to power.”
The SpiderCam operates similarly by way of a custom-built camera that shoots two images at the same time with minute differences in focus settings. Instead of running resource-heavy software on a conventional processor, the team built a custom algorithm directly into a specially programmed computer chip optimized for energy-efficient processing.
The chip prototype can process depth maps at 32.5 frames per second while using only 624 milliwatts of power – the team claims the SpiderCam is the first 3D camera to operate below one watt. The custom algorithm then interprets how the sharpness of the edges and textures of the two images differs, converting the data into depth measurements in real time.
Future plans include improving the SpiderCam’s optics, expanding its field of view, merging the technology into wearable devices and small robots, and redesigning the custom computer chip to reduce power usage even lower.
“I’m very interested in settings where you’re very resource-constrained and can’t just plug a camera into a wall,” Alexander says. “For example, [the SpiderCam] could be deployed in field settings with limited power. Separately, I also think it’s particularly exciting for applications like augmented reality where you’re interfacing with the physical world and need to know the locations of objects around you.”
And actually, this isn’t the first jumping-spider-inspired depth-sensing technology we've seen, as Harvard scientists have previously developed a device of their own.
Source: Northwestern University
