DARPA has announced the start of the Fast Event-based Neuromorphic Camera and Electronics (FENCE) program, which is designed to make computer vision cameras more efficient by mimicking how the human brain processes information. Three teams of scientists led by Raytheon, BAE Systems, and Northrop Grumman, are tasked with developing an infrared (IR) camera system that needs to process less data, operates faster, and uses less power.
Modern imaging cameras are growing increasingly sophisticated, but they are also becoming victims of their own success. While state-of-the-art cameras can capture high-resolution images and track objects with great precision, they do so by processing large amounts of data, which takes time and power.
According to DARPA, this is fine when the task is something like tracking an airplane in a clear blue sky, but if the background becomes cluttered or starts to change, as is often the case in military operations, these cameras can soon be overwhelmed.
What the FENCE program hopes to do is to create event-based cameras that are more intelligent thanks to the use of brain-mimicking or neuromorphic circuits. What these do is to drastically reduce the amount of data that needs to be handled by disregarding irrelevant parts of the image. Instead of dealing with an entire scene, the event-based camera focuses only on the pixels that have changed.
To achieve this, the FENCE team is working on a new low-latency, low-power, event-based infrared (IR) focal plane array (FPA), an asynchronous Read-Out Integrated Circuit (ROIC), and a processing layer that helps the ROIC identify relevant spatial and temporal signals. New digital signal processing and learning algorithms will also be needed to handle complex, changing backgrounds.
The result could be a FENCE sensor that uses less than 1.5 watts of power. Because the new technology is aimed at military applications that include autonomous vehicles, robotics, and IR search and tracking, the sensors will need to be flexible and adaptable.
"The goal is to develop a ‘smart’ sensor that can intelligently reduce the amount of information that is transmitted from the camera, narrowing down the data for consideration to only the most relevant pixels," says Dr. Whitney Mason, the program manager leading the FENCE program.
Source: DARPA
