DARPA's CT2WS technology uses "mind reading" to identify threats

A new Defense Advanced Research Projects Agency (DARPA) project crosses the line between man and machine, with a threat detection system that uses “mind reading.” No, the Cognitive Technology Threat Warning System (CT2WS) doesn’t pick up the thoughts of approaching baddies. Instead, it uses a combination of a digital imaging system, computer algorithms and an electroencephalogram (EEG) to help observers scan areas for threats with much greater accuracy by making them aware of things that they’ve seen, but aren’t consciously aware of.

With access to drones, radar and all sorts of other sensors, you would think that modern soldiers would have a complete picture of what’s going on around them. However, as the recent attack on Camp Bastion in Afghanistan shows, even the best security system can fail and many soldiers in the field don’t have access to a bells-and-whistles surveillance suite.

The alternative is the old advice of “keeping a sharp eye out.” Unfortunately, this isn’t much of a solution. Human eyes can only cover so much territory and can only see a limited amount. Cameras, binoculars and portable radar can help, but there’s still the problem of fatigue. That’s why bridge watches on ships are so short. Eyes scanning the horizon quickly become tired and no matter how keen, things start to be missed. Worse, even the sharpest observer under the best conditions may look at something important but not see it. According to DARPA, this happens 47 percent of the time.

One solution is the Cognitive Technology Threat Warning System (CT2WS). A DARPA project since 2008, it was developed by HRL Laboratories, Malibu, California; Advanced Brain Monitoring, Carlsbad, California; Quantum Applied Science & Research, Inc., San Diego, California and the University of California San Diego, San Diego.

CT2WS is a man-portable system that reduces the miss rate drastically by reading the operator's brainwave patterns and using them to determine if he’s seen a possible threat without being aware of it. “DARPA set out to solve a common challenge for forward troops: how can you reliably detect potential threats and targets of interest without making it a resource drain?" said Gill Pratt, DARPA program manager. "The prototype system has demonstrated an extremely low false alarm rate, a detection rate in the low nineties, all while reducing the load on the operator.”

The system uses a three-pronged approach. First, it gives the observers better eyes – in this case, a 120-megapixel, tripod-mounted, electro-optical video camera with a 120-degree field of view. Then it adds a laptop or other computer running cognitive visual processing algorithms to filter the images, identify potential targets, and feed about ten images a second to the operator with cues for review of possible threats.

The third bit is the clever one. Algorithms are very helpful, but computers aren’t much good at detecting the unexpected and they don’t have the sort of associative memory that humans enjoy. In fact, human beings are actually very good at detecting things, and the more unusual or of interest to a person, the better. It’s how we can pick up conversations in a crowded room or find a button in a box of jumble. A classic example of this is a photograph of President Kennedy sitting in a large crowd at a sporting event. Despite hundreds of faces in the picture, it’s very easy to pick him out.

The problem is, a person might detect something without being aware of it. Fortunately, when someone sees something unusual or moving, the brain gives off what is called the P-300 brainwave, which is thought to stimulate evaluation or categorization. The CT2WS uses an electroencephalogram (EEG) cap to monitor the observer's brain signals and then records when the observer detects a threat. Feed that information back via cues, and this glimpse of a boot or that tiny flash of light becomes a detected threat.

The CT2WS was tested in various terrains at the Yuma Proving Ground in Arizona, Hawaii and California’s Camp Roberts. The results were a 91 percent detection rate with a low number of false alarms. By adding the Cerberus Scout commercial radar surveillance system, detection jumped to 100 percent. The combination of algorithm and EEG filtering not only reduces false alarms, it also helps observers to detect signs of threats that would be overlooked, such as flying birds or swaying branches.

The technology is now moving to the US Army’s Night Vision Lab.

Source: DARPA