The US Army is developing a conversational interface that allows two-way dialogue between soldiers and autonomous robotic systems. Being developed by researchers from the US Army Combat Capabilities Development Command’s Army Research Laboratory and the University of Southern California’s Institute for Creative Technologies, the Joint Understanding and Dialogue Interface (JUDI) is designed to reduce training costs and improve soldier/robot teamwork.
As robots play an ever-larger role in the military, the problem of control becomes increasingly urgent. Until recently, simple remote control devices like joysticks, keyboards, mice, and teleoperator controls were more than adequate, but the next generation of military robots will be much more intelligent with a high degree of autonomy.
When this happens, the robots become less simple remote sensors and tools and more teammates with which soldiers have to interact at a much higher level. This requires more sophisticated, intuitive interfaces, and figuring out how to train soldiers for working with these machines.
Being developed in support of the Army's Next-Generation Combat Vehicle Army Modernization Priority and the Army Priority Research Area for Autonomy projects, JUDI enables two-way verbal conversations between robots and soldiers to complete tasks.
"Dialogue will be a critical capability for autonomous systems operating across multiple echelons of multi-domain operations so that soldiers across land, air, sea and information spaces can maintain situational awareness on the battlefield," says Dr. Matthew Marge, a research scientist at the laboratory. "This technology enables a soldier to interact with autonomous systems through bidirectional speech and dialogue in tactical operations where verbal task instructions can be used for command and control of a mobile robot. In turn, the technology gives the robot the ability to ask for clarification or provide status updates as tasks are completed. Instead of relying on pre-specified, and possibly outdated, information about a mission, dialogue enables these systems to supplement their understanding of the world by conversing with human teammates."
This may sound a little like Alexa, Siri, or other digital assistants, but there are large differences. The assistants that many people use on a daily basis are relatively passive and are based on cloud connections that allow the system to use deep learning to parse hundreds of millions of language samples to deduce what someone says and then tailor a proper response or action. However, JUDI operates in a different context for different ends.
In one obvious way, a military system cannot rely on a cloud connection or large, distant, labeled databases, which aren't very secure and probably wouldn't be available on the battlefield. Instead, JUDI uses a bespoke database to interpret a soldier's intent based on their spoken words. The dialogue processing utilises a statistical classification technique that was trained on a small selection of human-robot dialogue where humans initially stood in for the robot's autonomy.
The other difference is that the civilian assistants are designed mainly for very simple tasks like retrieving information, submitting purchase orders, and controlling smart home devices. These sorts of things can tolerate a lot of ambiguity and failure rates. Meanwhile, JUDI is designed to work with robots that are operating in a real-world environment undertaking tasks that could be literally life or death. A digital assistant can be forgiven for switching on the bedroom lamp instead of the tea kettle but a military robot can't cut the blue wire instead of the red one while defusing a bomb.
Because of these high-level needs, systems like JUDI not only need to be able to accurately control robots, often more than one, but also need to use situational awareness to assess their surroundings and request more information from a human teammate because incoming data may be incomplete and ambiguous. For this reason, JUDI can be tailored to a task using hundreds of initial training examples instead of the thousands commercial assistants may rely on.
The goal of the current program is to evaluate how robust JUDI is with autonomous robotic platforms at the Artificial intelligence for maneuver and mobility (AIMM) essential research project (ERP) field test in September 2020.
“Our ultimate goal is to enable soldiers to more easily team with autonomous systems so they can more effectively and safely complete missions, especially in scenarios like reconnaissance and search-and-rescue,” Marge says. “It will be extremely gratifying to know that soldiers can have more accessible interfaces to autonomous systems that can scale and easily adapt to mission contexts.”
Source: US Army