NASA trains pilots with Fused Reality

To gain proficiency, pilots need realistic training, but they also need to avoid needless cost and risk. Real aircraft provide the most obviously realistic training, but they're dangerous in inexperienced hands. Meanwhile, simulators can reproduce much of the look and feel of actual flying without the danger of losing an aircraft or pilot, but they aren't as successful when it comes to complex maneuvers like aerial-refueling. To square the circle, NASA is developing a technology called Fused Reality, which uses a special headset that combines real flying in a real aircraft with an overlaid simulation.

Fused Reality was named by its inventor, Ed Bachelder, the technical director at Systems Technology, Inc of Hawthorne, California, where development started in 2003. The systems works like augmented reality glasses like Google Glass or digital pilot goggles by combining the real-world with interactive computer graphics inside the viewer of the headset. The result is a view of the area around the plane that mixes the actual scene with computer generated elements that allow the trainee to practice complicated exercises, such as runway landing at altitude, formation flying, and tracking an aerial refueling drogue, without interfering with the regular operations of the aircraft.

"I’m seeing the real world through my camera, so I’m seeing mountains and clouds and the aircraft control panel, but I’m flying formation with a virtual tanker," says pilot Scott Howe, describing a simulated formation flight with KC-135 tanker. "I was just trying to keep station with that tanker and practice aerial refueling with the [Fused Reality] system."

The Fused Reality technology was tested by at NASA’s Armstrong Flight Research Center at Edwards, California, and the National Test Pilot School (NTPS) in Mojave, where three evaluation flights were made in 2012 using a highly modified Learjet in-flight simulator. A second testing phase was carried out in September 2014 with nine flights on board a Gippsland GA-8 Airvan research aircraft from the NTPS.

NASA sees Fused Reality as having applications not only in training, but also in evaluating new aircraft by allowing test pilots to gain hands-on experience of new configurations, aircraft control algorithms, and displays under actual conditions without the immediate need for the experimental aircraft. It may also find applications in astronaut training and carrier landing practice.

"Fused Reality allows all pilots to learn how to fly difficult and dangerous tasks such as aerial refueling, aircraft carrier landing, formation flight and aerial firefighting, which are usually taught in a ground based flight simulator, by putting the simulator in flight in the actual aircraft," says Armstrong project manager Bruce Cogan. “Virtual images of runways, aircraft carriers, and tanker aircraft are presented to the pilot in a helmet mounted display that reacts with the actual dynamics of the aircraft being flown."

The Fused Reality technology will undergo further testing next March at the Air Force Test Pilot School’s Test Management Program, where pilots will design a flight test using a Beechcraft King Air C-12 light aircraft, which will be used to compare the system with conventional handling qualities tasks.

The video below from the NASA Armstrong Flight Research Center walks us through the Fused Reality system.

Source: NASA