Aircraft

Augmented reality headset helps helicopter pilots see through the clouds

Augmented reality headset help...
A pilot wearing the augmented reality system inside a helicopter flight simulator
A pilot wearing the augmented reality system inside a helicopter flight simulator
View 4 Images
The research simulator at TUM’s Institute of Helicopter Technology
1/4
The research simulator at TUM’s Institute of Helicopter Technology
A pilot wearing the augmented reality system inside a helicopter flight simulator
2/4
A pilot wearing the augmented reality system inside a helicopter flight simulator
Inside the flight simulator
3/4
Inside the flight simulator
The LiDAR system renders tall thin structures like wind turbines and skyscrapers in red, and smaller structures and mountains in green
4/4
The LiDAR system renders tall thin structures like wind turbines and skyscrapers in red, and smaller structures and mountains in green

Unlike commercial airlines, helicopters often fly amidst serious obstacles like skyscrapers, power lines and mountains. So when heavy fog or clouds roll in, navigating can become a dangerous proposition. A new augmented reality vision system out of the Technical University of Munich (TUM) though, could help pilots see clearly through difficult weather using more than just their eyes.

The system primarily makes use of LiDAR, which stands for light detection and ranging and has been used in cars, to map the bottom of the sea, and on drones, among many other applications. LiDAR bounces pulses of laser light off the surrounding terrain, reads the signals that come back and creates a 3D map of the area. In the TUM system, LiDAR sensors would be attached to a helicopter's skids, the rails on which it lands.

We say "would be" because thus far, the system has only been tried out in simulations.

Using the research simulator at the TUM's Institute of Helicopter Technology, 16 pilots were outfitted with a special head-mounted display containing digital eyeglasses and then sent on virtual rides through difficult weather conditions. As their views were obscured by clouds and fog, they were able to turn to the display on the glasses which showed a digital version of their surroundings, indicating obstacles to avoid, in any direction the pilot looked. Red outlines showed objects like wind turbines and construction cranes, while green outlines showed mountains and houses.

The LiDAR system renders tall thin structures like wind turbines and skyscrapers in red, and smaller structures and mountains in green
The LiDAR system renders tall thin structures like wind turbines and skyscrapers in red, and smaller structures and mountains in green

"For ranges of sight below 800 m (about 2,625 ft), the pilots benefited measurably from the terrain and flight data displayed," says TUM. "They not only flew more quickly and more safely than without the head-mounted display, but they also felt that the flights had been less demanding both physically and mentally. The advantages of the system really came to bear when visibility was extremely poor, e.g. in visual ranges of just 100 to 400 m (about 328-1,312 ft)."

In addition to rendering the surroundings, the system can also display flight data including airspeed, position, course and altitude.

The results of the research were presented to the members of the American Helicopter Society in West Palm Beach, Florida.

Before the system can become commercially available to pilots, it will need to be moved from simulators to research helicopters, something one of the developers of the system, Franz Viertler, says will require support from the helicopter industry.

Source: TUM

3 comments
ericgrant
This isn't augmented reality; it's just a HUD.
windykites
Flightradar24.com is a website which enables you to see the synthesised view from any commercial airliner as it flies over the terrain. Surely this data could be used to give a proper visual appearance?
zr2s10
This could really save a lot of lives. Not only from preventing crashes, but it would potentially allow MedEvac helicopters to fly in poorer conditions than allowed now. This would save patients that otherwise would've died, due to the bad luck of having an accident on a foggy/snowy day. Excellent idea, not sure why they haven't already had this or FLIR systems available!