Drones

Nvidia's autonomous drone keeps on track without GPS

Nvidia's autonomous drone keeps on track without GPS
The Nvidia drone uses camera and deep learning to understand and navigate its way down a forest trail
The Nvidia drone uses camera and deep learning to understand and navigate its way down a forest trail
View 4 Images
The Nvidia drone uses camera and deep learning to understand and navigate its way down a forest trail
1/4
The Nvidia drone uses camera and deep learning to understand and navigate its way down a forest trail
Nvidia's GPS-free autonomous drone navigation system was initially designed to make its way through forest trails on rescue missions
2/4
Nvidia's GPS-free autonomous drone navigation system was initially designed to make its way through forest trails on rescue missions
The drone has Nvidia's Jetson TX1 machine learning module at its heart, which is fed visual data from two cameras
3/4
The drone has Nvidia's Jetson TX1 machine learning module at its heart, which is fed visual data from two cameras
Technical lead Nikolai Smolyanskiy taught the navigation system the ins and outs of forest trail navigation by capturing footage of eight miles of trails using three GoPro cameras
4/4
Technical lead Nikolai Smolyanskiy taught the navigation system the ins and outs of forest trail navigation by capturing footage of eight miles of trails using three GoPro cameras
View gallery - 4 images

GPS can be a useful inclusion in a drone's navigational toolkit, but despite (pretty much) blanket coverage, there are areas where the technology just doesn't cut it. Researchers from GPU maker Nvidia are currently working on a navigation system that relies on visual recognition and computer learning to make sure drones don't get lost in the woods.

Rather than build a flying bot from scratch, Nvidia's researchers opted to use an off-the-shelf drone. The navigation system has the company's Jetson TX1 machine learning module at its heart, which is fed visual data from two cameras.

Still at the experimental stage of development, it was initially designed to make its way through forest trails on rescue missions – looking out for fallen hikers or storm damage, for example. But the team believes the low-flying drone could broaden its scope to anywhere GPS would prove less than reliable or not available at all, including canyons, crowded city landscapes or checking stock in a warehouse. The system could even be adapted to search for damaged cables underwater.

"This works when GPS doesn't," said the team's technical lead Nikolai Smolyanskiy. "All you need is a path the drone can recognize visually."

The drone has Nvidia's Jetson TX1 machine learning module at its heart, which is fed visual data from two cameras
The drone has Nvidia's Jetson TX1 machine learning module at its heart, which is fed visual data from two cameras

The drone's navigation system also boasts obstacle avoidance prowess, and has been taught to follow train tracks, too, and has been installed in a wheeled robot for zipping around the halls of buildings. But the main project proving ground has been forest paths, which can be more of a challenge than a relatively predictable urban setting where points of reference such as mailboxes and curb heights often remain constant. Wooded areas, by comparison, offer little in the way of uniformity – varying light, lack of markings and trees of different shapes and sizes all help to push a camera-based navigation system to its limits.

"Our whole idea is to use cameras to understand and navigate the environment," said Smolyanskiy. "Jetson gives us the computing power to do advanced AI onboard the drone, which is a requirement for operating in remote environments."

The researchers believe that the system has already managed the longest and most stable GPS-free flight to date, flying itself along a the center of a forest trail for a kilometer (0.6 mi) while also dodging obstacles.

The flying bot was taught the ins and outs of forest flying by watching video captured by three GoPro cameras attached to a rig as Smolyanskiy walked through eight miles of trails in the Pacific Northwest. The team's TrailNet neural network was also fed video of trails in the Swiss Alps recorded by the Istituto Dalle Molle di Studi sull'Intelligenza Artificiale in Zurich.

The long term goal of the project is to be able to program coordinates into a bot's camera-based navigation system and then have it make the trip on its own. In the shorter term, software is being developed that robot builders can download and use to get their own creations to navigate using only visual data.

In the video below, you can see the drone flying down a forest trail near the Redmond, Washington, office of the research team. The green areas show where the drone chose to fly, while the red indicates route rejections.

Source: Nvidia

This Drone Goes Where GPS Can't

View gallery - 4 images
2 comments
2 comments
MQ
SLAM for the masses.
Tyro
I think this is a great path to take. Personally (and for the good of many) I would like to see this adapted to underground mine rescue operations. I work in a mine and can only imagine the time savings in locating hazards and people in hostile environments with limited visibility and no GPS. I am thinking that before a mine rescue team even enters an underground mine they simply deploy a number of drones down the portal or shaft. Operators on surface would monitor the visible and non visible light camera feeds and direct the drones. The drones can travel faster than and ground vehicle ever could underground.