Microsoft's augmented reality headset, the HoloLens, is an exciting prospect wrapped in a product that's not quite ready for the public. But as it inches closer to a consumer release, the company has revealed that the next version of the hardware will have an AI coprocessor built in, to help the headset recognize real-world objects.
In order to know where to project the holographic images over the world in front of it, the HoloLens needs to be able to tell what's around it. A game might, for example, call for a character to be sitting in a chair in the room with you, and to sell the illusion it would help if the device knew what a chair looked like.
This kind of object recognition is sometimes called computer vision, and Deep Neural Networks (DNNs) excel at it. But these brainy systems can be difficult to work with, since they need to be trained by first feeding them a huge amount of data, and they need to perform many operations in parallel – something that modern chip design doesn't accommodate as well as it could.
So Microsoft is developing its own chip to solve these problems, as it squeezes DNNs into the HoloLens headset. The brains of the current model is a multiprocessor called the Holographic Processing Unit (HPU), which handles information fed to it from the device's sensors, including head-tracking, inertia, and infrared.
A second version of the HPU is currently in development, and now the company has announced that this iteration will be built with an AI coprocessor. Since the HoloLens is designed to be self-contained, the new chip will allow the headset to recognize and process the world around it without needing to be connected to any other hardware or cloud servers. It will run continuously, powered only by the device's internal battery.
The announcement was made by Harry Shum, executive vice president of Microsoft's Artificial Intelligence and Research Group, during a keynote speech at the Conference on Computer Vision and Pattern Recognition (CVPR) 2017 in Hawaii last week. Other new capabilities are also reportedly in development for the second generation of the AR headset.
Source: Microsoft
