When the unmanned CRS-7 flight blew up in midair yesterday en route to the International Space Station (ISS), it destroyed a lot more than a shipment of freeze dried shrimp cocktail. It also meant the loss of dozens of experiments. One of more exotic of these was Sidekick; a project by NASA and Microsoft that uses the latter's HoloLens technology to provide astronauts with their own holographic augmented reality.
One problem that manned space missions have faced since the first Vostok and Mercury flights is tiny crews having to deal with a workload so great that some, such as on Skylab, have rebelled. Unlike on a naval vessel where experts and artificers in dozens of fields make up the crew, astronauts on the ISS and planned deep space missions are expected to handle tasks well outside their métier on a daily basis. One day they may be required to install a robot, the next tend a greenhouse, and the next fix a zero-gravity toilet. Even with the brightest personnel and the best training, that still means a lot of help from Earthside experts at mission control.
This could change once Sidekick is finally delivered to the ISS. Instead of relying on written instructions or being talked through a task by radio, astronauts could use the system to create an enhanced environment that would provide real-time expert consultation and reduce an already burdensome training regime.
According to NASA, Sidekick operates in two modes. In the Remote Expert Mode, the cameras in the HoloLens visor transmit what the astronaut sees, and the Earthside expert can superimpose text and graphics annotations over this view. Meanwhile, in Procedure Mode, the visor overlays real-world objects with interactive animated holograms.
Unlike the domestic version, the Sidekick HoloLens has been tested in free-fall aboard NASA’s Weightless Wonder C9 jet, and will also be included on the NASA Extreme Environment Mission Operations (NEEMO) 20 expedition starting July 21, which uses an undersea habitat to evaluate space technology and procedures.
Unfortunately, its application on the space station will have to wait until a second set can be delivered on a future mission.Source: NASA