Virtual reality can already sweep us away to weird and wonderful places, but how we interact with ordinary objects in those worlds could play an important role in selling the user experience. Disney researchers have been exploring these possibilities with a proof-of-concept system that enables users to catch a real ball while immersed in a VR environment, something they see as a first-step in developing more complex user-object interactions.
Fooling the senses and truly transporting the mind to another realm is the pot of gold at the end of the VR rainbow, and we are seeing some clever approaches to that end. The announcement of HTC's Vive Tracker in January, which allows baseball bats, firehoses, gloves and guns to be used as VR controllers, offered an interesting taste of what the future might hold for input devices. And Disney itself has made moves in the area with a system that alters the texture of real-world surfaces, and another that provides haptic feedback designed to imitate complex sensations like falling rain.
Disney's efforts in the space are much closer to the experimental research end of the spectrum than the consumer-ready, but they certainly have potential for those types of VR applications further down the track. The new system relies on a high-speed motion capture camera called OptiTrack Flex 13, which has been used previously to train orthopedic surgeons and by a quadriplegic to drive a race car using subtle head movements.
In adapting it to their purposes, the Disney researchers used the camera to track the motion of the ball along with the catcher's hands and head at 120 fps, with a lag time of 8.33 milliseconds. This allows the system to not only track the ball through the air, but predict its trajectory and translate that into a virtual rendering in real-time.
With the user strapped into a consumer version Oculus Rift headset, the team observed the system's success across three visualization modes. The first simply shows the user a virtual ball, a rendering of the physical ball as it moves through the air. The second uses a line to indicate the predicted trajectory of the ball, while the third indicates the target catching location as well, or where the ball will likely be snatched by the catcher.
These various modes were tested over 140 tosses in total, which included 132 catches. The team says catching was equally successful throughout the different modes, although the catching strategy did change when the users were shown the target location, where their hands reached the catch point much earlier. The team says this could offer game developers some unique possibilities.
"With VR, we can show you the future by pre-rendering where the ball is going to be," says Günter Niemeyer, senior research scientist at Disney. "For some types of interactions, game designers might choose to take advantage of VR to make certain tasks easier, just as using a net to catch balls might make some games more enjoyable."
As for what those interactions would be in future games, that isn't as clear. Perhaps local multiplayer experiences, where two players hand or toss objects to each other? VR arcades where staffers or machines throw objects you can interact with? Either way, virtual reality still has more basic problems to prioritize, including more consumer-friendly pricing.
The system can be seen in the video below, while a paper describing the research can be accessed online here.
Source: Disney