Last month, social media giant Facebook rebranded as Meta and revealed its vision for the future of online interaction. Much of the news so far has focused on what immersed users will see, but the company is also working on bringing touch to the Metaverse.
For the last seven years or so, Sean Keller has been building up a team at the company's Reality Labs development platform to work on a pair of soft, lightweight haptic gloves that can realistically mimic the sensation of touch in the virtual world.
We've seen plenty of developments in this area before of course – giving gamers a real feel for the action, helping to make virtual training appear more realistic, giving computer-generated objects some presence, and even bridging a physical divide in long-distance relationships. But Meta is looking to enable all of that and more.
"We’re creating almost everything about this discipline from scratch," said Keller. "We’re learning how people perceive the sensations of touch and how they complete tasks. We’re figuring out how to fit the whole variety of human hand shapes and sizes, while maintaining mechanical coupling to the user. We’re pushing the boundaries of what’s possible with soft robotics and instrumented tracking systems. And we’re inventing entirely new soft materials and manufacturing technologies – it’s a clean break from the past."
Coming up with a convincing wearable involves lining up a lot of technical dominoes so that the virtual experience is as close as possible to real-world tactile interaction. So far the project team has opted to tap into the world of soft robotics to drive the actual haptics, realizing early on that mounting a bunch of bulky motors and wiring to the gloves would not only add significant weight to any gloves, but would also generate uncomfortable amounts of heat.
The Reality Labs system employs a number of pads on the inside of the fingers, which are inflated with air to apply pressure against the user's skin. How much pressure varies according to the virtual object being touched and how it's being handled, but to help with real-time application the team is busy designing a new high-speed microfluidic processor to precisely control the actuators and make for a speedy, more natural response time.
Any computer system running the show would need to know more than simply where a wearer's hands are in a virtual space at any given moment, it would also need to register contact with objects, give them some sort of lifelike feel and determine how the user is interacting with them. As well as working on new hand-tracking technologies, the team is also building a new kind of rendering software to make for realistic real-time sensations, as well as software tools to allow creators to build haptic content with relative ease.
Another part of the project has been working on developing lightweight, flexible and comfortable materials for the gloves. Inexpensive customized polymers are being created that can be spun into fibers with included conductivity, capacitive and sensing properties. The materials team is also exploring ways to supply some sort of custom fit for wearers further down the line, as a one-size-fits-all approach is just not going to work for a mass-market product such as this.
Quite a bit of progress has been made already, but there's still a lot of work to do before a believable haptic experience leaves the lab and is made available to consumers – initially the system would be aimed at users wearing a VR headset, but the team is aiming to make such gloves play nice with AR glasses too. The experimental setup is currently something of a tech-packed monster, and includes a sizable tether, so will also need to shed a lot of bulk before any commercial launch.
"When we started the haptic glove project, we asked ourselves whether we could build a mass-producible, affordable consumer device that lets people experience any tangible interface anywhere," said Keller. "We couldn’t do it – not without inventing new materials, new sensors and actuators, new methods of integration and systems, new rendering algorithms, new physics engines, the list goes on. It just wasn’t possible, but we’ve forged a path that is plausible and could allow us to get there."
And the discoveries made along the way could find uses elsewhere.
"The possibilities for this research are immense," added the team's hardware engineering director, Tristan Trutna. "While we’re focused on building a haptic glove, the breakthroughs we’re making in fluidic switching and control – not to mention soft robotics – could lead to radical advances for the medical industry in lab-on-chip diagnostics, microfluidic biochemistry, and even wearable and assistive devices."
You can see team members having some fun with the current prototypes in a video posted to the project's Facebook page.
Source: Meta Reality Labs