Giving new meaning to "tilt to steer," Israeli tech startup Umoove has developed face- and eye-tracking software for mobile devices that translates gentle head tilts and nods into in-game movements. The company has released the Umoove Experience, a free app for iOS that demonstrates the technology, but hopes third party developers will integrate the technology into their own titles on both iOS and Android devices.
"Umoove is not coming to replace touch, it is coming to add another layer and it opens an opportunity for new types of interfaces," Yitzi Kempinski, Umoove's CEO, tells Gizmag. "A classic example is a first person shooter where you would shoot using touch and you could walk using a touch joystick, but you would aim and look around just by where you face."
We tried out the Flying Experience game included with Umoove Experience and found gameplay to be quite intuitive. At the start of the game, you're instructed to hold the device steady and train your Umoove skills by softly tilting your head to move one circle into another. Once that's done, you tap the screen to start Flying Experience, accompanied by strains of Arabian music. You're instantly given control of a flying character in a desert and tasked with collecting bottles of magic purple portion that help you stay airborne.
Tapping on the screen increases the character's flying speed, but if you don't manage to fly and grab a bottle in time, your character crashes and the game ends. It took a little getting used to, but the fine level of control you can achieve by simply turning your head was pretty impressive. There was no lag and the app was extremely sensitive and responsive to head movements on the iPad 2, both in bright and low light. Even switching off the light and plunging the room into semi-darkness while playing didn't affect the game play at all.
Flying Experience itself is very basic and there aren't any higher levels to go to, but that's by design. The app was released mainly to demonstrate what the company has achieved with its unique algorithms (15 patents on the technology have been filed). Kempinski says that the algorithms were built from scratch to cope with the changing environment of a mobile phone (lighting, shakiness, partial visibility of the face), while being capable enough to process videos obtained from even the most basic front-facing cameras.
"The tracking is both accurate and very sensitive, it senses movement as small as represented by one pixel of the image the camera captures," Kempinski says. "And we had to do it all at very low CPU, to not take over the device resources and leave room for the actual apps/games to run."
Getting the platform to consume only five percent of CPU resources in real time while processing raw frames from the camera took the company a long time to nail down. The idea behind the app is to transform natural head movements, and eventually eye movements, into meaningful actions, opening up intention-based interaction and allowing devices to respond to users without any additional intervention (like having a smartphone wake up when you look at it, for instance).
"Umoove is tracking the face and eyes," Kempinski explains, "These body parts have been moving and involved in the user's experiences even before technology began watching and tracking them – have you ever seen someone play a game such as a flight simulator with a frozen or still face and body? Users get involved in the experience and move their bodies even though it has no real effect on the game. So the key from a [user experience] perspective is to respond to movements users naturally do in the real world and not make the user start doing new types of movements."
Umoove has created an SDK for iOS that allows developers to integrate two-dimensional head-tracking capabilities into their own software (the company is also working on 3D head-tracking). An Android version is on the way, along with an eye-tracking SDK that is currently in the beta stage. This is designed to differentiate between different types of eye movements and allow smartphones to understand the user's intention by following what they are looking at. The company says this will enable developers to create a new dynamic breed of content that compels exploration, such as having a child’s eBook character smile and say something if the child looks at it closely.
"Think of an object or product on the 2D screen that acts as if it is 3D because it changes based on the angle you are looking at the screen," Kempinski tells us. "You can move around the object on screen as if you were looking at an object that is really in front of you."
Umoove says games aren't the only application for the technology. When integrated into other apps, the technology will allow smartphone users to do things like nod their head to scroll up and down a page, turn their head to flip eBook pages, and have a video pause when they glance away.
Flying Experience is available for free as part of the Umoove Experience app on the iTunes store.
The video below gives an idea of what the Umoove software is capable of.
Source: Umoove
Maybe the group is a small enough shop that it only had the resources to do one at a time. Perhaps the folks are more versed in iOS development so started their efforts where they are more comfortable/accomplished. The article doesn't provide that sort of background information. Point is, Umoove could have very good reasons for doing what they did and are doing. No need to get your knickers in a twist over it. Geez. Take a breath and find something about which grousing may actually be worth the effort. Bitching about a non-issue... that's idiocy.