Different Dimension Inc., a Japanese start-up company, will begin marketing an Augmented Reality robot doll that can take on the appearance of any virtual character. Essentially, a computer-generated image is projected in real-time onto a robot (covered in green screen material) that is viewable through a head-mounted display (HMD). It's a spin-off of a bizarre research project led by Michihiko Shoji at the Yokohama National University and the Japan Science and Technology Agency, originally developed in 2006.
The so-called "virtual humanoid" is an example of "mixed reality" – a technology which takes augmented reality to the next level by giving virtual objects a physical counterpart in the real world. Video images from the user's perspective – taken from the camera mounted to the HMD – are combined with a computer-generated character that moves in synch with the robot. The user can then touch and physically interact with the virtual character through the robot.
The original prototype, called U-Tsu-Shi-O-Mi, presented several challenges. It was a costly life-sized robot, and its green padding could be seen around the edges of thinner virtual characters. Additionally, the system only played back recorded voices, so it could not actually converse with the user. The new version has been shrunk to 60 percent of its original size (a height of 13.7 inches / 35 cm), and is only the top half of the body. It features a slimmer profile, which matches up better with the virtual characters, and the researchers have paid special attention to the texture of the green screen material, so that it feels more natural to the touch.
The new prototype can tilt and turn its neck and move its arms at the shoulders, elbows, and wrists. The servo motors which power each joint contain position sensors which read the joint angles – data which is fed to the software program controlling the virtual character – thereby synchronizing their movements. It's similar to the system adopted by the Qumarion, a puppet which can be posed to animate virtual characters.
The actual software used is called MMDAgent, which creates an interactive 3D anime-style character that can carry on conversations using speech recognition and a synthesized voice.
The research team is exploring possible applications for the technology, including education, training, and video games – ignoring the naughtier uses that might immediately spring to mind. Interested parties will be able to pre-order the virtual humanoid in March. It will be priced between ¥450,000 - ¥500,000 (US$4,800 - $5,300). The company, which will soon launch its website, expects to sell 120 units the first year, followed by 180 and then 270 units in the following years.
You can see the original U-Tsu-Shi-O-Mi and the MMDAgent software in action in the following videos.
Source: Wingover (Japanese) via MyNavi (Japanese)
