December's edition of Esquire magazine is a special "augmented reality" edition where readers use custom-designed software and a webcam to interact with the pages being viewed and get access to 3D animated video content.
The term augmented reality was first used by Tom Caudell in the early 1990s. Then, as now, it means adding audio, animation or graphic layers to live video and was used back then by Boeing workers to help them sort factory parts. It's since been used in everything from adverts to gaming. With the December edition of men's magazine Esquire, augmented reality has reached the printed page.
After experimenting with things like origami (May edition) and e-ink (October 2008 edition), Esquire began considering the possibility of making a whole interactive magazine edition in the Spring of this year when The Barbarian Group were called in to help.
After toying with lots of ideas, Esquire settled on creating a 3D cover, a weather-changing fashion portfolio, a time-sensitive funny joke from a beautiful woman, plus a song, a photo slideshow, and an ad from Lexus. The challenge was to bring them all together for one edition.
Robert Downey Jr. was asked to perform a routine in front of a green screen for animation studio Psyop. Then two more sequences were shot using Jeremy Renner and Gillian Jacobs before Psyop added the animation, frame-by-frame.
To use the magazine, readers need to download a C++ application from a page on Esquire's website. The software recognizes a black and white patterned marker on the printed page picked up via webcam and translates it into a video sequence enhanced by 3D animation on the computer screen.
Tilting the page towards and away from the webcam yields different situations. For instance, the cover shows Downey Jr. sitting on the box marker. Pointing the box straight at the webcam sees the Iron Man star jump off the box. Tilting and turning the magazine results in different scenarios being produced on-screen (see below), all controlled by the reader.
Other markers feature throughout the December issue and revealing them to a webcam initiates similar augmented reality sequences, but for the purposes of generating some intrigue, I'll not divulge any more. You'll have check them out for yourself.
Esquire says that this is just the beginning, plans are already underway for page recognition without the need for special markers and even mobile phone generated interaction.