Home Entertainment

Microsoft's IllumiRoom takes gaming visuals outside the box and onto the living room

Microsoft's IllumiRoom takes g...
Microsoft has revealed more details about how its IllumiRoom project will expand video games outside of the TV and even alter the appearance of your living room
Microsoft has revealed more details about how its IllumiRoom project will expand video games outside of the TV and even alter the appearance of your living room
View 16 Images
By overlaying objects in the room with corresponding bright colors and dark outlines for example, it can give everything a cartoonish look, or it could cancel the colors out for a black-and-white appearance
1/16
By overlaying objects in the room with corresponding bright colors and dark outlines for example, it can give everything a cartoonish look, or it could cancel the colors out for a black-and-white appearance
IllumiRoom also allows virtual objects to interact with the room to have balls bounce out of the screen or snow gather on the ground
2/16
IllumiRoom also allows virtual objects to interact with the room to have balls bounce out of the screen or snow gather on the ground
The developers hope to create a version of IllumiRoom that sits on a coffee table between the player and the television
3/16
The developers hope to create a version of IllumiRoom that sits on a coffee table between the player and the television
This expanded view allows for increased visuals with movies as well, but it only works with footage shot using a dual camera rig that captures narrow and wide FOVs simultaneously
4/16
This expanded view allows for increased visuals with movies as well, but it only works with footage shot using a dual camera rig that captures narrow and wide FOVs simultaneously
IllumiRoom can also set the visuals to appear differently from those on the TV, so only certain elements (edges of buildings, bullets, explosions, power-ups, etc.) are shown to give a distinctive style
5/16
IllumiRoom can also set the visuals to appear differently from those on the TV, so only certain elements (edges of buildings, bullets, explosions, power-ups, etc.) are shown to give a distinctive style
6/16
The system is even able to project only onto the wall behind the television, leaving out any surrounding furniture or objects and giving the appearance that the game extends beyond the wall
7/16
The system is even able to project only onto the wall behind the television, leaving out any surrounding furniture or objects and giving the appearance that the game extends beyond the wall
It's important to note that this is all just a proof-of-concept model (i.e. not necessarily a future consumer product), and carries a number of limitations with it, like the fact that the projection effect appears muted when the lights are on
8/16
It's important to note that this is all just a proof-of-concept model (i.e. not necessarily a future consumer product), and carries a number of limitations with it, like the fact that the projection effect appears muted when the lights are on
The device is able to calibrate itself to any room, with the Kinect detecting the colors and layout in front of it, while the projector uses that data to align the image properly around the TV
9/16
The device is able to calibrate itself to any room, with the Kinect detecting the colors and layout in front of it, while the projector uses that data to align the image properly around the TV
IllumiRoom can also alter the appearance of the room itself to match the aesthetics of the game by mapping visuals to its geometry
10/16
IllumiRoom can also alter the appearance of the room itself to match the aesthetics of the game by mapping visuals to its geometry
The team's initial prototype consists of a widescreen InFocus IN126ST projector and a Kinect sensor mounted above and behind the user
11/16
The team's initial prototype consists of a widescreen InFocus IN126ST projector and a Kinect sensor mounted above and behind the user
The system can even project the exact same features of the room and then distort them to match the gameplay - making the room ripple or shake with each gunshot, for instance
12/16
The system can even project the exact same features of the room and then distort them to match the gameplay - making the room ripple or shake with each gunshot, for instance
IllumiRoom also allows virtual objects to interact with the room to have balls bounce out of the screen or snow gather on the ground
13/16
IllumiRoom also allows virtual objects to interact with the room to have balls bounce out of the screen or snow gather on the ground
IllumiRoom can also set the visuals to appear differently from those on the TV, so only certain elements (edges of buildings, bullets, explosions, power-ups, etc.) are shown to give a distinctive style
14/16
IllumiRoom can also set the visuals to appear differently from those on the TV, so only certain elements (edges of buildings, bullets, explosions, power-ups, etc.) are shown to give a distinctive style
Microsoft has revealed more details about how its IllumiRoom project will expand video games outside of the TV and even alter the appearance of your living room
15/16
Microsoft has revealed more details about how its IllumiRoom project will expand video games outside of the TV and even alter the appearance of your living room
Microsoft has revealed more details about how its IllumiRoom project will expand video games outside of the TV and even alter the appearance of your living room
16/16
Microsoft has revealed more details about how its IllumiRoom project will expand video games outside of the TV and even alter the appearance of your living room

At CES in January, Microsoft Research teased its IllumiRoom concept, which involves projecting an image around a TV screen to enhance video games with additional visuals. Unfortunately, the company didn't offer much info beyond a short video that briefly showed it in action. But the team behind the project recently showed up at the CHI 2013 conference in Paris with some more in-depth details about how Illumiroom will not only expand the game screen, but completely alter the appearance of your living room.

The team's initial prototype consists of a widescreen InFocus IN126ST projector and a Kinect for Windows sensor mounted above and behind the user, though they hope to create a future version that sits on a coffee table between the player and the television. The device is able to calibrate itself to any room, with the Kinect detecting the colors and layout in front of it, while the projector uses that data to properly align the image displayed around the TV.

We've seen how this can extend the game environment beyond the boundaries of a TV screen, but that's just the simplest use for it. The system can also set the visuals to appear differently from those on the TV, so only certain elements (edges of buildings, bullets, explosions, power-ups, etc.) are shown for a more a distinctive style. It's even able to project only onto the wall behind the television, leaving out any surrounding furniture or objects and giving the appearance that the game extends beyond the wall.

IllumiRoom can also alter the appearance of the room and objects contained within to match the aesthetics of the game by mapping visuals to their geometry. By overlaying objects in the room with corresponding bright colors and dark outlines, for example, it can give everything a cartoonish look, or it could cancel the colors out almost entirely to give the room a black-and-white appearance.

By overlaying objects in the room with corresponding bright colors and dark outlines for example, it can give everything a cartoonish look, or it could cancel the colors out for a black-and-white appearance
By overlaying objects in the room with corresponding bright colors and dark outlines for example, it can give everything a cartoonish look, or it could cancel the colors out for a black-and-white appearance

The system can even project the exact same features of the room and then distort them to match the gameplay – making the room ripple or shake with each gunshot, for instance. It also allows virtual objects to interact with the room so that balls can bounce out of the screen onto the floor or falling snow gathers on the ground. Developers can essentially mix and match various effects to create more immersive games.

This expanded view allows for increased visuals with movies as well, either turning your whole wall into a projector screen or adding effects outside of the TV. The catch is it only works with footage shot using a custom dual camera rig that captures narrow and wide fields of view simultaneously.

The system is even able to project only onto the wall behind the television, leaving out any surrounding furniture or objects and giving the appearance that the game extends beyond the wall
The system is even able to project only onto the wall behind the television, leaving out any surrounding furniture or objects and giving the appearance that the game extends beyond the wall

It's important to note that this is all just a proof-of-concept model (i.e. not necessarily a future consumer product), and carries a number of limitations with it, like the fact that the projection effect appears muted when the lights are on. So while these new details arrive tantalizingly close to Microsoft's reveal of the next Xbox on May 21, it will probably be some time before the technology is available in a consumer device.

The video below demonstrates more examples of what's possible with IllumiRoom using some amazing unaltered footage of its visual effects.

Source: Microsoft Research, Brett R. Jones

IllumiRoom by Microsoft Research: Peripheral Projected Illusions for Interactive Experiences

6 comments
Gargamoth
I'd like to see an illiumi room for play station 4 in a bundle pack, to save money of course but it would make gamming awesome; especially in a prepared room specifically for this.
Cyberxbx
combine the illiumi room with this: http://www.gizmag.com/insect-eye-wide-angle-cameras/27349/ And we can mount the whole thing to the ceiling like a disco ball. I would love to see TOTAL immersion, 3d projectors so when a grenade is tossed on the ground near you, you will see it at your feet.
Nairda
Woud be even better if the intelligent projector could account for perspective from where you are sitting on the couch rather then projection point, so that corners and edges of walls could be made to dissapear.
Phyzzi
No thanks. I don't want to ceiling mount this thing and I don't want to have to keep a room dedicated to gaming and nothing else. Also, projectors still cost at least $200 (for the worst on the market) and often more than that for just the bulbs - how are they planning to keep this at the cost of a system add-on that will last through a normal living room and be usable by more than one person or in anything resembling daylight? Worst, this can't possibly work with the Kinnect, frankly MS's biggest innovation in the last decade. It's cool to keep innovating, but I can't see this being used in anything but commercial video-game parlors and the odd rich enthusiast's game room (those that don't already have 4k full wall projection units) - as such it makes much more sense as a computer peripheral than as a system add-on, if anything at all. Guess it's not for me...
ChuckTM
Wow really impressive tech by Microsoft I will say when the do bring out something they do always improve on it so this may be a super big thing later on. Looks like it may only be a Xbox one thing but if Sony pay up which would be unlikely they will be able to get for the ps4 consoles aswell. I'm looking at getting this product for the Xbox one if it is released. It will probably be a bundle deal or if not possible a separate expense, either way I don't mind
Steve Jones
It's going to be awesome - you'll feel fully immersed in the game world. The quality of the image doesn't have to be great in order to achieve the effect (just as well, because it won't be, unless you have an entirely-plain room) - it's only for your peripheral vision. The sense of movement from being surrounded by the game world which turns and scrolls with your character (or car, or whatever) will add to the involvement enormously, and the lighting effect it will bring (the room goes light when your character goes outside, dark when your car enters a tunnel etc) will be significant oo. Phyzzi, you have a point about the cost, but you don't have to dedicate a room to it - you can turn it off while you watch TV! You'll need the room to be dedicated to gaming, WHILE you're gaming, but how different is that to the situation now, when many of us can't play CoD if the family want to watch TV. And what makes you think it can't work with Kinect? Given how much Kinect means to MS, I suspect that it can.