Photography

Computational Zoom allows changes to photo composition post capture

Computational Zoom allows changes to photo composition post capture
The Computational Zoom system makes it possible to automatically combine wide-angle and telephoto perspectives into a single multi-perspective image
The Computational Zoom system  makes it possible to automatically combine wide-angle and telephoto perspectives into a single multi-perspective image
View 4 Images
The desired background (or wide angle) for the finished image
1/4
The desired background (or wide angle) for the finished image 
The two images are combined to create this composite
2/4
The two images are combined to create this composite 
The desired foreground for the image
3/4
The desired foreground for the image 
The Computational Zoom system makes it possible to automatically combine wide-angle and telephoto perspectives into a single multi-perspective image
4/4
The Computational Zoom system  makes it possible to automatically combine wide-angle and telephoto perspectives into a single multi-perspective image
View gallery - 4 images

Image stacking has opened the door for clever post-focusing capabilities in modern cameras like the Lumix GH5, but a team at UC Santa Barbara (UCSB) has now used a similar technique that allows the composition of a photo to be adjusted after a shot has been taken. Working with researchers from Nvidia, the UCSB team says it has developed a way to create images that, in some cases, simply couldn't be captured with a normal camera.

According to the researchers, their "Computational Zoom" system opens the door for novel image compositions by controlling the sense of depth in the scene, the sizes of different objects at different depths relative to each other, and the camera's perspective of them. This essentially allows photographers to manipulate the view of the foreground and background elements of an image without making the subjects look out-of-place.

Achieving this is a three-step process. Photographers take successive images as they gradually move closer to their subject without adjusting the focal length of the lens. The system takes the stack and uses an algorithm to estimate camera position (or depth) and where it was pointing, before a "novel multi-view 3D reconstruction method" is used to create a depth map for each image.

All this information is combined to create multi-perspective images, each of which has a different composition. Photographers are able to explore the different perspectives using a simple user interface.

Computational Zoom Result

The end result is the ability to automatically combine wide-angle and telephoto (zoomed-in) perspectives into the one image. Imagine you're taking a photo of your partner standing in front of a beautiful old building. Zooming in will give you a great view of the person, but will leave out most of the church. Shooting at a wide angle might allow you to squeeze both subjects into the image, but fisheye or wide lenses tend to suffer significant distortion at short focal lengths.

By using computation capture, you're able to combine the wide-angle shot of the building with a closer shot of the person. It sounds a bit sci-fi, but there's no arguing with the results in the video above. Although it doesn't look as polished or clear as a single shot, the final image squeezes the foreground and background into the one frame in a way that wouldn't have been possible with a single image.

It's only a research project at the moment, but the researchers have bigger plans for Computational Zoom. They're hoping to turn it into a plug-in for existing image processing software at some point. Should it get off the ground, the system has the potential to change the way we think about photography.

Rather than forcing photographers to settle on one particular composition – and rather than having that composition dictated by the limitations of the lens they're using – the system could open the door for unique perspectives and angles through post-capture composition.

The team presented their paper describing the system, which is demonstrated in the video below, at the ACM SIGGRAPH 2017 conference.

Source: UC Santa Barbara

Computational Zoom: A Framework for Post-Capture Image Composition (SIGGRAPH 2017)

View gallery - 4 images
1 comment
1 comment
Ralf Biernacki
Having this as a plug-in for photoshop clones would be nice, but keep in mind that it won't do "magic" on a regular photo. This system needs a series of snaps at various focal lengths, shot from the same position, as a starting point. So you'd need a new image format, a camera capable of capturing (sets of) photos in that format, and a "computational zoom" postprocessor algorithm, either built into the camera or as a plug-in to image editing software. And it would still take significantly longer to capture the metaimage, as it actually consists of a series of shots made while the camera zooms and refocuses. Even so, it's such an empowering idea that one could swallow the limitations, as long as the CZ-capable camera won't cost an arm and a leg.