Photography

Samsung raises the smartphone camera stakes with high-tech new sensor

Samsung raises the smartphone camera stakes with high-tech new sensor
A new Isocell GN2 camera sensor from Samsung takes some big steps forward in autofocus and imaging technology
A new Isocell GN2 camera sensor from Samsung takes some big steps forward in autofocus and imaging technology
View 3 Images
A new Isocell GN2 camera sensor from Samsung takes some big steps forward in autofocus and imaging technology
1/3
A new Isocell GN2 camera sensor from Samsung takes some big steps forward in autofocus and imaging technology
Samsung supplies cameras and other components to several other smartphone manufacturers, as well as its own Galaxy line
2/3
Samsung supplies cameras and other components to several other smartphone manufacturers, as well as its own Galaxy line
The Dual Pixel Pro system turns every pixel into a phase detect sensor using dual photodiodes
3/3
The Dual Pixel Pro system turns every pixel into a phase detect sensor using dual photodiodes
View gallery - 3 images

Smartphones have revolutionized photography, making it vastly easier than ever before to capture and share great images. And on the other side of the coin, camera advances are probably responsible for a lot of smartphone sales these days; phones have been more than good enough in most areas for many years now, and a terrific new camera provides a compelling reason to upgrade where a processing boost or RAM upgrade might not move the needle.

And they do keep getting better, thanks to a number of creative software and hardware developments that keep pushing super-compact imaging to new heights. Most people don't know or care how they're getting better; they don't need to understand. But the technically-minded will find some interest in something like the new Samsung Isocell GN2 sensor, currently in mass production, which pushes forward the state of the art with some clever ideas.

At the most basic level, it's a smartphone camera sensor with a monster 50-megapixel (8,160 x 6,144) resolution, far beyond what most of us will ever need unless we're cropping way in to look at fine details. Likewise, it can shoot video in massive 8K resolution, despite there not being all that many 8K screens to play it back on. Switch to 1080p HD, and it's so fast it can give you super-slow motion recorded at 480 fps, stretching time by a factor of 20 if you're watching it back at a cinematic 24 fps.

If 50 megapixels isn't enough for you, it can super-resolve the image "using an intelligent re-mosaic algorithm, creating three individual layers of 50Mp frames in green, red and blue. These frames are then up-scaled and merged to produce a single ultra-high 100Mp resolution photograph." It can perform this operation at an astonishing rate: 10 times per second. May your storage drives rest in peace.

In low light situations, it pretends to be a lower-resolution sensor with larger pixels, combining every four pixels into a single "bin" to soak up as much light as possible without introducing extra noise by jacking up the ISO sensitivity.

And from there, things get really fun.

Samsung supplies cameras and other components to several other smartphone manufacturers, as well as its own Galaxy line
Samsung supplies cameras and other components to several other smartphone manufacturers, as well as its own Galaxy line

Dual Pixel Pro: turning every pixel into a phase detect AF point

You've probably heard the term "phase detect" used to describe autofocus points. In typical camera sensors, what this means is that at strategic points around the sensor, special autofocus pixels are placed in the sensor matrix, with their right or left side blocked off so they pick up light from only one side of the lens. In this way, the images from right- and left-looking phase detect pixels can be compared, and the camera's processor can use phase differences to calculate the focus distance and make the necessary adjustments to bring the desired subjects into sharp focus.

The more of these phase detect pixels you put in your sensor, the greater the number of points the camera can use to build its understanding of the image and the faster it can focus, generally speaking. So phase detect points are exploding in numbers; the latest Sony flagship full-frame Alpha 1 will have 759 of them dotted through its 50-megapixel sensor, and it'll surely be a monster of a focus system.

The drawback, particularly for compact smartphone sensors using tiny pixels, is that every pixel devoted to phase detection is a pixel that's not drawing in light or color information, says Samsung, meaning the image suffers. So it's come up with a "Dual Pixel Pro" autofocus system that uses every pixel on the sensor as a phase detect point, while upgrading the concept to search for phase differences diagonally and vertically as well as just left/right horizontally.

To achieve this, each of the 50 million-odd pixels on the Isocell GN2 sensor has not one but two photodiodes in it. In the previous Samsung Dual Pixel system, these were uniformly divided into left and right looking diodes. But Samsung found that while this improved image quality and autofocus, there were still issues, particularly with patterns such as horizontal stripes.

So the Dual Pixel Pro system uses standard left/right splits for the red and blue pixels, but splits the green ones diagonally, giving the processor vertical phase shift information to work with as well as horizontal. Samsung says this gives the Isocell GN2 super-fast and accurate autofocus capabilities even in low light or with fast-moving subjects, while taking in light at every pixel for optimum image quality. Check out a video:

Dual Pixel Pro: Fast and accurate autofocus for ISOCELL Image Sensor | Samsung

Staggered HDR and Smart ISO Pro

HDR, or High Dynamic Range photography, seeks to address one of the failings of camera sensors, which is their limited dynamic range, or the difference in illumination between the brightest and darkest points they can capture in a single image. Canny photographers have been coming up with clever ways around this limitation for nearly 170 years, believe it or not. Check out this HDR image Gustave Le Gray created back in 1856, combining two exposures to give outstanding detail in both the bright sky and the dark ocean.

A very early piece of HDR photography. Gustave Le Gray used two negatives, one for the sky and another longer exposure for the sea, and combined the two in the darkroom to create this beautiful photo
A very early piece of HDR photography. Gustave Le Gray used two negatives, one for the sky and another longer exposure for the sea, and combined the two in the darkroom to create this beautiful photo

These days, of course, it's a lot easier. Most smartphones will automatically take several images and combine them to produce an image with a higher dynamic range than the sensor can do by itself. You never need to think about it, unless you want to fiddle with a slider somewhere and turn your image into a clownish, glowing monstrosity.

Samsung is achieving HDR in two ways with the new sensor. The first is staggered HDR, which uses rolling shutters over the same pixel arrays to capture short, middle and long exposures of the same scene. The key benefit here is in energy consumption; Staggered HDR uses nearly a quarter less energy than Samsung's previous real-time HDR mode.

Then there's Smart ISO Pro, which introduces dual conversion gain to Samsung's imaging lineup. This comes down to how the sensor reads the charge in each pixel's two photodiodes to work out what it's seeing. Dual conversion gain reads each photodiode's output twice, with two gain factors applied, thus effectively grabbing higher and lower ISO versions of each single shot. This means you can extend the camera's dynamic range without affecting your signal-to-noise ratio even before you start taking multiple exposures. The benefit is that you eliminate the ghosting and artefacts that can be introduced in multiple-exposure HDR images if a subject moves between exposures.

Smart ISO Pro also helps with low-light photography, shooting multiple images in the dark at high ISO and combining them all to deliver a single image Samsung says is close to 1 million ISO. We've seen this approach before with Google's Night Sight.

This sure looks like a pretty neat smartphone camera sensor; Samsung's cameras have been right up there with the best of Apple and Google for many years now, and while a lot of it will come down to how a company's camera software uses what it gets from the sensor, a high-tech upgrade like this certainly gives the image processors more and better information to start with.

Samsung says the Isocell GN2 sensor is already in mass production, and while nobody's officially announced a phone that uses it, Samsung is a major supplier for many brands. Leaks suggest the Xiaomi Mi 11 Ultra will be one of the first phones to get the sensor.

As we said at the top, the vast majority of smartphone buyers will never know or care what's going on under the hood, but I find it fascinating to see some of the work that goes into the ever-escalating smartphone camera war. We all get to enjoy the benefits as our cameras get smarter!

Source: Samsung

View gallery - 3 images
8 comments
8 comments
paul314
With so many pixels, each individual sensor is going to be way smaller than the resolving power of a cellphone lens. Which is probably a good thing. For decades, digital photos have been limited by the resolution of the sensors, but now (as with high-quality film images in the previous era) they'll be limited instead only by the lenses and processing.
P51d007
I have a better idea. How about making the SENSOR bigger. Make EACH pixel bigger. Akin to an APS-C size sensor. Instead of stuffing more pixels, more tricks, make the SIZE of the sensor bigger.
guzmanchinky
I love the new smartphone cameras. Once the Pro model of the iPhone came out I didn't really need to carry a compact around anymore.
dlj1875
Fine but, Samsung is such an intrusive system. I made a mistake once and purchased a samsung phone. I gave it away 2 months later. I don't care how good the camera is if they can't leave me alone as a user.
Nobody
When I am taking photos, I want a decent point and shoot camera that will catch the action in low light. Only the expensive ones with large sensors and little telephoto magnification will do that at a horrific price. Why can't they keep up with the tiny cell phone cameras? It seems crazy that I can buy a better, more feature packed and cheaper cell phone camera than a dedicated point and shoot and immediately text it to my friends.
WB
Samsung phones have so much shovelware in it and such a crappy user experience where they stuff their own SH*TTY syncs and apps down your throat.. I simply can't buy another one of their phones.. hardware is good but software and updates SUCK
Tord Eriksson
This kind of sensor in a Nikon 1 camera would create a sensation (naturally needing upgraded lenses, not failure-prone like the old ones). Giving the user a small compact mirrorless system with abilities far beyond most DSLRs!
BlueOak
“...camera advances are probably responsible for a lot of smartphone sales these days; phones have been more than good enough in most areas for many years now”

Exactly. Smartphones have, just like laptops and tablets, largely become mature products.

85% of users will find the functionality and performance of phones from the past 2-3 years to be perfectly fine for 5-7 years.

*If* they don’t break before then. That is the silliness of spending $800-1,200 on a new phone every 2 years - the risk of such a portable device breaking or getting stolen.