Early generation smartphones and digital cameras had the same problem… small image sensors that could only capture so much detail. Looking back, the resolution wasn’t all that great. Adobe got an idea on how to harness artificial intelligence to boost the resolution of old, smaller-resolution images and breathe new life into them. It’s called Super Resolution, and the results are pretty impressive.
According to Adobe, “the Super Resolution feature we built around this technology is very simple to use—press a button and watch your 10-megapixel photo transform into a 40-megapixel photo. It’s a bit like how Mario eats a mushroom and suddenly balloons into Super Mario, but without the nifty sound effects.”
Using machine learning to analyze millions of sample photographs, Adobe’s Super Resolution can enlarge photos without jagged edges or lost details. The result is that Super Resolution can take an image that has, say, 6 megapixels, and make it look like it came from a 100-megapixel image sensor. And it can do it with a simple touch of a button.
Bicubic resampling vs Super ResolutionCredit: Adobe
“Used in this way, Super Resolution is like having an advanced 'digital zoom' capability,” wrote Eric Chan on the Adobe Blog.
So how does Adobe take an early generation, low-resolution image and find details that would be lost when enlarging them? Adobe researchers Michael Gharbi and Richard Zhang created an algorithm to train a computer to analyze millions of both high-resolution and low-resolution images. Then, using artificial intelligence, Super Resolution figures out how to upsize those lower-res images while preserving as much of the sharpness and detail as possible.
The process, known as “Deep convolutional neural network,” states that what happens to any given pixel greatly depends on the pixels surrounding it. By looking at those surrounding pixels, the algorithm gains context in the image, and with that, it can gradually increase its resolution. Adobe then uses Bayer and X-Trans raw data to optimize the image quality. Along the way, the algorithm focuses on challenging examples with a lot of texture and details.
All that technical know-how translates into being able to convert a shot from an early generation digital camera, and upscale the resolution with fairly decent results, about double the linear resolution of the original image quality. If you do the math for both height and weight, that equates to four times the resolution. Super Resolution also saves the image in Raw DNG, so that all those details are preserved and can then be adjusted.
Super Resolution is currently limited to 65,000 pixels on the long side with 500 megapixels total, and it’s available right now with the latest version of Adobe Raw. Moreover, Adobe says it isn't stopping there, and that anything in an image is fair game for improvement. Sharpening, noise reduction, who knows what else?
Maybe video someday. Can you imagine Adobe taking a VHS video clip and improving its resolution to HD? I certainly can.
Source: Adobe
