Can Artificial Intelligence Create Low-Light Images Without Noise?

Noise is found in all imaging systems, but it becomes particularly challenging in low light. High ISO can be used to increase brightness, but it also amplifies noise. Post-processing can be applied, but it does not resolve the low signal-to-noise ratio due to low photon counts. Is artificial intelligence the answer to clean low-light images?

Researchers at the University of Illinois Urbana–Champaign and Intel are working on the low-light issue and have developed a deep neural network that brightens ultra-low light images without adding noise and other artifacts.

A new white paper says the neural network was trained using a “See-in-the-Dark” (SID) dataset that uses 5,094 raw short-exposure low-light and long-exposure image pairs. The result is a system that automatically brightens images at a much higher quality than traditional processing options.

The dataset contains both indoor and outdoor images. The outdoor images were generally captured at night, under moonlight or street lighting. The illuminance at the camera in the outdoor scenes is generally between 0.2 and 5 lux.

The indoor images are even darker. They were captured in closed rooms with regular lights turned off and with faint in-direct illumination set up for this purpose. The illuminance at the camera in the indoor scenes is generally between 0.03 and 0.3 lux.

The limitations of alternative denoising, deblurring and enhancement techniques result in high levels of noise that isn't present when using the machine learning technique. The researchers used images captured with a Fujifilm X-T2 and Sony a7S II, and also demonstrated the system on images taken with an iPhone X and Google Pixel 2 smartphone.

The researchers noted that fast, low-light imaging is a formidable challenge due to low photon counts and a low signal-to-noise ratio. Imaging in the dark, at video rates, in sub-lux conditions, was considered impractical with traditional signal processing techniques.

However, with the See-in-the-Dark (SID) dataset the team has developed a simple pipeline that improves upon traditional processing of low-light images. The presented pipeline is based on end-to-end training of a fully-convolutional network. Experiments demonstrate promising results, with successful noise suppression and correct color transformation on SID data.

“We expect future work to yield further improvements in image quality, for example by systematically optimizing the network architecture and training procedure,” the team wrote.

The researchers are using machine learning — artificial intelligence (AI) — to automatically enhance images in low light. Other companies are exploring AI as well to repair damage or to enhance effects. Since, all modern camera sensors are now used for both video and stills, AI could be a major future breakthrough for low-light videographers.

For the full white paper, go here.

The structure of different image processing pipelines. In illustration A, from top to bottom: a traditional image processing pipeline, the L3 pipeline and a burst imaging pipeline. In illustration B, the researcher’s pipeline.

You might also like...

Wi-Fi Gets Wider With Wi-Fi 7

The last 56k dialup modem I bought in 1998 cost more than double the price of a 28k modem, and the double bandwidth was worth the extra money. New Wi-Fi 7 devices are similarly premium-priced because early adaptation of leading-edge new technology…

NAB Show 2024 BEIT Sessions Part 2: New Broadcast Technologies

The most tightly focused and fresh technical information for TV engineers at the NAB Show will be analyzed, discussed, and explained during the four days of BEIT sessions. It’s the best opportunity on Earth to learn from and question i…

Standards: Part 6 - About The ISO 14496 – MPEG-4 Standard

This article describes the various parts of the MPEG-4 standard and discusses how it is much more than a video codec. MPEG-4 describes a sophisticated interactive multimedia platform for deployment on digital TV and the Internet.

Chris Brown Discusses The Themes Of The 2024 NAB Show

The Broadcast Bridge sat down with Chris Brown, executive vice president and managing director, NAB Global Connections and Events to discuss this year’s gathering April 13-17 (show floor open April 14-17) and how the industry looks to the show e…

Essential Guide: Next-Gen 5G Contribution

This Essential Guide explores the technology of 5G and its ongoing roll out. It discusses the technical reasons why 5G has become the new standard in roaming contribution, and explores the potential disruptive impact 5G and MEC could have on…