Learning to See in the Dark
Chen Chen, Qifeng Chen, Jia Xu and Vladlen Koltun
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018
Figure. Extreme low-light imaging by a Sony a7S II camera using ISO 8000, f/5.6, 1/30 second. Dark indoor environment. The illuminance at the camera is <0.1 lux.
Abstract
Imaging in low light is challenging due to low photon count and low SNR. Short-exposure images suffer from noise, while long exposure can lead to blurry images and is often impractical. A variety of denoising, deblurring, and enhancement techniques have been proposed, but their effectiveness is limited in extreme conditions, such as video-rate imaging at night. To support the development of learning-based pipelines for low-light image processing, we introduce a dataset of raw short-exposure night-time images, with corresponding long-exposure reference images. Using the presented dataset, we develop a pipeline for processing low-light images, based on end-to-end training of a fully-convolutional network. The network operates directly on raw sensor data and replaces much of the traditional image processing pipeline, which tends to perform poorly on such data. We report promising results on the new dataset, analyze factors that affect performance, and highlight opportunities for future work.
Code and dataset
Please check the [Github page] for instructions.
Paper
High resolution results (Click for details. Drag the slider bar.)
Compare with the traditional pipeline
Compare with the traditional pipeline followed by BM3D
Compare with the HDRNET [Gharbi et al. SIGGRAPH 2017]
Compare with commercial software Adobe Camera Raw and Sony Capture One on image colors (brightness adjusted in the software for visualization)
Controlled experiments (See paper for details)
Media
See in the Dark: a machine learning technique for producing astoundingly sharp photos in very low light
AI is Taking Low-Light Photography to the Next Level
Deep Learning Can Now Flawlessly Correct Photos Taken in Almost Complete Darkness
TRAINING AI TO SEE IN THE DARK AND TAKE LOW-LIGHT PHOTOGRAPHY TO A NEW LEVEL
A.I. could help cameras see in candlelight, research suggests
Machine learning corrects photos taken in complete darkness, turns them into amazingly sharp images
This neural network turns pitch black images into bright colorful photos
Researchers use AI to brighten ultra-low light images without adding noise
AI Turns Pitch Black Images Into Vibrant Photos
Perfect Pictures In Almost Zero Light
AI can significantly improve quality of low-light images (video)
Machine Learning Works Wonders On Low-Light Images
Artificial intelligence is learning to see in the dark
An AI learned to see in the dark
The Future: New AI Learns How To See In The Dark
How An AI Learned To See In The Dark?
(In Chinese) CVPR 2018：英特尔实验室推出在黑暗中也能清晰成像的系统
(In Chinese) CVPR 2018 | 无需额外硬件，全卷积网络让机器学习学会夜视能力
(In Chinese) 如何让摄像头变成“暗夜之眼”？英特尔开发了一套基于FCN的成像系统
(In Chinese) 我要这徕卡何用，AI技术能让手机、相机夜景亮如白昼
(In Japanese) イリノイ大学とIntel、機械学習を用いて暗い環境で撮られた画像を修正するためのデータセットと手法を発表
(In Spanish) Red neuronal ayuda a transformar fotos muy oscuras en imágenes estupendas
(In Dutch) Een virtuele flitslamp in nachtfoto’s
(In Hungarian) Döbbenetesen hatékony a MI éjjellátó/zajszűrő képessége!