Noise suppression algorithm in the process of high dynamic range image fusion

The dynamic range of real natural scenes is very large, and the illuminance is over 9 orders of magnitude from dark to daylight. Compared with low dynamic range (LDR) imaging, high dynamic range (HDR) imaging technology can record the more realistic and detailed scene information and provide users with a more realistic and natural visual experience. It has a broad application prospect in areas such as smart monitoring, film and television entertainment, and biomedical care. High dynamic range imaging, encoding and transmission and its display technology are the key to its applications. The existing high dynamic range imaging technologies are mainly divided into two categories: one is to use hardware to directly shoot real scenes through high dynamic range cameras, but the expensive price of this imaging device limits its application; The second is to reconstruct HDR images by using software by fusing multiple exposure LDR image sequences obtained by ordinary cameras shooting the same scene at different exposure times. In this mode, the multi-exposure image sequence is first registered; Then the camera response function is estimated according to the exposure information of the registered image. This function describes the relationship between the luminance values received by the sensor and the pixel values. The inverse function can be used to estimate the irradiance of the scene from the captured image. Finally, the multi-exposure image is properly weighted and fused to obtain the HDR image. However, because the multi-exposure image sequence captured in a low-illumination environment often contains large noise, it is necessary to preprocess and denoise the multi-exposure image sequence before image fusion.

    Professor Jiang Gangyi’s research team at Ningbo University proposed a noise suppression algorithm in the process of high dynamic range image fusion. Firstly, adaptive luminance partitioning is performed on the input LDR image, and the original image is decomposed into three images with different luminance ranges so as to effectively retain the middle luminance detail information of all the exposure images, as well as the high luminance region detail information of the low-exposure image and the low luminance region detail information of the high-exposure image. Afterwards, the above three images are decomposed into overlapping image blocks, a weak texture region is extracted using the structure tensor for each overlapping image block, and the noise level of the region is estimated using the image's covariance matrix so as to retain the texture details of the image as much as possible while suppressing noise. Because the reliability of the noise level estimated by each image block is inconsistent, and the noise level is related to the luminance, a “selection maximum” strategy is adopted for the low-luminance region image block. The middle luminance and high luminance region image blocks adopt the "average" strategy to obtain the final noise level and guide the sparse denoising of the image. Finally, the denoised three different luminance images are reconstructed and fused to generate the HDR image.

     
Noise suppression results     
 

About Team

Professor Jiang Gangyi's research team is affiliated with the School of Information Science and Engineering of Ningbo University, the Joint Key Laboratory of Zhejiang Province Embedded System, and the Multimedia Communication Engineering Center of the Ministry of Education. The team has been dedicated to research work in the fields of 3D video processing and communication, high dynamic range imaging and processing, and has published and co-published more than 100 papers in SCI journals such as IEEE Transactions and Signal Processing; and has completed the National Natural Science Foundation of China's key projects, general projects, international cooperation projects, etc., and cooperated to complete the national high-tech projects; and has authorized more than 50 Chinese invention patents and 5 U.S. invention patents, and some of the patent achievements have been successfully transferred. The research team has participated in winning 1 second prize of National Sscience and Technology Progress, and presided over 1 first prize, 2 second prize and 3 third prize of Provincial and Ministerial level Science and Technology Progress.


Article

Chen Y Y, Jiang G Y, Shao H, et al. Noise suppression algorithm in the process of high dynamic range image fusion[J]. Opto-Electronic Engineering, 2018, 45(7): 180083.
DOI:10.12086/oee.2018.180083