2 : (Rahoon Kang et al.: Image Filtering Method for an Effective Inverse Tone-mapping) (Special Paper) 24 2, (JBE Vol. 24, No. 2, March 2019) h

Transcription

1 2: (Rahoon Kang : Image Filtering Method for an Effective Inverse Tone-mapping) (Special Paper) 24 2, (JBE Vol. 24, No. 2, March 2019) ISSN (Online) ISSN (Print) a), a), a) Image Filtering Method for an Effective Inverse Tone-mapping Rahoon Kang a), Bumjun Park a), and Jechang Jeong a) (guided image filter: GIF) (convolutional neural network; CNN) (inverse tone-mapping). (low dynamic range; LDR) (high dynamic range; HDR). LDR HDR. (dynamic range).. (weighted guided image filter; WGIF). HDR. Abstract In this paper, we propose a filtering method that can improve the results of inverse tone-mapping using guided image filter. Inverse tone-mapping techniques have been proposed that convert LDR images to HDR. Recently, many algorithms have been studied to convert single LDR images into HDR images using CNN. Among them, there exists an algorithm for restoring pixel information using CNN which learned to restore saturated region. The algorithm does not suppress the noise in the non-saturation region and cannot restore the detail in the saturated region. The proposed algorithm suppresses the noise in the non-saturated region and restores the detail of the saturated region using a WGIF in the input image, and then applies it to the CNN to improve the quality of the final image. The proposed algorithm shows a higher quantitative image quality index than the existing algorithms when the HDR quantitative image quality index was measured. Keyword : Inverse Tone-mapping, HDR, CNN, Deep Learning, Guided Image Filter a) (Department of Electronics and Computer Engineering, Hanyang University) Corresponding Author : (Jechang Jeong) jjeong@hanyang.ac.kr Tel: ORCID: ICT SW ] This work was supported by the ICT R&D program of MSIP/IITP.[ , Software Platform for ICT Equipment] Manuscript received January 8, 2019; Revised March 18, 2019; Accepted March 18, Copyright 2019 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND ( which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.

2 (JBE Vol. 24, No. 2, March 2019).. (high dynamic range; HDR). HDR HDR. (low dynamic range ;LDR) (multi exposure fusion; MEF) [7]. HDR. (ghost artifact) [21].. LDR HDR LDR HDR. HDR LDR (inverse tone-mapping).. LDR HDR... LDR. (deep learning) [5,6].. (weighted guided image filter; WGIF). WGIF. WGIF (inverse tone-mapping).. Landis (power function) [1]. Akyüz [2].. Banterle, [3]. (tone-mapping operator; TMO)... - (edge-stopping) [4,5,6].. Eilersten (convolutional neural

3 2: (Rahoon Kang : Image Filtering Method for an Effective Inverse Tone-mapping) network; CNN) [5]. Marnerides ExpandNet LDR, HDR CNN HDR [6]. 2. (guided image filter; GIF) GIF [20]. GIF (halo artifact) [20]. Li WGIF [19]. WGIF GIF GIF. WGIF. WGIF ( 1). G , 1.. WGIF , Fig. 1. The flow chart of proposed algorithm

4 (JBE Vol. 24, No. 2, March 2019) ( 6).,,...,.. ( 1).... WGIF. LDR WGIF HDR. 1. HDR. (Weber s law) [8].. (gamma correction). HDR. LDR HDR ( 7).. WGIF. WGIF ( 6). 2. Eilersten. LDR. LDR 2. Fig 2. The structure of inverse tone-mapping using convolutional neural network

5 2: (Rahoon Kang : Image Filtering Method for an Effective Inverse Tone-mapping) HDR. (hybrid dynamic range autoencoder). Hinton, [9] ( 2)...,. Vincent [10]. Eilersten, LDR, HDR. Eilersten, LDR, HDR. skip-connection HDR. skip-connection He [11]. (fully connected layer) VGG16 [12]. VGG LDR HDR log. HDR HDR HDR log. (upsampling) (deconvolution) (bilinear interpolation). rectified linear unit (ReLU) [17], batch normalization. 8. HDR HDR.. log log.. HDR. ( 9). exp HDR c i. LDR. (sigmoid).. ( 10). maxmax %.. HDR LDR. HDR LDR. OpenCV HDR. L log log log log

6 (JBE Vol. 24, No. 2, March 2019),,, LDR. Fairchild Ward [13,14] ,,. 28 HDR 5% LDR.. 1. Table 1. The PU-PSNR of test dataset No. Akyüz Huo Eilersten Expand- Net Proposed algorithm avg Table 2. The PU-SSIM of test dataset No. Akyüz Huo Eilersten Expand- Net Proposed algorithm avg Eilersten [5], ExpandNet [6], Huo [18], Akyüz [2], HDR (percep- tually uniform encoding peak signal-to-noise ratio; PU- PSNR), (perceptually uniform encoding structural similarity; PU-SSIM) (high dynamic range visible difference predictor; HDR-VDP) 2.2 [15,16].

7 2: (Rahoon Kang : Image Filtering Method for an Effective Inverse Tone-mapping) (a) (b) (c) (d) (e) 3. HDR-VDP-2.2 (21 ), (a) Akyüz, (b) Huo, (c) Eilersten, (d) ExpandNet, (e) Fig 3. The comparison results of inverse tone-mapping methods(at no.21 image), (a) Akyüz, (b) Huo, (c) Eilersten, (d) ExpandNet, (e) proposed algorithm PSNR, SSIM VDP HDR. HDR RGB (perceptually uniform) [15]. Akyüz Huo. PU-PSNR PU-SSIM. ExpandNet Eiliersten. PU-PSNR ExpandNet PU-PSNR ExpandNet PU-PSNR 0.6. PU-SSIM ExpandNet 0.02 WGIF. 3 HDR-VDP-2.2. HDR-VDP. 21 ExpandNet. Eilersten. WGIF. 4 LDR,. (g-l).

8 (JBE Vol. 24, No. 2, March 2019) (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) (l) 4. (2 ), (a,g), (b,h) Eilersten, (c,i) ExpandNet, (d,j) Akyüz, (e,k) Huo, (f,l) Fig 4. The comparison of tone-mapping result subjective image quality of each method(at no.2 image), (a,g) ground truth, (b,h) Akyüz, (c,i) Huo, (d,j) Eilersten, (e,k) ExpandNet,, (f,l) proposed algorithm..... WGIF,, HDR. HDR. HDR LDR.,,,.

9 2: (Rahoon Kang : Image Filtering Method for an Effective Inverse Tone-mapping) (References) [1] H. Landis, Production-ready global illumination, SIGGRAPH Course Notes, Vol.16, pp , [2] A. O. Akyüz, R. Fleming, B. E. Riecke, E. Reinhard, and H. H. Bülthoff, Do HDR displays support LDR Content?: A psychophysical evaluation, ACM Transaction on Graphics, Vol.26, No.38, July [3] F. Banterle, A. Artusi, K. Debattista, and A. Chalmers, Advanced High Dynamic Range Imaging:theory and practice, A K Peters/CRC Press, New York, February [4] A. G. Rempel, M. Trentacoste, H. Seetzen, H. D. Young, W. Heidrich, L. Whiteheadm, and G. Ward, LDR2HDR: On-thefly reverse tone mapping of legacy video and photographs, ACM Transaction on Graphics, Vol.26, No.39, [5] G. Eilertsen, J. Kronander, G. Denes, R. Mantiuk, and J. Unger, "HDR image reconstruction from a single exposure using deep CNNs," ACM Transactions on Graphics, Vol.36, No.6, pp.1-15, [6] D. Marnerides, T. Bashford-Rogers, J. Hatchett, and K. Debattista, "ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content," Computer Graphics Forum, Vol.37, No.2, pp.37-49, 2018 [7] A. A. Goshtasby, Fusion of Multi-exposure Images, Image and Vision Computing, Vol.23, pp , June [8] S. Hecht, The visual discrimination of intensity and the Weber-Fechner law, The Journal of General Physiology, Vol.7, pp , [9] G. E. Hinton and R. Salakhutdinov, "Reducing the Dimensionality of Data with Neural Networks," Science, Vol.313, No.5786, pp , [10] P. Vincent, H. Larochelle, Y. Bengio and P. Manzagol, "Extracting and composing robust features with denoising autoencoders, Proceeding of 25th International Conference on Machine Learning(ICML), pp , [11] K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition", Arxiv.org, 2014, [12] K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, Proceeding of IEEE conference on computer vision and pattern recognition(cvpr), pp , [13] M. D. Fairchild, The HDR Photographic survey, Proceeding of Color and Imaging Conference, Vol.2007, No.1, pp , [14] G. Ward, Hight Dynamic Range Image Encodings, [15] T. O. Aydın, R. Mantiuk, and H. P. Seidel, Extending quality metrics to full luminance range images, Human Vision and Electronic Imaging XIII, Vol.6806, pp.68060b, March [16] M. Narwaria, R. Mantiuk, M. P. Da Silva, and P. Le Callet, HDR-VDP-2.2: a calibrated method for objective quality prediction of high-dynamic range and standard images, Journal of Electronic Imaging, Vol.24, No , [17] V. Nair and G. E. Hinton, Rectified linear units improve restricted boltzmann machines, Proceeding of the 27th International Conference on Machine Learning(ICML), pp , [18] Y. Huo, F. Yang, L. Domg, and V. Brost, Physiological inverse tone mapping based on retina response, The Visual Computer, Vol.30, pp , [19] Z. Liand J. Zheng, Z. Zhu, W. Yao, and S. Wu, Weighted Guided Image Filtering, IEEE Transaction on Image Processing, Vol.24, No.1, pp , [20] K. He, J. Sun, and X. Tang, Guided Image Filtering, Proceeding of European Conference on Computer Vision(ECCV), Berlin, Heidelberg, pp.1-14, [21] J. An, S. Lee, J. Kuk, and N. Cho, A multi-exposure image fusion algorithm without ghost effect, Proceeding of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), : ~ : - ORCID : - :,

10 (JBE Vol. 24, No. 2, March 2019) : ~ : - ORCID : - :, : : KAIST : ~ 1986 : KBS ( ) ~ 1991 : ( ) ~ 1995 : HDTV ~ : ( ) - ORCID : - :,