2: (Rahoon Kang : Image Filtering Method for an Effective Inverse Tone-mapping) (Special Paper) 24 2, 2019 3 (JBE Vol. 24, No. 2, March 2019) https://doi.org/10.5909/jbe.2019.24.2.217 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) a), a), a) Image Filtering Method for an Effective Inverse Tone-mapping Rahoon Kang a), Bumjun Park a), and Jechang Jeong a) (guided image filter: GIF) (convolutional neural network; CNN) (inverse tone-mapping). (low dynamic range; LDR) (high dynamic range; HDR). LDR HDR. (dynamic range).. (weighted guided image filter; WGIF). HDR. Abstract In this paper, we propose a filtering method that can improve the results of inverse tone-mapping using guided image filter. Inverse tone-mapping techniques have been proposed that convert LDR images to HDR. Recently, many algorithms have been studied to convert single LDR images into HDR images using CNN. Among them, there exists an algorithm for restoring pixel information using CNN which learned to restore saturated region. The algorithm does not suppress the noise in the non-saturation region and cannot restore the detail in the saturated region. The proposed algorithm suppresses the noise in the non-saturated region and restores the detail of the saturated region using a WGIF in the input image, and then applies it to the CNN to improve the quality of the final image. The proposed algorithm shows a higher quantitative image quality index than the existing algorithms when the HDR quantitative image quality index was measured. Keyword : Inverse Tone-mapping, HDR, CNN, Deep Learning, Guided Image Filter a) (Department of Electronics and Computer Engineering, Hanyang University) Corresponding Author : (Jechang Jeong) E-mail: jjeong@hanyang.ac.kr Tel: +82-2-2220-4372 ORCID: https://orcid.org/0000-0002-3759-3116.[2014-0-00670, ICT SW ] This work was supported by the ICT R&D program of MSIP/IITP.[ 2014-0-00670, Software Platform for ICT Equipment] 2018. Manuscript received January 8, 2019; Revised March 18, 2019; Accepted March 18, 2019. Copyright 2019 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND (http://creativecommons.org/licenses/by-nc-nd/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.
(JBE Vol. 24, No. 2, March 2019).. (high dynamic range; HDR). HDR HDR. (low dynamic range ;LDR) (multi exposure fusion; MEF) [7]. HDR. (ghost artifact) [21].. LDR HDR LDR HDR. HDR LDR (inverse tone-mapping).. LDR HDR... LDR. (deep learning) [5,6].. (weighted guided image filter; WGIF). WGIF. WGIF.. 2 3. 4 5.. 1. (inverse tone-mapping).. Landis (power function) [1]. Akyüz [2].. Banterle, [3]. (tone-mapping operator; TMO)... - (edge-stopping) [4,5,6].. Eilersten (convolutional neural
2: (Rahoon Kang : Image Filtering Method for an Effective Inverse Tone-mapping) network; CNN) [5]. Marnerides ExpandNet LDR, HDR CNN HDR [6]. 2. (guided image filter; GIF) GIF [20]. GIF (halo artifact) [20]. Li WGIF [19]. WGIF GIF GIF. WGIF. WGIF ( 1). G 3 3.. 1 1, 1.. WGIF....... 3, 4...... 1. Fig. 1. The flow chart of proposed algorithm
(JBE Vol. 24, No. 2, March 2019) ( 6).,,...,.. ( 1).... WGIF. LDR WGIF HDR. 1. HDR. (Weber s law) [8].. (gamma correction). HDR. LDR HDR ( 7).. WGIF. WGIF ( 6). 2. Eilersten. LDR. LDR 2. Fig 2. The structure of inverse tone-mapping using convolutional neural network
2: (Rahoon Kang : Image Filtering Method for an Effective Inverse Tone-mapping) HDR. (hybrid dynamic range autoencoder). Hinton, [9] ( 2)...,. Vincent [10]. Eilersten, LDR, HDR. Eilersten, LDR, HDR. skip-connection HDR. skip-connection He [11]. (fully connected layer) VGG16 [12]. VGG16 3 3. LDR HDR log. HDR HDR HDR log. (upsampling) (deconvolution) (bilinear interpolation). rectified linear unit (ReLU) [17], batch normalization. 8. HDR HDR.. log log.. HDR. ( 9). exp HDR c i. LDR. (sigmoid).. ( 10). maxmax 0 1. 0.9 10%.. HDR LDR. HDR LDR. OpenCV HDR. L log log log log
(JBE Vol. 24, No. 2, March 2019),,, LDR. Fairchild Ward [13,14]. 320 320,,. 28 HDR 5% LDR.. 1. Table 1. The PU-PSNR of test dataset No. Akyüz Huo Eilersten Expand- Net Proposed algorithm 1 7.695 7.783 32.341 28.381 32.352 2 10.328 12.639 33.027 30.205 33.227 3 8.771 9.782 33.127 30.987 33.192 4 7.238 6.730 31.265 27.888 31.113 5 8.097 8.317 33.855 23.942 33.823 6 6.973 6.183 28.535 24.745 28.569 7 8.542 9.960 32.773 27.873 32.727 8 11.395 14.836 40.795 37.788 40.662 9 9.698 11.005 35.014 38.394 35.078 10 8.882 9.760 35.330 39.514 35.333 11 9.520 10.617 37.899 38.085 37.900 12 8.444 9.551 36.211 28.776 36.191 13 7.381 7.110 30.564 36.996 30.569 14 8.912 10.558 30.186 30.254 30.170 15 7.398 7.088 25.828 26.366 25.868 16 9.044 10.112 36.757 37.056 36.854 17 8.658 9.648 31.982 33.272 31.984 18 8.112 8.518 35.991 35.970 35.997 19 8.398 9.200 36.472 35.643 36.424 20 7.466 7.164 27.995 38.269 28.015 21 8.250 8.535 36.686 37.626 36.699 22 8.402 9.051 34.734 38.707 34.761 23 9.682 11.032 41.989 39.646 41.840 24 8.294 8.897 32.166 33.744 32.235 25 7.402 7.126 32.550 26.980 32.568 26 7.291 7.059 26.801 25.034 26.842 27 7.480 7.262 32.596 39.307 32.606 28 7.408 6.708 35.390 26.389 35.350 avg. 8.399 9.008 33.531 32.780 33.534 2. Table 2. The PU-SSIM of test dataset No. Akyüz Huo Eilersten Expand- Net Proposed algorithm 1-0.1714-0.1541 0.7731 0.8265 0.7738 2-0.0910-0.1107 0.9602 0.8974 0.9593 3-0.1299-0.1268 0.8804 0.7894 0.8804 4-0.0917-0.1151 0.7471 0.6367 0.7480 5-0.1297-0.1232 0.7865 0.8104 0.7890 6-0.1288-0.1467 0.5357 0.8423 0.5403 7-0.1377-0.1428 0.9094 0.8152 0.9095 8-0.2057 0.0119 0.9515 0.8794 0.9570 9-0.2969-0.4613 0.9468 0.9686 0.9500 10-0.1831-0.2578 0.9187 0.8124 0.9186 11-0.2176-0.2662 0.9439 0.6693 0.9447 12-0.1680-0.1910 0.9008 0.7327 0.9021 13-0.1312-0.1401 0.6480 0.9402 0.6485 14-0.1654-0.2119 0.8859 0.9143 0.8861 15-0.1623-0.1644 0.6637 0.8037 0.6638 16-0.2125-0.2972 0.8915 0.8473 0.8977 17-0.2059-0.2659 0.8498 0.7754 0.8479 18-0.1402-0.1611 0.9110 0.7471 0.9133 19-0.1490-0.1776 0.8976 0.7811 0.9015 20-0.1775-0.1935 0.5911 0.9402 0.5919 21-0.1538-0.1578 0.6873 0.8349 0.6876 22-0.1490-0.1824 0.8310 0.9189 0.8298 23-0.0963-0.0044 0.9382 0.8205 0.9391 24-0.2119-0.2531 0.8254 0.8934 0.8265 25-0.1153-0.1149 0.8020 0.6467 0.8020 26-0.1655-0.1680 0.6970 0.7449 0.6980 27-0.1234-0.1317 0.6755 0.8371 0.6769 28-0.0663-0.0668 0.8951 0.4258 0.8986 avg. -0.1563-0.1705 0.8194 0.8054 0.8208 Eilersten [5], ExpandNet [6], Huo [18], Akyüz [2], 4. 1 2. 1 2 HDR (percep- tually uniform encoding peak signal-to-noise ratio; PU- PSNR), (perceptually uniform encoding structural similarity; PU-SSIM) (high dynamic range visible difference predictor; HDR-VDP) 2.2 [15,16].
2: (Rahoon Kang : Image Filtering Method for an Effective Inverse Tone-mapping) (a) (b) (c) (d) (e) 3. HDR-VDP-2.2 (21 ), (a) Akyüz, (b) Huo, (c) Eilersten, (d) ExpandNet, (e) Fig 3. The comparison results of inverse tone-mapping methods(at no.21 image), (a) Akyüz, (b) Huo, (c) Eilersten, (d) ExpandNet, (e) proposed algorithm PSNR, SSIM VDP HDR. HDR RGB (perceptually uniform) [15]. Akyüz Huo. PU-PSNR PU-SSIM. ExpandNet Eiliersten. PU-PSNR ExpandNet 20 10. PU-PSNR 28 9. ExpandNet PU-PSNR 0.6. PU-SSIM ExpandNet 0.02 WGIF. 3 HDR-VDP-2.2. HDR-VDP. 21 ExpandNet. Eilersten. WGIF. 4 LDR,. (g-l).
(JBE Vol. 24, No. 2, March 2019) (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) (l) 4. (2 ), (a,g), (b,h) Eilersten, (c,i) ExpandNet, (d,j) Akyüz, (e,k) Huo, (f,l) Fig 4. The comparison of tone-mapping result subjective image quality of each method(at no.2 image), (a,g) ground truth, (b,h) Akyüz, (c,i) Huo, (d,j) Eilersten, (e,k) ExpandNet,, (f,l) proposed algorithm..... WGIF,, HDR. HDR. HDR LDR.,,,.
2: (Rahoon Kang : Image Filtering Method for an Effective Inverse Tone-mapping) (References) [1] H. Landis, Production-ready global illumination, SIGGRAPH Course Notes, Vol.16, pp.87-101, 2002. [2] A. O. Akyüz, R. Fleming, B. E. Riecke, E. Reinhard, and H. H. Bülthoff, Do HDR displays support LDR Content?: A psychophysical evaluation, ACM Transaction on Graphics, Vol.26, No.38, July 2007. [3] F. Banterle, A. Artusi, K. Debattista, and A. Chalmers, Advanced High Dynamic Range Imaging:theory and practice, A K Peters/CRC Press, New York, February 2011. [4] A. G. Rempel, M. Trentacoste, H. Seetzen, H. D. Young, W. Heidrich, L. Whiteheadm, and G. Ward, LDR2HDR: On-thefly reverse tone mapping of legacy video and photographs, ACM Transaction on Graphics, Vol.26, No.39, 2007. [5] G. Eilertsen, J. Kronander, G. Denes, R. Mantiuk, and J. Unger, "HDR image reconstruction from a single exposure using deep CNNs," ACM Transactions on Graphics, Vol.36, No.6, pp.1-15, 2017. [6] D. Marnerides, T. Bashford-Rogers, J. Hatchett, and K. Debattista, "ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content," Computer Graphics Forum, Vol.37, No.2, pp.37-49, 2018 [7] A. A. Goshtasby, Fusion of Multi-exposure Images, Image and Vision Computing, Vol.23, pp. 611-618, June 2005. [8] S. Hecht, The visual discrimination of intensity and the Weber-Fechner law, The Journal of General Physiology, Vol.7, pp.235-267, 1924. [9] G. E. Hinton and R. Salakhutdinov, "Reducing the Dimensionality of Data with Neural Networks," Science, Vol.313, No.5786, pp.504-507, 2006. [10] P. Vincent, H. Larochelle, Y. Bengio and P. Manzagol, "Extracting and composing robust features with denoising autoencoders, Proceeding of 25th International Conference on Machine Learning(ICML), pp.1096-1103, 2008. [11] K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition", Arxiv.org, 2014, https://arxiv.org/abs/1409.1556 [12] K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, Proceeding of IEEE conference on computer vision and pattern recognition(cvpr), pp.770-778, 2016. [13] M. D. Fairchild, The HDR Photographic survey, Proceeding of Color and Imaging Conference, Vol.2007, No.1, pp.233-238, 2007. [14] G. Ward, Hight Dynamic Range Image Encodings, 2006. [15] T. O. Aydın, R. Mantiuk, and H. P. Seidel, Extending quality metrics to full luminance range images, Human Vision and Electronic Imaging XIII, Vol.6806, pp.68060b, March 2008. [16] M. Narwaria, R. Mantiuk, M. P. Da Silva, and P. Le Callet, HDR-VDP-2.2: a calibrated method for objective quality prediction of high-dynamic range and standard images, Journal of Electronic Imaging, Vol.24, No.010501, 2015. [17] V. Nair and G. E. Hinton, Rectified linear units improve restricted boltzmann machines, Proceeding of the 27th International Conference on Machine Learning(ICML), pp.807-814, 2010. [18] Y. Huo, F. Yang, L. Domg, and V. Brost, Physiological inverse tone mapping based on retina response, The Visual Computer, Vol.30, pp.507-517, 2014. [19] Z. Liand J. Zheng, Z. Zhu, W. Yao, and S. Wu, Weighted Guided Image Filtering, IEEE Transaction on Image Processing, Vol.24, No.1, pp.120-129, 2015. [20] K. He, J. Sun, and X. Tang, Guided Image Filtering, Proceeding of European Conference on Computer Vision(ECCV), Berlin, Heidelberg, pp.1-14, 2010. [21] J. An, S. Lee, J. Kuk, and N. Cho, A multi-exposure image fusion algorithm without ghost effect, Proceeding of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2011. - 2018 2 : - 2018 3 ~ : - ORCID : https://orcid.org/0000-0002-2291-2014 - :,
(JBE Vol. 24, No. 2, March 2019) - 2016 2 : - 2016 3 ~ : - ORCID : https://orcid.org/0000-0003-3783-8272 - :, - 1980 2 : - 1982 2 : KAIST - 1990 : - 1980 ~ 1986 : KBS ( ) - 1990 ~ 1991 : ( ) - 1991 ~ 1995 : HDTV - 1995 ~ : ( ) - ORCID : http://orcid.org/0000-0002-3759-3116 - :,