2: (Imjae Park et al.: Modified Exposure Fusiom with Improved Exposure Adjustment Using Histogram and Gamma Correction) (Special Paper) 22 3, 2017 5 (JBE Vol. 22, No. 3, May 2017) https://doi.org/10.5909/jbe.2017.22.3.327 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) a), a), a) Modified Exposure Fusion with Improved Exposure Adjustment Using Histogram and Gamma Correction Imjae Park a), Deajun Park a), and Jechang Jeong a).... MEF-SSIM,. Abstract Exposure fusion is a typical image fusion technique to generate a high dynamic range image by combining two or more different exposure images. In this paper, we propose block-based exposure adjustment considering unique characteristics of human visual system and improved saturation measure to get weight map. Proposed exposure adjustment artificially corrects intensity values of each input images considering human visual system, efficiently preserving details in the result image of exposure fusion. The improved saturation measure is used to make a weight map that effectively reflects the saturation region in the input images. We show the superiority of the proposed algorithm through subjective image quality, MEF-SSIM, and execution time comparison with the conventional exposure fusion algorithm. Keyword : Exposure fusion, Exposure adjustment, High dynamic range imaging, CIELAB color model, Human visual system a) (Department of Electronics and Computer Engineering, Hanyang University) Corresponding Author : (Jechang Jeong) E-mail: jjeong@hanyang.ac.kr Tel: +82-2-2220-4370 ORCID: http://orcid.org/0000-0002-3759-3116 2015 () (NRF-2015R1A2A2A01006004). Manuscript received March 6, 2017; Revised May 19, 2017; Accepted May 19, 2017. Copyright 2017 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND (http://creativecommons.org/licenses/by-nc-nd/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.
(JBE Vol. 22, No. 3, May 2017). (dynamic range) (scene) (image). (human visual system: HVS) (high dynamic range: HDR). (low dynamic range: LDR).....,. (exposure fusion)., (exposure adjustment). CIELAB...,... 1. Ferwerda [1]., 1. Fig. 1. Rough flow chart of proposed algorithm
2: (Imjae Park et al.: Modified Exposure Fusiom with Improved Exposure Adjustment Using Histogram and Gamma Correction).. Ferwerda,.... 2.. Debevec [2], (irradiance) (camera response function) (radiance). [3],. Mertens [4] (quality measure) (contrast), (saturation), (well-exposedness)., Burt [5]. [6]. [7] IMU,. Li [8] (gradient) (vector field), (quadratic optimization problem). Mertens. Ma [9] (patch), (signal strength), (signal structure), (mean intensity). (patch strength) (exposedness).. Peter [10].,. 3x3, LoG(Laplacian of Gaussian).,. Peter. 1.
(JBE Vol. 22, No. 3, May 2017)... 1,.. CIELAB. CIELAB L a, b. RGB CIELAB [11]. (3)~(5) XYZ CIELAB L 0 100 a b, 127 127. 8 0 255. 1..,,.. 1.1. L 32x24. RGB CIELAB (1) (2) XYZ. RGB XYZ, 0 1. XYZ CIELAB. for f or for for 1.2 3x3 Laplacian [12].., (threshold). max max
2: (Imjae Park et al.: Modified Exposure Fusiom with Improved Exposure Adjustment Using Histogram and Gamma Correction) n.,. m ax (bin). 30, 230.., 3x3 Laplacian. Laplacian., Laplacian.. n 3x3 Laplacian., Laplacian,. 1.3 (gamma correction)..,..,,. 1.0,,. (10). n,. m ax 255. n Laplacian. m ax,., 1.0. 3.0 2.5. 1.0., 0.1 0.2. (9). 2 (b). (distance weighting function). Duan [13]. Duan max if max max max if max
(JBE Vol. 22, No. 3, May 2017) 2. (a) (b) (c) Fig. 2. Exposure adjustment result (a) original input LDR image (b) result image with blocking artifacts (c) result image without blocking artifacts. (11)., 5x5 25. n, (9)..... Duan 20. 2 (c). LUT (look up table). 2. (quality measure processing). (constrast), (saturation).. Laplacian., (11) L. (saturat-
2: (Imjae Park et al.: Modified Exposure Fusiom with Improved Exposure Adjustment Using Histogram and Gamma Correction) ed).. (over-saturated). (under-saturated).. Laplacian, Gaussian., Burt Laplacian. a b (17), CIELAB., CIELAB a b. a b. a b 0 255 127...,,... 3. (fusion processing). Mertens.., MEF-SSIM,. Peter Yeganeh [14] 7, 14. Mertens Peter Visual C++ OpenCV 3.1, Window 8.1 64-bit, i7-4790 CPU, 8GB RAM. 3. Mertens, RGB. Peter..,. 3
334 방송공학회논문지 제22권 제3호, 2017년 5월 (JBE Vol. 22, No. 3, May 2017) 그림 3. 노출 합성 영상 부분 확대 (a) Mertens[4] 방식 (b) Peter[10] 방식 (c) 제안하는 알고리듬 Fig. 3. Locally extension image of exposure fusion result (a) Mertens[4] (b) Peter[10] (c) proposed algorithm
박임재 외 인 히스토그램과 감마보정 기반의 노출 조정을 이용한 다중 노출 영상 합성 기법 335 2 : (Imjae Park et al.: Modified Exposure Fusiom with Improved Exposure Adjustment Using Histogram and Gamma Correction) 첫 번째 영상의 경우 제안한 알고리듬의 결과에서 건물 외 벽과 구름의 디테일의 보존이 다른 알고리듬에 비해 우수 함을 확인할 수 있다. 표 1은 Mertens, Peter, 제안하는 노출 합성 알고리듬에 대해 MEF-SSIM 을 측정한 결과이다. MEF-SSIM은 노 출 합성 알고리듬의 성능을 평가하기 위해 사용되는 객관 [15] 적인 화질 측정 지표로써 결과 영상을 대비(contrast) 측면 과 구조(structure) 측면으로 분석한다. MEF-SSIM이 1.0 에 가까울수록 객관적 화질이 우수함을 나타낸다. MEF-SSIM 측면에서 총 21개의 실험 영상 중 13개의 영상에서 제안하 는 알고리듬이 우수함을 보인다. 표 2는 Mertens의 알고리듬을 기준으로 Mertens, Peter, 그림 4. 주관적 화질 비교 (a) Mertens[4] 방식 (b) Peter[5] 방식 (c) 제안하는 알고리듬 Fig. 4. Subjective comparison (a) Mertens[4] (b) Peter[5] (c) proposed algorithm
(JBE Vol. 22, No. 3, May 2017) 1. MEF-SSIM Table 1. MEF-SSIM of proposed and conventional algorithm Sequences MEF-SSIM [15] Mertens [4] Peter [10] Proposed Tintern abbey 0.812 0.827 0.879 Bristol bridge 0.883 0.871 0.912 Memorial 0.857 0.825 0.906 Clock building 0.889 0.881 0.916 Oaks 0.863 0.845 0.908 Tahoe1 0.859 0.842 0.898 Girl 0.850 0.832 0.883 Air stream sunrise 0.915 0.914 0.923 California highway 0.975 0.957 0.970 Carwall 0.952 0.932 0.946 Coffee shop 0.925 0.917 0.922 Popcorn counter 0.965 0.959 0.961 Egyptian 0.917 0.898 0.925 Fat cloud 0.972 0.951 0.964 Engines 0.960 0.954 0.957 Kitchen window 0.941 0.925 0.925 Mans Chinese 0.919 0.911 0.918 Berlin 0.958 0.938 0.967 Bremerhaven 0.947 0.923 0.961 Land scharft 0.918 0.904 0.943 Wald 0.917 0.897 0.956 Average 0.914 0.900 0.930. Mertens,,. Pedersen [16] (over- amplify). Peter. Peter 3x3, LoG. Mertens.,. Mertens. 3 Peter 2. Table 2. Comparing processing time of proposed and conventional algorithm Sequences Processing time Mertens [4] Peter [10] Proposed Tintern abbey 100% 159% 103% Bristol bridge 100% 160% 105% Memorial 100% 159% 100% Clock building 100% 158% 104% Oaks 100% 159% 94% Tahoe1 100% 155% 98% Girl 100% 157% 96% Air stream sunrise 100% 148% 109% California highway 100% 155% 109% Carwall 100% 159% 112% Coffee shop 100% 158% 112% Popcorn counter 100% 145% 104% Egyptian 100% 145% 99% Fat cloud 100% 138% 95% Engines 100% 132% 98% Kitchen window 100% 132% 100% Mans Chinese 100% 134% 100% Berlin 100% 140% 98% Bremerhaven 100% 128% 97% Land scharft 100% 130% 97% Wald 100% 134% 101% Average 100% 147% 104% 3. Table 3. Comparing exposure adjustment time of proposed and conventional Sequences Processing time Peter [10] Proposed Tintern abbey 100% 80% Bristol bridge 100% 79% Memorial 100% 77% Clock building 100% 79% Oaks 100% 76% Tahoe1 100% 82% Girl 100% 73% Air stream sunrise 100% 88% California highway 100% 85% Carwall 100% 91% Coffee shop 100% 97% Popcorn counter 100% 95% Egyptian 100% 89% Fat cloud 100% 88% Engines 100% 93% Kitchen window 100% 90% Mans Chinese 100% 87% Berlin 100% 88% Bremerhaven 100% 90% Land scharft 100% 86% Wald 100% 94% Average 100% 89%
2: (Imjae Park et al.: Modified Exposure Fusiom with Improved Exposure Adjustment Using Histogram and Gamma Correction). Peter 3x3 LoG.....,,.... (References) [1] J. A. Ferwerda, S, N. Pattanaik, P. Shirley, and D. P Greenberg, A model of visual adaptation for realistic image synthesis, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, ACM, pp.249-258, 1996. [2] P. E. Debevec, and J. Malik, Recovering high dynamic range radiance maps from photographs, SIGRAPTH 1997: Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pp. 369-378, 1997. [3] T. Park, and I. Park, HDR Image Acquisition from Two LDR Images, Journal of Broadcast Engineering, vol. 16, no. 2, pp.247-257, March 2011. [4] T. Mertens, J. Kautz, and F. Van Reeth, Exposure fusion: A simple and practical alternative to high dynamic range photography, Computer Graphics Forum, vol. 28, no. 1, pp. 161-171, 2009. [5] P. Burt, and E. Adelson, The Laplacian pyramid as a compact image code, IEEE Transactions on communications, vol. 31, no. 4, pp.532-540, 1983. [6] T. Kil, and N. Cho, Image Fusion using RGB and Near Infrared Image, Journal of Broadcast Engineering, vol. 21, no. 4, pp. 515-524, July 2016. [7] H. Ryu, and B. Song, Non-uniform Deblur Algorithm using Gyro Sensor and Different Exposure Image Pair, Journal of Broadcast Engineering, vol. 21, no. 2, pp.200-209, March 2016. [8] Z. G. Li, J. H. Zheng, and S. Rahardja, Detail-Enhanced Exposure Fusion, IEEE Transactions on Image Processing, vol. 21, no. 11, pp. 4672-4676, 2012. [9] K. Ma, and Z. Wang, Multi-exposure image fusion: A patch-wise approach, IEEE International Conference on Image Processing, pp.1717-1721, 2015. [10] S. Peter, D. Androutsos, and M. Kyan, Adaptive exposure fusion for high dynamic range imaging, IEEE International Conference on. Image Processing, pp. 4679-4683, 2015. [11] Color conversions, http://docs.opencv.org/ 3.1.0/de/d25/imgproc_color_conversions.html#color_convert_rgb_lab. [12] R. C. Gonzalez, and R. E. Woods. "Digital image processing, Pearson, New Jersey, 2010. [13] J. Duan, M. Bressan, C. Dance, and G. Qiu, Tone-mapping high dynamic range images by novel histogram adjustment, Pattern Recognition, vol. 43, no. 5, pp.1847-1862, 2010. [14] H. Yeganeh, and Z. Wang, Objective quality assessment of tone-mapped images, IEEE Transactions on Image Processing, vol. 22, no. 2, pp.657-667, 2013. [15] K. Ma, K. Zeng, and Z. Wang, Perceptual quality assessment for multi-exposure image fusion, IEEE Transactions on Image Processing, vol. 24, no.11, pp. 3345-3356, 2015. [16] M. Pedersen, Exposure fusion algorithm based on perceptual contrast and dynamic adjustment of well-exposedness, International Conference on Image and Signal Processing, Springer International Publishing, pp. 183-192, 2014.
338 방송공학회논문지 제22권 제3호, 2017년 5월 (JBE Vol. 22, No. 3, May 2017) 저자소개 박임재 년 월 강남대학교 컴퓨터미디어정보공학과 졸업 (학사) 년 월 현재 한양대학교 전자컴퓨터통신공학과 (석사과정) 주관심분야 영상처리 - 2016 2 : - 2016 3 ~ : - ORCID : http://orcid.org/0000-0002-4703-2849 :, HDR 박대준 년 월 한양대학교 전자통신컴퓨터공학부 졸업 (학사) 년 월 현재 한양대학교 전자컴퓨터통신공학과 석박사통합과정 (박사과정) 주관심분야 비디오 압축 영상처리 - 2011 2 : - 2011 3 ~ : - ORCID : http://orcid.org/0000-0001-9255-071x :,, HDR 정제창 - 년 2월 : 서울대학교 전자공학과 (학사) 년 2월 : KAIST 전기전자공학과 (석사) 년 : 미국 미시간대학 전기공학과 (공학박사) 년 ~ 1986년 : KBS 기술연구소 연구원 (디지털 및 뉴미디어 연구) 년 ~ 1991년 : 미국 미시간대학 전기공학과 연구교수 (영상 및 신호처리 연구) 년 ~ 현재 : 한양대학교 전자컴퓨터통신공학과 교수 (영상통신 및 신호처리 연구실) 년 12월 : 정보통신부장관상 수상 년 11월 : 과학기술자상 수상 년 : IEEE Chester Sall Award 수상 년 : ETRI Journal Paper Award 수상 년 5월 : 제 46회 발명의 날 녹조근정훈장 수훈 : http://orcid.org/0000-0002-3759-3116 주관심분야 : 영상처리, 영상압축, 3DTV 1980 1982 1990 1980 1990 1995 1990 1998 2007 2008 2011 ORCID