(JBE Vol. 0, No. 6, November 05) (Special Paper) 0 6, 05 (JBE Vol. 0, No. 6, November 05) http://d.doi.org/0.5909/jbe.05.0.6.88 ISSN 87-937 (Online) ISSN 6-7953 (Print) a), a) Elaborate Image Quality Assessment with a Novel Luminance Adaptation Effect Model Sung-Ho Bae a) and Munchurl Kim a) (Human Visual System: HVS) (Image Quality Assessment: IQA). HVS, (Luminance Adaptation: LA) HVS, (Weber s law) IQA. IQA LA /. IQA LA LA (LA effect-based Local weight Function: LALF). LALF SSIM(Structural SIMilarit PSNR (metric)., LALF SSIM SSIM 5%., LALF PSNR PSNR.5%. Abstract Recently, obective image quality assessment (IQA) methods that elaborately reflect the visual quality perception characteristics of human visual system (HVS) have actively been studied. Among those characteristics of HVS, luminance adaptation (LA) effect, indicating that HVS has different sensitivities depending on background luminance values to distortions, has widely been reflected into many eisting IQA methods via Weber s law model. In this paper, we firstly reveal that the LA effect based on Weber s law model has inaccurately been reflected into the conventional IQA methods. To solve this problem, we firstly derive a new LA effect-based Local weight Function (LALF) that can elaborately reflect LA effect into IQA methods. We validate the effectiveness of our proposed LALF by applying LALF into SSIM (Structural SIMilarit and PSNR methods. Eperimental results show that the SSIM based on LALF yields remarkable performance improvement of 5% points compared to the original SSIM in terms of Spear rank order correlation coefficient between estimated visual quality values and measured subective visual quality scores. Moreover, the PSNR (Peak to Signal Noise Ratio) based on LALF yields performance improvement of.5% points compared to the original PSNR. Keyword : Human visual system (HVS), luminance adaptation (LA), image quality assessment (IQA), power law, Weber s law
: (Sung-Ho Bae et al.: Elaborate Image Quality Assessment with a Novel Luminance Adaptation Effect Model). []., (Image Quality Assessment: IQA) [][3]., IQA, - [4][5]. (Full-Reference: FR) IQA, (Reduced-Reference: RR) IQA, (No-Reference: NR) IQA [6]. IQA(FR-IQA). PSNR(Peak to Signal Noise Ratio) MSE(Mean Squared Error), (,, ). FR-IQA (Human Visual System: HVS), []., SSIM(Structural SIMilarit a) (Korea Advanced Institute of Science and Technology, School of Electrical Engineering) Corresponding Author : (Munchurl Kim) E-mail: mkim@ee.kaist.ac.kr Tel: +8-4-350-749 ORCID:http://orcid.org/0000-0003-046-549 ( ) ( : 04RAAA000664). 05. Manuscript received September, 05; Revised September 3, 05 ;Accepted September 3, 05 FR-IQA [6]-[0]. (Weber s law) (Luminance Adaptation: LA) [6]-[9]. (Hu- man Visual System: HVS) [] (). d = DL L X d, DL =L y L (L ) (L y ) HVS. () DL (Just Noticeable Difference: JND), JND HVS JND []-[5]. () JND. FR-IQA LA LA,. (i) () [6]., Frese (power law), [6]. (ii) IQA (piel intensit. ( ) (luminance). (gamma correction function) LA [].
(JBE Vol. 0, No. 6, November 05) FR-IQA SSIM LA., (bg) (R ) 3 (. -(a), -(b), -(c)). 3 (bg) 8-7 (m), (R ) -(a), -(b), -(c) m = 0, 75, 45. (R ) (pseudo-addictive White Gaussian Noise: AWGN). -(d), -(e), -(f) -(a), -(b), -(c) SSIM, SSIM = 0.9348 AWGN. -(d), -(e), -(f), 3 SSIM R m = 75 -(e) R m = 45 -(f), R m = 0 -(d). SSIM LA. FR-IQA. LA FR-IQA,.,. (i) FR-IQA LA /. (ii) (i), FR-IQA LA LA (LA effect-based Local weight Function: LALF). FR-IQA, LA (local weight).. SSIM FR-IQA LA R =0) R =75) R ( =45) bg ( = 7) bg ( = 7) bg ( = 7) (a) (b) (c) (d) SSIM =0.9348 (e) SSIM =0.9348 (f) SSIM =0.9348. LA SSIM Fig.. An simple eample for the case that LA effect is not well-reflected into SSIM
: (Sung-Ho Bae et al.: Elaborate Image Quality Assessment with a Novel Luminance Adaptation Effect Model), / FR-IQA LA. 3 LALF, FR-IQA LA. 4 LALF LALF SSIM PSNR. 5... SSIM SSIM FR-IQA (mile stone) []., HVS FR-IQA [7]. SSIM HVS FR-IQA [8]-[0], SSIM. SSIM HVS (luminance), (contrast), (structure) 3 (local) SSIM. X Y X, y Y SSIM (). f ( = l( c( s( l( ), c( ), s( ),,.. m, m y y, C l( ) 0, C =(0.0 M) [7]( M ). (3) l( ) m m y, m, m y l( ). () c( ) s( ). s cs( = c( s( = s + y s y + C + C s, s y y, s y y PLCC(Pearson linear correlation coefficient). (4) C (3) C cs( ), C =(0.03 M) [7]. Y SSIM SSIM (pooling). J F( X, Y) = å f (, y ) J = -, J.. SSIM SSIM (3) FR-IQA [7]-[9]. FR-IQA., (3) C m ( C /m 0). (3) m y m y =m +Dm (6). m m + C l( = m m + C + y y ( + Dm m) l( = + ( + Dm m )
(JBE Vol. 0, No. 6, November 05) l( ) Dm/m ( () Dm/m ), SSIM FR-IQA. 3. FR-IQA LA (3) LA.., JND ( ). Frese [6]. Frese (power low), JND [6]. Frese (7). V V - ( h L ) L DL / L = h, z, L Ref. (), DL/L, DL/L L Ref., FR-IQA (piel intensity domain), ( ) (luminance domain)., ( ). []. (8). L = a + b m g a, b, g.. FR-IQA LA. LA FR-IQA, LA. (7) (8) -. (8) L m L +DL m +Dm, (first-order Taylor series epansion) (9). L + DL = a + b m g ( + Dm m ) g (7) DL (9), (8) L (0) LA JND (LA-JND ). JND a ( m ) = Dm = a m + a3 m + a4 a, a, a 3, a 4 LA-JND. (0) a {0, } JND m. (0) [5] JND. [5] (least square solution) (0), a = -.655, a = 0.959, a 3 = -.709, a 4 =.73. [5] JND (fitted) LA-JND., JND ( )
: (Sung-Ho Bae et al.: Elaborate Image Quality Assessment with a Novel Luminance Adaptation Effect Model) JND, ( ) JND. HVS,.. (0) LA-JND [5]. LA-JND JND. 5 0 5 Fitted curve Measured JND values - [], LALF () (). w - ( m ) p = b + JND b>0, p>0 LALF. (X) (Y) - LALF w, w y. - LALF w( w w y., FR-IQA, LALF LA FR-IQA. LALF FR-IQA (). 0 5 J F( X, Y) = å w(, y ) f (, y ) W = 0 0 50 00 50 00 50. [5] JND (0) LA-JND Fig.. The measured JND thresholds in [5], and their fitted curve by our LA-JND model in (0). LALF (0) LA-JND FR-IQA, LA (LALF).., HVS []-[5]. f( ) FR-IQA (, () SSIM), W =,,..., J- w(. LALF LALF. TID008 [7] (X i, i=, i- ) AWGN (Y i ) LALF (W i ). LALF, [0, 55]. LALF ( ) LALF ( ),
4 8 방송공학회논문지 제0권 제6호, 05년 월 (JBE Vol. 0, No. 6, November 05) 그림. 원본영상 영상(X i, 여기서 i=, ) 및 AWGN-왜곡 영상(Y i)에 대한 LALF 맵(Wi) Fig.. The LALF maps (Wi, where i=, ) obtained from the original images (X i) and their AWGN-distorted images (Y i) 확대한 그림이다. 그림 의 원본 영상 (a), (d) 및 왜곡 영상 듯 청색 박스 영 역)의 왜곡은 어둡거나 밝은 밝기를 가지는 영역(적색 박스 영역)의 왜곡보다 시각적으로 더욱 확연히 인지되며, 결과 적으로 이 영역들은 시각적 화질 열화 인지에 더 크게 기여 (b), (e)에서 보 이, 중간 밝기를 가지는 영역( 데이터로, 이는 5개의 참조 영상에 대 해 7 종류의 다른 왜곡 종류를 가지는 총 700개의 왜곡된 터는 TID008 IQA 영상과 이 왜곡 영상에 대해 측정한 주관적 화질 점수값 (Mean Opinion Score values)을 포함하고 있다[7]. 제안하 는 LALF는 그 성능을 검증하기 위해 SSIM과 PSNR에 적 한다. 그림 -(c) 및 -(f)의 LALF 맵들(W, W)은 위에서 용되었다. LALF가 적용된 SSIM과 PSNR은 원래의 방법들 분석한 HVS의 시각적 왜곡 민감도 특성과 상당히 일관된 과의 구분을 위해 LA-SSIM, LA-PSNR으로 표기된다. 결과를 보이는, 여기서 LALF는 시각적으로 왜곡에 SSIM의 경우 식 (3)의 광도 유사도 측정 모델이 부정확하 데 둔감 한 영역(적색 박스 영역)에 대해 상대적으로 낮은 값을 생 성하며, 시각적으로 왜곡에 민감한 영역(청색 박스 영역)에 대해서는 상대적으로 높은 값을 생성한다. 따라서 제안하 는 LALF는 HVS의 LA 효과를 FR-IQA 방법에 정교하게 반영할 수 있는 방법이 될 수 있다. 게 LA 효과를 반영하고 있기 때문에, 광도 유사도 측정 모 델을 LA-SSIM 계산에서 제거하였다. 본 논문에서는 실험 적으로 LA-SSIM에 대한 LALF의 모수를 b=.5 0-3, p = 로, LA-PSNR에 대한 LALF의 모수를 b = p = 0.55으로 설정하였다. 성능 검증을 위해, 본 논문에서는 4개의 성능 지표를 도 입하였다: SROC(Spearman Rank-Order Correlation co- Ⅳ. 실험 결과 efficient), KROC(Kendall Rank-Order Correlation coefficients), PLCC(Pearson Linear Correlation coefficient), 본 논문에서 제안하는 LALF의 성능을 검증하기 위해 주 관적 화질 예측 성능 실험을 수행한다. 실험에 사용한 데이 RMSE(Root Mean Squared Error). 여기서 SROC, KROC, PLCC는 IQA 방법을 이용해 예측한 화질 값과 실제 측정한
: (Sung-Ho Bae et al.: Elaborate Image Quality Assessment with a Novel Luminance Adaptation Effect Model), IQA. RMSE(Root Mean Squared Error) IQA, 0 IQA. FR-IQA -, [8] PLCC RMSE. 4, SROC FR-IQA [6]. TID008 SSIM, PSNR LA-SSIM, LA-PSNR. FR-IQA LALF FR-IQA.. TID008 SSIM(PSNR) LA-SSIM (LA-PSNR) Table. Performance comparison on TID03 database between SSIM (PSNR) and LA-SSIM (LA-PSNR) Measure \ Method SSIM LA-SSIM PSNR LA-PSNR SROC 0.7749 0.843 0.545 0.5509 KROC 0.5768 0.604 0.3696 0.3904 PLCC 0.773 0.870 0.5309 0.547 RMSE 0.870 0.7738.39.80, LA-SSIM LA-PSNR SSIM PSNR (SROC, KROC, PLCC RMSE). SSIM, LA-SSIM SROC 5%. LA-PSNR PSNR SROC.5%.. LA, LA -. LA LA FR-IQA LALF. LALF LALF SSIM PSNR., LALF SSIM PSNR. LA LALF LA FR-IQA. (References) [] Z. Wang and A. C. Bovik, Mean squared error: Love it or leave it? A new look at signal fidelity measures, IEEE Signal Process. Mag., vol. 6, no., pp. 98-7, Jan. 009. [] S.-H. Bae, J. Kim, M. Kim, S. H. Cho, and J. S. Choi, Assessments of subective video quality on HEVC-encoded 4K-UHD video for beyond-hdtv broadcasting services, IEEE Trans. on Broadcast., vol. 59, no., pp. 09-, Jun. 03. [3] J.-S. Choi, S.-H. Bae and M. Kim, Single image super-resolution based on self-eamples using contet-dependent subpatches, IEEE Int. Conf. on Image Proc, accepted for publication, Sept. 7-30, 05. [4] J.-S. Choi, S.-H. Bae and M. Kim, A no-reference perceptual blurriness metric based fast super-resolution of still pictures using sparse representation, Proc. SPIE, vol. 940, pp. 9400N.-9400N.7, Mar. 05. [5] J. Kim, S.-H. Bae, and M. Kim, An HEVC-compliant perceptual video coding scheme based on JND models for variable block-sized transform kernels, IEEE Trans. Circuits Syst. Video Technol., in press, Jan. 04. [6] L. Zhang, L. Zhang, X. Mou, and D. Zhang, A comprehensive evaluation of full reference image quality assessment algorithms, Proc. 9th IEEE Int. Conf. Image Process., pp. 477 480, Sep./Oct. 0. [7] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Trans. on Image Process., vol. 3, pp. 600-6, Apr. 004.
(JBE Vol. 0, No. 6, November 05) [8] Z. Wang and Q. Li, Information content weighting for perceptual image quality assessment, IEEE Trans. Image Process., vol. 0, no. 5, pp. 85-98, May 0. [9] Z. Wang, E. P. Simoncelli, and A. C. Bovik, Multiscale structural similarity for image quality assessment, Proc. 37th Asilomar Conf. Signals, Syst., Comput., pp. 398 40, Nov. 003. [0] S.-H. Bae and M. Kim, A novel image quality assessment based on an adaptive feature for image characteristics and distortion types, IEEE Video Comm. and Image Proc., accepted for publication, Dec. 3-6, 05. [] S.-H Bae and M. Kim, A novel DCT-based JND model for luminance adaptation effect in DCT frequency, IEEE Signal Process. Lett., vol. 0, no. 9, pp. 893-896, Sept. 03. [] Z. Wei and K. N. Ngan, Spatio-temporal ust noticeable distortion profile for grey scale image/video in DCT domain, IEEE Trans. Circuits Syst. Video Technol., vol. 9, no. 3, pp. 337-346, Mar. 009. [3] S.-H. Bae and M. Kim, A new DCT-based JND model of monochrome images for contrast masking effects with teture compleity and frequency, IEEE Int. Conf. on Image Proc, Melborne, Australia, Sept. 5-8, pp. 43-434, 03. [4] S.-H Bae and M. Kim, A novel generalized DCT-based JND profile based on an elaborate CM-JND model for variable block-sized transforms in monochrome images, IEEE Trans. on Image Process., vol. 3, no. 8, Aug. 04. [5] C.-H. Chou, Y.-C. Li, A perceptually tuned subband image coder based on the measure of ust-noticeable distortion profile, IEEE Trans. Circuits Syst. Video Technol. vol. 5, no. 6, pp. 467-476, Dec. 995. [6] T. Frese, C. A. Bouman, and J. P. Allebach. A methodology for designing image similarity metrics based on human visual system models, Proc. SPIE, vol. 306, pp. 47-483, 997. [7] N. Ponomarenko, V. Lukin, A. Zelensky, K. Egiazarian, M. Carli, and F. Battisti, TID008-A database for evaluation of full-reference visual quality assessment metrics, Adv. Modern Radioelectron., vol. 0, pp. 30 45, 009. [8] Final Report From the Video Quality Eperts Group on the Validation of Obective Models of Video Quality Assessment VQEG. Available: http://www.vqeg.org, 000. - 0 : - 0 8 : - 0 9 ~ : - ORCID : http://orcid.org/0000-0000-896-704 - : Human Visual Perception based Computational Vision - 989 : - 99 : University of Florida, Dept. of Electrical and Computer Engineering, - 996 8 : University of Florida, Dept. of Electrical and Computer Engineering, - 997 ~ 00 :, - 00 ~ 009 : / - 009 3 ~ : / - ORCID : http://orcid.org/0000-0003-046-549 - : Perceptual Video Coding, SDR/HDR Image/Video Quality Assessment and Modeling, Super-Resolution, Image/Video Analysis and Understanding, Pattern Recognition, Machine Learning