1 : (Sung-Ho Bae et al.: A Novel Fast and High-Performance Image Quality Assessment Metric using a Simple Laplace Operator) (Special Paper) 21 2, 2016

Size: px

Start display at page:

Download "1 : (Sung-Ho Bae et al.: A Novel Fast and High-Performance Image Quality Assessment Metric using a Simple Laplace Operator) (Special Paper) 21 2, 2016"

덕문 점
5 years ago
Views:

1 1: (Sung-Ho Bae et al.: A Novel Fast and High-Performance Image Quality Assessment Metric using a Simple Laplace Operator) (Special Paper) 21 2, (JBE Vol. 21, No. 2, March 2016) ISSN (Online) ISSN (Print) a), a) A Novel Fast and High-Performance Image Quality Assessment Metric using a Simple Laplace Operator Sung-Ho Bae a) and Munchurl Kim a), (Mean Squared Error: MSE) (, (metricability), (differentiability) (convexity)). MSE, (Image Quality Assessment: IQA). IQA MSE,., IQA. IQA, IQA. IQA, IQA., IQA IQA MSE,. Abstract In image processing and computer vision fields, mean squared error (MSE) has popularly been used as an objective metric in image quality optimization problems due to its desirable mathematical properties such as metricability, differentiability and convexity. However, as known that MSE is not highly correlated with perceived visual quality, much effort has been made to develop new image quality assessment (IQA) metrics having both the desirable mathematical properties aforementioned and high prediction performances for subjective visual quality scores. Although recent IQA metrics having the desirable mathematical properties have shown to give some promising results in prediction performance for visual quality scores, they also have high computation complexities. In order to alleviate this problem, we propose a new fast IQA metric using a simple Laplace operator. Since the Laplace operator used in our IQA metric can not only effectively mimic operations of receptive fields in retina for luminance stimulus but also be simply computed, our IQA metric can yield both very fast processing speed and high prediction performance. In order to verify the effectiveness of the proposed IQA metric, our method is compared to some state-of-the-art IQA metrics. The experimental results showed that the proposed IQA metric has the fastest running speed compared the IQA methods except MSE under comparison. Moreover, our IQA metric achieves the best prediction performance for subjective image quality scores among the state-of-the-art IQA metrics under test. Keyword : Human visual system, Simple Laplace operator, image quality assessment, computation complexity, mean squared error

2 (JBE Vol. 21, No. 2, March 2016). [1]., (Image Quality Assess- ment: IQA) [2]., IQA, (objective functions) (prior knowledge), - [3], [4]., IQA [5]-[9]. (Full- Reference: FR) IQA, (Reduced-Reference: RR) IQA, (No-Reference: NR) IQA [10]. IQA(FR-IQA). (Mean Squared Error: MSE) FR-IQA, (metricability), (differentiability) (convexity). a) (Korea Advanced Institute of Science and Technology, School of Electrical Engineering) Corresponding Author : (Munchurl Kim) mkim@ee.kaist.ac.kr Tel: ORCID: () ( : 2014R1A2A2A ) Manuscript received January 19, 2016; Revised February 15, 2016; Accepted March 9, FR-IQA (Human Visual System: HVS), [1]., FR-IQA [10]-[20]. Wang SSIM(Structural SIMilarity) [11]. SSIM FR-IQA,.,, FR-IQA [19],[20]. FR-IQA. Brunet SSIM, SSIM (, -(quasi-convex) ), SSIM(Modified SSIM: MoSSIM) [19]. Xue (smoothing), PAMSE(Perceptual fidelity Aware Mean Squared Error) [20]. PAMSE - (weighted MSE)., MoSSIM PAMSE. FR-IQA (Simple Laplace operator-based Quality Metric: SLQM). FR-IQA (Human Visual System: HVS)

3 1: (Sung-Ho Bae et al.: A Novel Fast and High-Performance Image Quality Assessment Metric using a Simple Laplace Operator) FR-IQA.. 2 FR-IQA. 3 FR-IQA, 4, II.,,, FR-IQA.,. FR-IQA ( ). MoSSIM [19] PAMSE [20] MSE. MoSSIM PAMSE. 1. Modified SSIM(MoSSIM) Wang SSIM HVS, FR-IQA [11]. SSIM (luminance), (contrast) (structure). SSIM, (, ( 0) ), (similarity form). SSIM (1), (2).,,. (2) 0. (2), (1). SSIM. (3).,. SSIM FR-IQA, SSIM., SSIM,,. 1 (2) SSIM,. 1, SSIM,

4 (JBE Vol. 21, No. 2, March 2016). (partially ordered condition)., MoSSIM,, (Pearson Linear Correlation Coefficient, PLCC). 2. Perceptual Fidelity Aware Mean Squared Error(PAMSE) 1. SSIM ( ) Fig. 1. An example of the similarity form in SSIM for, Brunet FR-IQA SSIM(Modi- fied SSIM, MoSSIM), SSIM, [19]. MoSSIM SSIM (1) (2) (4), (5).,,. (5) 0. (5) (Normalized Mean Squared Error, NMSE), (quasi-convex) [19]. MoSSIM. MoSSIM, Xue MSE FR-IQA, MSE FR-IQA (Perceptual fidelity Aware Mean Squared Error, PAMSE) [20]. PAMSE MoSSIM. PAMSE (6).,. (6),. (6) (7)., (7) PAMSE. PAMSE, MoSSIM., (7) ( )

5 1: (Sung-Ho Bae et al.: A Novel Fast and High-Performance Image Quality Assessment Metric using a Simple Laplace Operator), PAMSE. III. Simple Laplace operator based Quality Metric(SLQM) ( ) FR-IQA. Simple Laplace operator based Quality Metric (SLQM). SLQM, Luv ( 4 )., SLQM Luv. =,, (luminance) (chrominance). SLQM 3. SLQM, (8). HVS., HVS 1 (retina) (cons and rods), (optical nerve) (visual cortex)., ( ) [20]. FR-IQA HVS, SLQM. (8) (divergence) MSE ( )., (9).. (7) MSE PAMSE SLQM (9).,., (DC ). PAMSE., SLQM DC,.,, MSE. HVS, ( ) 4 (factor) (down-sampling, color sub-sampling). (10).

6 (JBE Vol. 21, No. 2, March 2016), 4. SLQM 3., SLQM (11).,., SLQM (11) MSE,. SLQM ( ). V. SLQM SLQM., i) SLQM ii) SLQM (10) (11). 4 (TID2013 [21], TID2008 [22], CSIQ [18], LIVE [23] ). 1., 4., SROC(Spearman Rank-Order Correlation coefficient), KROC(Kendall Rank-Order Correlation coefficients), PLCC(Pearson Linear Correlation coefficient) RMSE(Root Mean Squared Error). SROC, KROC, PLCC IQA, FR-IQA. RMSE(Root Mean Squared Error) FR-IQA, 0 FR-IQA. FR-IQA -, [24] PLCC RMSE Table 1. Information of eight publicly available IQA databases Databases The number of reference images The number of distorted images The number of distortion types The number of subjects TID TID CSIQ LIVE

7 1: (Sung-Ho Bae et al.: A Novel Fast and High-Performance Image Quality Assessment Metric using a Simple Laplace Operator) 2. (HSV, XYZ, RGB, YCbCr, YIQ) SLQM Table 2. Performance of SLQM for five different color spaces (HSV, XYZ, RGB, YCbCr, YIQ) Dataset Measure SLQM HSV SLQM XYZ SLQM RGB SLQM YIQ SLQM YCbCr SLQM Luv TID 2013 TID 2008 CSIQ LIVE Overall SROC KROC PLCC RMSE SROC KROC PLCC RMSE SROC KROC PLCC RMSE SROC KROC PLCC RMSE SROC KROC PLCC , SROC KROC FR-IQA [10]., SLQM, 5,, XYZ, HSV, RGB, YCbCr, Luv. SLQM SLQM (, YIQ SLQM SLQM YIQ ). 2. Luv. SLQM Luv. IQA,., SLQM (9) MSE (10) MSE. (9) (SLQM L ) (10) (SLQM uv ). 3 Luv (11) (SLQM Luv ), (9) SLQM L, (10) SLQM uv., 3 (L) Laplace MSE (u,v) MSE. 3 (9) (10) SLQM. SLQM (9) MSE (10) MSE (11).,

8 (JBE Vol. 21, No. 2, March 2016) 3. Luv (SLQM Luv), (SLQM L), (SLQM uv) Table 3. Performance comparison of SLQM Luv, SLQM L and SLQM uv Dataset Measure SLQM L SLQM uv SLQM Luv TID 2013 TID 2008 CSIQ LIVE (Overall) SROC KROC PLCC RMSE SROC KROC PLCC RMSE SROC KROC PLCC RMSE SROC KROC PLCC RMSE SROC KROC PLCC MSE L, u, v (9) MSE. SLQM mod1., 4 IQA (TID2013, TID2008, CSIQ, LIVE) SLQM mod1 SROC = , KROC = , PLCC = MSE SLQM SROC, KROC, PLCC 2.8%-, 3.2%- 2.1%-., u v (10) MSE (9) MSE. IQA., ( Bayes ) ( ) ( ).. u v (10) MSE. VI. SLQM. FR-IQA SLQM

9 1: (Sung-Ho Bae et al.: A Novel Fast and High-Performance Image Quality Assessment Metric using a Simple Laplace Operator) 3 FR- IQA (PSNR, MoSSIM [19], PAMSE [20] ). MoSSIM, PAMSE., SLQM (10)., MoSSIM PAMSE (10). MoSSIM PAMSE MoSSIMc, PAMSEc. IQA 4 (LIVE, CSIQ, TID2008, TID2013), 1. 4 FR-IQA., 4 SLQM FR-IQA ( ) SROC, KROC. PSNR, SROC KROC FR-IQA. PSNR MSE, MSE HVS. PAMSE LIVE (SROC, KROC ). PAMSE LIVE (,, JPEG ) [20]. SROC, KROC SLQM PAMSE PSNR 12%~15%. MoSSIM, SROC, KROC 5%-, 7%-. FR-IQA FR-IQA ( 4. Table 4. Performance of FR-IQA methods under test Dataset Measure PAMSE PAMSEc MoSSIM MoSSIMc SLQM TID 2013 TID 2008 CSIQ LIVE Overall SROC KROC PLCC RMSE SROC KROC PLCC RMSE SROC KROC PLCC RMSE SROC KROC PLCC RMSE SROC KROC PLCC

10 (JBE Vol. 21, No. 2, March 2016) ), (10) PAMSEc MoSSIMc (PAMSE, MoSSIM) 4. PAMSEc PAMSE SROC KROC 3.3%-, 2.3%-. MoSSIMc. MSE (10) FR-IQA. SLQM PAMSE MoSSIM., PAMSE, PAMSEc, MoSSIM, MoSSIMc. SLQM. 24 GB RAM 3.2 GHz Intel i7 TM, MATLAB TM R2013a. TID2013, (frames per second, fps). 5 FR-IQA (fps). SLQM PSNR 3 FR-IQA. PAMSE SLQM, MoSSIM ( 1.8). 5 PSNR PAMSE. FR-IQA SLQM MoSSIM, SLQM PAMSE, MoSSIM, SLQM ( SROC) (fps). 2 MoSSIM PAMSE. 5. FR-IQA Table 5. Average running speeds of four FR-IQA methods FR-IQA methods fps PSNR PAMSE MoSSIM SLQM PAMSE, MoSSIM, SLQM() (SROC) (fps) Fig. 2. Comparisons of PAMSE, MoSSIM, SLQM in running speed (fps) and prediction performance (SROC) VII. FR-IQA. HVS FR-IQA., SLQM FR-IQA,.,.

11 1: (Sung-Ho Bae et al.: A Novel Fast and High-Performance Image Quality Assessment Metric using a Simple Laplace Operator) (References) [1] Z. Wang and A. C. Bovik, Mean squared error: Love it or leave it? A new look at signal fidelity measures, IEEE Signal Process. Mag., vol. 26, no. 1, pp , Jan [2] S.-H. Bae, J. Kim, M. Kim, S. H. Cho, and J. S. Choi, Assessments of subjective video quality on HEVC-encoded 4K-UHD video for beyond-hdtv broadcasting services, IEEE Trans. on Broadcast., vol. 59, no. 2, pp , Jun [3] J.-S. Choi, S.-H. Bae and M. Kim, Single image super-resolution based on self-examples using context-dependent subpatches, IEEE Int. Conf. on Image Proc, Sept , [4] J.-S. Choi, S.-H. Bae and M. Kim, A no-reference perceptual blurriness metric based fast super-resolution of still pictures using sparse representation, Proc. SPIE, vol. 9401, pp N N.7, Mar [5] J. Kim, S.-H. Bae, and M. Kim, An HEVC-compliant perceptual video coding scheme based on JND models for variable block-sized transform kernels, IEEE Trans. Circuits Syst. Video Technol., vol. 25, no. 11, pp , Sept [6] S.-H Bae and M. Kim, A novel DCT-based JND model for luminance adaptation effect in DCT frequency, IEEE Signal Process. Lett., vol. 20, no. 9, pp , Sept [7] S.-H. Bae and M. Kim, A new DCT-based JND model of monochrome images for contrast masking effects with texture complexity and frequency, IEEE Int. Conf. on Image Proc, Melborne, Australia, Sept , pp , [8] S.-H Bae and M. Kim, A novel generalized DCT-based JND profile based on an elaborate CM-JND model for variable block-sized transforms in monochrome images, IEEE Trans. on Image Process., vol. 23, no. 8, Aug [9] S.-H. Bae and M. Kim, A DCT-based Total JND Profile for Spatio-Temporal and Foveated Masking Effects, IEEE Trans. Circuits Syst. Video Technol., to appear, [10] L. Zhang, L. Zhang, X. Mou, and D. Zhang, A comprehensive evaluation of full reference image quality assessment algorithms, Proc. 19th IEEE Int. Conf. Image Process., pp , Sep./Oct [11] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Trans. on Image Process., vol. 13, pp , Apr [12] Z. Wang and Q. Li, Information content weighting for perceptual image quality assessment, IEEE Trans. Image Process., vol. 20, no. 5, pp , May [13] Z. Wang, E. P. Simoncelli, and A. C. Bovik, Multiscale structural similarity for image quality assessment, Proc. 37th Asilomar Conf. Signals, Syst., Comput., pp , Nov [14] S.-H. Bae, M. Kim, A Novel SSIM Index for Image Quality Assessment using a New Luminance Adaptation Effect Model in Pixel Intensity Domain, IEEE Video Comm. and Image Proc., Dec , [15] S.-H. Bae and M. Kim, A novel image quality assessment based on an adaptive feature for image characteristics and distortion types, IEEE Video Comm. and Image Proc., Dec , [16] S.-H. Bae and M. Kim, Elaborate Image Quality Assessment with a Novel Luminance Adaptation Effect Model, Journal of Broadcast Engineering, vol. 20, no. 6, pp. 1-10, Nov [17] S.-H. Bae and M. Kim, A Novel Image Quality Assessment with Globally and Locally Consilient Visual Quality Perception, IEEE Trans. on Image Process., to appear, [18] E. C. Larson and D. M. Chandler, Most apparent distortion: Full-reference image quality assessment and the role of strategy, J. Electron. Imag., vol. 19, no. 1, pp : :21, Jan [19] D. Brunet, E. R Vrscay, and Z. Wang. On the mathematical properties of the structural similarity index, IEEE Trans. Image Process., vol. 21, no.4, pp , Oct [20] W. Xue, X. Mou, L. Zhang, X. Feng, Perceptual fidelity aware mean squared error, Proc. IEEE Int. Conf. Computer Vision, Dec. 2013, pp [21] N. Ponomarenko et al., Color image database TID2013: Peculiarities and preliminary results, Proc. 4th Eur. Workshop Vis. Inf. Process., Jun. 2013, pp [22] N. Ponomarenko, V. Lukin, A. Zelensky, K. Egiazarian, M. Carli, and F. Battisti, TID2008-A database for evaluation of full-reference visual quality assessment metrics, Adv. Modern Radioelectron., vol. 10, pp. 3045, [23] H.R. Sheikh, M.F. Sabir, and A.C. Bovik, A statistical evaluation of recent full reference image quality assessment algorithms, IEEE Trans. Image Process., vol. 15, no. 11, pp , Nov [24] Final Report From the Video Quality Experts Group on the Validation of Objective Models of Video Quality Assessment VQEG. Available:

(JBE Vol. 21, No. 2, March 2016) - 2011 2 : - 2012 8 : - 2012 9 ~ : - ORCID : http://orcid.

of Electrical and Computer Engineering, - 1996 8 : University of Florida, Dept.

12 (JBE Vol. 21, No. 2, March 2016) : : ~ : - ORCID : - : Human Visual Perception based Computational Vision : : University of Florida, Dept. of Electrical and Computer Engineering, : University of Florida, Dept. of Electrical and Computer Engineering, ~ :, ~ : / ~ : / - ORCID : - : Perceptual Video Coding, SDR/HDR Image/Video Quality Assessment and Modeling, Super-Resolution, Image/Video Analysis and Understanding, Pattern Recognition, Machine Learning

(JBE Vol. 20, No. 6, November 2015) (Special Paper) 20 6, (JBE Vol. 20, No. 6, November 2015) ISSN

(JBE Vol. 20, No. 6, November 2015) (Special Paper) 20 6, (JBE Vol. 20, No. 6, November 2015) ISSN (JBE Vol. 0, No. 6, November 05) (Special Paper) 0 6, 05 (JBE Vol. 0, No. 6, November 05) http://d.doi.org/0.5909/jbe.05.0.6.88 ISSN 87-937 (Online) ISSN 6-7953 (Print) a), a) Elaborate Image Quality Assessment