3 : S-JND HEVC (JaeRyun Kim et al.: A Perceptual Rate Control Algorithm with S-JND Model for HEVC Encoder) (Regular Paper) 21 6, (JBE Vol. 21,

Transcription

1 3 : S-JND HEVC (Regular Paper) 21 6, (JBE Vol. 21, No. 6, November 2016) ISSN (Online) ISSN (Print) S-JND HEVC a), a), a), a) A Perceptual Rate Control Algorithm with S-JND Model for HEVC Encoder JaeRyun Kim a), Yong-Jo Ahn a), Woong Lim a), and Donggyu Sim a) S-JND. S-JND (Saliency-Just Noticeable Difference). CTU (Coding Tree Unit) S-JND threshold. CTU threshold,. HM 16.9, CTC (Common Test Condition) RA (Random Access), Low-delay B Low-delay P Class B Class C., 2.3% BD-PSNR 0.07dB 0.06%. DSCQS (Double Stimulus Continuous Quality Scale), 0.03 MOS (Mean Opinion Score). Abstract This paper proposes the rate control algorithm based on the S-JND (Saliency-Just Noticeable Difference) model for considering perceptual visual quality. The proposed rate control algorithm employs the S-JND model to simultaneously reflect human visual sensitivity and human visual attention for considering characteristics of human visual system. During allocating bits for CTU (Coding Tree Unit) level in a rate control, the bit allocation model calculates the S-JND threshold of each CTU in a picture. The threshold of each CTU is used for adaptively allocating a proper number of bits; thus, the proposed bit allocation model can improve perceptual visual quality. For performance evaluation of the proposed algorithm, the proposed algorithm was implemented on HM 16.9 and tested for sequences in Class B and Class C under the CTC (Common Test Condition) RA (Random Access), Low-delay B and Low-delay P case. Experimental results show that the proposed method reduces the bit-rate of 2.3%, and improves BD-PSNR of 0.07dB and bit-rate accuracy of 0.06% on average. We achieved MOS improvement of 0.03 with the proposed method, compared with the conventional method based on DSCQS (Double Stimulus Continuous Quality Scale). Keyword : HEVC, S-JND, Rate control, Perceptual quality Copyright 2016 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND ( which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.

2 (JBE Vol. 21, No. 6, November 2016). Full HD 4K 8K. ISO/IEC MPEG (Moving Picture Experts Group) ITU-T VCEG (Video Coding Experts Group) JCT-VC (Joint Collabo- rative Team on Video Coding). JCT- VC H.264/AVC (Advanced Video Coding) 50%, 40% HEVC (High Effi- ciency Video Coding) version 1 [1][2][3][4]. Apple iphone 6 iphone 6 Plus HEVC FaceTime, HEVC 4K UHD TV HEVC. HEVC [5]. HEVC (Rate control) [6].. HEVC, (RDO: Rate-Distortion Optimization) [7]. a) (Dept. of Computer Engineering, Kwangwoon University) Corresponding Author : (Donggyu Sim) dgsim@kw.ac.kr Tel: ORCID: ( ) (NRF-2014R1A2A1A ) Manuscript received August 4, 2016; Revised September 13, 2016; Accepted September 20, 2016.,.,.,,. HEVC HM (HEVC reference Model). PSNR (Peak Signal to Noise Ratio), MSE (Mean Squared Error), HM PSNR., HM, [8][9][10][11]. S-JND (Saliency-Just Noticeable Difference) [8]. S-JND. S-JND CTU S-JND threshold, threshold CTU., CTU CTU CTU. CTU HEVC.. 2 HEVC, 3

3 3 : S-JND HEVC HEVC.. HEVC HM [12].. [13]. HM PSNR. HM. 1 HM.,.,,,.,. QP (Quantization Parameter). QP.,.., 1. HM Fig. 1. Rate control algorithm of HM

4 (JBE Vol. 21, No. 6, November 2016) H.264/AVC JM (Joint Model) HEVC HM [6][14]. HEVC. [15] URQ (Unified Rate-Quantization) QP [16]. HM [12] R-λ, λ QP [17]. R-λ GOP (Group Of Pictures), (Frame), CTU (Coding Tree Unit). GOP. GOP, CTU. CTU λ QP., R-λ (1).,,. λ (Lagrangian multiplier), HM.. R-λ (2).,. MSE. (2) λ., λ, (3). α β, (3) λ. λ (4) CTU QP. ln R-λ λ QP CTU, α β.. S-JND, HEVC HM PSNR... S-JND. S-JND, S-JND. HEVC.

5 3 : S-JND HEVC 1. S-JND (Saliency-Just Noticeable Difference) S-JND [8]. S-JND. [14]..., (Luminance adaptation).,. JND (Just Noticeable Difference) [18][19]. JND luminance JND value JND threshold,. 2., (Region of interest)..,, (Attention)., (Motion sensitivity). Saliency. S-JND JND Saliency. S-JND (5). JND threshold 2 luminance JND value Saliency 0 1. JND threshold Saliency. S-JND JND Saliency JND threshold S-JND threshold, (5). Saliency JND threshold, offset (5) α. α 0.5. S-JND threshold S-JND 2. JND Fig. 2. A graph of JND threshold depending on average background luminance HEVC. 3

6 (JBE Vol. 21, No. 6, November 2016). HM, S-JND. S-JND threshold, S-JND threshold. 4. 4, CTU. 3. S-JND Fig. 3. Proposed rate control algorithm based on S-JND model 4. Fig. 4. Overall flowchart of the proposed rate control algorithm

7 3 : S-JND HEVC S-JND CTU CTU S-JND threshold. S-JND threshold CTU CTU. (6), CTU CTU S-JND threshold S-JND threshold. CTU S-JND threshold, S-JND threshold. CTU (7). CTU, S-JND threshold CTU S-JND threshold., CTU S-JND threshold S-JND threshold. (7) CTU CTU (8) (9). (8) CTU., (9) CTU. (9) CTU CTU (10)., CTU CTU., HM 10%. 10% 10%. CTU, R-λ λ. λ (4) QP. CTU.. HEVC HM HEVC CTC (Common Test Condition) [20] RA (Random Access), Low-delay B, Low-delay P, CTC Class B Class C.

8 (JBE Vol. 21, No. 6, November 2016) 1. HEVC Table 1. Information about HEVC test sequences used for performance evaluation of proposed algorithm Class Resolution Sequence name Number of frame Frame rate (fps) Bit-depth Random Access B C Kimono ParkScene Cactus BasketballDrive BQTerrace RaceHorses BQMall PartyScene BasketballDrill Main 1 Class B Class C CTC RA, R1, R2, R3, R RA. 3 Anchor HM 16.9, Proposed HM BD-rate,. BD-rate 2.3%., %. BD-PSNR [21]. PSNR Table 2. Target bitrate and bitrate point for each test sequence Class (Resolution) Class B ( ) Class C ( ) Sequence name Kimono ParkScene Cactus BasketballDrive BQTerrace RaceHorses BQMall PartyScene BasketballDrill Target bitrate point 4868 R R R3 535 R R R R3 712 R R R R R R R R R R R R3 987 R R R R R R R R R R R R R R R R R4

9 3 : S-JND HEVC 3. RA Table 3. Comparing encoding performance with random access structure between conventional and proposed algorithm Class (Resolution) Class B ( ) Class C ( ) Sequence Kimono ParkScene Cactus Basketball Drive BQTerrace RaceHorses BQMall PartyScene Basketball Drill point Anchor Proposed BDrate (%) R BD- PSNR Bit accuracy (%) Time saving (sec) R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R Average dB,. S-JND threshold. S-JND threshold QP, S-JND threshold QP.

10 (JBE Vol. 21, No. 6, November 2016), Low-delay B. Low-delay B, BD-rate 0.26%, BD-PSNR 0.02dB, 0.33%,,. 4. LD-B Table 4. Comparing encoding performance with low-delay B structure between conventional and proposed algorithm Class (Resolution) Class B ( ) Class C ( ) Sequence Kimono ParkScene Cactus Basketball Drive BQTerrace RaceHorses BQMall PartyScene Basketball Drill point Anchor Proposed BDrate (%) R BD- PSNR Bit accuracy (%) R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R Average

11 3 : S-JND HEVC 5. LD-P Table 5. Comparing encoding performance with low-delay P structure between conventional and proposed algorithm Class (Resolution) Class B ( ) Class C ( ) Sequence Kimono ParkScene Cactus Basketball Drive BQTerrace RaceHorses BQMall PartyScene Basketball Drill point Anchor Proposed BDrate (%) BD- PSNR Bit accuracy (%) R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R Average Low-delay P, BD-rate 0.27%, BD-PSNR 0.01dB. 0.29%., 6 JND [22]. RA, [22]., [22] GOP

12 (JBE Vol. 21, No. 6, November 2016) 6. RA JND Table 6. Comparing encoding performance with random access structure between rate control based on JND and proposed algorithm Class (Resolution) Class B ( ) Sequence Kimono ParkScene Cactus Basketball Drive BQTerrace point JND based RC Proposed BDrate (%) R BD- PSNR Bit accuracy (%) R R R R R R R R R R R R R R R R R R R Average , [22] JND, S-JND. 7. Table 7. Experimental information for subjective quality assessment Display Samsung 47LG50FD Type and size 47 inch 16:9 Resolution UHD Number of subjects 11 Viweing distance 1m (0.75H). ITU-R BT.500 [23] DSCQS (Double Stimulus Continuous Quality Scale) MOS (Mean Opinion Score)., Class B CTC RA. 5. MOS, 0.03 MOS., MOS,. 10%., 10%

13 3 : S-JND HEVC 5. MOS Fig. 5. MOS comparison result between two encoded sequences of conventional rate control and proposed rate control,.. S-JND. S-JND CTU. HM 16.9 CTC RA Class B Class C, 2.3% BD-PSNR 0.07dB 0.06%. Low-delay B Low-delay P.,, 0.03 MOS.,.. (References) [1] G. J. Sullivan, J. Ohm, W. J. Han and T. Wiegand, Overview of the High Efficiency Video Coding (HEVC) standard, in Circuits and Systems for Video Technology, IEEE Transactions on, vol. 22, no. 12, pp , Dec [2] H. H. Jo, E. K. Ryu, J. H. Nam, D. G. Sim, D. H. Kim and J. H. Song, Parallel method for HEVC deblocking filter based on coding unit depth information, Journal of Broadcast Engineering, vol. 17, no. 5, pp , Sep [3] Y. J. Ahn, T. J. Hwang, D. K. Lee, S. M. Kim, S. J. Oh and D. G. Sim,

14 (JBE Vol. 21, No. 6, November 2016) Study of parallelization methods for real-time HEVC encoder implementation, Journal of Broadcast Engineering, vol. 18, no. 6, pp , Nov [4] Y. J. Ahn and D. G. Sim, Intra block copy analysis to improve coding efficiency for HEVC screen content coding, Journal of Broadcast Engineering, vol. 20, no. 1, pp , Jan [5] Y. J. Ahn, S. J. Park and D. G. Sim, Implementation and commercialization techniques of HEVC codec, The Journal of Korea Institute of Electronics Engineers, vol. 41, no. 19, pp , Oct [6] D. G. Sim and H. H. Jo, Understanding of HEVC standard and technology, HongRung Publishing Company, Korea, [7] Q. Lin and G. Feng, The bit allocation and RDO mode based rate control algorithm, 2010 International Conference on Anti-Counterfeiting Security and Identification in Communication (ASID), pp , Jul [8] J. Kim, W. Lim, J. Y. Lim and D. G. Sim, A perceptual rate control algorithm based on S-JND model, in Winter Conference on The Korean Institute of Communications and Information Sciences, Jan [9] W. Lim, H. Jo and D. Sim, Perceptual coding based on JND model for High Efficiency Video Coding, in International Conference on 3 Dimensional Systems and Applications, Seoul, Korea, [10] Guan-Lin Wu, Yu-Jie Fu, Sheng-Chieh Huang and Shao-Yi Chien, Perceptual quality-regulable video coding system with region-based rate control scheme, IEEE Transactions on Image Processing, vol. 22, no. 6, Jun [11] X. Wang, L. Su, Q. Huang, C. Liu and L. Duan, Motion based perceptual distortion and rate optimization for video coding, 2012 IEEE International Conference on Multimedia and Expo. (ICME), pp , Jul [12] B. Li, H. Li, L. Li and J. Zhang, Rate control by R-lambda model for HEVC, JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/ WG11, JCTVC-K0103, Shanghai, China, Oct [13] G. J. Sullivan and R. Wiegand, Rate-distortion optimization for video compression, IEEE Signal Processing Magazine, vol. 15, no. 6, pp , Nov [14] W. Lim and D. G. Sim, Coding unit-level multi-loop encoding method based on JND for perceptual coding, Journal of the Institute of Electronics and Information Engineers, vol. 52, no. 5, pp , May [15] H. Choi, J. Nam, J. Yoo, D. Sim and I. V. Bajić, Rate control based on unified RQ model for HEVC, JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, JCTVC-H0213, San Jose, CA, Jan [16] H. Choi, J. Yoo, J. Nam, D. Sim and I. V. Bajic, Pixel-wise unified rate-quantization model for multi-level rate control, IEEE Journals & Magazines, vol. 7, no. 6, pp , [17] B. Lin, H. Li, L. Li and J. Zhang, Lambda domain rate control algorithm for High Efficiency Video Coding, IEEE Transactions on Image Processing, vol. 23, no. 9, pp , [18] C. Chou and Y. Li, A perceptually tuned subband image coder based on the measure of Just-Noticeable-Distortion profile, in Circuits and Systems for Video Technology, IEEE Transactions on, vol. 5, no. 6, pp , Dec [19] Z. Wei and K. N. Ngan, Spatio-temporal Just Noticeable Distortion profile for grey scale image/video in DCT domain, in IEEE Transactions on Circuits and Systems for Video Technology, vol. 19, no. 3, pp , Mar [20] F. Bossen, Common test conditions and software reference configurations, JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/ WG11, JCT-VC H1100, San Jose, CA, Jan [21] G. Bjøntegaard, Calculation of average PSNR differences between RD-curves, ITU-T SG16/Q.6, VCEG-M33, Austin, TX, Apr [22] J. Kim, W. Lim and D. G. Sim, Rate control encoding method of HEVC for improving perceptual visual quality, in Summer Conference on The Institute of Electronics and Information Engineers, June [23] ITU-R BT , Methodology for the subjective assessment of the quality of television pictures, Jan : ~ : - ORCID : - :,,

15 김재련 외 인 모델을 사용한 주관적인 율 제어 알고리즘 기반의 부호화 방법 : S-JND HEVC 저자소개 안용조 - 년 2월 : 광운대학교 컴퓨터공학과 학사 년 2월 : 광운대학교 컴퓨터공학과 석사 년 8월 : 광운대학교 컴퓨터공학과 박사 : 주관심분야 : 영상압축, 최적화 및 병렬화 ORCID 임웅 - 년 2월 : 광운대학교 컴퓨터공학과 학사 년 2월 : 광운대학교 컴퓨터공학과 석사 년 2월 : 광운대학교 컴퓨터공학과 박사 년 4월 ~ 현재 : 한국전자통신연구원(ETRI) 방송통신융합연구부문 연구원 : 주관심분야 : 영상압축, 컴퓨터비전, 영상신호처리 ORCID 심동규 - 년 2월 : 서강대학교 전자공학과 공학사 년 2월 : 서강대학교 전자공학과 공학석사 년 2월 : 서강대학교 전자공학과 공학박사 년 3월 ~ 2000년 8월 : 현대전자 선임연구원 년 9월 ~ 2002년 3월 : 바로비젼 선임연구원 년 4월 ~ 2005년 2월 : University of Washington Senior research engineer 년 3월 ~ 현재 : 광운대학교 컴퓨터공학과 교수 : 주관심분야 : 영상신호처리, 영상압축, 컴퓨터비전 ORCID