1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC) (Regular Paper) 22 3, (JBE Vol. 22, No. 3, May 2017)

Transcription

1 1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC) (Regular Paper) 22 3, (JBE Vol. 22, No. 3, May 2017) ISSN (Online) ISSN (Print) S-JND HEVC a), a) S-JND based Perceptual Rate Control Algorithm of HEVC JaeRyun Kim a) and Donggyu Sim a) HEVC (High Efficiency Video Coding). HEVC. CTU,, CTU. HEVC 16.9 CTC (Common Test Condition) Class B BD-rate 3.12% BD-PSNR 0.08dB 0.07%. HEVC DSCQS Abstract In this paper, the perceptual rate control algorithm is studied for HEVC (High Efficiency Video Coding) encoder with bit allocation based on perceived visual quality. This paper proposes perceptual rate control algorithm which could consider perceived quality for HEVC encoding method. The proposed rate control algorithm employs adaptive bit allocation for frame and CTU level using the perceived visual importance of each CTU. For performance evaluation of the proposed algorithm, the proposed algorithm was implemented on HM 16.9 and tested for sequences in Class B under the CTC (Common Test Condition) RA (Random Access) case. Experimental results show that the proposed method reduces the bitrate of 3.12%, and improves BD-PSNR of 0.08dB and bitrate accuracy of 0.07% on average. And also, we achieved MOS improvement of 0.16 with the proposed method, compared with the conventional method based on DSCQS (Double Stimulus Continuous Quality Scale). Keyword : HEVC, Perceptual rate control, S-JND a) (Dept. of Computer Engineering, Kwangwoon University) Corresponding Author : (Donggyu Sim) dgsim@kw.ac.kr Tel: ORCID: ( ) (NRF-2014R1A2A1A ). Manuscript received January 20, 2017; Revised March 13, 2017; Accepted April 13, Copyright 2017 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND ( which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.

2 (JBE Vol. 22, No. 3, May 2017). 4G.,,, VoIP,, Cisco [1]. Full HD 4K 8K, H.264/AVC (Advanced Video Coding) [2]. ISO/IEC MPEG (Moving Picture Experts Group) ITU-T VCEG (Video Coding Experts Group) JCT-VC (Joint Collaborative Team on Video Coding). JCT-VC H.264/AVC 50% HEVC (High Efficiency Video Coding) [3][4]. HEVC. H.264/AVC (MB: Macro Block), HEVC (CU: Coding Unit),, (PU: Prediction Unit) (TU: Transform Unit) [5]. HEVC 35 H.264/AVC [6], 8-tap, (Merge) AMVP (Advanced Motion Vector Prediction) [7]., H.264/AVC HEVC (Deblocking filter) (SAO: Sample Adaptive Offset) (Blocking artifact) (Ringing artifact) [8]. H.264/AVC HEVC, HEVC. Apple iphone 6 iphone 6 Plus HEVC Facetime, HEVC, HEVC. HEVC [9]. HEVC. HEVC HM (HEVC reference Model) [10] (Rate control algorithm).,, [11][18]. HM (RDO: Rate-Distortion Optimization) [12].,. PSNR (Peak Signal to Noise Ratio), MSE (Mean Squared Error), MOS (Mean Opinion Score)., HM PSNR.

3 1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC) [13][14][15][16][17].. S-JND (Saliency-Just Noticeable Di- fference). S-JND JND (Just Noticeable Difference) Saliency,, [16]. S-JND... S-JND HEVC.. 2 HM S-JND, 3. 4 HM, 5.. HM S-JND, HEVC H.264/AVC HEVC H.264/AVC 50% 2. HEVC [9]. H.264/AVC HEVC,, 8-tap, AMVP, (In-loop filter). H.264/AVC 50%. 1 HEVC HM. 2 S-JND. 1. HM HEVC HEVC HM... HEVC,.,,.,..,

4 (JBE Vol. 22, No. 3, May 2017)., SAD (Sum of Absolute Difference), SSE (Sum of Squared Error), MSE PSNR.,. HEVC PSNR., MOS., ITU-R (International Telecommunication Union Radiocommunication sector) BT [19]. BT , DSIS (Double Stimulus Im- pairment Scale) DSCQS (Double Stimulus Continuous Quality Scale).,,., [13][14][15][16][17]., HM PSNR. HM. 1 HM. (Input sequence), (Output bit- stream),., (Target bit-rate), 1. HM Fig. 1. Block diagram of rate control algorithm for HM

5 1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC) (HRD: Hyperthetical Reference Decoder) (Buffer constraint parameter) (Occurred bit-rate), (Reconstructed sequence)., (Complexity calculation).,. (QP: Quantization Parameter).. 1,. HEVC URQ (Unified Rate-Quantization) [20][21], λ R-λ [22]. HEVC HM [22] R-λ, GOP (Group Of Pictures), (Frame), CTU (Coding Tree Unit)., GOP GOP, GOP GOP. CTU. CTU λ. R-λ (1). (1),, (Lagrangian multiplier).. HM, (Rate-Distortion cost). (1) (2). (2) MSE., (2) λ. (3). (3), λ., (4) λ CTU QP. ln R-λ λ QP CTU., CTU. R-λ. 2. S-JND (Saliency-Just Noticeable Difference)

6 (JBE Vol. 22, No. 3, May 2017) S-JND. HEVC.,, -,.,,. (Luminance adaptation) Saliency [17]., JND. 2 JND threshold, JND threshold.,.. (ROI: Region Of Interest).,,, (Motion sensitivity).. S-JND [16]. S-JND JND Saliency, (5). JND threshold Saliency 0 1. JND threshold Saliency, S-JND S-JND threshold. S-JND S-JND threshold (5), JND threshold JND threshold Saliency 1. JND threshold Saliency, offset., 0.5. (5) S-JND threshold JND JND threshold Fig. 2. A graph for relation of average background luminance and JND threshold

7 1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC).. S-JND HEVC. CTU, CTU.,. 1, S-JND HEVC. 3. HM, S-JND. S-JND threshold, S-JND threshold..,,.,. [16] CTU, CTU, CTU. S-JND S-JND threshold, 3. Fig. 3. Block diagram of proposed rate control algorithm

8 (JBE Vol. 22, No. 3, May 2017) 3.2. GOP CTU. GOP R-λ,,. CTU, CTU.,,. CTU, GOP, R-λ GOP. 2., GOP 4. Fig. 4. Proposed target bit decision model for frame level

9 1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC),., 4. 4., GOP, GOP.. [16],. GOP. GOP (9). (9) GOP. GOP, (10). (6) CTU CTU S-JND threshold., (7) (6) CTU,. CTU GOP, GOP. (8) GOP., [16] CTU. GOP (10) GOP. (10). GOP GOP, 10%. CTU, (11) (12). (11) (12) [16] CTU, (9), (10),

10 (JBE Vol. 22, No. 3, May 2017)... HEVC HEVC HM Class B CTC (Common Test Condition) [25] RA (Random Access). 1. RA Low-delay All intra. 1. Table 1. Experimental environments CPU OS RAM Intel (R) Core (TM) i7-3960x CPU 3.30GHz Windows 7 Ultimate K 16.0 GB CTC Class Class B, 2 Class B. 2. CTC Table 2. Information of CTC test sequences used for performance evaluation of proposed algorithm Class Resolution Sequence name Number of frame B Frame rate (fps) Kimono ParkScene Cactus BasketballDrive BQTerrace Bit-depth 3 8. CTC RA, BP1, BP2, BP3, BP Table 3. Target bit-rate and bit-rate point Class (Resolution) Class B ( ) Sequence name Kimono ParkScene Cactus BasketballDrive BQTerrace Target bit-rate (Kbps) Bit-rate point 4868 BP BP BP3 535 BP BP BP BP3 712 BP BP BP BP BP BP BP BP BP BP BP BP3 987 BP4.,. 4 2 HM 16.9 CTU, RA.

11 1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC) 4. Table 4. Comparing encoding performance between conventional and proposed rate control algorithm Sequence Kimono ParkScene Cactus BasketballDrive BQTerrace Bitrate point Anchor 1 Proposed BDrate Bitrate PSNR-Y Bitrate PSNR-Y (Kbps) (db) (Kbps) (db) (%) BP BD- PSNR (db) Bit accuracy (%) BP BP BP BP BP BP BP BP BP BP BP BP BP BP BP BP BP BP BP Average CTU. 4 Anchor 1 HM 16.9, Proposed HM 16.9 CTU. BD-rate,. 4, BD-rate 3.12%, 0.07%. BD-PSNR [26]. PSNR. PSNR, 0.08dB. S-JND threshold. S-JND threshold QP, S-JND threshold QP. BD-rate., BD-PSNR.,, BD-rate BD-PSNR.

12 (JBE Vol. 22, No. 3, May 2017) 5. [16] CTU Table 5. Comparing encoding performance between conventional algorithm for CTU level ([16]) and proposed rate control algorithm Sequence Kimono ParkScene Cactus BasketballDrive BQTerrace Bitrate point Bitrate (Kbps) Anchor 2 PSNR-Y (db) Bitrate (Kbps) Proposed PSNR-Y (db) BP BDrate (%) BD- PSNR (db) Bit accuracy (%) BP BP BP BP BP BP BP BP BP BP BP BP BP BP BP BP BP BP BP Average Anchor 2 [16] CTU, Proposed 4 HM 16.9 CTU. BD-rate [16] CTU 0.26%., 5 [16] 0.01%. 4 PSNR, 0.01dB,. 6 [15] JND. RC based on JND JND [15], Proposed HM 16.9., JND., JND, [15],.

13 1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC) 6. [15] JND RA Table 6. Comparing encoding performance with random access structure between rate control based on JND ([15]) and proposed algorithm Sequence Kimono ParkScene Cactus BasketballDrive BQTerrace Bitrate point RC based on JND Bitrate (Kbps) PSNR-Y (db) Bitrate (Kbps) Proposed PSNR-Y (db) BP BDrate (%) BD- PSNR (db) Bit accuracy (%) BP BP BP BP BP BP BP BP BP BP BP BP BP BP BP BP BP BP BP Average Table 7. Experimental information for subjective quality assessment Display Samsung 47LG50FD Type and size 47 inch 16:9 Resolution UHD Number of subjects 11 Viewing distance 1m (0.75H) ITU-R BT [19] DSCQS, MOS HM 16.9.,. 5, MOS, 0.16 MOS., MOS,.

14 (JBE Vol. 22, No. 3, May 2017) 5. Fig. 5. Experimental result of perceived visual quality for conventional and proposed rate control method 10%., 10%,.. S-JND. S-JND, CTU. HM 16.9 CTC RA Class B, 3.12% BD-PSNR 0.08dB 0.07%., DSCQS, 0.16 MOS.,.. (References) [1] Cisco, Cisco visual networking index: Global mobile data traffic forecast update, Cisco White Paper, Feb [2] T. Wiegand, G. J. Sullivan, G. Bjontegaard and A. Luthra, Overview of the H.264/AVC video coding standard, IEEE transactions on Circuits and Systems for Video Technology, vol. 13, no. 7, pp , Jul [3] B. Bross, W. J. Han, G. J. Sullivan, J. R. Ohm and T. Wiegand, High

15 1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC) Efficiency Video Coding (HEVC) text specification draft 10 for FDIS & last call, Joint Collaborative Team on Video Coding (JCT-VC), JCTVC-L1003_v34, Geneva, CH, Jan [4] J. R. Ohm, G. J. Sullivan, H. Schwarz, T. K. Tan and T. Wiegand, Comparison of the coding efficiency of video coding standards-including High Efficiency Video Coding (HEVC), IEEE transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp , Dec [5] I. K. Kim, J. Min, T. Lee, W. J. Han and J. Park, Block partitioning structure in the HEVC standard, IEEE transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp , Dec [6] J. Lainema, F. Bossen, W. J. Han, J. Min and K. Ugur, Intra coding of the HEVC standard, IEEE transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp , Dec [7] G. Sullivan, J. Ohm, W. Han and T. Wiegand, Overview of the High Efficiency Video Coding (HEVC) standard, IEEE transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp , Dec [8] A. Norkin, G. Bjøntegaard, A. Fuldseth, M. Narroschke, M. Ikeda, K. Andersson, M. Zhou and G. Auwera, HEVC deblocking filter, IEEE transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp , Dec [9] Y. J. Ahn, S. J. Park and D. G. Sim, Implementation and commercialization techniques of HEVC codec, The Magazine of the Institute of Electronics and Information Engineers, vol. 41, no. 19, pp , Oct [10] HM 16.9 S/W. [11] D. G. Sim and H. H. Jo, Understanding of HEVC standard and technology, HongRung Publishing Company, Korea, [12] Q. Lin and G. Feng, The bit allocation and RDO mode based rate control algorithm, 2010 International Conference on Anti-Counterfeiting Security and Identification in Communication (ASID), Chengdu, China, pp , Jul [13] C. Chou and Y. Li, A perceptually tuned subband image coder based on the measure of Just-Noticeable-Distortion profile, IEEE transactions on Circuits and Systems for Video Technology, vol. 5, no. 6, pp , Dec [14] Z. Wei and K. N. Ngan, Spatio-temporal Just Noticeable Distortion profile for grey scale image/video in DCT domain, IEEE transactions on Circuits and Systems for Video Technology, vol. 19, no. 3, pp , Mar [15] W. Lim and D. G. Sim, Coding unit-level multi-loop encoding method based on JND for perceptual coding, Journal of the Institute of Electronics and Information Engineers, vol. 52, no. 5, pp , May [16] J. Kim, Y. J. Ahn, W. Lim and D. G. Sim, A perceptual rate control algorithm with S-JND model for HEVC encoder, Journal of Broadcast Engineering, vol. 21, no. 6, pp , Nov [17] S. Goferman, L. Zelnik-Manor, A. Tal, Context-aware saliency detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 10, pp , Oct [18] W. Lim, J. Nam and D. Sim, Scalable multi-view video coding based on HEVC, IEIE transactions on Smart Processing and Computing, vol. 4, no. 6, pp , Dec [19] Recommendation ITU-R BT , Methodology for the subjective assessment of the quality of television pictures, Recommendation ITU-R BT , ITU-Telecommunication Standardization Sector, no. 22, pp , Jan [20] H. Choi, J. Nam, J. Yoo, D. Sim and I. V. Bajić, Rate control based on unified RQ model for HEVC, Joint Collaborative Team on Video Coding (JCT-VC), JCTVC-H0213, San Jose, CA, Jan [21] H. Choi, J. Yoo, J. Nam, D. Sim and I. V. Bajić, Pixel-wise Unified Rate-Quantization model for multi-level rate control, IEEE Journal of Selected Topics in Signal Processing, vol. 7, no. 6, pp , Dec [22] B. Li, H. Li, L. Li and J. Zhang, Rate control by R-lambda model for HEVC, Joint Collaborative Team on Video Coding (JCT-VC), JCTVC-K0103, Shanghai, China, Oct [23] V. Sze, M. Budagavi and G. J. Sullivan, High Efficiency Video Coding (HEVC): algorithms and architectures, Springer International Publishing, Sep [24] Mathias Wien, High Efficiency Video Coding (HEVC): coding tools and specification, Springer International Publishing, Sep [25] F. Bossen, Common test conditions and software reference configurations, Joint Collaborative Team on Video Coding (JCT-VC), JCTVC-L1100, Geneva, CH, Jan [26] G. Bjøntegaard, Calculation of average PSNR differences between RD-curves, ITU-Telecommunications Standardization Sector, VCEG-M33, pp , Austin, TX, Apr

16 396 방송공학회논문지 제22권 제3호, 2017년 5월 (JBE Vol. 22, No. 3, May 2017) 저자소개 김재련 년 월 광운대학교 컴퓨터공학과 학사 년 월 광운대학교 컴퓨터공학과 석사 주관심분야 영상신호처리 영상압축 컴퓨터비전 : : - ORCID : :,, 심동규 - 년 2월 : 서강대학교 전자공학과 공학사 년 2월 : 서강대학교 전자공학과 공학석사 년 2월 : 서강대학교 전자공학과 공학박사 년 3월 ~ 2000년 8월 : 현대전자 선임연구원 년 9월 ~ 2002년 3월 : 바로비젼 선임연구원 년 4월 ~ 2005년 2월 : University of Washington Senior research engineer 년 3월 ~ 현재 : 광운대학교 컴퓨터공학과 교수 : 주관심분야 : 영상신호처리, 영상압축, 컴퓨터비전 ORCID