1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC) (Regular Paper) 22 3, 2017 5 (JBE Vol. 22, No. 3, May 2017) https://doi.org/10.5909/jbe.2017.22.3.381 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) S-JND HEVC a), a) S-JND based Perceptual Rate Control Algorithm of HEVC JaeRyun Kim a) and Donggyu Sim a) HEVC (High Efficiency Video Coding). HEVC. CTU,, CTU. HEVC 16.9 CTC (Common Test Condition) Class B BD-rate 3.12% BD-PSNR 0.08dB 0.07%. HEVC DSCQS 0.16. Abstract In this paper, the perceptual rate control algorithm is studied for HEVC (High Efficiency Video Coding) encoder with bit allocation based on perceived visual quality. This paper proposes perceptual rate control algorithm which could consider perceived quality for HEVC encoding method. The proposed rate control algorithm employs adaptive bit allocation for frame and CTU level using the perceived visual importance of each CTU. For performance evaluation of the proposed algorithm, the proposed algorithm was implemented on HM 16.9 and tested for sequences in Class B under the CTC (Common Test Condition) RA (Random Access) case. Experimental results show that the proposed method reduces the bitrate of 3.12%, and improves BD-PSNR of 0.08dB and bitrate accuracy of 0.07% on average. And also, we achieved MOS improvement of 0.16 with the proposed method, compared with the conventional method based on DSCQS (Double Stimulus Continuous Quality Scale). Keyword : HEVC, Perceptual rate control, S-JND a) (Dept. of Computer Engineering, Kwangwoon University) Corresponding Author : (Donggyu Sim) E-mail: dgsim@kw.ac.kr Tel: +82-2-940-5470 ORCID: http://orcid.org/0000-0002-2794-9932 2014 ( ) (NRF-2014R1A2A1A11052210). Manuscript received January 20, 2017; Revised March 13, 2017; Accepted April 13, 2017. Copyright 2017 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND (http://creativecommons.org/licenses/by-nc-nd/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.
(JBE Vol. 22, No. 3, May 2017). 4G.,,, VoIP,, Cisco [1]. Full HD 4K 8K, H.264/AVC (Advanced Video Coding) [2]. ISO/IEC MPEG (Moving Picture Experts Group) ITU-T VCEG (Video Coding Experts Group) JCT-VC (Joint Collaborative Team on Video Coding). JCT-VC 2013 1 H.264/AVC 50% HEVC (High Efficiency Video Coding) [3][4]. HEVC. H.264/AVC (MB: Macro Block), HEVC (CU: Coding Unit),, (PU: Prediction Unit) (TU: Transform Unit) [5]. HEVC 35 H.264/AVC [6], 8-tap, (Merge) AMVP (Advanced Motion Vector Prediction) [7]., H.264/AVC HEVC (Deblocking filter) (SAO: Sample Adaptive Offset) (Blocking artifact) (Ringing artifact) [8]. H.264/AVC HEVC, HEVC. Apple iphone 6 iphone 6 Plus HEVC Facetime, HEVC, HEVC. HEVC [9]. HEVC. HEVC HM (HEVC reference Model) [10] (Rate control algorithm).,, [11][18]. HM (RDO: Rate-Distortion Optimization) [12].,. PSNR (Peak Signal to Noise Ratio), MSE (Mean Squared Error), MOS (Mean Opinion Score)., HM PSNR.
1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC) [13][14][15][16][17].. S-JND (Saliency-Just Noticeable Di- fference). S-JND JND (Just Noticeable Difference) Saliency,, [16]. S-JND... S-JND HEVC.. 2 HM S-JND, 3. 4 HM, 5.. HM S-JND, HEVC H.264/AVC 2013 1. HEVC H.264/AVC 50% 2. HEVC [9]. H.264/AVC HEVC,, 8-tap, AMVP, (In-loop filter). H.264/AVC 50%. 1 HEVC HM. 2 S-JND. 1. HM HEVC HEVC HM... HEVC,.,,.,..,
(JBE Vol. 22, No. 3, May 2017)., SAD (Sum of Absolute Difference), SSE (Sum of Squared Error), MSE PSNR.,. HEVC PSNR., MOS., ITU-R (International Telecommunication Union Radiocommunication sector) BT.500-11 [19]. BT.500-11, DSIS (Double Stimulus Im- pairment Scale) DSCQS (Double Stimulus Continuous Quality Scale).,,., [13][14][15][16][17]., HM PSNR. HM. 1 HM. (Input sequence), (Output bit- stream),., (Target bit-rate), 1. HM Fig. 1. Block diagram of rate control algorithm for HM
1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC) (HRD: Hyperthetical Reference Decoder) (Buffer constraint parameter) (Occurred bit-rate), (Reconstructed sequence)., (Complexity calculation).,. (QP: Quantization Parameter).. 1,. HEVC URQ (Unified Rate-Quantization) [20][21], λ R-λ [22]. HEVC HM [22] R-λ, GOP (Group Of Pictures), (Frame), CTU (Coding Tree Unit)., GOP GOP, GOP GOP. CTU. CTU λ. R-λ (1). (1),, (Lagrangian multiplier).. HM, (Rate-Distortion cost). (1) (2). (2) MSE., (2) λ. (3). (3), λ., (4) λ CTU QP. ln R-λ λ QP CTU., CTU. R-λ. 2. S-JND (Saliency-Just Noticeable Difference)
(JBE Vol. 22, No. 3, May 2017) S-JND. HEVC.,, -,.,,. (Luminance adaptation) Saliency [17]., JND. 2 JND threshold, 0 255 JND threshold.,.. (ROI: Region Of Interest).,,, (Motion sensitivity).. S-JND [16]. S-JND JND Saliency, (5). JND threshold 2 0 255. Saliency 0 1. JND threshold Saliency, S-JND S-JND threshold. S-JND S-JND threshold (5), JND threshold JND threshold Saliency 1. JND threshold Saliency, offset., 0.5. (5) S-JND threshold 0 1 2. JND JND threshold Fig. 2. A graph for relation of average background luminance and JND threshold
1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC).. S-JND HEVC. CTU, CTU.,. 1, 2. 1. S-JND HEVC. 3. HM, S-JND. S-JND threshold, S-JND threshold..,,.,. [16] CTU, CTU, 64 64 CTU. S-JND S-JND threshold, 3. Fig. 3. Block diagram of proposed rate control algorithm
(JBE Vol. 22, No. 3, May 2017) 3.2. GOP CTU. GOP R-λ,,. CTU, CTU.,,. CTU, GOP, R-λ GOP. 2., GOP 4. Fig. 4. Proposed target bit decision model for frame level
1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC),., 4. 4., GOP, GOP.. [16],. GOP. GOP (9). (9) GOP. GOP, (10). (6) CTU CTU S-JND threshold., (7) (6) CTU,. CTU GOP, GOP. (8) GOP., [16] CTU. GOP (10) GOP. (10). GOP GOP, 10%. CTU, (11) (12). (11) (12) [16] CTU, (9), (10),
(JBE Vol. 22, No. 3, May 2017)... HEVC HEVC HM 16.9. Class B CTC (Common Test Condition) [25] RA (Random Access). 1. RA Low-delay All intra. 1. Table 1. Experimental environments CPU OS RAM Intel (R) Core (TM) i7-3960x CPU 3.30GHz Windows 7 Ultimate K 16.0 GB CTC Class Class B, 2 Class B. 2. CTC Table 2. Information of CTC test sequences used for performance evaluation of proposed algorithm Class Resolution Sequence name Number of frame B 1920 1080 Frame rate (fps) Kimono 240 24 ParkScene 240 24 Cactus 500 50 BasketballDrive 500 50 BQTerrace 600 60 Bit-depth 3 8. CTC RA, BP1, BP2, BP3, BP4.. 3. Table 3. Target bit-rate and bit-rate point Class (Resolution) Class B (1920 1080) Sequence name Kimono ParkScene Cactus BasketballDrive BQTerrace Target bit-rate (Kbps) Bit-rate point 4868 BP1 2228 BP2 1080 BP3 535 BP4 7789 BP1 3414 BP2 1561 BP3 712 BP4 18374 BP1 5882 BP2 2724 BP3 1357 BP4 17616 BP1 6147 BP2 2836 BP3 1435 BP4 39754 BP1 7377 BP2 2321 BP3 987 BP4.,. 4 2 HM 16.9 CTU, RA.
1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC) 4. Table 4. Comparing encoding performance between conventional and proposed rate control algorithm Sequence Kimono ParkScene Cactus BasketballDrive BQTerrace Bitrate point Anchor 1 Proposed BDrate Bitrate PSNR-Y Bitrate PSNR-Y (Kbps) (db) (Kbps) (db) (%) BP1 4870.73 41.30 4870.78 41.37 BD- PSNR (db) Bit accuracy (%) BP2 2229.00 39.29 2229.27 39.42-0.01-3.2 0.10 BP3 1084.21 36.86 1080.86 36.95 0.31 BP4 546.635 34.35 544.90 34.35 0.32 BP1 7791.76 39.76 7792.57 39.84 BP2 3415.30 37.23 3415.42 37.29 0.00-1.9 0.06 BP3 1563.02 34.44 1561.78 34.51 0.08 BP4 733.05 31.83 731.97 31.89 0.15 BP1 18378.00 38.37 18377.86 38.50 0.00-0.01 BP2 5884.66 36.58 5884.82 36.66 0.00-3.6 0.08 BP3 2731.01 34.43 2725.94 34.52 0.19 BP4 1378.94 32.15 1376.58 32.18 0.17 BP1 17625.08 39.09 17623.07 39.17 BP2 6150.67 37.21 6150.43 37.28 0.00-3.0 0.07 BP3 2838.10 35.19 2838.25 35.28-0.01 BP4 1438.74 33.06 1436.81 33.11 0.13 BP1 39756.46 37.32 39756.27 37.43 BP2 7379.35 35.15 7379.77 35.22-0.01-3.9 0.07 BP3 2338.57 33.44 2337.46 33.47 0.05 BP4 1019.15 31.58 1018.34 31.63 0.08 Average 6457.62 35.93 6456.66 36.00-3.12 0.08 0.07 0.00 0.01 0.00 4 CTU. 4 Anchor 1 HM 16.9, Proposed HM 16.9 CTU. BD-rate,. 4, BD-rate 3.12%, 0.07%. BD-PSNR [26]. PSNR. PSNR, 0.08dB. S-JND threshold. S-JND threshold QP, S-JND threshold QP. BD-rate., BD-PSNR.,, BD-rate BD-PSNR.
(JBE Vol. 22, No. 3, May 2017) 5. [16] CTU Table 5. Comparing encoding performance between conventional algorithm for CTU level ([16]) and proposed rate control algorithm Sequence Kimono ParkScene Cactus BasketballDrive BQTerrace Bitrate point Bitrate (Kbps) Anchor 2 PSNR-Y (db) Bitrate (Kbps) Proposed PSNR-Y (db) BP1 4870.96 41.36 4870.78 41.37 BDrate (%) BD- PSNR (db) Bit accuracy (%) BP2 2229.25 39.41 2229.27 39.42 0.00-0.4 0.01 BP3 1080.87 36.95 1080.86 36.95 0.00 BP4 544.62 34.34 544.90 34.35-0.05 BP1 7793.07 39.83 7792.57 39.84 BP2 3415.39 37.29 3415.42 37.29 0.00-0.3 0.01 BP3 1561.78 34.51 1561.78 34.51 0.00 BP4 732.15 31.85 731.97 31.89 0.03 BP1 18377.90 38.49 18377.86 38.50 BP2 5885.05 36.65 5884.82 36.66 0.00-0.5 0.01 BP3 2725.91 34.50 2725.94 34.52 0.00 BP4 1379.55 32.16 1376.58 32.18 0.22 BP1 17622.88 39.18 17623.07 39.17 BP2 6150.64 37.27 6150.43 37.28 0.00-0.2 0.01 BP3 2838.25 35.27 2838.25 35.28 0.00 BP4 1436.77 33.10 1436.81 33.11 0.00 BP1 39756.32 37.43 39756.27 37.43 BP2 7379.66 35.22 7379.77 35.22 0.00 0.1 0.00 BP3 2337.10 33.48 2337.46 33.47-0.02 BP4 1018.18 31.63 1018.34 31.63-0.02 Average 6456.82 36.00 6456.66 36.00-0.26 0.01 0.01 0.00 0.01 0.00 0.00 0.00 5 Anchor 2 [16] CTU, Proposed 4 HM 16.9 CTU. BD-rate [16] CTU 0.26%., 5 [16] 0.01%. 4 PSNR, 0.01dB,. 6 [15] JND. RC based on JND JND [15], Proposed HM 16.9., JND., JND, [15],.
1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC) 6. [15] JND RA Table 6. Comparing encoding performance with random access structure between rate control based on JND ([15]) and proposed algorithm Sequence Kimono ParkScene Cactus BasketballDrive BQTerrace Bitrate point RC based on JND Bitrate (Kbps) PSNR-Y (db) Bitrate (Kbps) Proposed PSNR-Y (db) BP1 4868.68 41.38 4870.78 41.37 BDrate (%) BD- PSNR (db) Bit accuracy (%) BP2 2227.89 39.56 2229.27 39.42-0.05 7.3-0.22 BP3 1066.24 37.21 1080.86 36.95 1.19 BP4 535.15 34.82 544.90 34.35-1.82 BP1 7795.28 39.91 7792.57 39.84 BP2 3415.97 37.49 3415.42 37.29 0.02 8.9-0.28 BP3 1561.41 34.89 1561.78 34.51-0.02 BP4 716.56 32.31 731.97 31.89-2.16 BP1 18379.98 38.36 18377.86 38.50 BP2 5882.41 36.79 5884.82 36.66-0.04 9.4-0.20 BP3 2722.45 34.93 2725.94 34.52-0.01 BP4 1356.80 32.69 1376.58 32.18-1.43 BP1 17615.06 39.05 17623.07 39.17 BP2 6145.53 37.41 6150.43 37.28-0.03 6.9-0.16 BP3 2834.81 35.55 2838.25 35.28-0.04 BP4 1435.20 33.48 1436.81 33.11-0.11 BP1 39763.40 37.39 39756.27 37.43 BP2 7378.09 35.29 7379.77 35.22-0.02 9.4-0.13 BP3 2321.79 33.75 2337.46 33.47-0.68 BP4 1006.48 32.00 1018.34 31.63-1.20 Average 6451.46 36.21 6456.66 36.00 8.38-0.20-0.32-0.04 0.03 0.01-0.03 0.02 7. Table 7. Experimental information for subjective quality assessment Display Samsung 47LG50FD Type and size 47 inch 16:9 Resolution UHD Number of subjects 11 Viewing distance 1m (0.75H). 7. 5 6 11 ITU-R BT.500-11 [19] DSCQS, MOS. 5. 5 HM 16.9.,. 5, MOS, 0.16 MOS., MOS,.
(JBE Vol. 22, No. 3, May 2017) 5. Fig. 5. Experimental result of perceived visual quality for conventional and proposed rate control method 10%., 10%,.. S-JND. S-JND, CTU. HM 16.9 CTC RA Class B, 3.12% BD-PSNR 0.08dB 0.07%., DSCQS, 0.16 MOS.,.. (References) [1] Cisco, Cisco visual networking index: Global mobile data traffic forecast update, 2015 2020 Cisco White Paper, Feb. 2016. [2] T. Wiegand, G. J. Sullivan, G. Bjontegaard and A. Luthra, Overview of the H.264/AVC video coding standard, IEEE transactions on Circuits and Systems for Video Technology, vol. 13, no. 7, pp. 560-576, Jul. 2003. [3] B. Bross, W. J. Han, G. J. Sullivan, J. R. Ohm and T. Wiegand, High
1 : S-JND HEVC (JaeRyun Kim et al.: S-JND based Perceptual Rate Control Algorithm of HEVC) Efficiency Video Coding (HEVC) text specification draft 10 for FDIS & last call, Joint Collaborative Team on Video Coding (JCT-VC), JCTVC-L1003_v34, Geneva, CH, Jan. 2013. [4] J. R. Ohm, G. J. Sullivan, H. Schwarz, T. K. Tan and T. Wiegand, Comparison of the coding efficiency of video coding standards-including High Efficiency Video Coding (HEVC), IEEE transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1668-1683, Dec. 2012. [5] I. K. Kim, J. Min, T. Lee, W. J. Han and J. Park, Block partitioning structure in the HEVC standard, IEEE transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1697-1706, Dec. 2012. [6] J. Lainema, F. Bossen, W. J. Han, J. Min and K. Ugur, Intra coding of the HEVC standard, IEEE transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1792-1801, Dec. 2012. [7] G. Sullivan, J. Ohm, W. Han and T. Wiegand, Overview of the High Efficiency Video Coding (HEVC) standard, IEEE transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1649-1668, Dec. 2012. [8] A. Norkin, G. Bjøntegaard, A. Fuldseth, M. Narroschke, M. Ikeda, K. Andersson, M. Zhou and G. Auwera, HEVC deblocking filter, IEEE transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1746-1754, Dec. 2012. [9] Y. J. Ahn, S. J. Park and D. G. Sim, Implementation and commercialization techniques of HEVC codec, The Magazine of the Institute of Electronics and Information Engineers, vol. 41, no. 19, pp. 939-947, Oct. 2014. [10] http://hevc.kw.bbc.co.uk/trac/browser/tags/hm-16.9, HM 16.9 S/W. [11] D. G. Sim and H. H. Jo, Understanding of HEVC standard and technology, HongRung Publishing Company, Korea, 2015. [12] Q. Lin and G. Feng, The bit allocation and RDO mode based rate control algorithm, 2010 International Conference on Anti-Counterfeiting Security and Identification in Communication (ASID), Chengdu, China, pp. 154-157, Jul. 2010. [13] C. Chou and Y. Li, A perceptually tuned subband image coder based on the measure of Just-Noticeable-Distortion profile, IEEE transactions on Circuits and Systems for Video Technology, vol. 5, no. 6, pp. 467-476, Dec. 1995. [14] Z. Wei and K. N. Ngan, Spatio-temporal Just Noticeable Distortion profile for grey scale image/video in DCT domain, IEEE transactions on Circuits and Systems for Video Technology, vol. 19, no. 3, pp. 337-346, Mar. 2009. [15] W. Lim and D. G. Sim, Coding unit-level multi-loop encoding method based on JND for perceptual coding, Journal of the Institute of Electronics and Information Engineers, vol. 52, no. 5, pp. 147-154, May 2015. [16] J. Kim, Y. J. Ahn, W. Lim and D. G. Sim, A perceptual rate control algorithm with S-JND model for HEVC encoder, Journal of Broadcast Engineering, vol. 21, no. 6, pp. 929-943, Nov. 2016. [17] S. Goferman, L. Zelnik-Manor, A. Tal, Context-aware saliency detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 10, pp. 1915-1926, Oct. 2012. [18] W. Lim, J. Nam and D. Sim, Scalable multi-view video coding based on HEVC, IEIE transactions on Smart Processing and Computing, vol. 4, no. 6, pp. 434-442, Dec. 2015. [19] Recommendation ITU-R BT.500-11, Methodology for the subjective assessment of the quality of television pictures, Recommendation ITU-R BT.500-11, ITU-Telecommunication Standardization Sector, no. 22, pp. 25-34, Jan. 2002. [20] H. Choi, J. Nam, J. Yoo, D. Sim and I. V. Bajić, Rate control based on unified RQ model for HEVC, Joint Collaborative Team on Video Coding (JCT-VC), JCTVC-H0213, San Jose, CA, Jan. 2012. [21] H. Choi, J. Yoo, J. Nam, D. Sim and I. V. Bajić, Pixel-wise Unified Rate-Quantization model for multi-level rate control, IEEE Journal of Selected Topics in Signal Processing, vol. 7, no. 6, pp. 1112-1123, Dec. 2013. [22] B. Li, H. Li, L. Li and J. Zhang, Rate control by R-lambda model for HEVC, Joint Collaborative Team on Video Coding (JCT-VC), JCTVC-K0103, Shanghai, China, Oct. 2012. [23] V. Sze, M. Budagavi and G. J. Sullivan, High Efficiency Video Coding (HEVC): algorithms and architectures, Springer International Publishing, Sep. 2014. [24] Mathias Wien, High Efficiency Video Coding (HEVC): coding tools and specification, Springer International Publishing, Sep. 2014. [25] F. Bossen, Common test conditions and software reference configurations, Joint Collaborative Team on Video Coding (JCT-VC), JCTVC-L1100, Geneva, CH, Jan. 2013. [26] G. Bjøntegaard, Calculation of average PSNR differences between RD-curves, ITU-Telecommunications Standardization Sector, VCEG-M33, pp. 290-294, Austin, TX, Apr. 2001.
396 방송공학회논문지 제22권 제3호, 2017년 5월 (JBE Vol. 22, No. 3, May 2017) 저자소개 김재련 년 월 광운대학교 컴퓨터공학과 학사 년 월 광운대학교 컴퓨터공학과 석사 주관심분야 영상신호처리 영상압축 컴퓨터비전 - 2015 2 : - 2017 2 : - ORCID : http://orcid.org/0000-0002-8735-0542 :,, 심동규 - 년 2월 : 서강대학교 전자공학과 공학사 년 2월 : 서강대학교 전자공학과 공학석사 년 2월 : 서강대학교 전자공학과 공학박사 년 3월 ~ 2000년 8월 : 현대전자 선임연구원 년 9월 ~ 2002년 3월 : 바로비젼 선임연구원 년 4월 ~ 2005년 2월 : University of Washington Senior research engineer 년 3월 ~ 현재 : 광운대학교 컴퓨터공학과 교수 : http://orcid.org/0000-0002-2794-9932 주관심분야 : 영상신호처리, 영상압축, 컴퓨터비전 1993 1995 1999 1999 2000 2002 2005 ORCID