5 : HEVC GOP R-lambda (Dae-Eun Kim et al.: R-lambda Model based Rate Control for GOP Parallel Coding in A Real-Time HEVC Software Encoder) (Special Paper) 222, 2017 3 (JBE Vol. 22, No. 2, March 2017) https://doi.org/10.5909/jbe.2017.22.2.193 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) HEVC GOP R-lambda a), a), a), b), b), b) R-lambda Model based Rate Control for GOP Parallel Coding in A Real-Time HEVC Software Encoder Dae-Eun Kim a), Yongjun Chang a), Munchurl Kim a), Woong Lim b), Hui Yong Kim b), and Jin Wook Seok b) 4K UHD GOP IDR R-λ. R-λ (bit allocation). GOP IDR, B (bit budget)., R-λ, GOP, GOP B.,. HEVC,. Abstract In this paper, we propose a rate control method based on the R-λ model that supports a parallel encoding structure in GOP levels or IDR period levels for 4K UHD input video in real-time. For this, a slice-level bit allocation method is proposed for parallel encoding instead of sequential encoding. When a rate control algorithm is applied in the GOP level or IDR period level parallelism, the information of how many bits are consumed cannot be shared among the frames belonging to a same frame level except the lowest frame level of the hierarchical B structure. Therefore, it is impossible to manage the bit budget with the existing bit allocation method. In order to solve this problem, we improve the bit allocation procedure of the conventional ones that allocate target bits sequentially according to the encoding order. That is, the proposed bit allocation strategy is to assign the target bits in GOPs first, then to distribute the assigned target bits from the lowest depth level to the highest depth level of the HEVC hierarchical B structure within each GOP. In addition, we proposed a processing method that is used to improve subjective image qualities by allocating the bits according to the coding complexities of the frames. Experimental results show that the proposed bit allocation method works well for frame-level parallel HEVC software encoders and it is confirmed that the performance of our rate controller can be improved with a more elaborate bit allocation strategy by using the preprocessing results. Keyword : HEVC, Rate control, R-lmabda model, GOP parallelism, real time encoder Copyright 2017 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND (http://creativecommons.org/licenses/by-nc-nd/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.
(JBE Vol. 22, No. 2, March 2017). HEVC (High Efficiency Video Coding) [1][2] H.264 AVC (Advanced Video Coding) [3][4] [2]. HEVC.,. 1920 1080 full HD 3840 2160 4K UHD(Ultra HD).. [5]-[7], [8]. [8] B.. a) (The School of Electrical Engineering, Korea Advanced Institute of Science and Technology) b) AV (Realistic AV Research Group Media Research Division Broadcasting Media Research Laboratory Electronics and Telecommunications Research Institute) Corresponding Author : (Munchurl Kim) E-mail: mkimee@kaist.ac.kr Tel: +82-42-350-7419 ORCID: http://orcid.org/0000-0003-0146-5419 2016 ( ) (B0101-16-0295, UHD / / ). Manuscript received January 13, 2017; Revised March 3, 2017; Accepted March 3, 2017.. (bit budget).,,, (QP, quantization parameter). [9]-[12],.,.., GOP(Group of Picture) IDR(Instantaneous Decoder Refresh),., HEVC HM [13] R-λ [9][10] GOP IDR.,. GOP IDR HEVC 4K UHD.. HEVC R-λ, GOP IDR.
5 : HEVC GOP R-lambda (Dae-Eun Kim et al.: R-lambda Model based Rate Control for GOP Parallel Coding in A Real-Time HEVC Software Encoder).,. ln ln ln ln ln. HEVC R-λ HEVC HM R-λ.. 1. R-λ. (quantization parameter, QP). R-λ QP R-Q. [10] HEVC,, QP λ R-λ. [10] λ.,. [10]. [10].,,. bpp(bit per pixel) λ λ QP, λ QP [10]. ln (5) QP, R-λ bpp. 2., GOP.,,,, (smooth window), GOP. GOP GOP. (6),. GOP GOP,.
(JBE Vol. 22, No. 2, March 2017) GOP, GOP. 1. 1. Table 1. depending on target bpp and frame level Level bpp>0.2 0.2 bpp>0.1 0.1 bpp>0.05 other wise 1 15 20 25 30 2 5 6 7 8 3 4 4 4 4 4 1 1 1 1., 1 GOP IDR,,. GOP IDR.. GOP IDR HEVC 1 (b) 3 (b) 5 (b) 7 (b) 2 (B) 6 (B) 4(B) 0 (I) 8 (B) 1. HEVC GOP Fig. 1. GOP structure of HEVC random access 2. GOP Fig. 2. Encoding order of random access GOP
5 : HEVC GOP R-lambda (Dae-Eun Kim et al.: R-lambda Model based Rate Control for GOP Parallel Coding in A Real-Time HEVC Software Encoder). HEVC. [8] HEVC. 1 HEVC (random access) GOP. 1 I B b, (dependency). 2 1 GOP, POC. 2 6 2, 1, 3, 5, 7., [8] 6 2 GOP. 3 [8] GOP. 3 6 2 2 6 1 (b) 3 (b) 5 (b) 7 (b) 2 (B) 6 (B) 4(B) 0 (I) 8 (B) 3. GOP Fig. 3. GOP structure for frame level parallelism 4. GOP Fig. 4. GOP structure for frame level parallelism
(JBE Vol. 22, No. 2, March 2017).. GOP,. IDR IDR. 4 3 GOP IDR. 4. 0 0 8, 16, 24 1, 4, 12, 20 2. 0, 1, 2 3 4. 4 4. 4 1., GOP 1. 5. 5 GOP 1( 8, 16, 24) GOP. 3 5 GOP, 5 IDR, IDR. IDR, GOP, 3-5.,,, 5,, 5 IDR () (sequence)... 5. Fig. 5. Coding order of a serial encoder and a parallel encoder
5 : HEVC GOP R-lambda (Dae-Eun Kim et al.: R-lambda Model based Rate Control for GOP Parallel Coding in A Real-Time HEVC Software Encoder) 6. Fig. 6. Flow chart for proposed bit allocation method 1.,,...,. 6. 6 IDR. 1 IDR., 4 IDR 32 GOP 8 GOP 1 GOP 4 GOP,,,,. (frame per second) IDR.. IDR IDR 4 GOP. 1. (9)-(12) IDR IDR GOP. GOP. GOP
(JBE Vol. 22, No. 2, March 2017), GOP GOP GOP. GOP, GOP GOP. GOP 1,. 8. 8 4., 2 6. 2 6. 1, 3, 5, 7. (15) 1, 3, 5, 7., GOP.. 4 (14)-(17). 4. 4 GOP 1. 4 GOP 7 16. GOP 4 GOP. 4 GOP 7, 16. GOP (coding dependency) 1 (8, 16, 24 ) GOP, 4.. 2.,. R-λ.
5 : HEVC GOP R-lambda (Dae-Eun Kim et al.: R-lambda Model based Rate Control for GOP Parallel Coding in A Real-Time HEVC Software Encoder). IDR, 4,.,,.. 4 1, 3, 5, 7 3 9, 11 1 3.. 3.,.. IDR 16 16.,, SAD (sum of absolute difference). SAD, SAD 1., 1, GOP., 4K UHD (3840 2160) HEVC. 3840 2160 4K UHD 6.. 6... 6 IDR 32 IDR 192(= 6 32). SAD 192. 1.. 4K UHD (3840 2160) HEVC. 10 4. 2. 3 2.
(JBE Vol. 22, No. 2, March 2017) 2. Table 2. Experimental conditions Sequence 10 Number of encoded frames 32 GOP size 8 Frame rate 60 Bit depth 10 YUV format 4:2:0 Max CU size 64 64 Profile Main10 3 10 %.. 20 %. 1 IDR 32, R-λ R-λ. 10...3 4K UHD 6, SAD. 4,, 3. Table 3. Experimental result Seq. Target Output Error Target Output Error Target Output Error Target Output Error (kbps) (kbps) (%) (kbps) (kbps) (%) (kbps) (kbps) (%) (kbps) (kbps) (%) 1 34183 29755 13 11779 7777 34 6629 5201 22 3942 3670 7 2 44924 53022 18 21553 24883 15 3942 3670 7 4256 4480 5 3 38810 34054 12 16229 16802 4 44924 53022 18 4533 4689 3 4 38119 38527 1 18104 19000 5 21553 24883 15 4660 4047 13 5 41088 49059 19 21749 26546 22 10071 11134 11 4670 5081 9 6 39927 38167 4 18283 16988 7 4256 4480 5 3939 3459 12 7 37277 36365 2 16960 17864 5 38810 34054 12 3670 4852 32 8 39086 27737 29 12684 8803 31 16229 16802 4 3960 3692 7 9 53409 57761 8 30951 32921 6 8165 7846 4 5374 7150 33 10 6261 6695 7 3383 3781 12 4533 4689 3 1005 2434 142 4. Table 4. Performance of Encoding with respect to the use of preprocessing Seq. 1 Target (kbps) without preprocessing Output (kbps) PSNR-Y (db) Target (kbps) with preprocessing Output (kbps) PSNR-Y (db) 23,092.42 39.9070 19,999.997 18,454.8 39.2781 2 22,035.87 38.0683 19,999.996 19,081.01 38.2650 3 22,051.26 38.6307 19,999.997 19,278.99 39.1030 4 20,077.96 34.8815 19,999.998 19,624.38 34.6913 5 20,851.31 36.4836 19,999.998 21,196.38 37.1567 19,999.998 6 21,675.8 41.8547 19,999.996 17,835.84 41.3384 7 20,879.06 37.1371 19,999.997 19,455.69 36.3078 8 24,939.65 41.7070 19,999.998 17,146.08 43.2591 9 20,079.23 33.2931 19,999.997 20,873.22 32.2692 10 81,360.24 38.2269 19,999.996 28,370.42 41.7008
5 : HEVC GOP R-lambda (Dae-Eun Kim et al.: R-lambda Model based Rate Control for GOP Parallel Coding in A Real-Time HEVC Software Encoder). 4 20Mbps. 6 3,333.333Mbps 6 19,999.998kbps. 20Mbps, 6. 6 PSRN-Y 6 PSNR. 7. 7 (a) Without preprocessing (b) With preprocessing 7. Fig. 7. Result image without/with preprocessing (a) Without preprocessing (b) With preprocessing 8. Fig. 8. Result image without/with preprocessing
(JBE Vol. 22, No. 2, March 2017). 8. 8,.. 8.., GOP IDR HEVC. R-λ. R-λ. GOP, B, GOP... (References) [1] B. Bross, W.-J. Han, G. J. Sullivan, J.-R. Ohm, and T. Wiegand, High Efficiency Video Coding (HEVC) Text Specification Draft 9, document JCTVC-K1003, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Oct. 2012. [2] G. J. Sullivan, J. Ohm, W.-J. Han, and T. Wiegand, Overview of the high efficiency video coding (HEVC) standard, IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 12, pp. 1649 1668, Dec. 2012. [3] ITU-T, Advanced Video Coding for Generic Audio-Visual Services, ITU-T Rec. H.264 and ISO/IEC 14496-10 (AVC), ITU-T and ISO/IEC JTC 1, May 2003 (and subsequent editions). [4] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, Overview of the H.264/AVC video coding standard, IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 560 576, Jul. 2003. [5] J. H. Jang, and C. E. Rhee, Down Sampling for Fast Rough Mode Decision for a Hardware-based HEVC Intra-frame encoder, Journal of Broadcast Engineering, vol. 21, no. 3, pp.341-348, May 2016. [6] S. Jeon, N. Kim, and B. Jeon, CU Depth Decision Based on FAST Corner Detection for HEVC Intra Prediction, Journal of Broadcast Engineering, vol. 21, no. 4, pp. 484-492, July 2016. [7] J. Lim, Y.-J. Ahn, and D. Sim, Study on Fast HEVC Encoding with Hierarchical Motion Vector Clustering, Journal of Broadcast Engineering, vol. 21, no. 4, pp.578-591, July 2016. [8] Y. Kim, J. Seok, S.-h. Jung, H. Kim, and J. S. Choi Tile-level and Frame-level Parallel Encoding for HEVC, Journal of Broacast Engineering, vol. 20, no. 3, pp. 388-397, May 2015. [9] B. Li, H.Li, L. Li and J. Zhang, Rate control by R-lambda model for HEVC, document JCT-VC K0103, Shanghai, CN, Oct. 2012. [10] B. Li, H.Li, L. Li and J. Zhang, λ Domain Rate Control Algorithm for High Efficiency Video Coding, IEEE Trans. Image Processing, Vol. 23, No. 9, Sept. 2014. [11] H. Choi, J. Nam, J. Yoo, D. Sim, and I. Baji c, Rate Control Based on Unified RQ Model for HEVC, document Rec. JCTVC-H0213, San Jose, CA, USA, Feb. 2012. [12] J. Si, S. Ma, W. Gao, and M. Yang, Adaptive Rate Control for HEVC, document Rec. JCTVC-I0433, Geneva, Switzerland, Apr./May 2012. [13] (2017, Jan. 12). HM, HEVC Test Model [Online]. Available: http://hevc.hhi.fraunhofer.de/svn/svn_hevcsoftware/.
5 : HEVC GOP R-lambda (Dae-Eun Kim et al.: R-lambda Model based Rate Control for GOP Parallel Coding in A Real-Time HEVC Software Encoder) - 2011 2 : - 2014 2 : - 2014 3 ~ : - ORCID : hjttp://orcid.org/0000-0003-0948-7049 - : HDR, - 1997 8 : - 2000 8 : - 2012 8 : - 2012 4 ~ 2015 6 : / - 2015 7 ~ 2016 7 : - 2016 8 ~ : - ORCID : hjttp://orcid.org/0000-0001-8953-3986 - :,, - 1989 2 : - 1992 12 : University of Florida, Dept. of Electrical and Computer Engineering, - 1996 8 : University of Florida, Dept. of Electrical and Computer Engineering, - 1997 1 ~ 2001 1 :, - 2001 2 ~ 2009 2 : / - 2009 3 ~ : / - ORCID : hjttp://orcid.org/0000-0003-0146-5419 - : Perceptual Video Coding, SDR/HDR Image/Video Quality Assessment and Modeling, Super-Resolution, Image/Video Analysis and Understanding, Pattern Recognition, Machine Learning - 2008 2 : - 2010 2 : - 2016 2 : - 2016 4 ~ : AV - ORCID : hjttp://orcid.org/0000-0002-1772-0683 - :, - 1994 8 : (KAIST) () - 1998 2 : (KAIST) ( ) - 2004 2 : (KAIST) ( ) - 2003 8 ~ 2005 10 : ( ) - 2006 9 ~ 2010 8 : (UST) - 2013 9 ~ 2014 8 : Univ. of Southern California(USC) - 2005 11 ~ : (ETRI) AV - ORCID : hjttp://orcid.org/0000-0001-7308-133x - :,,,
(JBE Vol. 22, No. 2, March 2017) - 1993 : - 1995 : - 1998 : - 2000 ~ : AV - ORCID : hjttp://orcid.org/0000-0001-5318-1237 - :,, UHDTV