2 : HEVC (Yongtae Kim et al.: Performance Analysis of Scalable HEVC Coding Tools) (Special Paper) 20 4, 2015 7 (JBE Vol. 20, No. 4, July 2015) http://dx.doi.org/10.5909/jbe.2015.20.4.497 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) HEVC a), a), a) Performance Analysis of Scalable HEVC Coding Tools Yongtae Kim a), Jinhyuk Choi a), and Haechul Choi a) HD(High Definition) UHD(Ultra High Definition),, PC,. (Scalability),,, Scalable Video Coding(SVC). ISO/IEC MPEG(Moving Picture Experts Group) ITU-T VCEG(Video Coding Experts Group) Joint Collaborative Team on Video Coding(JCT-VC) Scalable High Efficiency Video Coding(SHVC).,, SHVC SHVC, SHVC.. Abstract Current communication networks consist of channels with various throughputs, protocols, and packet loss rates. Moreover, there are also diverse user multimedia consumption devices having different capabilities and screen sizes. Thus, a practical necessity of scalability on video coding have been gradually increasing. Recently, The Scalable High Efficiency Video Coding(SHVC) standard is developed by Joint Collaborative Team on Video Coding(JCT-VC) organized in cooperation with MPEG of ISO/IEC and VCEG of ITU-T. This paper introduces coding tools of SHVC including adopted and unadopted tools discussed in the process of the SHVC standardization. Furthermore, the individual tool and combined tool set are evaluated in terms of coding efficiency relative to a single layer coding structure. This analysis would be useful for developing a fast SHVC encoder as well as researching on a new scalable coding tool. Keyword : scalable video, SHVC, video coding, performance analysis, HEVC a) (Hanbat National University) Corresponding Author : (Haechul Choi) E-mail: choihc@hanbat.ac.kr Tel: +82-42-821-1149 ORCID: http://orcid.org/0000-0002-7594-0828 [B0126-15-1013, Ultra-wide viewing ] 2013 ( ) (NRF-2013R1A1A1010344). Manuscript received May 18, 2015; revised June 12, 2015; accepted June 12, 2015.
(JBE Vol. 20, No. 4, July 2015).,, HD (High Definition) UHD (Ultra High Definition),. (format) (bit-rate) (single layer video coding).,,,. (Heterogeneous),, (scalabili- ty) Scalable Video Coding(SVC) [1]. ISO/IEC MPEG (Moving Picture Experts Group) ITU-T VCEG (Video Coding Experts Group) Joint Collaborative Team on Video Coding (JCT-VC) Scalable High Efficiency Video Coding (SHVC) [2]. SHVC (Spatial Scalability), (Temporal Scalability) (SNR Scalability).. (Base Layer) HEVC, (Enhancement Layer),, SHVC 1.5 2......,. SHVC Reference Index IntraBL. (inter-layer prediction),. Reference Index IntraBL. SHVC, IntraBL Reference Index. Reference Index HEVC high level syntax,, IntraBL. SHVC. HEVC (Single Layer)
2 : HEVC (Yongtae Kim et al.: Performance Analysis of Scalable HEVC Coding Tools) SHVC, Reference Index IntraBL. SHVC Reference Index, SHVC. 2 SHVC, 3 2, 4..... SHVC 2. SVC Fig. 2. single-loop decoding 1. SHVC 1., 3. SVC Fig. 3. multi-loop decoding 1. SHVC high-level [3] Fig. 1. High-level block diagram of a SHVC encoder [3] H.264/AVC SVC. 2. Intra_BL, (residual). SHVC. 3., [4]. SVC
(JBE Vol. 20, No. 4, July 2015), SHVC / HEVC., SHVC. SHVC Reference Index IntraBL. 2-2, 2-3. 2. Reference Index SHVC Reference Index SHVC (Inter-layer Reference Picture, ILP). 2.1 (Inter-layer texture prediction) Reference Index SHVC HEVC block level syntax, high level syntax., POC.. SHVC 4. L0. L1 HEVC,, long term. P-Slice, L0, B-Slice, 4. Fig. 4. type and construction of the reference picture L0 L1., SHVC. Reference Index.. Ÿ REF_IDX_ME_ZEROMV [5] PU (Motion Vector, MV) HEVC. AMVP(Advanced Motion Vector Prediction) MVP(Motion Vector Predictor) ( ). MVP. PU 0(ZeroMV). Ÿ ENCODER_FAST_MODE [6] SHVC AI(All Intra mode) simulcast. I_Slice P_Slice,
2 : HEVC (Yongtae Kim et al.: Performance Analysis of Scalable HEVC Coding Tools) ZeroMV., (inter) 2Nx2N., ZeroMV 2Nx2N (uni-prediction). Ÿ MAX_TID_REF_PRESENT_FLAG [7] random access picture(rap) sub-layer, sub-layer. 2. Ÿ MAX_TID_FOR_REF_LAYERS [8] 2,.,, MAX_TID_FOR_REF_LAYERS temporal sub-layer(tid) flag. sub-layer. 5 Tid=2 Layer ID=1, Layer ID=2. HEVC,, trade-off.. Ÿ FAST_INTRA_SHVC [9] CU CU, PU (intra),. angular 33 2 4. 5. Fig. 5. Inter layer prediction using direct reference layer 2.2 (Inter-layer motion prediction) Reference Index SHVC Merge/AMVP TMVP(Temporal Motion Vector Predic- tor). 1 TMVP ILP. 1. SHVC TMVP Table 1. TMVP candidates of SHVC Merge candidate list Left Above Above right Left bottom Above left TMVP(ILP) Combined candidates Zero candidates AMVP candidate list Left Above TMVP(ILP) Zero candidates, TMVP block level Motion Field Mapping(MFM)
(JBE Vol. 20, No. 4, July 2015). Ÿ REF_IDX_MFM [10] ILP. ILP 16x16. Ÿ POSITION_ROUNDING_OFFSET [11] 6 ILP ILP (xrl, yrl). (xrl, yrl) 16x16 1. (xrl, yrl) 4 4. (1). R=4, S=4.. 3.1 (Inter-layer texture prediction) IntraBL SHVC, CU, HEVC IntraBL. IntraBL,. Ÿ INTRA_BL_DST4x4 [12][13] IntraBL 4x4 luma TU(Transform Unit) Discrete Sine Transform, 8x8 luma TU Discrete Cosine Transform. Ÿ NO_RESIDUAL_FLAG_FOR_BLPRED [14] HEVC TU, 0. TU cbf(coded block flag). SHVC TU HEVC. 6. Fig. 6. motion derivation using collocated base layer 3. IntraBL SHVC IntraBL SHVC Ÿ INTRA_BL_CTX_CHANGE [15] IntraBL IntraBL flag., HEVC CABAC.. IntraBL CABAC CU., CU (ctdepth) (ctxidxinc). 2 ctxidxinc CtDepth.
2 : HEVC (Yongtae Kim et al.: Performance Analysis of Scalable HEVC Coding Tools) 3. CU CABAC Table 3. CABAC context initialization in CU-size dependent CtDepth 0, 1(MaxCuDepth-1) ctxidxinc Note (corresponding CU size) 0 64x64/32x32 2(MaxCuDepth-1) 1 16x16 3(MaxCuDepth) 2 8x8 3.2 (Inter-layer motion prediction) HEVC merge mode AMVP. Merge, PU, motion vector difference(mvd). IntraBL 3 merge.. 4. IntraBL merge Table 4. merge candidates derivation for IntraBL based framework 5. merge Table 5. simplified redundancy check for merge mode candidate SHVC Merge candidate construction 0 colbase 1 A1(Left) : if(a1! = colbase) 2 B1(Above) : if(b1!=a1)&&(b1!=colbase)) 3 B0(Above right) : if(b0!=b1) A0(Left bottom) : if(candnum<4) 4 if(a0!=a1) B2(Above left) : if(candnum<4) 5 if(b2!=b1&&b2!=a1) 3.3 (Inter-layer residual prediction) GRP(Generalized Residual Prediction) [17]. predictor, GRP predictor. 7 (BEL) (PEL), (MVEL). Merge candidate list Base layer MV Left Above Above right Left bottom(if candnum<4) Above left(if candnum<4) TMVP Combined candidates Zero candidates Ÿ IL_MRG_SIMPLIFIED_PRUNING [16] Merge.., 4 (A1), (B1) (colbase) 3 (B0, A0, B2). 7. Generalized Residual Prediction Fig. 7. Generalized Residual Prediction
(JBE Vol. 20, No. 4, July 2015) MVEL, BEL (BBL) (PBL). BBL PBL (BBL-PBL), (ω = 0, 0.5, 1) predictor (P EL). SHVC.,.. SHVC SHVC SHM 6.0, IntraBL SHM 2.0. simulcast HEVC Test Model (HM)14.1. PC 2.4 GHz zeon E5 2665, 128GB RAM. SHVC Common Test Condition(CTC) [18][19] A traffic B Baskerball- Drive., A 1280x800, 2560x1600, B 960x540, 1920x1080. QP(Quantization Parameter) CTC 22, 26, 30, 34., A 2560x1600, B 1920x1080. QP 26, 30, 34, 38, QP 6 20, 24, 28, 32. Random-Access mode(ra), Low-Delay B mode(ld), All-Intra mode(ai). 4. Reference Index IntraBL. simulcast. 6. SHVC Table 6. SHVC experimental conditions scalability 2x spatial scalability SNR scalability class A class B class A class B Traffic Basketball Drive 1280 x 800 960 x 540 2560 x 1600 1920 x 1080 Traffic 2560 x 1600 Basketball Drive 1920 x 1080 1. Reference Index SHVC 22 26 30 34 26 30 34 38 QP 22 26 30 34 22 26 30 34 Reference Index. Reference Index IntraBL texture prediction. Reference Index. Reference Index 6., texture prediction. Texture prediction AI 0.1%, 42.3%. ENCODER_FAST_MODE. RA LD REF_IDX_ZEROMV. REF_IDX_ ZEROMV MVP AMVP, (Zero).
2 : HEVC (Yongtae Kim et al.: Performance Analysis of Scalable HEVC Coding Tools) ENCODER_FAST_MODE AI I P.. motion prediction. motion prediction MFM MFM motion prediction. AI.. Reference Index texture prediction motion prediction. prediction prediction. 2. IntraBL SHVC IntraBL texture prediction motion prediction. 7., texture prediction. motion prediction, texture prediction. INTRABL_DST_4X4 4x4 luma TU DST x2 AI 7. Reference Index Table 7. Performance Comparison for tool of Reference Index based Framework Texture Prediction Motion Prediction Prediction REF_IDX_ZEROMV + ENCODER_FAST_MODE REF_IDX_ZEROMV ENCODER_FAST_MODE REF_IDX_MFM POSITION_ROUNDING_OFFSE T texture + motion prediction Sequence AI 2x RA LD RA SNR LD Traffic -0.1-0.3 0.0-0.5-0.1 BasketballDrive -0.1-1.2-0.4-1.3-0.4 Enc. time 42.3 92 95.1 94.5 96.6 Traffic -0.5-0.2 0.1-0.3 0.0 BasketballDrive -0.3-0.4-0.2-0.2 0.1 Enc. time 90.1 90.4 92.9 93.3 95.3 Traffic 0.7 0.3 0.1-0.1 0.0 BasketballDrive 1.6-0.8-0.2-1.2-0.4 Enc. time 44.3 101 101.8 100.8 101.2 Traffic 0.0-0.3-0.1-1.4-1.3 BasketballDrive 0.0-1.9-0.6-2.3-1.2 Enc. time 100.1 98.4 99 97.8 99.1 Traffic 0.0-0.9-0.7-1.4-1.3 BasketballDrive 0.0-2.0-0.7-2.3-1.2 Enc. time 100 98.4 99.1 97.6 99 Traffic -0.1-1.1-0.7-1.8-1.4 BasketballDrive -0.1-2.8-0.9-3.1-1.4 Enc. time 42.5 89.6 92.9 91.5 95.1
(JBE Vol. 20, No. 4, July 2015). NO_RESIDUAL_ FLAG_FOR_BLPRED AI,. cbf. INTRABL_ CTX_CHANGE class A, class B. CU 64x64 32x32. motion prediction. texture prediction. motion prediction. IL_MRG_SIMPLIFIED_PRUNING merge. IntraBL texture prediction motion prediction,. IntraBL prediction RA 20%. 3. simulcast SHVC Simulcast. 8 8. IntraBL Table 8. Performance Comparison for tool of IntraBL based Framework Texture Prediction Motion Prediction Prediction INTRABL_DST_4X4 + NO_RESIDUAL_FLAG_FOR_BLPRED + INTRABL_CTX_CHANGE INTRABL_DST_4X4 NO_RESIDUAL_FLAG_FOR_BLPRED INTRABL_CTX_CHANGE IL_MRG_SIMPLIFIED_PRUNING texture + motion prediction Sequence AI 2x RA LD RA SNR LD Traffic -0.3-0.1-0.1 0.1-0.3 BasketballDrive -0.5 0.0-0.1 0.1 0.0 Enc. time 101.6 99.8 99.6 98.6 99.5 Traffic -0.2-0.1 0.0-0.3-0.3 BasketballDrive -0.3-0.1-0.1-0.2-0.1 Enc. time 100.1 100.0 100.0 99.4 99.5 Traffic -0.3 0.1 0.1 0.3 0.1 BasketballDrive -0.3-0.1-0.3 0.1-0.1 Enc. time 101.5 99.9 99.8 99.3 99.8 Traffic 0.0-0.1 0.0 0.0-0.2 BasketballDrive 0.1 0.2 0.2 0.1 0.1 Enc. time 100.3 99.9 99.9 98 99.5 Traffic 0.0 0.2 0.3 0.3 0.4 BasketballDrive 0.0 0.2 0.1 0.2 0.2 Enc. time 100.0 100.0 100.1 100.6 100.2 Traffic -33.8-17.1-0.5-22.6-8.2 BasketballDrive -20.3-19.4-13.8-23.9-17.3 Enc. time 114.3 102 101 100.6 100.9
2 : HEVC (Yongtae Kim et al.: Performance Analysis of Scalable HEVC Coding Tools) 8. simulcast SHVC Traffic(class A)-x2-RA Fig. 8. Traffic(class A)-x2-RA, Performance comparison of simulcast and SHVC framework 9 Reference Index IntraBL, SHVC simulcast 20%. SHVC HEVC high level syntax Reference Index HEVC,. 3D-HEVC.. SHVC. SHVC.. SHVC 9. simulcast SHVC BasketballDrive(class B)-x2-RA Fig. 9. BasketballDrive(class B)-x2-RA, Performance comparison of simulcast and SHVC framework.. (References) [1] Haechul Choi, Kyung Il Lee, Jung Won Kang, Seong-Jun Bae, and Jeong-Ju Yoo, "Overview and Performance Analysis of the Emerging Scalable Video Coding", JBE Vol.12, no. 6, November 2007. [2] J. Chen, J. Boyce, Y. Ye, M. M. Hannuksela, G. J. Sullivan and Y.-K. Wang, "High efficiency video coding (HEVC) scalable extension Draft 6", Q-1008_v5, 17th JCT-VC meeting, Valencia, Mar. 2014. [3] Jianle Chen, Jill Boyce, Yan Ye, Miska M. Hannuksela, "SHVC Test Model 2 (SHM 2)", Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 13th Meeting, JCTVC-M1007, Incheon, KR, 18-26 Apr. 2013. [4] Hyomin Choi, Junghak Nam, Donggyu Sim, and Ivan V. Bajic, Scalable Video Coding Based on High Efficiency Video Coding (HEVC), Communications, Computers and Signal Processing (PacRim), 2011 IEEE Pacific Rim Conference on. [5] Xiaoyu Xiu, Yuwen He, Yong He, Yan Ye, "TE A2: Inter-layer reference picture placement", JCTVC-L0051, Geneva, CH, 14-23 Jan. 2013 [6] Peng Yin, Taoran Lu and Tao Chen, "Non-TE2: Inter-layer reference picture placement", JCTVC-L0174, Geneva, CH, 14-23 Jan. 2013 [7] Hahun Lee, Jung Won Kang, Jinho Lee, Jin Soo Choi, MV-HEVC/SHVC HLS: Signalling for Inter-Layer prediction indication, JCTVC-N0120, Vienna, AT, 25 July 2 Aug. 2013 [8] Krishna Rapaka, Ye-kui Wang, Jianle Chen, Vadim Seregin, Hendry, Marta Karczewicz, MV-HEVC/SHVC HLS: On inter-layer RPS deri-
(JBE Vol. 20, No. 4, July 2015) vation and sub-layer inter-layer dependency, JCTVC-O0225, Geneva, CH 23 Oct. - 1 Nov. 2013 [9] Edouard Francois, ShaSha Shi, Chistophe Gisquet, Guilaume Laroche, Patrice Onno, Non-SCE1: simplification of intra mode coding in SHVC, JCTVC-M0115, Incheon, KR, 18-26 Apr. 2013 [10] Jianle Chen, Vadim Seregin, Liwei Guo, Marta Karczewicz, "Non-TE5: on motion mapping in SHVC", JCTVC-L0336, Geneva, CH, 14-23 Jan. 2013 [11] Chistophe Gisquet, Patrice Onno, Edouard Francois, Guilaume Laroche, SCE2: Results on test 2.1, JCTVC-N0139, Vienna, AT, 25 July 2 Aug. 2013 [12] Chuohao Yeo, Yih Han Tan, "On transform selection for IntraBL mode in SHVC", JCTVC-L0067, Geneva, CH, 14-23 Jan. 2013 [13] Vadim Seregin, Tzu-Der Chuang, Do-Kyoung Kwon, Fabrice Le Leannec, Patrice Onno, "AHG11: DST transform for Intra-BL mode", JCTVC-L0204, Geneva, CH, 14-23, Jan. 2013 [14] Tzu-Der Chuang, Yu-Wen Huang, Shawmin Lei, Chulkeun Kim, Joonyoung Park, Byeongmoon Jeon, "TE2-3.1.7: Inter-layer intra prediction signaling with residual skip flag", JCTVC-L0069, Geneva, CH, pp. 14-23, Jan. 2013 [15] Tomoyuki Yamamoto, "On TextureRL flag context", JCTVC-M0075, Incheon, KR, 18-26 Apr. 2013 [16] Tzu-Der Chuang, Yu-Wen Huang, Shawmin Lei, SCE5.1.5: Inter-Layer motion information prediction, JCTVC-M0124, Incheon, KR, 18-26 Apr. 2013 [17] Jianle Chen, Krishna Rapaka, Xiang Li, Vadim Seregin, Liwei Guo, Marta Karczewicz, Geert Van der Auwera, Joel Sole, Xianglin Wang, Chengjie Tu, Ying Chen, Description of scalable cideo coding technology proposal by Qualcomm, JCTVC-K0035, Shanghai, CN, 10-19 Oct. 2012 [18] Xiang Li, Jill Boyce, Patrice Onno, Yan Ye, Common SHM test conditions and software reference configurations, JCTVC-M1009, Incheon, KR, 18-26 Apr. 2013 [19] Vadim, Yong He, Common SHM test conditions and software reference configurations, JCTVC-Q1009, Valencia, Es, 27 March 4 April 2014-1998 2 : - 2000 2 : ( ) - 1999 12 ~ 2000 6 : 3G - 2000 8 ~ 2010 2 : (ETRI) - 2010 4 ~ 2012 1 : - 2012 11 ~ : - ORCID: http://orcid.org/0000-0001-8847-3061 - :,, - 2012 2 () - 2013 3 ~ 2015 2 ( ) - ORCID: http://orcid.org/0000-0002-0221-8092 - :,, - 1997 2 : - 1999 2 : ( ) - 2004 8 : ( ) - 2004 9 ~ 2010 2 : (ETRI) - 2010 3 ~ : - ORCID: http://orcid.org/0000-0002-7594-0828 - :,,