1 : 3D-HEVC (Regular Paper) 19 5, 2014 9 (JBE Vol. 19, No. 5, September 2014) http://dx.doi.org/10.5909/jbe.2014.19.5.713 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) 3D-HEVC a), a) Performance Analysis of 3D-HEVC Video Coding Daemin Park a), and Haechul Choi a) 3. 3.,,.. JCT-3V(Joint Collaborative Team on 3D Video Coding Extension Development) 3. JCT-3V HEVC(High Efficiency Video Coding) 3D-HEVC.. Abstract Multi-view and 3D video technologies for a next generation video service are widely studied. These technologies can make users feel realistic experience as supporting various views. Because acquisition and transmission of a large number of views require a high cost, main challenges for multi-view and 3D video include view synthesis, video coding, and depth coding. Recently, JCT-3V (joint collaborative team on 3D video coding extension development) has being developed a new standard for multi-view and 3D video. In this paper, major tools adopted in this standard are introduced and evaluated in terms of coding efficiency and complexity. This performance analysis would be helpful for the development of a fast 3D video encoder as well as a new 3D video coding algorithm. Keyword : 3D video, Multi-view video, video coding, performance analysis, HEVC a) (Multimedia Engineering, Hanbat National University) Corresponding Author : (Haechul Choi) E-mail: choihc@hanbat.ac.kr Tel: +82-42-821-1149 (2012H1B8A2025982) (NRF-2013 R1A1A1010344). Manuscript received July 22, 2014 Revised August 18, 2014 Accepted August 18, 2014
(JBE Vol. 19, No. 5, September 2014).. 3DV(3D Video) [1]. 3 HD(High Definition) 2 2D [2]. 3D (Stereoscopic),,.,, [3]. ISO/IEC JTC1/SC29/WG11 MPEG(Mov-ing Picture Experts Group) 3DV.,,.,. 3DV,., 3D, [4]. 3 (Call For Proposal) [5] 2011 4 4 96 MPEG, 2011 10 JCT-3V(Joint Collaborative Team on 3D Video Coding Extension Development) 3. 1. 3 [5] Table 1. Standardization schedule of JCT-3V [5] 2011 01 2011 04 2012 04 2014 07 2014 10 2015 06 (Working Draft) MVC compatible extension including depth AVC compatible video-plus-depth extension HEVC 3D extensions 1 3. MVC compatible extension including depth H.264/AVC multi-view video coding (MVC) 2014 7. (syntax) 3. AVC(Advanced Video Coding) 3D 2014 10 AVC compatible video-plus-depth extension [6]. 2D HEVC(High Efficiency Video Coding) 3D 2015 6 HEVC 3D extension [3][6]. HEVC 3D 3D-HEVC. JCT-3V 3D-HEVC.,
1 : 3D-HEVC.,,. 3D-HEVC,., 3D-HEVC.,.. 2 3D-HEVC,,, HTM(HEVC Test Model Ver. 10.0 ) [7],. 3 HTM,. 4. 2. 3D-HEVC 1 3D-HEVC (View). (View) (View 0) T0-D0-T1-D1-T2-D2, T D, T D. HEVC. (Non-Base view) 3D-HEVC. (Texture first coding)., [7]. 2.1.,. 3D-HEVC, [7][8]. 1. 3D-HEVC [7] Fig. 1. 3D-HEVC encoder and prediction structure [7] 2.1.1 IVMP(Inter-View Motion Parameter Prediction) 3D-HEVC DV-MCP(Disparity vector motion compensated prediction) DCP(Disparity compensated prediction). MCP. DV-MCP. DCP
(JBE Vol. 19, No. 5, September 2014) MCP.,. 3D-HEVC (, ) (IVMP, Inter-View Motion Prediction). 3D-HEVC IVMP 4, 1 2 3 4. 3, 4 Do-NBDV(Depth oriented neighbouring block based disparity vector). 3 4,. 3 CU(Coding Unit) 4. 4. 1 2.. 2 2. 3 MCP. 3. MCP IVMP Fig. 3. The inter-view predicted motion vector of a MCP coded block 3 MCP IVMP,. [7][9]. 4 Do-NBDV (Virtual depth), Virtual depth., 4 3, 2. Fig. 2. Location of spatial and temporal neighbour blocks 4. Fig. 4. Derivation of the virtual depth block
1 : 3D-HEVC., 3 [7][9]. 2.1.2 IVRP(Inter-View Residual Parameter Prediction) IVRP(Inter-View Residual Parameter Prediction). 5, Dc Dr (, MCP ), Dc Bc MCP, Dr Dc Dc Bc. Dc Bc, [7][10]. 6. (Advanced Residual Prediction) [11] Fig. 6. Advanced Residual Prediction [11] Dc VD Bc Br, Bc Br Dr Dc [7][11]. 2.1.4 IC(Illumination Compensation) (view). 3D-HEVC IC(Illumination compensation),. 7, CU PU (1) 5. [11] Fig. 5. Inter-view residual prediction [11] 2.1.3 ARP(Advanced Residual Prediction) ARP IVRP. 5 Bc Dc IVRP. ARP. 6, 7. [8] Fig. 7. Neighbouring samples for the derivation of illumination compensation parameters [8]
(JBE Vol. 19, No. 5, September 2014) a b.,. [7][12]. 2.2 2.2.1 MPI(Motion Parameter Inheritance) MPI(Motion Parameter Inheritance). 3D-HEVC HEVC. CU CU. 3D-HEVC CU. CU. CU [7][14]. 2.2.2 QTLPC(Quadtree limitation Predictive Coding) 3D-HEVC QTLPC(Quadtree limitation Predictive Coding) CU CU..,.,, CU CU CU. QTLPC PU PU. 8 PU PU., PU 2N 2N, 2N N, N 2N 2N 2N PU PU [8][15]. 8. PU PU [15] Fig. 8. Texture partitions and corresponding possible depth partitions [15] 2.2.3 DMM(Depth Modeling Mode) DMM. HEVC PU 35, DC. (Wedgelet) (Contour). 9
1 : 3D-HEVC [7]. DC., 10 P1 P1 DC, P2 P2 DC. DC 10 2 [7]. 9. ( ), ( ) [13] Fig. 9. Wedgelet(up), Contour(down) [13].. DMM 1 4 4. 1 3, 4. 1 ( 9 S) ( 9 E). 2 HEVC DMM. 1. 3. 4.. CTLB(co-located texture luma block) CTLB, 10. [7] Fig. 10. Selection of reference samples for difference partition pattern cases [7] 3. IVMP, ARP, IC MPI, DMM, QTLPC., HTM, 2.. 3D-HEVC 3DV-HTM 10.0.
(JBE Vol. 19, No. 5, September 2014) PC Intel Sandy Bridge 2.4GHz E5 2665, RAM 128GB RAM. 3D-HEVC CTC(Common Test Condition) [16] A(1024x 768) Balloons, Kendo, Newpaper C (1920x1088) GT_Fly, Poznan_Street. CTC (QP: quantization parameter) 25, 30, 35, 40, 34, 39, 42, 45.. 2,,. 2. Table 2. Test sequences GT_Fly 1920x1088 5-9-1 250 Poznan_Street 1920x1088 5-1-9 250 Newspaper 1024x768 4-2-6 300 Balloons 1024x768 3-1-5 300 3. 3D-HEVC Table 3. Coding efficiency of 3D-HEVC compared with simulcast (BD-rate) View 1 View 2 GT_Fly -30.4% -29.8% Poznan_Street -18.5% -19.2% Newspaper -21.7% -20.4% Balloons -33.7% -32.9% Kendo -30.9% -33.1% Average -27.04% -27.08%. 3D-HEVC. 3D-HEVC. 11 12 3 Y-PSNR bitrate. Balloons view 1, Poznan_ Street view1. 3D-HEVC,. Kendo 1024x768 3-1-5 300. 3 3D-HEVC. (simulcast) HEVC.. 3, View 0 (base view), HEVC. 3 simulcast 3D-HEVC 27% 11. 3 Balloons - Fig. 11. R-D Curve for sequence Balloons that has the best coding efficiency in Table 3
1 : 3D-HEVC 12. 3 Poznan_Street - Fig. 12. R-D Curve for sequence Poznan_Street that has the worst coding efficiency in Table 3.. 4 4. 3D-HEVC Table 4. Coding efficiency of inter-view prediction tools in 3D-HEVC compared with simulcast GT_Fly Poznan_ Street Newspap er Balloons Kendo Average (BD-rate) IC ARP IVMP IC+ ARP+ IVMP View 1 0.0% -3.0% -21.7% -21.9% View 2 0.0% -2.3% -21.6% -21.8% View 1 0.3% -6.3% -11.7% -15.1% View 2-0.9% -6.5% -11.6% -16.0% View 1-0.8% -8.0% -14.4% -19.6% View 2-2.1% -7.3% -12.9% -18.6% View 1-3.1% -13.8% -24.0% -31.5% View 2-2.9% -13.0% -23.0% -30.4% View 1-4.7% -11.8% -18.5% -28.6% View 2-6.3% -12.9% -18.2% -30.5% View 1-1.66% -8.58% -18.06% -23.34% View 2-2.26% -8.4% -17.46% -23.66% II.2-1 IVMP, ARP, IC 3. IC Kendo View1, 2-4.7% -6.3% BD-rate, GT_Fly View1, 2. Kendo IC, GT_Fly. IC. ARP Kendo View1, 2-13.8%, -13.0%, GT_Fly -3.0% -2.3%. GT_Fly. IVMP Balloons View1, 2-24%, -23%, Poznan_Street View1, 2-11.7%, -11.6%. 4 Balloons IVMP. 3,. 13, 14 3 IC, ARP, IVMP 3, -. Balloons, Poznan_Street. IVMP..,. 3D-HEVC MPI.
(JBE Vol. 19, No. 5, September 2014) 5. 3D-HEVC Table 5. Coding efficiency of depth map coding tools when inter-view coding tools are turned on (BD-rate) MPI DMM QTLPC 13. 4 Kendo - Fig. 13. R-D Curve for sequence Kendo that has the best coding efficiency in Table 4 14. 4 Poznan_Street - Fig. 14. R-D Curve for sequence Poznan_Street that has the worst coding efficiency in Table 4 II.2-2 3D-HEVC MPI, DMM, QTLPC 5. 5 T, D. HEVC. 3 MPI GT_Fly Poznan_ Street Newspaper Balloons Kendo Average View 1(T) -6.7% -8.5% -4.5% View 2(T) -8.6% -7.1% -3.9% View 0(D) -13.1% -8.6% -1.4% View 1(D) -8.5% -5.4% -3.7% View 2(D) -9.4% -3.6% -3.0% View 1(T) -4.4% -4.3% -10.0% View 2(T) -3.9% -3.9% -9.4% View 0(D) -10.6% -1.7% -0.3% View 1(D) -11.2% 0.7% -4.4% View 2(D) -8.5% -0.1% -4.3% View 1(T) -2.5% -2.5% -2.5% View 2(T) -2.0% -2.1% -2.0% View 0(D) -7.1% -2.2% 5.3% View 1(D) -5.6% -2.0% 1.4% View 2(D) -4.8% -1.6% 4.4% View 1(T) -3.2% -3.3% -3.2% View 2(T) -3.7% -3.6% -3.8% View 0(D) -6.5% -0.7% 3.3% View 1(D) -7.0% -0.9% 1.7% View 2(D) -9.4% -0.3% 1.9% View 1(T) -3.4% -3.5% -3.3% View 2(T) -4.0% -3.8% -4.1% View 0(D) -3.2% 0.0% 3.6% View 1(D) -5.8% 0.6% 3.7% View 2(D) -5.5% -0.6% 5.5% View 1(T) -4.04% -3.24% -2.81% View 2(T) -4.14% -3.93% -4.26% View 0(D) -6.93% -2.97% 0.33% View 1(D) -6.36% -1.00% 0.84% View 2(D) -7.03% -0.71% 1.70%.. MPI.
1 : 3D-HEVC GT_Fly GT_Fly - (zoom-in).,, MPI., DMM GT_Fly. GT_Fly.. DMM. QTLPC CU PU GT_Fly Poznan_ Street,. QTLPC CU PU. QTLPC.. 15, 16 5 16. 5 Newspaper - Fig. 16. R-D Curve for sequence Newspaper(Texture) that has the worst coding efficiency in Table 5 6. 3D-HEVC QTLPC Table 6. complexity efficiency of QTLPC QTLPC GT_Fly 74.1% Poznan_Street 74.0% Newspaper 87.9% Balloons 80.2% Kendo 87.2% Average 80.68% 15. 5 GT_Fly - Fig. 15. R-D Curve for sequence GT_Fly(Texture) that has the best coding efficiency in Table 5 -. Poznan_street View 1, Newspaper View 2.. 17, 18 5 -. GT_Fly View 0, Kendo View 0., 15 16
(JBE Vol. 19, No. 5, September 2014). 18, DMM MPI. MPI, DMM. QTLPC. QTLPC (execution time). QTLPC 6. HEVC CU. QTLPC CU 80.68%.. 3D-HEVC. 3D-HEVC,.. 3. 3,. (References) [1] A. Smolic and P. Kauff, Interactive 3D video representation and coding technologies," Proc. of IEEE, Special Issue on Advances in Video Coding and Delivery, pp. 98-110, Jan. 2005. [2] ISO/IEC JTC1/SC29/WG11, Reference software of depth estimation and views synthesis for FTV/3DV, M15836, Oct. 2008. [3] Y. Ho, 3D Video Compression MPEG 3DV Standardization, IDEC Newsletter, pp. 10-13, May 2012. [4] ISO/IEC JTC1/SC29/WG11, Applications and requirements on 3D video coding, N12035, Geneva, Switzerland, March 2011. [5] ISO/IEC JTC1/SC29/WG11, Call for proposals on 3D video coding technology," N12036, Geneva, Switzerland, March 2011. [6] ISO/IEC JTC1/SC29/WG11, MPEG time line," N14404, Valencia, Spain, April 2014. [7] Y. Chen, G. Tech, K. Wegner, and S. Yea, Test model 8 of 3D-HEVC and MV-HEVC," JCT3V-H1003, Valencia, Spain, April 2014. [8] L. Zhang, G. Tech, K. Wegner, and S. Yea, Test model 7 of 3D-HEVC and MV-HEVC," JCT3V-G1005, San Jose, US, Jan. 2014. [9] J. An, Y.-W. Chen, J.-L. Lin, Y.-W. Huang, and S. Lei, CE 5 related: Inter-view motion prediction for HEVC-based 3D video coding," JCT3V-A0049, Stockholm, Sweden, July 2012. [10] G. Tech, K. Wegner, Y. Chen, and S. Yea, 3D-HEVC test model 2, JCT3V-B1005, Shanghai, China, Oct. 2012. [11] X. Li, L. Zhang, and Y. Chen, Advanced residual prediction in 3D-HEVC, Proc. of 20th IEEE International Conference Image Processing (ICIP), pp. 1747-1751, Sept. 2013. [12] J. Lee, Description of Core Experiment 2: Illumination compensation," JCT3V-A1102, Stockholm, Sweden, July 2012. [13] K. Muller, H. Schwarz, D. Marpe, C. Bartnik, S. Bosse, H. Brust, T. Hinz, H. Lakshman, P. Merkle, F. H. Rhee, G. Tech, M. Winken, and T. Wiegand, 3D high-efficiency video coding for multi-view and depth data, IEEE Trans. on Image Processing, Vol. 22, No. 9, pp. 3366-3378, Sept. 2013. [14] J. Jung and K. Viswanathan, Depth quadtree prediction for HTM," JCT3V-A0044, Stockholm, Sweden, July 2012. [15] X. Zhang, K. Zhang, J. An, J.-L. Lin, and S. Lei, CE2 related: A texture-partition-dependent depth partition for 3D-HEVC," JCT3V-G0055, San Jose, US, Jan. 2014. [16] D. Rusanovskyy, K. Mueller, and A. Vetro, Common test conditions of 3DV core experiments, JCT3V-E1100, Vienna, Austria, Aug. 2013.
1 : 3D-HEVC - 2013 2 : () - 2013 3 ~ : ( ) - :,, 3D - 1997 2 : - 1999 2 : ( ) - 2004 8 : ( ) - 2004 9 ~ 2010 2 : (ETRI) - 2010 3 ~ : - :,,