(JBE Vol. 22, No. 2, March 2017) (Regular Paper) 22 2, 2017 3 (JBE Vol. 22, No. 2, March 2017) https://doi.org/10.5909/jbe.2017.22.2.240 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) MV-HEVC a), a) Fast Disparity Motion Vector Searching Method for the MV-HEVC Jae-Yung Lee a) and Jong-Ki Han a) High Efficiency Video Coding(HEVC), disparity compensation prediction(dcp). MV-HEVC.. 90.78%.,. Abstract Multi-view video codec based on the High Efficiency Video Coding (MV-HEVC) has high encoding complexity because it exploits an additional reference picture for disparity compensation prediction (DCP) when the picture of dependent view is encoded. In this paper, we propose an efficient method to reduce the complexity of disparity motion vector search for the MV-HEVC. The proposed method includes the initial search point decision method using affine transform and the adaptive search range decision method. The simulation results show that the proposed method reduces the complexity of disparity motion vector search up to 90.78% with negligible coding efficiency degradation. Also the results show that the proposed method outperforms other conventional techniques reducing complexity. Keyword : Multi-view HEVC (MV-HEVC), Disparity Searching, Multiview Video Coding (MVC), Search Range, Affine Transform a) (Sejong University) Corresponding Author : (Jong-Ki Han) E-mail: hjk@sejong.edu Tel: +82-3408-3739 ORCID: http://orcid.org/0000-0002-5036-7199 2015 (No. 2015R1A2A2A01006193). Manuscript received February 3, 2017; Revised March 15, 2017; Accepted March 17, 2017.. 2013 1 VCEG(Video Coding Expert Group) MPEG(Moving Picture Expert Group) Joint Collaborative Team on Video Coding (JCT-VC) Copyright 2017 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND (http://creativecommons.org/licenses/by-nc-nd/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.
1 : MV-HEVC (Jae-Yung Lee et al.: Fast Disparity Motion Vector Searching Method for the MV-HEVC) High Efficiency Video Coding (HEVC) [1][2]. VCEG MPEG Joint Collaborative Team on 3D Video(JCT-3V) HEVC MV- HEVC [3][4] 2014 10, 3D-HEVC [5][6] 2015 2. MV-HEVC Disparity Compensation Prediction(DCP). HEVC., 3D-HEVC. 3D-HEVC HEVC. HEVC, MV- HEVC,. HEVC Motion Compensation Prediction(MCP), MV-HEVC MCP DCP., DCP /,. MV-HEVC HEVC. [7][8][9]. [7], [8] Temporal layer. MV-HEVC. RAP(Random Access Point).., MV-HEVC DCP. MV-HEVC, MV-HEVC... HEVC, MV-HEVC HEVC, HEVC. MV-HEVC 2 1. 1 2, (View 0) (View 1), HEVC.,.
242 방송공학회논문지 제22권 제2호, 2017년 3월 (JBE Vol. 22, No. 2, March 2017) 그림 1. 다시점 영상 부호화의 예측 구조 Fig. 1. Prediction structure of multiview video coding 한 영상을 부호화 할 때, 부호화되는 영상과 동일한 시간에 상을 이용하여 화면 간 예측을 사용하는 경우, 움직임 벡터 대응하는 독립 시점의 영상을 종속 시점 영상의 참조 영상 (Motion Vector, MV)를 사용하여 MCP를 하며, 시점 간 참 리스트에 추가적인 참조 영상으로 사용하는 것이다. 따라 조 영상을 이용하여 화면 간 예측을 사용하는 경우, 변위 서, 독립 시점의 비디오 데이터는 다른 시점의 복호 여부와 움직임 벡터(Disparity Motion Vector, DV)를 사용하여 상관없이 독립적으로 복호가능한 반면에 종속 시점의 비디 DCP를 한다. 오 데이터는 독립 시점의 비디오 데이터가 복호된 이후에 기반 다시점 영상 부호화에서 움직임 탐색 복호가 가능하다. 종속 시점의 영상 부호화에 있어서, 참조 1. HEVC 영상의 종류를 2개로 구분할 수 있다. 구분되는 참조 영상 종류는 부호화되는 영상과 동일한 시점이나 다른 시간에 MV-HEVC에서는 그림 2와 같이 현재 블록이 종속 시점 있는 참조 영상인 시간적 참조 영상과 부호화되는 영상과 의 영상에 존재하는 경우, 시간적 참조 영상에서 시간적 움 동일한 시간이나 다른 시점에 있는 참조 영상인 시점 간 직임 벡터를 찾고, 시점 간 참조 영상에서 변위 움직임 벡터 참조 영상이다. 현재 부호화되는 영상에서 시간적 참조 영 를 찾는다. 이때, 시점 간 참조 영상에서 변위 움직임 벡터 그림 2. 다시점 영상 부호화에서 시간적 움직임 벡터와 변위 움직임 벡터 Fig. 2. Temporal motion and disparity motion vector for multiview video coding
1 : MV-HEVC (Jae-Yung Lee et al.: Fast Disparity Motion Vector Searching Method for the MV-HEVC).,. 3 MV-HEVC. 3,. MV-HEVC AMVP(Ad- vanced Motion Vector Prediction). AMVP SAD (Sum of Absolute Difference) SAD AMVP. 2 MV-HEVC. 1 77.59%, 22.41%. 2 1 19.40%, 1. 1 MV-HEVC Table 1 Comparison between temporal motion estimation complexity and disparity motion vector estimation complexity Sequence Temporal Motion Estimation (%) Complexity disparity motion vector Estimation (%) Balloons 77.10% 22.90% Kendo 77.20% 22.80% PozanaHall2 77.78% 22.22% PozanaStreet 77.71% 22.29% 3. MV-HEVC Fig. 3. Initial search point for disparity motion vector estimation in the MV-HEVC, (Motion Cost). MV-HEVC 1/4. 2. HEVC 1 2 GhostTownFly 77.72% 22.28% Shark 77.83% 22.17% UndoDancer 77.77% 22.23% Average 77.59% 22.41% 1, MV-HEVC., MV-HEVC. Camera Calibration [10~15]. MV- HEVC Camera Calibration
(JBE Vol. 22, No. 2, March 2017)., Camera Calibration.,...,,. 1. Affine Motion Model.., 4 4. Fig.4. The Disparity motion vector between two neighboring views, (1),. (1) (2). (3),...,., 5 RAP(Random Access Point) Key picture, Key picture RAP Key picture., Key picture B-picture QP. Key picture.
1 : MV-HEVC (Jae-Yung Lee et al.: Fast Disparity Motion Vector Searching Method for the MV-HEVC) 5. Fig.5. Initial search point decision using the affine transform 6 Key picture k, (3).. Fast DE. (4) Key picture. LMS(Least Mean Square),,,,,,., Key picture Fast DE,, Key picture Key picture 6. Key Fig.6 Disparity motion vectors in the key picture
(JBE Vol. 22, No. 2, March 2017) Key picture (3)..,. 1/4 7 Fast DE., MV-HEVC 1/4 1/4 7. Fig. 7. Initial search point for disparity motion vector estimation in the proposed method 2...,. 8 Key picture. 8 k, k., 8. Fig. 8. Search range decision method for disparity motion vector estimation
1 : MV-HEVC (Jae-Yung Lee et al.: Fast Disparity Motion Vector Searching Method for the MV-HEVC)., Fast DE. Key picture, 9 Fast DE. 9. Fig. 9. Proposed search range for disparity motion vector estimation.. 2, JCT-3V HTM16.2 [16]. 2, Full search. JCT-3V Com- mon Test Condition(CTC) [17]. 2. Table 2. Simulation condition Coding structure QP setting Random Access Independent view : defualt QP Dependent view : defualt QP+3 Intra period 24 Number of frames 100 Search method Full search Search range 64 Sequence Balloons(1024x768) Kendo(1024x768) GTFly UndoDancer PoznanHall2 PoznanStreet Shark 10 RD- curve. 10 (a), (b), (c), (d) Balloons, Kendo, GhosTownFly, PoznanHall2 RD-curve, MV-HEVC. 11 MV-HEVC. 11 Anchor MV-HEVC, Ref[7] Ref[8]. Proposed. 11 MV-HEVC. (8) MV-HEVC,, MV-HEVC Full Search
(JBE Vol. 22, No. 2, March 2017) (a) Balloons (b) Kendo (c) GhostTownFly (d) PoznanHall2 10. R-D Fig. 10. Rate-Distortion performance comparison 11. Fig. 11. Disparity motion vector Estimation Complexity
1 : MV-HEVC (Jae-Yung Lee et al.: Fast Disparity Motion Vector Searching Method for the MV-HEVC),., 11. 3. 3 BD-rate(Y) [18] Y-component BD-rate,. DE time[%] (8). 3 MV- HEVC 90.78% 0.10%., 19.10%~31.37%. 3, GTFly. GTFly SKIP 60%., GTFly. GTFly SKIP. 4. 4 JCT-3V CTC, TZ-Search., 64., 4., JCT-3V TZ-Search. 4 Key. 3. Table 3. Performance evaluation of proposed method Sequence Balloons (1024x768) Kendo (1024x768) PoznanHall2 PoznanStreet GTFly Shark UndoDancer [7] [8] Proposed method BD-rate(Y) DE time[%] BD-rate(Y) DE time[%] BD-rate(Y) DE time[%] 0.04% 27.87% -0.01% 45.26% -0.04% 8.85% 0.01% 27.83% 0.81% 22.33% -0.13% 7.67% -0.16% 27.61% 0.14% 38.81% -0.37% 10.50% 0.13% 27.67% 0.06% 25.74% -0.02% 10.68% 0.02% 27.56% 0.06% 49.23% 1.26% 8.23% 0.00% 29.50% -0.10% 46.71% 0.04% 7.90% -0.01% 30.23% -0.09% 56.05% -0.02% 10.72% Average 0.00% 28.32% 0.12% 40.59% 0.10% 9.22%
(JBE Vol. 22, No. 2, March 2017) (9), Key, Key.,.,. 0.36%,. 3 4,,. (10),. 4, 0.24%,., Key.. MV-HEVC 90.78%, 0.10%., Camera calibration,. 4. Affine Rigid Table 4. Performance comparison between affine motion model and rigid motion model Sequence Balloons (1024x768) Kendo (1024x768) PoznanHall2 PoznanStreet GTFly Shark UndoDancer BD-rate(Y) Affine Motion Model KeyPic Enc Time (%) Enc.Time (%) BD-rate(Y) Rigid Motion Model KeyPic Enc Time (%) Enc.Time (%) -0.01% 101.45% 101.15% 0.17% 101.72% 100.88% -0.25% 100.66% 100.50% 0.22% 102.97% 100.45% -0.44% 100.99% 99.38% -0.40% 100.76% 99.38% 0.01% 101.55% 100.24% 0.33% 101.00% 99.85% 0.06% 101.21% 100.83% 0.56% 101.87% 100.93% -0.12% 99.59% 98.98% -0.01% 99.82% 98.49% 0.00% 99.87% 98.97% 0.08% 99.72% 98.68% Average -0.11% 100.76% 100.01% 0.13% 101.12% 99.81%
1 : MV-HEVC (Jae-Yung Lee et al.: Fast Disparity Motion Vector Searching Method for the MV-HEVC) Key., Key,..,. (References) [1] High Efficiency Video Coding, Rec. ITU-T H.265 and ISO/IEC 23008-2, Jan. 2013. [2] G. J. Sullivan, J. Ohm, W.-J. Han, and T. Wiegand, Overview of the high efficiency video coding (HEVC) standard, IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 12, pp. 1649 1668, Dec. 2012. [3] G. Sullivan, J. Boyce, Y. Chen, J.-R. Ohm, C. Segall, and A. Vetro, Standardized extensions of high efficiency video coding (hevc), IEEE Journal of Selected Topics in Signal Processing, vol. 7, no. 6, pp. 1001 1016, Dec. 2013. [4] G. Tech, K. Wegner, Y. Chen, M. M. Hannuksela, and J. Boyce, MV-HEVC draft text 5, in Joint Collaborative Team on 3D Video Coding Extensions (JCT-3V) Document JCT3V-E1004, 5th Meeting: Vienna, Austria, Jul. 27 Aug. 2 2013. [5] G. Tech, K. Wegner, Y. Chen, and S. Yea, 3D-HEVC draft text 1, in Joint Collaborative Team on 3D Video Coding Extensions (JCT-3V) Document JCT3V-E1001, 5th Meeting: Vienna, Austria, July 27 Aug. 2 2013. [6] G. Tech, Y. Chen, K. Muller, J.-R Ohm, A, Vetro, and Y. Wang "Overview of the Multiview and 3D Extensions of High Efficiency Video Coding", IEEE Trans. on CSVT, Vol. 26, no. 1, pp. 35-49, Jan. 2016. [7] Y.-T. Chang and W.-H. Chung, An adaptive search range algorithm for multiview motion and disparity estimation, Information Science, Signal Processing and their Applications (ISSPA), 2012 11th International Conference on, pp.550-554, 2-5 July 2012. [8] S. Khattak, R. Hamzaoui and S. Ahmad, Low-complexity multiview video coding, Picture Coding Symposium (PCS), pp. 97-100, Krakow, Poland, 7-9 May 2012. [9] Y. Chen, X. Zhao, L. Zhang and J. -W. Kang, Multiview and 3D Video Compression Using Neighboring Block based Disparity Vectors, IEEE Transaction on Multimedia, vol.18, no. 4, pp.576-589, Apr. 2016. [10] Dong-Hoon Han, Suk-Hee Cho, Nam-Ho Hur,Yung-Lyul Lee, Fast Mode Decision using Global Disparity Vector for Multi-view Video Coding, JBE, Vol. 13, No. 3, pp. 328-338, May, 2008. [11] Yoon Jin Lee, Dong In Bae, Gwang Hoon Park, HEVC Encoder Optimization using Depth Information, JBE, 19, No. 5, pp. 640-655, September, 2014. [12] Daemin Park, Haechul Choi, Performance Analysis of 3D-HEVC Video Coding, JBE, Vol. 19, No. 5, pp. 713-725, September, 2014. [13] N. Boonthep, W. Chiracharit and K. Chamnongthai, Adaptive search range determination for geometry based disparity and motion estimation of MVC, Asia-Pacific Signal and Information Processing Association (APSIPA), 2014 Annual Summit and Conference on, pp.1-5, 9-12 Dec. 2014. [14] B. W. Micallef, C. J. Debono and R. A. Farrugia, Low complexity disparity estimation for immersive 3D video transmission, Communications Workshops (ICC), 2013 IEEE International Conference on, pp.612-616, 9-13 June 2013. [15] B. W. Micallef, C. J. Debono, and R. A. Farrugia, Fast disparity estimation for Multi-view plus depth video coding, 2011 Visual Communications and Image Processing (VCIP), pp. 1-4, 6-9 Nov. 2011. [16] https://hevc.hhi.fraunhofer.de/svn/svn_3dvcsoftware/tags/htm- 16.2/ [17] K. Müller and A. Vetro, Common Test Conditions of 3DV Core Experiments in Joint Collaborative Team on 3D Video Coding Extensions (JCT-3V) Document JCT3V-G1100, 7th Meeting, San José, US, 11 17 Jan. 2014. [18] G. Bjontegaard, Calculation of Average PSNR Differences Between RDCurves, document VCEG-M33, ITU-T Q.6/SG16 VCEG, Apr. 2001.
5 2 2 방송공학회논문지 제22권 제2호, 2017년 3월 (JBE Vol. 22, No. 2, March 2017) 저자소개 이재영 - 년 : 세종대학교 정보통신공학과 학사 년 : 세종대학교 정보통신공학과 공학석사 년 3월 ~ 현재 : 세종대학교 정보통신공학과 박사과정 년 12월 ~ 2016년 1월 : UC San Diego Visiting Scholar : http://orcid.org/0000-0001-5049-2574 주관심분야 : 비디오 코덱, 영상 신호처리, 정보 압축, 방송 시스템 2011 2013 2013 2014 ORCID 한종기 - 년 : KAIST 전기및전자공학과 공학사 년 : KAIST 전기및전자공학과 공학석사 년 : KAIST 전기및전자공학과 공학박사 년 3월 ~ 2001년 8월 : 삼성전자 DM 연구소 책임연구원 년 9월 : 세종대학교 정보통신공학과 교수 년 9월 ~ 2009년 8월 : UC San Diego Visiting Scholar : http://orcid.org/0000-0002-5036-7199 주관심분야 : 비디오 코덱, 영상 신호처리, 정보 압축, 방송 시스템 1992 1994 1999 1999 2001 2008 ORCID