(JBE Vol. 18, No. 6, November 2013) (Regular Paper) 18 6, 2013 11 (JBE Vol. 18, No. 6, November 2013) http://dx.doi.org/10.5909/jbe.2013.18.6.850 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) 3D a),c), b), c) A Method of Frame Synchronization for Stereoscopic 3D Video Youngsoo Park a),c), Dohoon Kim b), and Namho Hur c) 3D., (rectification),., (mean of absolute difference(mad)) 3D., MAD 3D, H.264/AVC. Abstract In this paper, we propose a method of frame synchronization for stereoscopic 3D video to solve the viewing problem caused by synchronization errors between a left video and a right video using the temporal frame difference image depending on the movement of objects. Firstly, we compute two temporal frame difference images from the left video and the right video which are corrected the vertical parallax between two videos using rectification, and calculate two horizontal projection profiles of two temporal frame difference images. Then, we find a pair of synchronized frames of the two videos by measuring the mean of absolute difference (MAD) of two horizontal projection profiles. Experimental results show that the proposed method can be used for stereoscopic 3D video, and is robust against Gaussian noise and video compression by H.264/AVC. Keyword : stereoscopic, 3D, synchronization, projection a) (UST) b) (HAFS) c) (ETRI) Corresponding Author : (Namho Hur) E-mail: namho@etri.re.kr Tel: +82-42-860-6270 3D UHDTV (KCA-2011-11921- 02001). Manuscript received March 28, 2013 Revised September 13, 2013 Accepted October 21, 2013. [1]. (inter ocular distance; IOD),,, 65mm [2,3].
2 : 3D (Youngsoo Park et al. : A Method of Frame Synchronization for Stereoscopic 3D Video) (binocular field),, (monocular sector) [4]. [4]., (cyclopean image; Odyssey Cyclops, ) [1,4]. (binocular disparity) [5,6]. 2D, (,,,, ) [7,8]. 2D 3D [1]. 3D 3D 3D 3DTV [8,9]. 3D, 3D [5,8].,, (ghost effect) 3D [8]. 1/120, [10]., 3D,. (synchronization signal generator) (generation locking; gen-lock), (time code), [9],, 3D. 3D [11, 12],. 3D 3D.. 2, (horizontal projection profile) [12], 3.. 1. 3D,
(JBE Vol. 18, No. 6, November 2013),., [7]. (cepstrum) [14], (funda- mental matrix) (rectification) [15,16] [17]. [17].,. 2. 3D [18].,. 3D,,.,,,.., ( ). 3. mean absolute difference (MAD). MAD. ( ).. arg (6).
박영수 외 인 스테레오스코픽 동영상을 위한 동기화 방법 2 : 3D (Youngsoo Park et al. : A Method of Frame Synchronization for Stereoscopic 3D Video) (a) (b) (a) (b) (c) (d) (c) (d) (e) (f) (e) (f) 그림 1. 스테레오스코픽 3D 실험 동영상 : (a) Basket; (b) Boxers; (c) Hall; (d) Lab; (e) News report; (f) Phone call Fig. 1. Test stereoscopic 3D video contents : (a) Basket; (b) Boxers; (c) Hall; (d) Lab; (e) News report; (f) Phone call 853 그림 2. 비월주사방식의 스테레오스코픽 3D 실험 동영상 : (a) 100m; (b) Athletic; (c) ra-ra-ra; (d) f(x); (e) After school; (f) 5dolls Fig. 2. Test interlaced stereoscopic 3D video contents : ((a) 100m; (b) Athletic; (c) ra-ra-ra; (d) f(x); (e) After school; (f) 5dolls (a) (b) (c) (d) (e) (f) 그림 3. 제안 방법에 따른 MAD 결과 그래프 : (a) Basket; (b) Boxers; (c) Hall; (d) Lab; (e) News report; (f) Phone call Fig. 3. MAD graphs of proposed method : (a) Basket; (b) Boxers; (c) Hall; (d) Lab; (e) News report; (f) Phone call
(JBE Vol. 18, No. 6, November 2013) III. 1. 3D 1 6 3D - Basket, Boxers, Hall, Lab, News report, Phone call [19]. 6 3D 100m, athletic, ra-ra-ra, f(x), After school, 5dolls. Intel i5 CPU, 4GB RAM MATLAB. 4 -, (interlaced),,, H.264/AVC JM 18.4. 2.,,.. 3D [9], (1/16) HDTV [20] 4. 3 6 3D. 6, (a) (b) (c) (d) (e) (f) 4. MAD : (a) 100m; (b) Athletic; (c) ra-ra-ra; (d) f(x); (e) After school; (f) 5dolls Fig. 4. MAD graphs of proposed method in interlaced scan mode: (a) 100m; (b) Athletic; (c) ra-ra-ra; (d) f(x); (e) After school; (f) 5dolls
2 : 3D (Youngsoo Park et al. : A Method of Frame Synchronization for Stereoscopic 3D Video) (a) (b) (c) (d) (e) (f) 5. MAD ( -, - ) : (a) Basket; (b) Boxers; (c) Hall; (d) Lab; (e) News report; (f) Phone call Fig. 5. MAD graphs of robustness of proposed method against Gaussian noise (blue line - Gaussian noise added video, red dotted line - original video) : (a) Basket; (b) Boxers; (c) Hall; (d) Lab; (e) News report; (f) Phone call (d) (e) (f) (d) (e) (f) 6. (H.264/AVC, QP=32) MAD ( -, - ) : (a) Basket; (b) Boxers; (c) Hall; (d) Lab; (e) News report; (f) Phone call Fig. 6. MAD graphs of robustness of proposed method against video compression (H.264/AVC, QP=32) (green line - compressed video, red dotted line - original video) : (a) Basket; (b) Boxers; (c) Hall; (d) Lab; (e) News report; (f) Phone call
(JBE Vol. 18, No. 6, November 2013) MAD. 1.. 1. Table 1. The processing time of proposed method () () () Basket 2.00 0.52 2.52 Boxers 2.33 0.51 2.84 Hall 5.04 0.50 5.54 Lab 3.23 0.53 3.76 News report 2.04 0.52 2.56 Phone call 1.68 0.52 2.20 2.72 0.52 3.24 3. (progressive) () (interlaced) [21].,, [20].. 8. 4,. MAD. 4.. (additive white Gaussian noise; AWGN) [22], ( PSNR = 24.29 db). (high definition television; HDTV) PSNR = 35 db [23] H.264/AVC, ( PSNR = 35.12 db). 5, 6, 6 MAD 3D. 5. 3D,,.,.. IV. 3D
2 : 3D (Youngsoo Park et al. : A Method of Frame Synchronization for Stereoscopic 3D Video),.,,. 3D,. ( ) (H.264/AVC, ). (References) [1] M. J. Tovee, An Introduction to the Visual System, Cambridge University Press, UK, 2008. [2] M. J. Hirsch and F. W. Weymouth, Distance discrimination I Theoretic consideration, Archives of Ophthalmology, Vol.39, pp.210-223, 1948. [3] N. Dodgson, Validation and extrema of human interpupillary distance, Proc. SPIE Stereoscopic Displays and Virtual Reality Systems XI, Vol.5291, pp. 19-22, San Jose, USA, Jan 2004. [4] I. P. Howard and B. J. Rogers, Binocular vision and stereopsis, Oxford university Press, USA, 1995. [5] H. Harashima, 3D Image and human science, Ohmsa, Japan, 2010. [6] B. Mendiburu et al., 3D TV and 3D Cinema : Tools and Processes for Creative Stereoscopy, Focal Press, USA, 2011. [7] J. Kim et al., How three dimensional technology changes future, ETNEWS 2010. [8] B. Kim, The theory of 3D movie, MJ media, 2011. [9] Y. Chol et al., Stereoscopic 3D production workbook, KOCCA, 2010. [10] Y. Cho and J. Kim, The basic visual communications, Chang-ji co,, 1999. [11] H. Lee, "Stereo-video Synchronization for 3D Video Transmission," The Journal of KOREA Information and Communications Society, Vol. 34.4, pp.349-359, 2009. [12] J. Kim, S. Lee and J. Kim, "Synchronization Issues for Stereoscopic High-Definition Video Delivery over IP Networks.", HCI, pp. 1373-1378, 2006. [13] A. Zramdini et al., Optical font recognition from projection profile, Electronic Publishing, Vol. 6(3), pp. 249-260, Sep 1993. [14] P. W. Smith, An improved power cepstrum based stereo correspondence method for textured scene, Pattern Analysis and Machine Intelligence, IEEE Transactions on, Vol. 18, Issue 3, pp. 338-348, Mar 1996. [15] R. Hartley et al., Multiple view Geometry in Computer Vision, Cambridge Univ. Press, UK, 2nd, 2003. [16] J. Mallon, Projective rectification from the fundamental matrix, Image and Vision Computing, Vol.23, Issue 7, pp.643-650, Jul 2005. [17] Y. Park and N. Hur, A method of Pseudo-stereoscopic image detection and correction, KSBE conference in summmer, pp. 245-248, 2010. [18] A. J. Woods et al., Image distortions in Stereoscopic video system, In Stereoscopic Displays and Application IV, Vol. 1915, pp. 36-48, USA, 2005. [19] M. Urvoy et al., "NAMA3DS1-COSPAD1 : Subjective video quality assessment database on coding conditions introducing freely available high quality 3D stereoscopic sequences", Fourth International Workshop on Quality of Multimedia Experience, Yarra Valley, pp. 109-114, 2012. [20] KBS technical research institute, The study of broadcasting technology the trends of DTV transmission, 2007. [21] J. Kim et al., Introduction of Visual Communication, Hongrung Publishing Company, 2010. [22] S. Kim and K. Lim, Training-based noise reduction method considering noise correlation for visual quality improvement of recorded analog video, Journal of the IEEK, vol. 47, no. 6, pp. 28-38, 2010. [23] M. A. Isnardi, Historical overview of video compression in consumerelectronic devices, Proc. ICCE, pp. 1-2, 2007.
858 방송공학회논문지 제18권 제6호, 2013년 11월 (JBE Vol. 18, No. 6, November 2013) 저자소개 박영수 년 2월 : 전북대학교 전자정보공학부 학사 년 9월 ~ 현재 : 과학기술연합대학원대학교 (UST) 석, 박사 통합과정 주관심분야 3DTV - 2005-2005 - 김도훈 년 2월 : 한국외국어대학교 HAFS 졸업 예정 주관심분야 : 입체방송, 경영학 - 2014 - 허남호 - 년 2월 : 포항공과대학교 전기전자공학과 학사 년 2월 : 포항공과대학교 대학원 전기전자공학과 석사 년 2월 : 포항공과대학교 대학원 전기전자공학과 박사 년 4월 ~ 현재 : 한국전자통신연구원 방송시스템연구부 부장 년 9월 ~ 현재 : 과학기술연합대학원대학교 (UST) 겸임교수 주관심분야 차세대 방송기술 1992 1994 2000 2000 2005