Depth layer partition 2D 3D a), a) 3D conversion of 2D video using depth layer partition Sudong Kim a) and Jisang Yoo a) depth layer partition 2D 3D. 2D (depth map). (edge directional histogram). depth layer partition.. Abstract In this paper, we propose a 3D conversion algorithm of 2D video using depth layer partition method. In the proposed algorithm, we first set frame groups using cut detection algorithm. Each divided frame groups will reduce the possibility of error propagation in the process of motion estimation. Depth image generation is the core technique in 2D/3D conversion algorithm. Therefore, we use two depth map generation algorithms. In the first, segmentation and motion information are used, and in the other, edge directional histogram is used. After applying depth layer partition algorithm which separates objects(foreground) and the background from the original image, the extracted two depth maps are properly merged. Through experiments, we verify that the proposed algorithm generates reliable depth map and good conversion results. Keywords : 2D/3D conversion, 3D conversion, 2D video, segmentation, motion information, depth layer partition, depth map generation. 3D a) Electronic Engineering Dept. Kwangwoon University : (jsyoo@kw.ac.kr) 2010 ( ) (-2010-0026245) (2010 9 10 ),(1 :2010 11 10,2 :12 2 ), (2010 12 20 ), 3D,,., LG, TV 3D TV,, 3D. 3D. 3D
,. 3D 2D 3D. 3D. 3D.. 3D 90. Ross. Ross [1]. T. Okino MTD(modified time difference) [2], Y. Matsumoto (motion parallax) [3]. [4], [5,6,7,8]. 2. 2 2D/3D. [9] [10]. MTD. [9], [10].,. depth layer partition. [11] [12]...,, depth layer partition, 3D... 2, 3 3D. 4 5.. 2D/3D 1. 3D,, (binocular disparity).,,. 1 Ross [1]...
,. 1. Ross 3D Fig. 1. Basic principle of 3D conversion based Ross phenomenon Ross.. 2 2D/3D. TV, DVD 2D. 2D 3D 2D/3D..,,, (delayed image).,,.. 3D. 3D 2D. (),., (parallax). 2. 2D/3D 3 2 3D 2D/3D... 3D 3 2D/3D. 2D. (depth map). 2. 2D/3D Fig. 2. Principle of 2D/3D conversion
,.,.. depth layer partition, 3D.. [12]. (1) [11]. i j (0 ~ 255) T.. i,. 3. 2D/3D Fig. 3. Flow chart of proposed 2D/3D conversion algorithm 1.,, (2) SAD(sum of absolute difference) Y. (2) n Y n+1 Y. dx dy.. [13]. 7x7, spiral search [14],. 4 i (i+1), i (i+2). (i+2)
4. Fig. 4. Change of search frame depending on existence of motion information.,,., i (i+1), (i-1). (global motion intensity) (local motion intensity). [15]. 2. 2.1 [9]..,,. (region merging). Y, U, V (3)., Y, Y. U, V. (3) Y, U, V. (0 ~ 5)... (4),.
(5). (x, y). x, y., [9]. 5. [10].,,,, 4. 6. (6) Sobel. dx dy i, j i, j = p = p i-1, j+ 1 i+ 1, j-1 + 2 p + 2 p i, j+ 1 i+ 1, j + p + p i+ 1, j+ 1 i+ 1, j+ 1 - p - p i-1, j-1 i-1, j-1-2 p - 2 p i, j-1 i-1, j - p - p i+ 1, j-1 i-1, j+ 1 2.2, (edge directional histogram),., [10]. (vanishing point) dxi,j dyi,j,. dxi,j dyi,j (7). r Ang ( D 180 æ dy ö r < è ø 0 i, j 0, ) arctanç i j =, Ang ( D, ) 90 i j p dxi, j 7 5 (a) (b) (c) 5. (a) (b) (c) Fig. 5. Process of segmentation and depth value aligning (a) segmentation (b) depth information (c) aligning mean value 6. - (a) (b) (c) (d) Fig. 6. Initial depth map image follow position of vanishing point - Position of vanishing point : (a) left (b) up (c) right (d) Object image
[10]. 7. Fig. 7. Range of edge direction histogram 1. 1.. 8.,. [10]. - Range0 : - Range1 : - Range2 : - Range3 : - Range4 : 2.3 Depth layer partition..,. 8. Fig. 8. Result image of separation to object and back ground,. 3. 3D 9 9., Fig. 9. Method of Left and right image generation
....,, 3D [16].... MTD [2] H.264 [4], [10]. DSCQS 10,, [17]. 720x480 (Walking with beasts, BBC, 2001) (,,, 2007) (, /,, 2008) ( -,,, 2002) 10. (a) (b) (c) 3D Fig. 10. Final result image (a) Original image (b) Depth map image (c) Generated 3D image
4, 21. 32 3D 1m.,. 1~5 1.. 1,,. MTD, H.264,.. depth layer partition 2D/3D. 3D. 10 3D. 1. (5 ) Table 1. Result of DSCQS MTD H.264 2.3 2.2 2.6 2.9 2 2.1 1.9 3.3 1.9 2.1 1.5 2.4 2.1 2.1 2 2.7. depth layer partition 2D 3D.,., 3D..,.,, depth layer partition. 3D,... [1] Ross, J., "Stereopsis by binocular delay," Nature, vol. 248, pp. 354-364, 1974. [2] T. Okino and et. al, "New television with 2D/3D image conversion techniques," Proceedings of SPIE, vol. 2653, pp. 96-103, 1995. [3] Y. matsumoto, H. Terasaki, K. Sugimoto and T. Arakawa, "Conversion system of monocular image sequence to stereo using motion parallax," Proceedings of SPIE Stereoscopic Displays and Virtual Reality Systems, vol. 3012, pp. 108-115, May 1997. [4],,,,, 2D H.264 3D
,, 3112C, pp. 1208-1215, 2006 12. [5] D. F. McAllister, Stereo computer graphics and other true 3D technologies, Princeton, NJ:Princrton Univercity Press, 1993. [6] S. Battiato, S. Curti, M. LaCascia, E. Scordato, M. Tortora, "Depth-Map Generation by Image Classification," Proceedings of SPIE Electronic Imaging 2004, Three-Dimensional Image Capture and Applications VI, vol. 5302, pp 95-104, San Jose, California, USA, Jan. 2004. [7] C. Fehn, "Depth-Image-Based Rendering (DIBR), Compression and Transmission for a New Approach on 3D-TV," Proceedings of SPIE Stereoscopic Displays and Virtual Reality Systems XI, pp. 93-104, San jose, CA, USA, Jan. 2004. [8] Man-Bae Kim, Mun-Sup Song, Do-Kyoon Kim, Kwang-Chul Choi, "Stereoscopic conversion of monoscopic video by the transformation of vertical-to-horizontal disparity," Proceedings of SPIE, vol. 3295, pp. 65-75, 1998. [9],,,, 2,, 33 1, pp. 53-61, 2008 1. [10],, " 2 3," 34 8, pp. 782-789, 2009 8. [11],,,,, 29 9C, pp. 1269-1278, 2004 9. [12],,,,,,,, 2010 7. [13] Y. Wang, J. Ostermann, Y. Zhang, Video processing and communication, Prentice Hall, 2002. [14] Y. Fisher, Fractal Image Compression: Theory & Application, Springer, 1995. [15] KuK-Jin Yoon, In-So Kweon, Moving Object Segmentation with Accurate Boundary Using Color and Motion," in Workshop on Image Processing and Image Understanding, pp. 150-155, 2000. [16] Lenny Lipton, "Stereo-vision formats for video and computer graphics," Proceedings of SPIE, vol. 3012, pp. 239-244, 1997. [17] Methodology for subjective assessment of the quality of television picture," ITU-R Recommendation BT.500-11. - 2009 2 : - 2009 3 ~ : - : 3,, 2D/3D - 1985 : - 1987 : - 1993 : Purdue - 1997 ~ : - : 3D /,,