(Regular Paper) 19 6, 2014 11 (JBE Vol. 19, No. 6, November 2014) http://dx.doi.org/10.5909/jbe.2014.19.6.866 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) RGB-Depth - a), a), b), a) Real-Virtual Fusion Hologram Generation System using RGB-Depth Camera Joongseok Song a), Jungsik Park a), Hanhoon Park b), and Jong-Il Park a) (CG) 3. RGB-Depth - 3, GPU (CGH). RGB-Depth, 3. RGB-Depth Z, GPU CGH., 3 3 0.5138%, 99%, - GPU. Abstract Generating of digital hologram of video contents with computer graphics(cg) requires natural fusion of 3D information between real and virtual. In this paper, we propose the system which can fuse real-virtual 3D information naturally and fast generate the digital hologram of fused results using multiple-gpus based computer-generated-hologram(cgh) computing part. The system calculates camera projection matrix of RGB-Depth camera, and estimates the 3D information of virtual object. The 3D information of virtual object from projection matrix and real space are transmitted to Z buffer, which can fuse the 3D information, naturally. The fused result in Z buffer is transmitted to multiple-gpus based CGH computing part. In this part, the digital hologram of fused result can be calculated fast. In experiment, the 3D information of virtual object from proposed system has the mean relative error(mre) about 0.5138% in relation to real 3D information. In other words, it has the about 99% high-accuracy. In addition, we verify that proposed system can fast generate the digital hologram of fused result by using multiple GPUs based CGH calculation. Keyword : Digital hologram, parallel processing, computer graphics a) (Department of Computer Software, Hanyang University) b) (Department of Electronic Engineering, Pukyong National University) Corresponding Author : (Jong-Il Park) E-mail: jipark@hanyang.ac.kr Tel: +82-2-2220-0368 IT. (NIPA-2014-H0401-14-1001) Manuscript received August 4, 2014 Revised Ooctober 16, 2014 Accepted Ooctober 16, 2014
3 : RGB-Depth - (Joongseok Song et al. : Real-Virtual Fusion Hologram Generation System using RGB-Depth Camera). (CG: computer graphics) [1-2]. CG CG. CG., CG. CG. CG 3D. 3D (stereoscopic). 3D.. 3D, [3]. 3D..., Brown Lohmann Computer-Generated Holography(CGH) [4]... CGH 3D (x, y, z), CG 3D CG 3D. 3D. RGB-Depth 3D, GPU CGH 3D.. - 1..., RGB- 1. - Fig. 1. Real-virtual fusion hologram generation system
2. - Fig. 2. The flowchart of real-virtual fusion hologram generation system Depth. RGB-Depth. RGB-Depth Z [5]. GPU CGH. 2 -. 1. - CG 3D. 3D... 1., s, [6].. [7]., RGB-Depth. RGB-
3 : RGB-Depth - (Joongseok Song et al. : Real-Virtual Fusion Hologram Generation System using RGB-Depth Camera) Depth. OpenGL [12] ARToolkit [7]. 0 1. RGB-Depth,. Z [5], 0 1 Z. 2 0 1. far near, RGB-Depth. Depth. far near. 3 far near 3. Near Far Fig. 3. Near plane and Far plane. near, far., far near. 0 1, Z. Z 3D,.. 4 Z () 3D 4. 3D Z Fig. 4. 3D object and Z buffer () 3D Z
[8]. Z, occlusion. (3). 5 3D CGH,. CGH. [9-10], GPU CGH [10]. 2. - 1. CGH Table 1. CGH Optics Parameter, CGH. (4) CGH. cos parameter contents digital hologram Index 3D object Index digital hologram 3D object coordinates of digital hologram coordinates of 3D object pixel pitch 3D, x, y z 3D.. 5. Fig. 5. Generation of digital hologram CG RGB-Depth. 6 RGB-Depth Kincet, 640 480. RGB-Depth 800 4000, (2) near 800, far 4000. 6 [7]. 100, ARToolkit. 3D GPU -
3 : RGB-Depth - (Joongseok Song et al. : Real-Virtual Fusion Hologram Generation System using RGB-Depth Camera) () RGB-Depth, 6. Fig. 6. Experimental environment. 1. 3. 6-() 255. 255. RGB-Depth,. (5) (mean relative error).. RGB-Depth, 400. RGB-Depth () 400.. 7-(),()., 0.5138%,. 2., 6-(). 8. RGB-Depth. ARToolkit,.
() () () () () 7. Fig. 7. The comparison of depth data between real and virtual object () () 8. Fig. 8. The virtual object RGB-Depth Z. 9 (),() RGB-Depth, 10-(),() Z.
3 : RGB-Depth - (Joongseok Song et al. : Real-Virtual Fusion Hologram Generation System using RGB-Depth Camera) () 1 () 2 9. Fig. 9. Reality space image city model and marker () 1 () 2 () 1 () 2 () 1 () 2 10. Fig. 10. Experimental results
GPU CGH [10]. 200 200 8bit, 1,024 1,024pixel. (λ) 532, 1000, Pixel pitch(p) 8um 8um. CGH GPU GTX 680, 2, CUDA GPU CGH [10]. 10-(),() CGH, 10-(),(). 10-(),(), RGB-Depth. RGB-Depth (hole) Joint bilateral filter [11]. 10-(),() occlusion.,. 11, Fresnel transform.,. 12,. CGH, 1,024 1,024 200 200. 2 1000 1,024 1,024. CPU GPU () 900mm () 1000mm () 1100mm 11. Fig. 11. The reconstruction of digital hologram with various reconstruction distance () - () 256 256 () 512 512 () 1024 1024 () 2048 2048 12. Fig. 12. The resolution of digital hologram and quality of reconstruction
3 : RGB-Depth - (Joongseok Song et al. : Real-Virtual Fusion Hologram Generation System using RGB-Depth Camera) 11,710. 2. CGH Table 2. CGH computational time computational time (ms) CPU 86,075.50 Multiple GPU 7.35 Speed ratio 11,710.9 : 1. RGB-Depth., 3 3 0.5138% 99%, - GPU., CGH. (References) [1] Debevec, Paul E., Camillo J. Taylor, and Jitendra Malik. "Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach." Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. ACM, 1996. [2] Debevec, Paul. "Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography." ACM SIGGRAPH 2008 classes. ACM, 2008. [3] Gabor, Dennis. "Theory of communication. Part 1: The analysis of information." Journal of the Institution of Electrical Engineers-Part III: Radio and Communication Engineering 93.26 (1946): 429-441. [4] Brown, Bryon R., and Adolf W. Lohmann. "Complex spatial filtering with binary masks." Applied Optics 5.6 (1966): 967-969. [5] Foley, James D., et al. "Computer Graphics Principles and Practice, Assison-Wesley.", pp 668, Massachusetts (1996). [6] Zhang, Zhengyou. "A flexible new technique for camera calibration." Pattern Analysis and Machine Intelligence, IEEE Transactions on 22.11 (2000): 1330-1334. [7] Kato, Hirokazu, and Mark Billinghurst. "Marker tracking and hmd calibration for a video-based augmented reality conferencing system." Augmented Reality, 1999.(IWAR'99) Proceedings. 2nd IEEE and ACM International Workshop on. IEEE, 1999. [8] www.felixgers.de/teaching/jogl/depthbufferalgo.html [9] Lucente, Mark E. "Interactive computation of holograms using a look-up table." Journal of Electronic Imaging 2.1 (1993): 28-34. [10] Song, Joongseok, et al. "Real-time generation of high-definition resolution digital holograms by using multiple graphic processing units." Optical Engineering 52.1 (2013): 015803-015803. [11] Kopf, Johannes, et al. "Joint bilateral upsampling." ACM Transactions on Graphics (TOG). Vol. 26. No. 3. ACM, 2007. [12] OpenGL reference pages : https://www.opengl.org/sdk/docs/man/ - 2010 : - 2010 ~ 2012 : - 2012 ~ : - : 3 /,, GPGPU
- 2010 : - 2010 ~ 2012 : - 2012 ~ : - : 3,, GPGPU - 2000 2 : - 2002 2 : - 2007 8 : - 2007 9 ~ 2008 10 : BK21-2008 11 ~ 2011 10 : NHK - 2011 11 ~ 2012 2 : - 2012 3 ~ : - :,, 3 / - 1987 : - 1989 : - 1995 : - 1992 ~ 1994 : NHK - 1995 ~ 1996 : - 1996 ~ 1999 : ATR - 1999 ~ : - :,, 3,