High Resolution Disparity Map Generation Using TOF Depth Camera In this paper, we propose a high-resolution disparity map generation method using a low-resolution Time-Of- Flight (TOF) depth camera and a stereo camera. The TOF depth camera is efficient since it measures range information of objects using Infra-Red (IR) signal in real-time. It also quantizes the range and then provides depth images. However, there are some problems of the TOF depth camera such as noise and lens distortion. Moreover, the output resolution of the TOF depth camera is too small for 3D applications. Therefore, it is essential to not only reduce the noise and distortion but also enlarge the output resolution of the TOF depth image. The proposed method generates a disparity map for a color image using the TOF depth camera and the stereo camera that is used together. We reduce the noise and lens distortion in the TOF depth image and then warp the depth value at each pixel to the color image position. The color image is segmented using mean-shift segmentation, and the warped depth values are translated to the disparity values and filled the segmented regions. Experimental results show the generated disparity maps and reconstructed virtual view images, and verify that the proposed method efficiently generate the disparity maps. Keywords: Depth camera, Time-of-flight camera, Disparity map generation, 3DTV
I II
III
Z f B d fb Z (1) d
d(x,y) Σλ k d k (x,y) (2) k d(x,y) xy d k λ d k 1 k k d k (x,y) Σ Σ d t (x i, y j) (3) n k i kj k 2k d t d k n k
λ λ λ IV
k 3 V [1] C. Fehn, R. Barre, and S. Pastoor, "Interactive 3-DTV- Concepts and Key Technologies," Proceeding of the IEEE, Vol. 94, No. 3, Mar. 2006, pp. 524-538. [2],, 3DTV 3,, 2010. [3] ISO/IEC JTC/SC29/WG11 N8944, "Preliminary
FTV Model and Requirements," 2007. [4] P.F. Felzenszwalb and D.P. Huttenlocher, "Efficient Belief Propagation for Early Vision," International Journal of Computer Vision, Vol. 70, No. 1, Oct. 2006, pp. 41-54. [5] S.A. Gudmundsson, H. Aanaes, and R. Larsen, "Fusion of Stereo Vision and Time-of-Flight Imaging for Improved 3D Estimation," International Journal of Intelligent Systems Technologies and Applications, Vol. 5, No. 3, Nov. 2008, pp. 425-433. [6] J. Zhu, L. Wang, R. Yang, and J. Davis, "Fusion of Time-of-Flight Depth and Stereo for High Accuracy Depth Maps," Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2008, pp. 231-236. [7] K. D. Kuhnert and M. Stommel, "Fusion of Stereo- Camera and PMD-Camera Data for Real-time Suited Precise 3D Environment Reconstruction," Proc. of the IEEE/RSJ Int'l Conference on Intelligent Robots and Systems, Mar. 2006, pp. 4780-4785. [8] http://www.mesa-imaging.ch, SwissRanger SR 4000. [9] http://www.baslerweb.com, Basler Pylon GigE Camera pia I1900-32gm/gc. [10] Z. Zhang, "A Flexible New Technique for Camera Calibration," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 11, Nov. 2000, pp. 1330-1334. [11] http://www.vision.caltech.edu/bouguetj, Camera Calibration Toolbox for MATLAB. [12] A. Fusiello, E. Trucco, and A. Verri, "A Compact Algorithm for Rectification of Stereo Pairs," Machine Vision and Application, Vol. 12, No. 1, Jul. 2000, pp. 16-22. [13] D. Comaniciu and P. Meer, "Mean Shift: A Robust Approach Toward Feature Space Analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, No. 4, May 2002, pp. 603-619. [14] A. Wang, T. Qiu, and L. Shao, "A Simple Method of Radial Distortion Correction with Centre of Distortion Estimation," Journal of Mathematical Imaging and Vision, Vol. 35, No. 3, Jul. 2009, pp. 165-172. [15] G. Gilboa, N. Sochen, and Y.Y. Zeevi, "Regularized Shock Filters and Complex Diffusion", Proc. of European Conference on Computer Vision, Jun. 2002, pp. 399-313.