: 565 (Special Paper) 7 4, 0 7 (JBE Vol. 7, No. 4, July 0) http://dx.doi.org/0.5909/jbe.0.7.4.565 a), b), a) Depth Map Denoising Based on the Common Distance Transform Sung-Yeol Kim a), Manbae Kim b), and Yo-Sung Ho a). (joint bilateral filter).... Abstract During depth data acquisition and transmission, the quality of depth maps is usually degraded by physical noise and coding error. In this paper, a new joint bilateral filter based on the common distance transform is presented to enhance the low-quality depth map. The proposed method determines the amount of exploitable color data according to distance transform values of depth and color pixels. Consequently, the proposed filter minimizes noise in the depth map while suppressing visual artifacts of joint bilateral filtering. Experimental results show that our method outperforms other conventional methods in terms of noise reduction and visual artifact suppression. Keyword : depth map, denoising, distance transform, joint bilateral filter.. a) (Gwangju Institute of Science and Technology, School of Information & Communications) b) (Kangwon National University, IT College, Dept. of Computer & Communications) : (Manbae Kim) E-mail: manbae@kangwon.ac.kr Tel: +8--50-690, Fax: +8--5-690. [KI00058, IP ] (057),(077),(077). []., [].,
(JBE Vol. 7, No. 4, July 0)., [].,,. [4,5,6] [7,8,9]., (bilateral filter, BF) [4,5]. BF., BF,. (joint bilateral filter, JBF) [7,8]. JBF,., BF., JBF,. JBF (visual artifacts) [,]. (common distance transform) JBF. (modal similarity)...,.,,. JBF ().,. JBF () (spatial weighting function) (color weighting function),. (). exp, exp.,..,., BF () (4). (range weighting function), (5). exp.,., (adaptive joint bilateral filter, A-JBF) [9, 4] (6),,.
: 567..,.. JBF. ) (edge map). ). ). 4). 5),,. 6) JBF. [].,, [5].,, 0., - (7). min,.,.. (a) (b) -. (a) 4. 5.. (homogeneous area). 0,. (non-homogeneous area). 4 4 5 6 4 0 0 0 0 0 0 깊이맵의경계화소 5 4 0 0 0 0 0 5 4 0 4 0 0 0 0 색상영상의경계화소 0 0 0 (a) (b) (c) 0 0 0 4 0 0 4 4 5 5 4 5 6 6 0 0 0 0 0 0 0 0 4 0. (a) (b) (c) Fig.. Generation of the common distance transform map (a) distance transform of a depth map, (b) distance transform of a color image, and (c) common distance transform values,.,. i f i f,. (c) =, =, (a) (b)., -, 6a, a.
(JBE Vol. 7, No. 4, July 0). JBF. (c) 0,. A-JBF.,,., JBF BF.,.,. JBF () (9). i f (weighting control function). (), (5). (8)., (0). i f log exp i f,..,.. (6) A-JBF.,.,.,.,,. (9)..., Middlebury Stereo [0] Baby, Bowling, Cone, Midd, Monopoly, Teddy. (ground truth depth map). (Gaussian filter, GF) [6], BF [5], JBF [7], A-JBF [9, 4].,, (peak signal to noise ratio, PSNR).,,,,,,.5, 6,,,, 0., 0., -. PSNR., σ=0, GF, BF, JBF, A-JBF, PSNR.77 db, 7.4 db,.9 db, 8.00 db, 8. db. PSNR GF, BF, JBF, A-JBF 6.56 db, 0.9 db, 6.04 db, 0. db. σ=0, PSNR 5. db, 0.6 db, 4.4 db, 0.48 db. PSNR GF, BF, JBF, A-JBF 5.88 db, 0.75 db, 5. db, 0.4 db. Teddy Monopoly. (a) (σ=0). (b), BF., Teddy., (c) (d), JBF A-JBF., Monopoly., JBF A-JBF
: 569., (e),. (i) (ii). PSNR (σ=0, : db) Table. PSNR comparison(σ=0, unit: db) GF BF JBF A-JBF Baby 5.00 8.65 5.5 9.6 9.59 Bowling.77 8.75.6 9.59 9.64 Cone 9.0 6.86 9.45 7.74 7.90 Midd.49 7.55.46 8.6 8.44 Monopoly.85 8. 4.9 9.0 9.49 Teddy 8.0 4.65 8.54 4.5 5.00.77 7.4.9 8.00 8.. PSNR (σ=0, : db) Table. PSNR comparison(σ=0, unit: db) GF BF JBF A-JBF Baby.78 5.9 4.66 5.79 6.0 Bowling.89 5.78.7 6.7 6.40 Cone 8.8 4.0 9.06.78 4.6 Midd.48 4.67.70 4.57 4.97 Monopoly.0 5.76.58 5.69 6.44 Teddy 8.6 7.5 8.87 8.7 8.54.00 5.60.77 5.7 6.0 GB Ram.0 GHz GPU PC.. 6 5.. BF, JBF, A-JBF 0.6, 4., 4.7. BF. JBF A-JBF.. (iii) (iv) (a) (b) (c) (d) (e)., (i), (iii) Teddy, Monopoly, (ii), (iv), (a) (b) BF (c) JBF (d) A-JBF (e). Fig.. Results of depth map denoising, (a) input depth map and ground truth data, (b) results of bilateral filter and difference image, (c) results of joint bilateral filter, (d) results of adaptive joint bilateral filter, and (e) results of the proposed method., MPEG DVC Undo_ Dancer [7]. Undo_Dancer 9, 50. Undo_Dancer. Undo_Dancer DV- ATM 0. [8], 5 IP.. 4.,. Undo_Dancer 0 PSNR. 50 BF JBF.0 db.6 db PSNR. 4 Undo_Dancer 0. 4(a), 4(b). 4(c)
(JBE Vol. 7, No. 4, July 0).,.. BF.,... Undo_Dancers PSNR Fig.. PSNR comparison of Undo_Dancer (i).. Middlebury,,,, 5.88 db, 0.75 db, 5. db, 0.4 db PSNR., MPEG DV Undo_Dancer,. (ii) (a) (b) (c) 4. Undo_Dancer (a) (b) (c) Fig. 4. Results of Undo_Dancer (a) ground truth depth map and its color image, (b) input depth map and color image, and (c) common distance transform map and output depth map enhanced by the proposed method 4(c) (i)...., [] C. Fehn, A D-TV system based on video plus depth information, in Proc. of ACSSC, vol., pp. 59-5, 00. [] C. Fehn, R. Barré, and S. Pastoor, Interactive -D TV- concepts and key technologies, Proceedings of the IEEE, vol. 94, no., pp. 54-58, 006. [] S.U. Yoon and Y.S. Ho, Multiple color and depth video coding using a hierarchical representation, IEEE Trans. on Circuits and Systems for Video Technology, vol. 7, no.. pp.450-460, 007. [4] C. Tomasi and R. Manduchi, Bilateral filtering for gray and color images, in Proc. of IEEE International Conference on Computer Vision, pp. 89-846, 998. [5] M. Elad, On the origin of the bilateral filter and ways to improve it, IEEE Trans. on Image Processing, vol., no. 0, pp.4-50, 00. [6] H. Takeda, S. Farsiu, and P. Milanfar, "Kernel regression for image processing and reconstruction", IEEE Trans. on Image Processing, vol. 6, no., pp. 49-66, 007. [7] G. Petschnigg, M. Agrawala, H. Hoppe, R. Szeliski, M. Cohen, and K. Toyama, Digital photography with flash and no-flash image pairs, ACM Trans. on Graphics, vol., no., pp. 664-67, 004. [8] J. Kopf, M.F. Cohen, D. Lischinski, and M. Uyttendaele, Joint bi-
: 57 lateral upsampling, ACM Trans. on Graphics, vol. 6, no., pp.-6, 007. [9] S.Y. Kim, J. Cho, A. Koschan, and M. Abidi, Spatial and temporal enhancement of depth images captured by a time-of-flight depth sensor, in Proc. of IEEE International Conference of Pattern Recognition, pp. 58-6, 00. [0] D. Scharstein and R. Szeliski, A taxonomy and evaluation of dense two-frame stereo correspondence algorithms, International Jour. of Computer Vision, vol. 47, no. -, pp. 7-4, 00. [] O.P. Gangwal and B. Djapic, Real-time implementation of depth map post-processing for D-TV in dedicated hardware, in Proc. of International Conference of Consumer Electronics, pp. 7-74, 00. [] S.Y. Kim, W. Cho, A. Koschan, and M. Abidi, Depth map enhancement using adaptive steering kernel regression based on distance transform, in Proc. of International Symposium and Visual Computing, Lecture Notes in Computer Science, vol. 698, pp. 9-00, 0. [] G. Borgefors, Hierarchical chamfer matching: a parametric edge matching algorithm, IEEE Trans. on Patten Analysis and Machine Intelligence, vol. 0, no. 6, pp. 849-865, 988. [4] J. Cho, S.Y. Kim, Y.S. Ho, and K.H. Lee, Dynamic D human actor generation method using a time-of-flight depth camera, IEEE Trans. on Consumer Electronics, vol. 54, no. 4, pp.54-5, 008. [5] J. Canny, A computational approach to edge detection, IEEE Trans. on Patten Analsys and Machine Intelligence, vol. 8, no. 6, pp. 679-698, 986. [6] S.M. Kim, J. Cha, J. Ryu, and K.H. Lee, Depth video enhancement for haptic interaction using a smooth surface reconstruction, IEICE Trans. on Information and Systems, vol. E89-D, no., pp. 7-44, 006. [7] ISO/IEC JTC/SC9/WG, Call for proposals on D video coding technology, MPEG document N06, 0. [8] ISO/IEC JTC/SC9/WG, Test model under consideration for AVC-based D video coding (DV-ATM), MPEG document N49, 0. - 00 : - 00 : - 008 : - 009 ~ 0 : University of Tennessee - 0 ~ : - :, D, TV, - 98 : - 986 : University of Washington - 99 : University of Washington - 99 ~ 998 : - 998 ~ : - : D,, - 977 ~ 98 : - 98 ~ 98 : - 98 ~ 995 : - 985 ~ 989 : University of California, - 990 ~ 99 : - 00 ~ 0 : - 995 ~ : - :, TV, MPEG, TV,