... K-vision Fig.. K-vision camera tracking screen Drummond [3] 3. 3 (lines), (edge) 3. (target). (homography perspective transform) [4]. (drifting).

Size: px

Start display at page:

Download "... K-vision Fig.. K-vision camera tracking screen Drummond [3] 3. 3 (lines), (edge) 3. (target). (homography perspective transform) [4]. (drifting)."

선화 영
8 years ago
Views:

1 a,b), b) Real-Time Camera Tracking for Markerless Augmented Reality Juhyun Oh a,b) and Kwanghoon Sohn b). SURF(speeded up robust features), (multi-scale). (normalized cross correlation, NCC).. (pose).. Abstract We propose a real-time tracking algorithm for an augmented reality (AR) system for TV broadcasting. The tracking is initialized by detecting the object with the SURF algorithm. A multi-scale approach is used for the stable real-time camera tracking. Normalized cross correlation (NCC) is used to find the patch correspondences, to cope with the unknown and changing lighting condition. Since a zooming camera is used, the focal length should be estimated online. Experimental results show that the focal length of the camera is properly estimated with the proposed online calibration procedure. Keywords: Machine vision, real time systems, tracking, TV cameras I. (augmented reality, AR) HCI(human computer interaction), (visual servoing) a) KBS Technical Research Institute, KBS b) School of Electrical and Electronic Engineering, Yonsei University : (khsohn@yonsei.ac.kr) (0 3 7 ),(0 6 ), (0 6 5 )., Lepetit []. Park [] 6 (tracking by detection). K-vision PC. (binarization) (orientation)

2 ... K-vision Fig.. K-vision camera tracking screen Drummond [3] 3. 3 (lines), (edge) 3. (target). (homography perspective transform) [4]. (drifting). (a) (b), (c). Vacchetti [5] (homography chaining) (jitter) (drift),., (pose).. Wang [6] (Bayesian).,. Comport [7] (moving edges). Pressigout [8] (texture) (hybrid). D'fusion [9]. (a) (b) (c).. (a) (b) (c) Fig.. Drift in a homography chaining experiment. (a) The object image. (b) One frame of the input video. (c) The object image transformed by the chained homography.

. (keypoints) affine training set, randomized trees ferns (classification) Lepetit [0] [,]. training phase. Wagner [3,4]. SIFT(scale invariant feature transform) [5] ferns [].., Yu [6] 6 CPU. PC.

3 . (keypoints) affine training set, randomized trees ferns (classification) Lepetit [0] [,]. training phase. Wagner [3,4]. SIFT(scale invariant feature transform) [5] ferns [].., Yu [6] 6 CPU. PC. Wagner [3,4],...., , , 5.. II.,. SIFT [5], ferns [], SURF [7], SURF. 3 SURF. (a) (b) (top), (c) (bottom) 3. SURF. (a). (b). (c). Fig. 3. The object detected by SURF. (a) The feature correspondences and the obtained object. (b) The object features. (c) The object pattern warped by the estimated homography (Hessian).. 50.

4 .,.. - (lookup) [8,9].,.., Zhang [0]. SURF 3 3 H. Z=0 H. éxù éx ù s = ê y H ê Y êë û êë û 3 (projection equation),. éxù s ê y = K êë û éx ù ê Y êz ë û éx ù ê Y ê 0 ë û [ R t] = K[ r r r t] = K[ r r t] éx ù êë û ' 3 Y K skew s = 0, aspect ratio a =. (principal point). f. () (), [ h h h ] = K[ r r t] H = 3 l h H. (orthogonal),. - T - T ( K h ) K h = h 0. = ωh l l T r = r.. T T r = r r = r h (5) (6) T ωh = h T ωh ω = K -T K - (the image of the absolute conic [] ) (symmetric) 6 b = [ w, w, w, w, w w ] T 3 3, 33. (5) (6). Vb = 0 t r R (column vector). K. V 6. b K. K K H.

5 r = lk r r 3 = lk t = lk - - = r r - h h h 3 (orientation). (normalized cross correlation, NCC)... III.....,. (warping). (patch).,.. [3,4] (affine approximation) (full perspective transformation).. SAD(sum of absolute differences) (cross correlation).,. [3,4], 7 4. Levenberg-Marquardt (nonlinear least squares optimization) [] r, t, f.. N { r t, f } = arg min ( Proj( m, θ x ) θ =, å r i ) - i i m i i x i NCC. ρ, M (M-estimator). Geman-McClure.. (feature descriptors),.

6 r x 4. M Geman-McClure Fig. 4. The Geman-McClure function used as an M-estimator 5. Fig. 5. The algorithm outline 5... H RANSAC [3]

. r, t f (0)... NCC.., NCC (threshold). threshold=90. IV. Wagner [3,4]. 6. 6 RANSAC.. 6(a)., (residual error). 6(b),.. 7.. -68 (perspective). 7.. 4 PC 640 480 38.

7 . r, t f (0)... NCC.., NCC (threshold). threshold=90. IV. Wagner [3,4] RANSAC.. 6(a)., (residual error). 6(b), (perspective) PC Hz. Wagner. (a) 6.. (a) Wagner, (b) Fig. 6. Camera tracking results. (a) Wagner et al. (b) The proposed online camera calibration (b)

오주현 외 : 마커 없는 증강현실을 위한 실시간 카메라 추적 6 그림 7. 잔여 오류 Fig. 7. Residual error 있듯이 현재의 구현에서는 투영변환에 많은 시간이 소요 되고 있는데, 향후 OpenGL 등을 이용하여 GPU에서 처리 함으로써 상당한 개선이 가능할 것으로 보인다. 표. 추적 모듈의 계산 시간 Table.

8 오주현 외 : 마커 없는 증강현실을 위한 실시간 카메라 추적 6 그림 7. 잔여 오류 Fig. 7. Residual error 있듯이 현재의 구현에서는 투영변환에 많은 시간이 소요 되고 있는데, 향후 OpenGL 등을 이용하여 GPU에서 처리 함으로써 상당한 개선이 가능할 것으로 보인다. 표. 추적 모듈의 계산 시간 Table. Computation time for the tracking module 절차 (procedure) 투영 변환 다운샘플링 NCC 정합 RANSAC을 이용한 호모그래피 추정 카메라 파라미터의 비선형 최적화 합계 소요 시간 (밀리초) V. 결론 기존의 인위적인 마커를 사용하지 않아 방송용 증강현실 제작에 적합한 카메라 추적 알고리듬을 제안하였다. SURF 를 이용하여 객체를 검출함으로써 카메라 추적을 초기화하 고, 다층 구조를 사용하여 안정적인 실시간 카메라 추적이 가능하게 하였다. 알려져 있지 않은 조명 환경에서의 특징 정합을 위해 정규상호상관도를 사용하였다. 대부분 데스크 톱 웹캠 환경을 가정하고 있는 기존 증강현실 카메라 추적 연구들과 달리 본 논문에서는 줌 카메라에의 적용을 위해 온라인 카메라 보정 방법을 제안하였다. 실험 결과는 제안 된 온라인 보정 방법에 의해 카메라의 초점거리가 정확하 게 추정되는 것을 보여주었으며, 모든 과정이 실시간으로 처리 가능함을 확인하였다. 그러나 패턴이 카메라의 광축 에 수직인 경우와 같이 초점거리의 변화와 패턴의 이동을

9 (degenerate case),., (optical flow). HD (high definition),.. [] V. Lepetit and P. Fua, Monocular Model-based 3D Tracking of Rigid Objects, Now Publishers Inc, 005. [],,,,, 7, pp.-9, [3] T. Drummond and R. Cipolla, "Real-time visual tracking of complex structures," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.4, pp , Jul. 00. [4] G. Simon, A. Fitzgibbon, and A. Zisserman, "Markerless tracking using planar structures in the scene," Int. Symposium on Augmented Reality (ISAR), pp.0-8, Oct [5] L. Vacchetti, V. Lepetit, and P. Fua, "Stable Real-Time 3D Tracking Using Online and Offline Information," IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(0), pp , 004. [6] Q. Wang, W. W. Zhang, X. O. Tang, and H. Y. Shum, "Real-time Bayesian 3-D pose tracking," IEEE Transactions on Circuits and Systems for Video Technology, vol.6, pp , Dec [7] A. I. Comport, E. Marchand, M. Pressigout, and F. Chaumette, "Real-time markerless tracking for augmented reality: The virtual visual servoing framework," IEEE Transactions on Visualization and Computer Graphics, vol., pp.65-68, 006. [8] M. Pressigout and E. Marchand, "Real-time hybrid tracking using edge and texture information," International Journal of Robotics Research, vol.6, pp , Jul [9] "Total Immersion: Augmented reality software solutions with D'Fusion," [0] V. Lepetit and P. Fua, Keypoint recognition using randomized trees," IEEE Transactions on Patteren Analysis and Machine Intelligence, 8(9), pp , 006. [] M. Özuysal, P. Fua, and V. Lepetit, "Fast keypoint recognition in ten lines of code," in IEEE Conference on Computer Vision and Pattern Recognition, pp. -8, 007. [] Z. Chen and X. Li, "Markerless tracking based on natural feature for autmented reality," Int. Conf. on Educational and Information Technology (ICEIT), Sep. 00. [3] D. Wagner, G. Reitmayr, A. Mulloni, T. Drummond, and D. Schmalstieg, "Pose tracking from natural features on mobile phones," Int. Symposium on Mixed and Augmented Reality, pp.5-34, 008. [4] D. Wagner, G. Reitmayr, A. Mulloni, T. Drummond, and D. Schmalstieg, "Real-Time Detection and Tracking for Augmented Reality on Mobile Phones," IEEE Transactions on Visualization and Computer Graphics, vol.6, pp , May-Jun. 00. [5] D. Lowe, "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, vol.60, pp.9-0, 004. [6] J. Yu, J. Kim, H. Kim, I. Choi, and I. Jeong, "Real-time camera tracking for augmented reality," Int. Conf. on Advanced Communication Technology (ICACT), Feb [7] H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, "Speeded-Up Robust Features (SURF)," Computer Vision and Image Understanding, vol.0, pp , Jun [8] J. Oh and K. Sohn, "Semiautomatic zoom lens calibration based on the camera's rotation," SPIE Journal of Electronic Imaging, 0(), Apr-Jun. 0. [9] W. Liu, Y. Wang, J. Chen, and J. Guo, "An efficient zoom tracking method for pan-tilit-zoom camera," IEEE Int. Conf. on Computer Science and Information Technology (ICCSIT), pp , Jul. 00. [0] Z. Y. Zhang, "A flexible new technique for camera calibration," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol., pp , Nov [] R. Hartley and A. Zisserman, Multiple view geometry in computer vision, Cambridge University Press, 003. [] M. Lourakis, "A brief description of the Levenberg-Marquardt algorithm implemented by levmar," matrix, 3(), 005. [3] M. Fischler and R. Bolles, "Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography," Communications of the ACM, vol.4, pp , 98.

- 997 : - 999 : - 999 ~ : KBS - :, 3-983 : - 985 : University of Minnesota, MSSE - 99 : North Carolina State University, Ph.D.

10 - 997 : : ~ : KBS - :, : : University of Minnesota, MSSE - 99 : North Carolina State University, Ph.D ~ 993 : : Georgetown University, Post Doctoral Fellow ~ : ~ : Nanyang Technological University, Visiting Professor - : 3,

High Resolution Disparity Map Generation Using TOF Depth Camera In this paper, we propose a high-resolution disparity map generation method using a lo

High Resolution Disparity Map Generation Using TOF Depth Camera In this paper, we propose a high-resolution disparity map generation method using a low-resolution Time-Of- Flight (TOF) depth camera and