(JBE Vol. 25, No. 2, March 2020) (Special Paper) 25 2, (JBE Vol. 25, No. 2, March 2020) ISSN

Size: px

Start display at page:

Download "(JBE Vol. 25, No. 2, March 2020) (Special Paper) 25 2, (JBE Vol. 25, No. 2, March 2020) ISSN"

일화 화
3 years ago
Views:

1 (JBE Vol. 25, No. 2, March 2020) (Special Paper) 25 2, (JBE Vol. 25, No. 2, March 2020) ISSN (Online) ISSN (Print) CPU a), a), a) Towards Real-time Multi-object Tracking in CPU Environment Kyung Hun Kim a), Jun Ho Heo a), and Suk-Ju Kang a) Abstract Recently, the utilization of the object tracking algorithm based on the deep learning model is increasing. A system for tracking multiple objects in an image is typically composed of a chain form of an object detection algorithm and an object tracking algorithm. However, chain-type systems composed of several modules require a high performance computing environment and have limitations in their application to actual applications. In this paper, we propose a method that enables real-time operation in low-performance computing environment by adjusting the computational process of object detection module in the object detection-tracking chain type system. Keyword : multi object tracking, data association, real time object tracking a) (Dept. of Electronic Engineering, Sogang University) Corresponding Author : (Suk-Ju Kang) sjkang@sogang.ac.kr Tel: ORCID: () (No. 2018R1D1A1B ), ( 19PQWO-B ), ICT (IITP ) This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2018R1D1A1B ) and a grant(19pqwo-b ) from Smart road lighting platform development and empirical study on test-bed Program funded by Ministry of the Interior and Safety of Korean government, and the MSIT(Ministry of Science and ICT), Korea, under the ITRC(Information Technology Research Center) support program(iitp ) supervised by the IITP(Institute for Information & communications Technology Promotion). Manuscript received January 14, 2020; Revised March 3, 2020; Accepted March 3, Copyright 2020 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND ( which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.

2 2: CPU (Kyung Hun Kim et al.: Towards Real-time Multi-object Tracking in CPU Environment). [1], [2], [3],. (Multi Object Track-ing; MOT) (Multi Target Tracking; MTT)..... [5].,. [4].,,.,. -, [9]... Convolutional Neural Network(CNN) [9]. CNN CPU Fig. 1. Multi-object tracking video result XOR,, Restricted Boltz-

3 (JBE Vol. 25, No. 2, March 2020) mann Machine(RBM). CNN (Feature Extraction).,.,. CNN. [12]. CNN 2014 Region-CNN(R-CNN) [13]. R-CNN Scale Invariant Feature Transform(SIFT) [15], Histo- gram of Oriented Gradient(HOG) [14], Optical Flow [16], haar-like features [17], low-level feature. R-CNN,. R-CNN R-CNN, SPP-Net [18], Fast R-CNN [19], Faster R-CNN [20]. R-CNN.,,. Single Shot Multi-box Detector(SSD) [21], YOLO [10]. 2..,,..,.,..,. 2 ( x,y 2. Fig. 2. Kalman filter-based Multi-Object Tracking System Overview

4 2: CPU (Kyung Hun Kim et al.: Towards Real-time Multi-object Tracking in CPU Environment) a, h) (x,y,a,h ).. -(IOU) [22]., IOU min IOU.. DEEP SORT [9] [11] IOU distance Mahalanobis distance CNN Deep cosine metric [23]. YOLO-v3 [10].. 1. CNN [9]. 3 CNN.,. 3. Fig. 3. Deep Learning-based Multi-Object Tracking System Overview ( 4. ) Fig. 4. Overview of tracking method using Kalman filter and deep learning 2. -, (1 ) 10 30, trade-off. 5 (%) (frame per second; fps). 5 1, 4 1, 3 1.

5 (JBE Vol. 25, No. 2, March 2020) 5. Fig. 5. Accuracy and Speed Correlation with Detection Frame Rate 1 1, 2 1, [9] 7. Fig. 7. Comparison between existing and proposed methods when two objects have similar image features. 7 Frame 3. 7.,. ( 2. ). (1). O, O, id. feature feature id id. deep cosine metric [23], feature. 6. Fig. 6. Method to update previous to previous frame maxo O O O

6 2: CPU (Kyung Hun Kim et al.: Towards Real-time Multi-object Tracking in CPU Environment) 1.. MOT16 [8]. Intel(R) Xeon(R) CPU E5-2687W 3.00GHz 12 core CPU. CLEAR MOT [6] 7. GT ground truth Table 1. Metrics used for benchmarking Metric Better Perfect Description MOTA higher 100% Overall Tracking Accuracy. MOTAL higher 100% Log Tracking Accuracy. MOTP higher 100% Percentage alignment of predicted bounding box and ground truth. FP lower 0 Number of false positives. FN lower 0 Number of false negatives. IDsw lower 0 FPS higher Inf. Identity switches, see [80] for details. Processing speed in frames per second. log Table 2. Experimental results applying the original method and the two proposed method METHOD VIDEO MOTA MOTP MOTAL FP FN IDs FPS original frame skip frame skip + previous to previous frame MOT MOT MOT MOT MOT MOT MOT AVERAGE MOT MOT MOT MOT MOT MOT MOT AVERAGE MOT MOT MOT MOT MOT MOT MOT AVERAGE

7 (JBE Vol. 25, No. 2, March 2020). CPU 4 fps fps, 0.3,., MOT16-13.,...., trade-off (MOTA). ( 2. ( 2 1 ).. - CPU. 3 1,.,. (References) [1] Betke, Margrit, Esin Haritaoglu, and Larry S. Davis. "Real-time multiple vehicle detection and tracking from a moving vehicle." Machine vision and applications 12.2 (2000): [2] Hu, Weiming, et al. "A survey on visual surveillance of object motion and behaviors." IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 34.3 (2004): [3] Lu, Wei-Lwun, et al. "Learning to track and identify players from broadcast sports videos." IEEE transactions on pattern analysis and machine intelligence 35.7 (2013): [4] Murray, Samuel. "Real-time multiple object tracking-a study on the importance of speed." arxiv preprint arxiv: (2017). [5] Luo, Wenhan, et al. "Multiple object tracking: A literature review." arxiv preprint arxiv: (2014). [6] Bernardin, Keni, and Rainer Stiefelhagen. "Evaluating multiple object tracking performance: the CLEAR MOT metrics." EURASIP Journal on Image and Video Processing 2008 (2008): [7] Ristani, Ergys, et al. "Performance measures and a data set for multi-target, multi-camera tracking." European Conference on Computer Vision. Springer, Cham, [8] Milan, Anton, et al. "MOT16: A benchmark for multi-object tracking." arxiv preprint arxiv: (2016). [9] Wojke, Nicolai, Alex Bewley, and Dietrich Paulus. "Simple online and realtime tracking with a deep association metric." 2017 IEEE international conference on image processing (ICIP). IEEE, doi: /ICIP [10] Redmon, Joseph, et al. "You only look once: Unified, real-time object detection." Proceedings of the IEEE conference on computer vision and pattern recognition [11] A. Bewley, Z. Ge, L. Ott, F. Ramos and B. Upcroft, "Simple online and realtime tracking," 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, 2016, pp doi: /ICIP [12] LeCun, Yann, et al. "Gradient-based learning applied to document recognition." Proceedings of the IEEE (1998): doi: / [13] Girshick, Ross, et al. "Rich feature hierarchies for accurate object detection and semantic segmentation." Proceedings of the IEEE conference on computer vision and pattern recognition [14] Dalal, Navneet, and Bill Triggs. "Histograms of oriented gradients for human detection." 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05). Vol. 1. IEEE, [15] Lowe, David G. "Object recognition from local scale-invariant features." Proceedings of the seventh IEEE international conference on computer vision. Vol. 2. Ieee, [16] Horn, Berthold KP, and Brian G. Schunck. "Determining optical flow." Techniques and Applications of Image Understanding. Vol. 281.

2: CPU (Kyung Hun Kim et al.: Towards Real-time Multi-object Tracking in CPU Environment) International Society for Optics and Photonics, 1981. [17] Viola, Paul, and Michael Jones.

[18] He, Kaiming, et al. "Spatial pyramid pooling in deep convolutional networks for visual recognition." IEEE transactions on pattern analysis and machine intelligence 37.9 (2015): 1904-1916.

8 2: CPU (Kyung Hun Kim et al.: Towards Real-time Multi-object Tracking in CPU Environment) International Society for Optics and Photonics, [17] Viola, Paul, and Michael Jones. "Rapid object detection using a boosted cascade of simple features." Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR Vol. 1. IEEE, [18] He, Kaiming, et al. "Spatial pyramid pooling in deep convolutional networks for visual recognition." IEEE transactions on pattern analysis and machine intelligence 37.9 (2015): [19] Girshick, Ross. "Fast r-cnn." Proceedings of the IEEE international conference on computer vision [20] Ren, Shaoqing, et al. "Faster r-cnn: Towards real-time object detection with region proposal networks." Advances in neural information processing systems [21] Liu, Wei, et al. "Ssd: Single shot multibox detector." European conference on computer vision. Springer, Cham, [22] Kuhn, Harold W. "The Hungarian method for the assignment problem." Naval research logistics quarterly 2.12 (1955): [23] Wojke, Nicolai, and Alex Bewley. "Deep cosine metric learning for person re-identification." 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, : : - ORCID : - :,, : ~ 2020 : - ORCID : - :, : : ~ 2012 : LG Display ~ : - ORCID : - :,,

6 : (Gicheol Kim et al.: Object Tracking Method using Deep Learing and Kalman Filter) (Regular Paper) 24 3, (JBE Vol. 24, No. 3, May 2019) http

6 : (Gicheol Kim et al.: Object Tracking Method using Deep Learing and Kalman Filter) (Regular Paper) 24 3, (JBE Vol. 24, No. 3, May 2019) http (Regular Paper) 24 3, 2019 5 (JBE Vol. 24, No. 3, May 2019) https://doi.org/10.5909/jbe.2019.24.3.495 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) a), a), a), b), b), b), a) Object Tracking Method using