6 : (Gicheol Kim et al.: Object Tracking Method using Deep Learing and Kalman Filter) (Regular Paper) 24 3, (JBE Vol. 24, No. 3, May 2019) http

Size: px

Start display at page:

Download "6 : (Gicheol Kim et al.: Object Tracking Method using Deep Learing and Kalman Filter) (Regular Paper) 24 3, (JBE Vol. 24, No. 3, May 2019) http"

현시 음
5 years ago
Views:

1 (Regular Paper) 24 3, (JBE Vol. 24, No. 3, May 2019) ISSN (Online) ISSN (Print) a), a), a), b), b), b), a) Object Tracking Method using Deep Learning and Kalman Filter Gicheol Kim a), Sohee Son a), Minseop Kim a), Jinwoo Jeon b), Injae Lee b), Jihun Cha b), and Haechul Choi a) CNN(Convolutional Neural Networks), RNN(Recurrent Neural Networks). CNN., CNN R-CNN, YOLO(You Only Look Once), SSD(Single Shot Multi-box Detector)... YOLO v2, YOLO v2 7.7% IoU FHD 20 fps. Abstract Typical algorithms of deep learning include CNN(Convolutional Neural Networks), which are mainly used for image recognition, and RNN(Recurrent Neural Networks), which are used mainly for speech recognition and natural language processing. Among them, CNN is able to learn from filters that generate feature maps with algorithms that automatically learn features from data, making it mainstream with excellent performance in image recognition. Since then, various algorithms such as R-CNN and others have appeared in object detection to improve performance of CNN, and algorithms such as YOLO(You Only Look Once) and SSD(Single Shot Multi-box Detector) have been proposed recently. However, since these deep learning-based detection algorithms determine the success of the detection in the still images, stable object tracking and detection in the video requires separate tracking capabilities. Therefore, this paper proposes a method of combining Kalman filters into deep learning-based detection networks for improved object tracking and detection performance in the video. The detection network used YOLO v2, which is capable of real-time processing, and the proposed method resulted in 7.7% IoU performance improvement over the existing YOLO v2 network and 20 fps processing speed in FHD images. Keyword : YOLO, Kalman filter, Object tracking, CNN, Deep learning Copyright 2016 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND ( which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.

2 (JBE Vol. 24, No. 3, May 2019). ICT(Information and Communications Technologies), , 10% [1][2].,,.,.,.,.,.. a) (Information of Departments, Hanbat National University) b) (ETRI) Corresponding Author : (Haechul Choi) choihc@hanbat.ac.kr Tel: ORCID: () (No. 19PCRD-C , EO/IR ). This research was supported by a grant from Police Science and Technology R&D Program funded by Korean National Police Agency. [No.19PCRD-C , Development and Field Demonstration Test of Surveillance System using radar and EO/IR for detecting illegal Flight of UAVs] Manuscript received April 15, 2019; Revised May 24, 2019; Accepted May 24, [3][4]..,,. YOLO(You Only Look Once) v2 (Kalman filter). YOLO v2 (Deep learning) [8][9]. YOLO v2,. GPU fps.,.. CNN(Convolutional Neural Networks) [5][6][7] YOLO v2, [10][11][12][13].,. YOLO v2.. 2 CNN. 3, 4. 5.

3 . 1. [14][15] XOR, [16], RBM (Restricted Boltzmann Machine) [17]. CNN (Feature Extraction).,.,. CNN.. CNN 2014 R-CNN(Region-CNN) [18]. R-CNN SIFT (Scale Invariant Feature Transform) [19], HOG(Histogram of Oriented Gradient) [20], Optical Flow [21], haar-like features [22], low-level feature. R-CNN,. R-CNN R-CNN, SPP-Net [23], Fast R-CNN [24], Faster R-CNN [25]. R-CNN.,,. SSD(Single Shot Multi-box Detector) [26], YOLO [8], YOLO v2 [9]. YOLO v2. YOLO v2. YOLO v2 1. Table 1. The detection system network Layer Filters Size Input Output 0 conv / max 2 2/ conv / max 2 2/ conv / conv 64 11/ conv / max 2 2/ conv / conv / conv / max 2 2/ conv / conv / conv / conv / conv / max 2 2/ conv / conv / conv / conv / conv / conv / conv / route conv / reorg / route conv / conv / detection

4 (JBE Vol. 24, No. 3, May 2019),,. YOLO v2 1.,., 6.. ( ). 2..,,..,.,..,, 1., 1 1 ( ) ( ) ( ) ( ) Fig. 1. Overall flowchart of Kalman filter

2 H 1, ( ) (R) ( ).,. 3 2. ( ) 0 1, ( ) ( ).,.,,. 1, 2. 4.. 1...,.... 2. CNN YOLO v2.

5 2 H 1, ( ) (R) ( )., ( ) 0 1, ( ) ( ).,.,,. 1, , CNN YOLO v2. 2., YOLO v2.., 2. Fig. 2. System flowchart

4 FHD, DJI Phantom 4 Professional. 16.04 Intel i7-6850k 3.

6 (JBE Vol. 24, No. 3, May 2019) 3. Fig. 3. Flowchart of proposed method.. 3. YOLO v2 confidence ,..,. 4,000, FHD, DJI Phantom 4 Professional Intel i7-6850k 3.6GHz CPU 32GB RAM NVIDIA geforce GTX. YOLO v2,,, 4. Fig. 4. Examples of training data

1080 Ti GPU, python 3.5.5, tensorflow [28] 1.10.0, opencv 3.4.0 [29].

5. Calculation of IoU and evaluation of the various IoU values 6 YOLO v2,

6. YOLO v2 IoU Fig. 6. IoU results of YOLO v2 network 7.

7 1080 Ti GPU, python 3.5.5, tensorflow [28] , opencv [29]. IoU(Intersection over Union). IoU, 5., (ground truth) IoU. IoU 3, 5 IoU IoU IoU Fig. 5. Calculation of IoU and evaluation of the various IoU values 6 YOLO v2, IoU FHD((Full High Definition) 24 fps. 7 YOLO v2 IoU. YOLO v2. 500, YOLO v2 75, YOLO v2 IoU Fig. 6. IoU results of YOLO v2 network 7. YOLO v2 IoU Fig. 7. Comparison of proposed method and YOLO v2 with IoU results

8 (JBE Vol. 24, No. 3, May 2019) 8. YOLO v2 Fig. 8. Examples of detection failure (YOLO v2) and tracking success (Proposed method). YOLO v2. IoU FHD 20 fps. 8 YOLO v2. YOLO v2 YOLO v2.. CNN YOLO v2.. YOLO v2.,.,., YOLO v2 7.7%. FHD

9 YOLO v2 4 fps 20 fps. YOLO v2... DB.,. YOLO v2, ( 1km )..,.,. (References) [1] Teal Group, 2014 Market Profile and Forecast, World Unmanned aerial Vehicle Systems, 2014 [2] Choi Youngchul, Ahn Hyosung. (2015). Dron's current and technology development trends and prospects. The world of electricity, 64(12), [3] Eric N. Johnson, Anthony J. Calise, Yoko Watanabe, Jincheol Ha, and James C. Neidhoefer, 2007, Real-Time Vision-Based Relative Aircraft Navigation, Journal of Aerospace Computing, Information, and Communication, Vol.4, pp [4] John Lai, Luis Mejias, and Jason J. Ford, 2011, Airborne Vision-Based Collision-Detection System, Journal of Field Robotics, Vol.28, Issue 2, pp [5] Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems [6] Schmidhuber, Jurgen. "Deep learning in neural networks: An overview." Neural networks 61 (2015): [7] Gidaris, Spyros, and Nikos Komodakis. "Object detection via a multi-region and semantic segmentation-aware cnn model." Proceedings of the IEEE International Conference on Computer Vision [8] Redmon, Joseph, et al. "You only look once: Unified, real-time object detection." Proceedings of the IEEE conference on computer vision and pattern recognition [9] Redmon, Joseph, and Ali Farhadi. "YOLO9000: better, faster, stronger." Proceedings of the IEEE conference on computer vision and pattern recognition [10] Brown, Robert Grover, and Patrick YC Hwang. Introduction to random signals and applied Kalman filtering. Vol. 3. New York: Wiley, [11] Ristic, Branko, Sanjeev Arulampalam, and Neil Gordon. "Beyond the Kalman filter." IEEE Aerospace and Electronic Systems Magazine 19.7 (2004): [12] Haykin, Simon. Kalman filtering and neural networks. Vol. 47. John Wiley & Sons, [13] Peterfreund, Natan. "Robust tracking of position and velocity with Kalman snakes." IEEE transactions on pattern analysis and machine intelligence 21.6 (1999): [14] LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. "Deep learning." nature (2015): 436. [15] Deng, Li, and Dong Yu. "Deep learning: methods and applications." Foundations and Trends in Signal Processing (2014): [16] Mayer-Schönberger, Viktor, and Kenneth Cukier. Big data: A revolution that will transform how we live, work, and think. Houghton Mifflin Harcourt, [17] Nair, Vinod, and Geoffrey E. Hinton. "Rectified linear units improve restricted boltzmann machines." Proceedings of the 27th international conference on machine learning (ICML-10) [18] Girshick, Ross, et al. "Rich feature hierarchies for accurate object detection and semantic segmentation." Proceedings of the IEEE conference on computer vision and pattern recognition [19] Lowe, David G. "Object recognition from local scale-invariant features." Computer vision, The proceedings of the seventh IEEE international conference on. Vol. 2. Ieee, [20] Dalal, Navneet, and Bill Triggs. "Histograms of oriented gradients for human detection." Computer Vision and Pattern Recognition, CVPR IEEE Computer Society Conference on. Vol. 1. IEEE, [21] Bouguet, Jean-Yves. "Pyramidal implementation of the affine lucas kanade feature tracker description of the algorithm." Intel Corporation (2001): 4. [22] Lienhart, Rainer, and Jochen Maydt. "An extended set of haar-like features for rapid object detection." Proceedings. International Conference on Image Processing. Vol. 1. IEEE, [23] He, Kaiming, et al. "Spatial pyramid pooling in deep convolutional networks for visual recognition." IEEE transactions on pattern analysis and machine intelligence 37.9 (2015): [24] Girshick, Ross. "Fast r-cnn." Proceedings of the IEEE international

(JBE Vol. 24, No. 3, May 2019) conference on computer vision. 2015. [25] Ren, Shaoqing, et al.

[26] Liu, Wei, et al. "Ssd: Single shot multibox detector." European conference on computer vision. Springer, Cham, 2016.

[28] Abadi, Martín, et al. "Tensorflow: A system for large-scale machine learning.

Learning OpenCV: Computer vision with the OpenCV library." O'Reilly Media, Inc.", 2008. - 2017 2 : ( ) - 2019 2 : ( ) - ORCID : https://orcid.

10 (JBE Vol. 24, No. 3, May 2019) conference on computer vision [25] Ren, Shaoqing, et al. "Faster r-cnn: Towards real-time object detection with region proposal networks." Advances in neural information processing systems [26] Liu, Wei, et al. "Ssd: Single shot multibox detector." European conference on computer vision. Springer, Cham, [27] Everingham, Mark, et al. "The pascal visual object classes (voc) challenge." International journal of computer vision 88.2 (2010): [28] Abadi, Martín, et al. "Tensorflow: A system for large-scale machine learning." 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16) [29] Bradski, Gary, and Adrian Kaehler. Learning OpenCV: Computer vision with the OpenCV library." O'Reilly Media, Inc.", : ( ) : ( ) - ORCID : - :,,, : ( ) : ( ) ~ : - ORCID : - :,,, : ( ) ~ : - ORCID : - :,, : : ~ : - ORCID : - :,,,

- 1997 : - 2001 : - 2001 ~ : - ORCID : https://orcid.

11 : : ~ : - ORCID : - :,,, : : : ~ : / - ORCID : - :,,, : : : ~ : (ETRI) ~ : - ORCID : - :,,,

09권오설_ok.hwp

09권오설_ok.hwp (JBE Vol. 19, No. 5, September 2014) (Regular Paper) 19 5, 2014 9 (JBE Vol. 19, No. 5, September 2014) http://dx.doi.org/10.5909/jbe.2014.19.5.656 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) a) Reduction