4: Optical Flow CCTV (Jihae Kim et al.: Optical Flow Based Vehicle Counting and Speed Estimation in CCTV Videos) (Regular Paper) 22 4, 2017 7 (JBE Vol. 22, o. 4, July 2017) https://doi.org/10.5909/jbe.2017.22.4.1 ISS 2287-9137 (Online) ISS 1226-7953 (Print) Optical Flow CCTV a), a), a), a), b) Optical Flow Based Vehicle Counting and Speed Estimation in CCTV Videos Jihae Kim a), okyung Shin a), Jaekyung Kim a), Cheolhee Kwon a), and Hyeran Byun b) CCTV. (IPM, Inverse Perspective Mapping), 1), 2). (Optical flow). CCTV 106,993, 88.94%. Abstract This paper proposes a vehicle counting and speed estimation method for traffic situation analysis in road CCTV videos. The proposed method removes a distortion in the images using Inverse perspective Mapping, and obtains specific region for vehicle counting and speed estimation using lane detection algorithm. Then, we can obtain vehicle counting and speed estimation results from using optical flow at specific region. The proposed method achieves stable accuracy of 88.94% from several CCTV images by regional groups and it totally applied at 106,993 frames, about 3 hours video. Keyword : Vehicle Counting, Vehicle Speed Estimation, Inverse Perspective Mapping, Optical Flow, Traffic Situation Analysis a) LIG.Project 5(LIG ex1 Avionics R& Lab) b) (Yonsei University, ept. Computer Science) Corresponding Author : (Hyeran Byun) E-mail: hrbyun@yonsei.ac.kr Tel: +82-2-2123-3876 ORCI: http://orcid.org/0000-0002-3082-3214 2017 () (o.2016-0-00152,eep Learning ) Manuscript received April 3, 2017; Revised July 7, 2017; Accepted July 7, 2017. Copyright 2017 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-C- (http://creativecommons.org/licenses/by-nc-nd/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.
(JBE Vol. 22, o. 4, July 2017). 10 (ITS, Intelligent Transportation System) [1]-[3],. ITS.. [4],[5],[6],[7],[8].,..... [4]-[6],[9]... [4],[5]. (OF, Optical Flow), (F, Frame ifference), (BS, Background Subtraction)., - [10] (Vision) [6],[7],[9],[11],[12].. CCTV (IPM, Inverse Perspective Mapping) [13] -(Top-view). -.. 1.. 1. Fig. 1. The flow of the proposed method. 2, 3 IPM.. 4, 5.
4: Optical Flow CCTV (Jihae Kim et al.: Optical Flow Based Vehicle Counting and Speed Estimation in CCTV Videos)..,. 1.. (F, Frame ifference), (BS, Background Subtraction), (OF, Optical Flow) [4],[5],[14]-[16]. (F, Frame ifference) [4],[5].,,. (BS, Background Subtraction) [14],[16].. (background modeling).,,. [14],[16] CCTV. (OF, Optical Flow) [15],[17].. 2. /, -, - [6],[7],[9],[11],[12],[18].. Inertial avigation System(IS) (wheel speed sensor)..,. height-precision GPS. GPS,. - [18] kinematic dynamics model... - [6],[7],[9],[11],[12]. [7],[11]. [12] Lucas- Kanade Horn-Schunck.. CCTV CCTV. CCTV
(JBE Vol. 22, o. 4, July 2017) 1) (IPM, Inverse Perspective Mapping), 2), 3) (OF, Optical Flow), 4) 4. (perspective distortion) IPM -,.., IPM IPM -.... 1. (IPM, Inverse Perspective Mapping). 2, 3. [19]. 2 (resampling). 2 (Euclidean). (), ( ) ( ), (), () (2) 2 3. CCTV... (IPM, Inverse Perspective Mapping) [13] -. tan cos tan sin (a) Input image (b) Vanishing point detection (c) Result of IPM 2. Fig. 2. Examples of applying IPM
4: Optical Flow CCTV (Jihae Kim et al.: Optical Flow Based Vehicle Counting and Speed Estimation in CCTV Videos) 2 IPM., IPM. 2. IPM -. [13],[20] 3. 3(a) 3.1 IPM -. -., -(top-hat). 3(b), 3(c). [21] (structuring element),. 3(d). (column), 5 (bin). 3(e) - 3(f). 2/3, 20. 3(g). -. (3) n-1, n ( ) 0.6,. 3. Fig. 3 Flow of lane detection algorithm
(JBE Vol. 22, o. 4, July 2017) 3. CCTV. IPM. Farneback (dense optical flow)..,. (sparse optical flow), (dense optical flow). Lucas-kanade,.. Farneback. Farneback. 4 step. 3.2,. 4. Fig. 4. Example of motion measuring regionounting 5. 3.2 5 Fig. 5 The flow of vehicle counting
4: Optical Flow CCTV (Jihae Kim et al.: Optical Flow Based Vehicle Counting and Speed Estimation in CCTV Videos). (maxflow) (4). max max max,,.. 4.. ( ) ( ) (5), (6).,.. 4. CCTV. Lucas-Kanade. 3.2 Lucas-Kanade..,. 3.2. Lucas-Kanade,.... ( ) 1000cm(10m). (7).,,,. (7). (8)., (7), FPS C km/h. (8). 1.. CCTV.
(JBE Vol. 22, o. 4, July 2017). 1. 2, 3, 4. (ay) (ight) 16., 106,993. 2 16. 1. Table 1. Test set composition for experiment CCTV ay ight total Busan 7,000 6,218 13,218 Goyang 7,000 7,000 14,000 Gimpo1 7,000 7,000 14,000 Gimpo2 5,575 6,787 12,362 Yangsan 7,000 7,000 14,000 Ceongdo 7,000 7,000 14,000 Suseong 5,873 5,813 11,686 amyangju 6,727 7,000 13,727 total 53,175 53,818 106,993 2. Table 2. Composition of testset for evaluation # lane CCTV 4 G Busan_ Busan_ oyang_ Goyang_ 3 Gimpo1_ Gimpo2_ Yangsan_ Gimpo1_ Gimpo2_ Yangsan_ 2 Ceongdo_ Suseong_ amyangju_ Ceongdo_ Suseong_ amyangju_
4: Optical Flow CCTV (Jihae Kim et al.: Optical Flow Based Vehicle Counting and Speed Estimation in CCTV Videos) 2.. ground truth. (9) (10)., G ground truth, C. (9) AE (10).. 3. 3 5 ground truth(g), experimental result(c),,. 3 4, 4 5 3, 2. 6 3 5. ground truth(g), experimental result(c),. 16 88.94%. 6. 90.55%, 87.32%.. 7 (background subtraction) [14], (optical flow) Lucas-kanade.,,, 56.94%, 63.57%, 88.94%.. Gimpo2 3. 4 Table 3. Experiment result of 4 lanes of video CCTV Lane 1 Lane 2 Lane 3 Lane 4 total ground truth 67 114 157 111 449 experimental result 74 122 165 108 469 accuracy(%) 89.55 92.98 94.90 97.30 93.68 Busan average speed(km/h) 82 84 78 77 80.25 ground truth 39 87 112 85 323 experimental result 43 97 125 90 355 accuracy(%) 89.74 88.51 88.39 94.12 90.19 average speed(km/h) 62 76 72 68 69.5 ground truth 51 82 88 74 295 experimental result 51 81 98 73 303 accuracy(%) 100.00 98.78 88.64 98.65 96.52 Goyang average speed(km/h) 73 73 73 71 72.5 ground truth 25 32 41 23 121 experimental result 28 37 41 23 129 accuracy(%) 88.00 84.38 100.00 100.00 93.09 average speed(km/h) 61 58 55 52 56.5
(JBE Vol. 22, o. 4, July 2017) 4. 3 Table 4. Experiment result of 3 lanes of video CCTV Lane 1 Lane 2 Lane 3 total Gimpo1 Gimpo2 Yangsan ground truth 23 30 18 71 experimental result 21 28 19 68 accuracy(%) 91.30 93.33 94.44 93.03 average speed(km/h) 70 56 50 58.67 ground truth 18 26 15 59 experimental result 19 28 17 64 accuracy(%) 94.44 92.31 86.67 91.14 average speed(km/h) 62 47 39 49.33 ground truth 21 31 34 86 experimental result 18 29 33 80 accuracy(%) 85.71 93.55 97.06 92.11 average speed(km/h) 89 81 64 78 ground truth 27 29 33 89 experimental result 27 30 35 92 accuracy(%) 85.71 96.55 93.94 92.07 average speed(km/h) 64 62 57 61 ground truth 83 90 46 219 experimental result 81 89 46 216 accuracy(%) 97.59 98.89 100.00 98.83 average speed(km/h) 91 81 82 84.67 ground truth 34 48 18 100 experimental result 34 47 18 99 accuracy(%) 100.00 97.92 100.00 99.31 average speed(km/h) 66 73 78 72.33 5. 2 Table 5. Experiment result of 2 lanes of video CCTV Lane 1 Lane 2 total ground truth 55 72 127 experimental result 54 74 128 accuracy(%) 98.18 97.22 97.70 Ceongdo average speed(km/h) 50 49 49.50 ground truth 46 57 103 experimental result 46 57 103 accuracy(%) 100.00 100.00 100.00 average speed(km/h) 50 48 49 ground truth 18 36 54 experimental result 18 38 56 accuracy(%) 100.00 94.44 97.22 Suseong average speed(km/h) 61 68 64.50 ground truth 15 28 43 experimental result 18 29 47 accuracy(%) 80 96.43 88.21 average speed(km/h) 78 65 71.50 ground truth 47 35 82 experimental result 46 35 81 accuracy(%) 97.87 100.00 98.94 amyangju average speed(km/h) 89 76 82.50 ground truth 34 43 77 experimental result 35 43 78 accuracy(%) 97.06 100.00 98.53 average speed(km/h) 83 66 74.50
4: Optical Flow CCTV (Jihae Kim et al.: Optical Flow Based Vehicle Counting and Speed Estimation in CCTV Videos) 6. Table 6. Accuracy of vehicle counting on day and night video Accuracy(%) ay 90.55 ight 87.32 total 88.94 6. Fig. 6. Result of vehicle counting experiment 7. Fig. 7. Result of vehicle counting with comparative experiment
(JBE Vol. 22, o. 4, July 2017) 8. Fig. 8. Result of vehicle speed estimation... Farneback, Lucas-kanade.. Farneback. 8.,... CCTV. (IPM, Inverse Perspective Mapping),..,, CCTV..,
4: Optical Flow CCTV (Jihae Kim et al.: Optical Flow Based Vehicle Counting and Speed Estimation in CCTV Videos). (References) [1] EunJu Lee, Jae-Yeal am, ByoungChul Ko, Speed-limit Sign Recognition Using Convolutional eural etwork Based on Random Forest, The Korean Institute of Broadcast and Media Engineers, pp.938-949, 2015. [2] Kum, Chang-Hoon, Cho, ong-chan, Kim, Whoi-Yul, evelopment of Lane etection System using Surrounding View Image of Vehicle, The Korean Institute of Broadcast and Media Engineers, pp.331-334, 2013. [3] Hyung-Sub Kang, ong-chan Cho and Whoi-Yul Kim, Passing Vehicle etection using Local Binary Pattern Histogram, The Korean Institute of Broadcast and Media Engineers, pp.261-264, 2010. [4] A. Tourani and A. Shahbahrami, Vehicle Counting Method Based on igital Image Processing Algorithms, IEEE Transactions on International Conference on Pattern Recognition and Image Analysis, pp. 1-6, 2015. [5] Y. Xia, X. Shi, G. Song, Q. Geng and Y. Liu, Towards imporving quality of video-based vehicle counting method for traffic flow estimation, Signal Processing, pp. 672-681, 2016. [6] X. Qimin, L. Xu, W. Mingming, L. Bin and S. Xianghui, A Methodology of Vehicle Speed Estimation Based on Optical Flow, IEEE International Conference on Service Operations and Logistics and Informaitcs, pp. 33-37, 2014. [7] J. Lan, J. Li, G. Hu, B. Ran, L. Wang, Vehicle speed measurement based on gray constraint optical flow algorithm, International Journal of Light and Eletron Optics, pp. 289-295, 2014. [8] M.S. Shirazi and B. Morris, A Typical Video-based Framework for Counting, Behavior and Safety Analysis at Intersections, IEEE Transactions on Intelligent Vehicles Symposium, pp. 1264-1269, 2015. [9] S.C. iamantas and P. asgupta, Active Vision Speed Estimation from Optical Flow, Towardss Autonomous Robotic Systems, pp. 173-184, 2014. [10] X. Yu, X. Gao, Review of Vehicle State Estimation Problem under riving Situation, Chinese Journal of Mechanical Engineering, pp. 20-33, 2009. [11].C. Luvizon, B.T. assu and R. Minetto, Vehicle speed estimation by license plate detection and tracking, IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 6563-6567, 2014. [12] E. Patel and. Shukla, Comparison of Optical Flow Algorithms for Speed etermination of Moving Objects, International Journal of Computer Applications, Vol. 63, o. 5, pp. 32-37, 2013. [13]. ing, J.S. Yoo, J.K. Jung and S. Kwon, An Urban Lane etection Method Based on Inverse Perspective Mapping, GCIT, Advanced Science and Technology Letters, vol. 63 pp.53-58, 2014. [14] S. Aslani and H. Mahdavi-asab, Optical Flow Based Moving Object etection and Tracking for Traffic Surveillance, International Journal of Electrical, Computer, Energetic, Electronic and Communication Engineering, Vol. 7, o. 9, pp. 1252-1256, 2013. [15] A. Glowacz, Z. Mikrut and P. Pawlik, Video etection Algorithm Using an Optical Flow Calculation Method, Multimedia Communications, Services and Security, Vol. 287, pp. 118-129, 2012. [16] H.Y. Cheng and S.H. Hsu, Intelligent Highway Traffic Surveillance With Self-iagnosis Abilities, IEEE Transactions on Intelligent Transportation Systems, Vol. 12, o. 4, pp. 1462-1472, 2011. [17] M. Mizushima, Y. Taniquchi, G. hasegawa, H. akano and M. Matsuoka, Counting Pedestrians Passing through a Line in Video Sequences based on Optical Flow Extraction, Recent Advances in Circuits, Systems and Automatic Control, pp. 129-136, 2013. [18] K. Jo, K. Chu, K. Lee, M. Sunwoo, Integration of Multiple Vehicle Models with IMM Filter for Vehicle Localization, IEEE Transactions on Intelligent Vehicles Symposium, pp. 746-751, 2010. [19] M. Bertozzi, A. Broggi and A. Fascioli, An extension to the Inverse Perspective Mapping to handle non-flat roads, IEEE International Conference on Intelligent Vehicle, pp. 305-310, 1998. [20] M. Aly, Real time etection of Lane Markers in Urban Streets, IEEE Transactions on Intelligent Vehicles Symposium, 2008. [21] S. Suzuki and K. Abe, Topological Structural Analysis of igitized Binary Images by Border Following, Computer Vision, Graphics, and Image Processing, Vol. 30, o. 1, pp. 32-46, 1985. [22] L. Imsland, T.A. Johansen, T.I. Fossen, et al, Vehicle velocity estimation using nonliear observer, Automatica, pp. 2091-2103, 2006. [23] R. Zhao and X. Wang, Counting Vehicles from Semantic Rigions, IEEE Transactions on Intelligent Transportation Systems, Vol. 14, o. 2, pp. 1016-1022, 2013.
(JBE Vol. 22, o. 4, July 2017) - 2017 : () - 2017 ~ : LIG - ORCI : http://orcid.org/0000-0003-0640-3022 - :,,, - 2008 2 : () - 2015 2 : () - 2015 3~ : LIG - ORCI : http://orcid.org/0000-0001-9918-7132 - :,,, - 2012 8 : CW() - 2000 12 ~ : LIG - ORCI : http://orcid.org/0000-0002-8535-3184 - :,, CW - 2000 2 : () - 2000 1 ~ : LIG - ORCI : http://orcid.org/0000-0002-5811-1622 - : RF,, - 1980 : () - 1983 : () - 1987 : University of Illinois, Computer Science(M.S.) - 1993 : Purdue University, Computer Science(Ph..) - 1994 ~ 1995 : - 1995 ~ 1998 : - 1998 ~ 2003 : - 2003 ~ : - ORCI : http://orcid.org/0000-0002-3082-3214 - :,,