(JBE Vol. 22, o. 4, July 2017) (Regular Paper) 22 4, 2017 7 (JBE Vol. 22, o. 4, July 2017) https://doi.org/10.5909/jbe.2017.22.4.448 ISS 2287-9137 (Online) ISS 1226-7953 (Print) Optical Flow CCTV a), a), a), a), b) Optical Flow Based Vehicle Counting and Speed Estimation in CCTV Videos Jihae Kim a), okyung Shin a), Jaekyung Kim a), Cheolhee Kwon a), and Hyeran Byun b) CCTV. (IPM, Inverse Perspective Mapping), 1), 2). (Optical flow). CCTV 106,993, 88.94%. Abstract This paper proposes a vehicle counting and speed estimation method for traffic situation analysis in road CCTV videos. The proposed method removes a distortion in the images using Inverse perspective Mapping, and obtains specific region for vehicle counting and speed estimation using lane detection algorithm. Then, we can obtain vehicle counting and speed estimation results from using optical flow at specific region. The proposed method achieves stable accuracy of 88.94% from several CCTV images by regional groups and it totally applied at 106,993 frames, about 3 hours video. Keyword : Vehicle Counting, Vehicle Speed Estimation, Inverse Perspective Mapping, Optical Flow, Traffic Situation Analysis a) LIG.Project 5 (LIG ex1 Avionics R& Lab) b) (Yonsei University, ept. Computer Science) Corresponding Author : (Hyeran Byun) E-mail: hrbyun@yonsei.ac.kr Tel: +82-2-2123-3876 ORCI: http://orcid.org/0000-0002-3082-3214 2017 ( ) (o.2016-0-00152,eep Learning ). Manuscript received April 3, 2017; Revised July 7, 2017; Accepted July 7, 2017. Copyright 2017 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-C- (http://creativecommons.org/licenses/by-nc-nd/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.
4 : Optical Flow CCTV (Jihae Kim et al.: Optical Flow Based Vehicle Counting and Speed Estimation in CCTV Videos). 10 (ITS, Intelligent Transportation System) [1]-[3],. ITS.. [4],[5],[6],[7],[8].,..... [4]-[6],[9]... [4],[5]. (OF, Optical Flow), (F, Frame ifference), (BS, Background Subtraction)., - [10] (Vision) [6],[7],[9],[11],[12].. CCTV (IPM, Inverse Perspective Mapping) [13] - (Top-view). -.. 1.. 1. Fig. 1. The flow of the proposed method. 2, 3 IPM.. 4, 5.
(JBE Vol. 22, o. 4, July 2017)..,. 1.. (F, Frame ifference), (BS, Background Subtraction), (OF, Optical Flow) [4],[5],[14]-[16]. (F, Frame ifference) [4],[5].,,. (BS, Background Subtraction) [14],[16].. (background modeling).,,. [14],[16] CCTV. (OF, Optical Flow) [15],[17].. 2. /, -, - [6],[7],[9],[11],[12],[18].. Inertial avigation System(IS) (wheel speed sensor)..,. height-precision GPS. GPS,. - [18] kinematic dynamics model... - [6],[7],[9],[11],[12]. [7],[11]. [12] Lucas- Kanade Horn-Schunck.. CCTV CCTV. CCTV
4 : Optical Flow CCTV (Jihae Kim et al.: Optical Flow Based Vehicle Counting and Speed Estimation in CCTV Videos) 1) (IPM, Inverse Perspective Mapping), 2), 3) (OF, Optical Flow), 4) 4. (perspective distortion) IPM -,.., IPM IPM -.... 1. (IPM, Inverse Perspective Mapping). 2, 3. [19]. 2 (resampling). 2 (Euclidean). ( ), ( ) ( ), ( ), () (2) 2 3. CCTV... (IPM, Inverse Perspective Mapping) [13] -. tan cos tan sin (a) Input image (b) Vanishing point detection (c) Result of IPM 2. Fig. 2. Examples of applying IPM
(JBE Vol. 22, o. 4, July 2017) 2 IPM., IPM. 2. IPM -. [13],[20] 3. 3(a) 3.1 IPM -. -., - (top-hat). 3(b), 3(c). [21] (structuring element),. 3(d). (column), 5 (bin). 3(e) - 3(f). 2/3, 20. 3(g). -. (3) n-1, n ( ) 0.6,. 3. Fig. 3 Flow of lane detection algorithm
4 : Optical Flow CCTV (Jihae Kim et al.: Optical Flow Based Vehicle Counting and Speed Estimation in CCTV Videos) 3. CCTV. IPM. Farneback (dense optical flow)..,. (sparse optical flow), (dense optical flow). Lucas-kanade,.. Farneback. Farneback. 4 step. 3.2,. 4. Fig. 4. Example of motion measuring regionounting 5. 3.2 5 Fig. 5 The flow of vehicle counting
(JBE Vol. 22, o. 4, July 2017). (maxflow) (4). max max max,,.. 4.. ( ) ( ) (5), (6).,.. 4. CCTV. Lucas-Kanade. 3.2 Lucas-Kanade..,. 3.2. Lucas-Kanade,.... ( ) 1000cm(10m). (7).,,,. (7). (8)., (7), FPS C km/h. (8). 1.. CCTV.
김지혜 외 4인: Optical Flow 기반 CCTV 영상에서의 차량 통행량 및 통행 속도 추정에 관한 연구 (Jihae Kim et al.: Optical Flow Based Vehicle Counting and Speed Estimation in CCTV Videos) 를 위해 사용한 테스트 셋은 도로 중간의 철탑에 설치된 카메라로 촬영되었다. 표 1은 실험을 위해 구성한 테스트 셋을 보여주고 있다. 테스트 셋 영상은 2차선, 3차선, 4차선 영상으로 구성되어 표 1. 실험을 위한 테스트 셋 구성 Table 1. Test set composition for experiment CCTV ay ight total Busan 6,218 13,218 Goyang 14,000 있다. 각 지역별로 주간영상(ay)과 야간영상(ight)을 모 Gimpo1 14,000 두 Gimpo2 5,575 6,787 12,362 Yangsan 14,000 Ceongdo 14,000 Suseong 5,873 5,813 11,686 amyangju 6,727 13,727 total 53,175 53,818 106,993 포함하여 총 16개의 영상에 대하여 실험을 진행하였다. 제공받은 영상을 프레임단위로 저장하여 사용하였고, 총 106,993 프레임에 대하여 실험을 진행하였다. 표 2에서는 구성된 16개 영상의 테스트 셋에 대한 예시를 보여주고 있 다. 표 2. 성능 평가를 위한 테스트 셋 구성 Table 2. Composition of testset for evaluation # lane CCTV 4 G Busan_ 3 2 Busan_ oyang_ Goyang_ Gimpo1_ Gimpo2_ Yangsan_ Gimpo1_ Gimpo2_ Yangsan_ Ceongdo_ Suseong_ amyangju_ Ceongdo_ Suseong_ amyangju_ 455
(JBE Vol. 22, o. 4, July 2017) 2.. ground truth. (9) (10)., G ground truth, C. (9) AE (10).. 3. 3 5 ground truth(g), experimental result(c),,. 3 4, 4 5 3, 2. 6 3 5. ground truth(g), experimental result(c),. 16 88.94%. 6. 90.55%, 87.32%.. 7 (background subtraction) [14], (optical flow) Lucas-kanade.,,, 56.94%, 63.57%, 88.94%.. Gimpo2 3. 4 Table 3. Experiment result of 4 lanes of video CCTV Lane 1 Lane 2 Lane 3 Lane 4 total ground truth 67 114 157 111 449 experimental result 74 122 165 108 469 accuracy(%) 89.55 92.98 94.90 97.30 93.68 Busan average speed(km/h) 82 84 78 77 80.25 ground truth 39 87 112 85 323 experimental result 43 97 125 90 355 accuracy(%) 89.74 88.51 88.39 94.12 90.19 average speed(km/h) 62 76 72 68 69.5 ground truth 51 82 88 74 295 experimental result 51 81 98 73 303 accuracy(%) 100.00 98.78 88.64 98.65 96.52 Goyang average speed(km/h) 73 73 73 71 72.5 ground truth 25 32 41 23 121 experimental result 28 37 41 23 129 accuracy(%) 88.00 84.38 100.00 100.00 93.09 average speed(km/h) 61 58 55 52 56.5
4 : Optical Flow CCTV (Jihae Kim et al.: Optical Flow Based Vehicle Counting and Speed Estimation in CCTV Videos) 4. 3 Table 4. Experiment result of 3 lanes of video CCTV Lane 1 Lane 2 Lane 3 total Gimpo1 Gimpo2 Yangsan ground truth 23 30 18 71 experimental result 21 28 19 68 accuracy(%) 91.30 93.33 94.44 93.03 average speed(km/h) 70 56 50 58.67 ground truth 18 26 15 59 experimental result 19 28 17 64 accuracy(%) 94.44 92.31 86.67 91.14 average speed(km/h) 62 47 39 49.33 ground truth 21 31 34 86 experimental result 18 29 33 80 accuracy(%) 85.71 93.55 97.06 92.11 average speed(km/h) 89 81 64 78 ground truth 27 29 33 89 experimental result 27 30 35 92 accuracy(%) 85.71 96.55 93.94 92.07 average speed(km/h) 64 62 57 61 ground truth 83 90 46 219 experimental result 81 89 46 216 accuracy(%) 97.59 98.89 100.00 98.83 average speed(km/h) 91 81 82 84.67 ground truth 34 48 18 100 experimental result 34 47 18 99 accuracy(%) 100.00 97.92 100.00 99.31 average speed(km/h) 66 73 78 72.33 5. 2 Table 5. Experiment result of 2 lanes of video CCTV Lane 1 Lane 2 total ground truth 55 72 127 experimental result 54 74 128 accuracy(%) 98.18 97.22 97.70 Ceongdo average speed(km/h) 50 49 49.50 ground truth 46 57 103 experimental result 46 57 103 accuracy(%) 100.00 100.00 100.00 average speed(km/h) 50 48 49 ground truth 18 36 54 experimental result 18 38 56 accuracy(%) 100.00 94.44 97.22 Suseong average speed(km/h) 61 68 64.50 ground truth 15 28 43 experimental result 18 29 47 accuracy(%) 80 96.43 88.21 average speed(km/h) 78 65 71.50 ground truth 47 35 82 experimental result 46 35 81 accuracy(%) 97.87 100.00 98.94 amyangju average speed(km/h) 89 76 82.50 ground truth 34 43 77 experimental result 35 43 78 accuracy(%) 97.06 100.00 98.53 average speed(km/h) 83 66 74.50
(JBE Vol. 22, o. 4, July 2017) 6. Table 6. Accuracy of vehicle counting on day and night video Accuracy(%) ay 90.55 ight 87.32 total 88.94 6. Fig. 6. Result of vehicle counting experiment 7. Fig. 7. Result of vehicle counting with comparative experiment
4 : Optical Flow CCTV (Jihae Kim et al.: Optical Flow Based Vehicle Counting and Speed Estimation in CCTV Videos) 8. Fig. 8. Result of vehicle speed estimation... Farneback, Lucas-kanade.. Farneback. 8.,... CCTV. (IPM, Inverse Perspective Mapping),..,, CCTV..,
(JBE Vol. 22, o. 4, July 2017). (References) [1] EunJu Lee, Jae-Yeal am, ByoungChul Ko, Speed-limit Sign Recognition Using Convolutional eural etwork Based on Random Forest, The Korean Institute of Broadcast and Media Engineers, pp.938-949, 2015. [2] Kum, Chang-Hoon, Cho, ong-chan, Kim, Whoi-Yul, evelopment of Lane etection System using Surrounding View Image of Vehicle, The Korean Institute of Broadcast and Media Engineers, pp.331-334, 2013. [3] Hyung-Sub Kang, ong-chan Cho and Whoi-Yul Kim, Passing Vehicle etection using Local Binary Pattern Histogram, The Korean Institute of Broadcast and Media Engineers, pp.261-264, 2010. [4] A. Tourani and A. Shahbahrami, Vehicle Counting Method Based on igital Image Processing Algorithms, IEEE Transactions on International Conference on Pattern Recognition and Image Analysis, pp. 1-6, 2015. [5] Y. Xia, X. Shi, G. Song, Q. Geng and Y. Liu, Towards imporving quality of video-based vehicle counting method for traffic flow estimation, Signal Processing, pp. 672-681, 2016. [6] X. Qimin, L. Xu, W. Mingming, L. Bin and S. Xianghui, A Methodology of Vehicle Speed Estimation Based on Optical Flow, IEEE International Conference on Service Operations and Logistics and Informaitcs, pp. 33-37, 2014. [7] J. Lan, J. Li, G. Hu, B. Ran, L. Wang, Vehicle speed measurement based on gray constraint optical flow algorithm, International Journal of Light and Eletron Optics, pp. 289-295, 2014. [8] M.S. Shirazi and B. Morris, A Typical Video-based Framework for Counting, Behavior and Safety Analysis at Intersections, IEEE Transactions on Intelligent Vehicles Symposium, pp. 1264-1269, 2015. [9] S.C. iamantas and P. asgupta, Active Vision Speed Estimation from Optical Flow, Towardss Autonomous Robotic Systems, pp. 173-184, 2014. [10] X. Yu, X. Gao, Review of Vehicle State Estimation Problem under riving Situation, Chinese Journal of Mechanical Engineering, pp. 20-33, 2009. [11].C. Luvizon, B.T. assu and R. Minetto, Vehicle speed estimation by license plate detection and tracking, IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 6563-6567, 2014. [12] E. Patel and. Shukla, Comparison of Optical Flow Algorithms for Speed etermination of Moving Objects, International Journal of Computer Applications, Vol. 63, o. 5, pp. 32-37, 2013. [13]. ing, J.S. Yoo, J.K. Jung and S. Kwon, An Urban Lane etection Method Based on Inverse Perspective Mapping, GCIT, Advanced Science and Technology Letters, vol. 63 pp.53-58, 2014. [14] S. Aslani and H. Mahdavi-asab, Optical Flow Based Moving Object etection and Tracking for Traffic Surveillance, International Journal of Electrical, Computer, Energetic, Electronic and Communication Engineering, Vol. 7, o. 9, pp. 1252-1256, 2013. [15] A. Glowacz, Z. Mikrut and P. Pawlik, Video etection Algorithm Using an Optical Flow Calculation Method, Multimedia Communications, Services and Security, Vol. 287, pp. 118-129, 2012. [16] H.Y. Cheng and S.H. Hsu, Intelligent Highway Traffic Surveillance With Self-iagnosis Abilities, IEEE Transactions on Intelligent Transportation Systems, Vol. 12, o. 4, pp. 1462-1472, 2011. [17] M. Mizushima, Y. Taniquchi, G. hasegawa, H. akano and M. Matsuoka, Counting Pedestrians Passing through a Line in Video Sequences based on Optical Flow Extraction, Recent Advances in Circuits, Systems and Automatic Control, pp. 129-136, 2013. [18] K. Jo, K. Chu, K. Lee, M. Sunwoo, Integration of Multiple Vehicle Models with IMM Filter for Vehicle Localization, IEEE Transactions on Intelligent Vehicles Symposium, pp. 746-751, 2010. [19] M. Bertozzi, A. Broggi and A. Fascioli, An extension to the Inverse Perspective Mapping to handle non-flat roads, IEEE International Conference on Intelligent Vehicle, pp. 305-310, 1998. [20] M. Aly, Real time etection of Lane Markers in Urban Streets, IEEE Transactions on Intelligent Vehicles Symposium, 2008. [21] S. Suzuki and K. Abe, Topological Structural Analysis of igitized Binary Images by Border Following, Computer Vision, Graphics, and Image Processing, Vol. 30, o. 1, pp. 32-46, 1985. [22] L. Imsland, T.A. Johansen, T.I. Fossen, et al, Vehicle velocity estimation using nonliear observer, Automatica, pp. 2091-2103, 2006. [23] R. Zhao and X. Wang, Counting Vehicles from Semantic Rigions, IEEE Transactions on Intelligent Transportation Systems, Vol. 14, o. 2, pp. 1016-1022, 2013.
김지혜 외 4인: Optical Flow 기반 CCTV 영상에서의 차량 통행량 및 통행 속도 추정에 관한 연구 (Jihae Kim et al.: Optical Flow Based Vehicle Counting and Speed Estimation in CCTV Videos) 김지혜 저자소개 년 연세대학교 컴퓨터과학과 졸업 공학석사 년 현재 넥스원 항공연구소 주관심분야 영상처리 영상인식 컴퓨터비전 지능형 자동차 - 2017 : ( ) - 2017 ~ : LIG - ORCI : http://orcid.org/0000-0003-0640-3022 :,,, 신도경 년 2월 : 한양대학교 일반대학원 컴퓨터공학과(공학석사) 년 2월 : 한양대학교 일반대학원 컴퓨터공학과(공학박사) - 2015년 3월~현재 : LIG넥스원 항공연구소 - ORCI : http://orcid.org/0000-0001-9918-7132 - 주관심분야 : 영상처리, 컴퓨터비전, 패턴인식, 인지공학 - 2008-2015 김재경 년 월 아주대학교 일반대학원 학과 공학석사) 년 월 현재 넥스원 항공연구소 주관심분야 네트워크 미들웨어 영상처리 - 2012 8 : CW ( - 2000 12 ~ : LIG - ORCI : http://orcid.org/0000-0002-8535-3184 :,, CW 권철희 년 월 고려대학교 일반대학원 제어계측공학과(공학석사) 년 월 현재 넥스원 항공연구소 주관심분야 신호처리 디지털 신호처리 융합네트워크 - 2000 2 : - 2000 1 ~ : LIG - ORCI : http://orcid.org/0000-0002-5811-1622 : RF,, 변혜란 - 년 : 연세대학교 수학과 졸업(이학사) 년 : 연세대학교 대학원 수학과 졸업(이학석사) 년 : University of Illinois, Computer Science(M.S.) 년 : Purdue University, Computer Science(Ph..) 년 ~ 1995년 : 한림대학교 정보공학과 조교수 년 ~ 1998년 : 연세대학교 컴퓨터과학과 조교수 년 ~ 2003년 : 연세대학교 컴퓨터과학과 부교수 년 ~ 현재 : 연세대학교 컴퓨터과학과 교수 : http://orcid.org/0000-0002-3082-3214 주관심분야 : 패턴인식, 영상처리, 영상인식 1980 1983 1987 1993 1994 1995 1998 2003 ORCI 6 4 1