(JBE Vol. 22, No. 3, May 2017) (Special Paper) 22 3, 2017 5 (JBE Vol. 22, No. 3, May 2017) https://doi.org/10.5909/jbe.2017.22.3.282 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) a), a), a), a), a) Vehicle Headlight and Taillight Recognition in Nighttime using Low-Exposure Camera and Wavelet-based Random Forest Duyoung Heo a), Sang Jun Kim a), Choong Sub Kwak a), Jae-Yeal Nam a), and Byoung Chul Ko a),. ROI (Region of Interest) FROI (Front ROI) BROI (Back ROI). ROI, 2. (redness) Haar-like. SVM(Support Vector Machine) CNN(Convolutional Neural Network). (Pairing),.,. Abstract In this paper, we propose a novel intelligent headlight control (IHC) system which is durable to various road lights and camera movement caused by vehicle driving. For detecting candidate light blobs, the region of interest (ROI) is decided as front ROI (FROI) and back ROI (BROI) by considering the camera geometry based on perspective range estimation model. Then, light blobs such as headlights, taillights of vehicles, reflection light as well as the surrounding road lighting are segmented using two different adaptive thresholding. From the number of segmented blobs, taillights are first detected using the redness checking and random forest classifier based on Haar-like feature. For the headlight and taillight classification, we use the random forest instead of popular support vector machine or convolutional neural networks for supporting fast learning and testing in real-life applications. Pairing is performed by using the predefined geometric rules, such as vertical coordinate similarity and association check between blobs. The proposed algorithm was successfully applied to various driving sequences in night-time, and the results show that the performance of the proposed algorithms is better than that of recent related works. Keyword : intelligent headlight control, region of interest, adaptive thresholding, random forest, pairing, low-exposure camera Copyright 2017 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND (http://creativecommons.org/licenses/by-nc-nd/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.
4 : (Duyoung Heo et al.: Vehicle Headlight and Taillight Recognition in Nighttime using Low-Exposure Camera and Wavelet-based Random Forest). (Advanced Driver Assistant System, ADAS) (Intelligent Headlight Control, IHC). 2011 [1], 50% 31.4%. (headlight).. IHC.. IHC. IHC (taillight),,. [2]. IHC. IHC a) (Dept. of Computer Engineering, Keimyung University) Corresponding Author : (ByoungChul Ko) E-mail: niceko@kmu.ac.kr Tel: +82-53-580-6275 ORCID: http://orcid.org/0000-0002-7284-0768 Manuscript received February 23, 2017; Revised April 18, 2017; Accepted April 26, 2017..,, [3]..... CMOS(Complementary Metal Oxide Semiconductor)... (Automatic Exposure) IHC [2][4][5][6], [4] [6]. SVM (Support Vector Machine) Kalman filtering [2]. IHC. (Low Exposure) [7][8]... [9].
(JBE Vol. 22, No. 3, May 2017). (Front Region- Of-Interest, FROI) FROI.. IHC. 1 [9] FROI BROI(Back ROI). (Random Forest). (Kalman filtering) (Hungarian) (pairing).. FROI BROI... ROI 1. FROI BROI. ROI 1. Overview Fig 1. Proposed algorithm overview
4 : (Duyoung Heo et al.: Vehicle Headlight and Taillight Recognition in Nighttime using Low-Exposure Camera and Wavelet-based Random Forest) FROI ROI BROI. FROI BROI. 2 ROI FROI BROI. 1. RGB YCbCr Y Cr. 3 Cr Y, Y Cr. (1). ROI Y Cr (1). 2. ROI Fig 2. ROI composition of input image 2. ROI (Adaptive Thresholding) ROI (1) Y Cr.,. ROI, FROI BROI. K K3 1.5 3. YCr color model Fig 3. Brightness difference between headlight and tail light in YCr color model 4. K Fig 4. Binarization of car light according to a constant value K'
(JBE Vol. 22, No. 3, May 2017). 4 K 1.5 K 1.5. ( 2). (2) i, (, ). 3. (optical flow) ROI IHC,, (optical flow) ROI. 0.5 (Down-sampling) 5 (a) (dense point)... ( 5 (b)). 5 (c) ROI.. 1. Haar-Like (Redness). ( ) (3). (3) r, g, b r,g,b i. (3) 0~1 5. ROI Fig 5. ROI movement using optical flow
4 : (Duyoung Heo et al.: Vehicle Headlight and Taillight Recognition in Nighttime using Low-Exposure Camera and Wavelet-based Random Forest).. R 70%. 0.13.. 0~1-2. 0.13 98.6%.,,. Haar- Like [10] [11]. SVM(Support Vector Machine) Adaboost. Haar-Like. 6(a) 20 Haar- Like 6(b) Haar-Like. 6. (a) Haar-Like 20, (b) Haar-Like Fig 6. (a) 20-patterns of Haar-Like features, (b) Haar-Like feature vector (a) 7. Fig 7. Classification of head light and tail light (b)
(JBE Vol. 22, No. 3, May 2017) Haar-like. 7 (a) Haar-like,. 7 (b). 3.2. 2. 2.1 OCS-LBP(oriented center-sym- metric local binary feature) (texture). 8. (wavelet transform). 8,,.. 3 high pass (LH, HL, HH) OCS-LBP [12] (Oriented Center Symmetric-Local Binary Pattern). OCS-LBP. 9 OCS-LBP, HOG LBP. OCS-LBP. 8. (a), (b), (c) Fig 8. Results of wavelet transform according to the lights, (a) headlight, (b) street light, (c) reflection light of traffic sign 9. OCS-LBP Fig 9. OCS-LBP feature extraction from a candidate region
4 : (Duyoung Heo et al.: Vehicle Headlight and Taillight Recognition in Nighttime using Low-Exposure Camera and Wavelet-based Random Forest) 2.2 OCS-LBP OCS-LBP.. R, G, B. 7 (b) OCS-LBP.. / 1. (Paring), 2.., (Hungarian)., 4..... (Similarity score). 3,,. (Size) 2 max min. 1. (Verti- cal overlap) 10 Y 2 Y (length) (overlap). (Aspect ratio) / 2 max min 1.. min max min max. 10. V(vertical overlap) Fig 10. Vertical overlap component
(JBE Vol. 22, No. 3, May 2017) (a) (b) 11., (a) (b) Fig 11. Vehicle detection through paring and high beam control, (a) blob matching using Hungarian matching, (b) high beam control algorithm 2. (High beam),.,. (N=5),. N, (queue)... 1. Aptina MT9V CMOS. 752x480 1/3., 7 10. 247 239. 100 94. 1. 1. Table 1. Training data composition of head and tail light detection algorithm Category Feature Number of Training Positive : 105 Color Negative : 105 Head light Positive : 142 Wavelet Negative : 134 Tail light Haar-Like Positive : 100 Negative : 94 10
4 : (Duyoung Heo et al.: Vehicle Headlight and Taillight Recognition in Nighttime using Low-Exposure Camera and Wavelet-based Random Forest). 2. Table 2. Test video configuration Video No. Total frames Video 1 211 Video 2 402 Video 3 87 Video 4 101 Video 5 68 Video 6 250 Video 7 200 Video 8 198 Video 9 86 Video 10 144 Data description ( C : Car ) C1 : Tail light Road sign C1, C2, C3 : Head light Road sign C1, C2, C3 : Head light Speed bumper C1, C2 : Head light Street light, AD sign C1 : Head light Street light C1 : Tail light Street light, Traffic sign C1 : Tail light, C2 : Head light Street light, Traffic sign, Traffic lamp C1 : Head light Street light, Traffic sign C1 : Head light, C2 : Tail light Street light C1 : Head light, C2 : Tail light Street light, Traffic lamp, Method1 [9], Method2 [3], Method3 [13] precision recall. CPU 3 GPU CNN (Convolutional Neural Networks). / 4. Method1 [9] : RGB Laplacian Of Gaussian (LOG), Method2 [3] : Real-AdaBoost Method3 [13] :, Method4 [14] : CNN (Deep Learning) YOLO Proposed : Precision 98.6% Method2 1.7%. Recall 87.8% Method2 22.6%. CNN Method4 Precision Recall. Recall. 12 Fig 12. Performance evaluation
(JBE Vol. 22, No. 3, May 2017) (miss). Precision Recall. 3. ( ms) Table 3. Processing time evaluation (ms) Category Method 1 Method 2 Method 3 Method 4 Proposed AVG 12.3 13.9 11.6 11.8 12.5 3.. Method1 Method3 CPU Method4 GPU. Method3 Precision Recall, 12.5ms Precision Recall. CNN Method4 Precision Recall, 11.8ms GPU. 13,. IV. Haar-like OCS-LBP,., ROI. ROI FROI BROI.. N 13. (a) Video1,(b) Video3, (c) Video4, (d) Video9 Fig 13. Vehicle light detection result of the proposed algorithm
4 : (Duyoung Heo et al.: Vehicle Headlight and Taillight Recognition in Nighttime using Low-Exposure Camera and Wavelet-based Random Forest) (5).,. ROI, ROI ROI. i7-4790 CPU @ 3.60GHz 752x480 11ms..,,,,.,. (References) [1] National Highway Traffic Safety Administration, Traffic safety facts 2011: A compilation of motor vehicle crash data from the fatality analysis reporting system and the general estimates system, NHTSA Annual Report, 2011. [2] P. F. Alcantarilla, L. M. Bergasa, P. Jimenez, M. A. Sotelo, I. Parra, D. Fernandez, Night time vehicle detection for driving assistance lightbeam controller, IEEE Intelligent Vehicle Symposium, pp. 291-296, 2008. [3] A. López, J. Hilgenstock, A. Busse,R. Baldrich, F. Lumbreras, and J. Serrat, "Nighttime vehicle detection for intelligent headlight control," Lecture Note in Computer Science, vol. 5259, pp. 113-124, 2008. [4] P. F. Alcantarilla, L. M. Bergasa, P. Jimenez, M. A. Sotelo, I. Parra, D. Fernandez, M. A. Sotelo, S. S. Mayoral, Automatic lightbeam controller for driver assistance, Machine Vision and Application. vol. 22, pp. 819-835,2011. [5] J. H. Connell, B. W. Herta, S. PanKani, H. Hess, and S. Pliefke, A fast and robust intelligent headlight controller for vehicles, IEEE Intelligent Vehicle Symposium, pp. 703-708, 2011. [6] W. Zhang, Q. M. J. Wu, G. Wang, and X. You,"Tracking and Pairing Vehicle Headlight in Night Scenes," IEEE Transaction on Intelligent Transportation System, vol. 13, pp. 140-153, 2011. [7] R. O Malley, M. Glavin, and E. Jones, Vision-based detection and tracking of vehicles to the rear with perspective correction in low-light conditions, IET Intelligent Transportation System, vol. 5, pp. 1-10, 2011. [8] D. Heo, C.-S. Kwak, S. Kim, B. C. Ko, and J. Y. Nam, Intelligent high beam control of a vehicle for driving assistance in nighttime," International Workshop on Advanced Image Technology, pp. 1-4, 2017. [9] S. Eum, H. G. Jung, Enhancing light blob detection for intelligent headlight control using lane detection, IEEE Transaction on Intelligent Transportation System, vol. 14, pp. 1003-1011, 2013. [10] Kamal Nasrollahi and Thomas B. Moeslund, Haar-Like Features for Robust Real-Time Face Recognition, IEEE Int. Conf. Image Process., pp. 3073 3077, 2013. [11] M. Jeong, B. C. Ko, and J.-Y. Nam, Early Detection of Sudden Pedestrian Crossing for Safe Driving during Summer Nights, IEEE Trans. Circuits Syst. Video Technol., vol. 1, pp. 1 13, 2016. [12] B. C. Ko, J.-Y. Kwak, and J.-Y. Nam, Human tracking in thermal images using adaptive particle filters with online random forest learning, Opt. Eng. vol. 52, no. 11, pp. 113-105, 2013. [13] S. Zhou, J.Li, Z.Shen, L.Ying A Night time Application for a Real-Time Vehicle Detection Algorithm Based on Computer Vision Science, Engineering and Technology vol. 5, no.10, pp. 3037-3043, 2013. [14] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, YOLO-You only look once: Unified, real-time object detection, IEEE Conf. on Computer Vision, pp.1-10, 2016.
2 94 방송공학회논문지 제22권 제3호, 2017년 5월 (JBE Vol. 22, No. 3, May 2017) 저자소개 허두영 년 계명대학교 컴퓨터공학과 졸업 공학사 년 계명대학교 대학원 컴퓨터공학 석사과정 주관심분야 비전기반 화재감지 영상검색 머신러닝 - 2017 : ( ) - 2017 : - ORCID : http://orcid.org/0000-0002-3456-6884 : ADAS,,, 김상준 년 계명대학교 컴퓨터공학과 졸업 공학사 년 계명대학교 대학원 컴퓨터공학 석사과정 주관심분야 비전기반 화재감지 보행자 추적, 머신러닝 - 2017 : ( ) - 2017 : - ORCID : http://orcid.org/0000-0002-7548-9651 : ADAS,, 곽충섭 년 계명대학교 컴퓨터공학과 졸업 공학사 주관심분야 비전기반 화재감지 영상검색 머신러닝 - 2017 : ( ) - ORCID : http:/orcid.org/0000-0003-4978-8778 : ADAS,,, 남재열 - 년 : 경북대학교 전자공학과 졸업(공학사) 년 : 경북대학교 대학원 전자공학(공학석사) 년 : University of Texas at Arlington 전기공학(공학박사) 년 5월 ~ 1987년 7월 : 한국전자통신연구소 연구원 년 9월 ~ 1995년 2월 : 한국전자통신연구소 선임연구원 년 3월 ~ 현재 : 계명대학교 컴퓨터공학과 교수, 산학 부총장 : http://orcid.org/0000-0001-7288-283x 주관심분야 : 영상압축, 영상통신, 멀티미디어 시스템 1983 1985 1991 1985 1991 1995 ORCID 고병철 - 년 : 경기대학교 전자계산학과 졸업(이학사) 년 : 연세대학교 대학원 컴퓨터과학과 졸업(공학석사) 년 : 연세대학교 대학원 컴퓨터과학과 졸업(공학박사) 년 3월 ~ 2005년 8월 : 삼성전자 통신연구소 책임연구원 년 9월 ~ 현재 : 계명대학교 컴퓨터공학과 교수 : http://orcid.org/0000-0002-7284-0768 주관심분야 : ADAS, 비전기반 화재감지, 영상검색, 의료영상처리 1998 2000 2004 2004 2005 ORCID