(JBE Vol. 20, No. 6, November 2015) (Regular Paper) 20 6, 2015 11 (JBE Vol. 20, No. 6, November 2015) http://dx.doi.org/10.5909/jbe.2015.20.6.938 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) a), a), a) Speed-limit Sign Recognition Using Convolutional Neural Network Based on Random Forest EunJu Lee a), Jae-Yeal Nam a), and ByoungChul Ko a). CNN (Convolutional neural network). CNN MLP(Multi-layer perceptron) (fully-connected). 2 CNN (Random forest). GTSRB(German Traffic Sign Recognition Benchmark) 8 SVM (Support Vector Machine) MLP. Abstract In this paper, we propose a robust speed-limit sign recognition system which is durable to any sign changes caused by exterior damage or color contrast due to light direction. For recognition of speed-limit sign, we apply CNN which is showing an outstanding performance in pattern recognition field. However, original CNN uses multiple hidden layers to extract features and uses fully-connected method with MLP(Multi-layer perceptron) on the result. Therefore, the major demerit of conventional CNN is to require a long time for training and testing. In this paper, we apply randomly-connected classifier instead of fully-connected classifier by combining random forest with output of 2 layers of CNN. We prove that the recognition results of CNN with random forest show best performance than recognition results of CNN with SVM (Support Vector Machine) or MLP classifier when we use eight speed-limit signs of GTSRB (German Traffic Sign Recognition Benchmark). Keyword : Convolutional Neural Network, Random forest, speed-limit sign recognition, feature extraction, ADAS a) (Dept. of Computer Engineering, Keimyung University) Corresponding Author : (ByoungChul Ko) E-mail: niceko@kmu.ac.kr Tel: +82-53-580-6275 ORCID: http://orcid.org/0000-0002-7284-0768 (B0008866). Manuscript received September 12, 2015; revised November 4, 2015; accepted November 4, 2015.
2 : (EunJu Lee et al.: Speed-limit Sign Recognition Using Convolutional Neural Network Based on Random Forest). (Advanced Driver Assistant System, ADAS) [1].,,, ADAS. ADAS,. EU [2] 10 50km 31km 33% 5%.. GPS.. HOG (Histogram of Gradient) [3] RSD(Radial Symmetry Detection) [4], SVM [5] (Neural Networks) [6], [7]. Mathias [8] HOG Multi-Class SVM. HOG. Barns [9] RSD(Radial Symmetry Detector), (Cross Correlation).,,. Aoyagi Asakura [10].,. 1990 CNN (Deep Learning),. CNN Caffe [11] MNIST(Mixed National Institute of Standards and Technology database) [12]. MNIST 0~9. CNN,,. CNN,, (parameters). CNN 2
(JBE Vol. 20, No. 6, November 2015) CNN 4.. l l CNN l MLP. 2 CNN... 2 CNN 1. CNN CNN MNN (Multi-layer Neural Network). MNN. (fully-connected). 1) : MNN (hidden layer). 2) :. 3) : 2. LeCun CNN [13] - 1998. 1 [13] CNN. CNN ILSVRC (Imagenet Large Scale Visual Recognition Challenge) [9]. CNN (Local receptive field), (Shared weight), (Subsampling, Pooling), (Convolution 1. CNN [13] Fig. 1. Traditional CNN structure designed for handwriting recognition
2 : (EunJu Lee et al.: Speed-limit Sign Recognition Using Convolutional Neural Network Based on Random Forest) layer) (Fully Connected Layer)... 1.1 (Convolution layer) MNN (Local) 2 N N., (Feature map).. 2 [14]. 2 N N ( - +1). 32 32 3 3 30 30. 1.2 (Subsampling, Pooling layer) (Subsampling, Pooling). 2 2 2D., 2 2. Max-pooling. 1.3 (Fully connected (FC) layer),. (Correlation) [15], [14], SVM [5], [16] (Error Correcting Output Code) [17]. (Decision tree). 2., (a) (b) (c) Fig. 2. Convolution Operation using a filter, (a) Receptive field, (b) Convolution layer, (c) Output
(JBE Vol. 20, No. 6, November 2015) 2. 2 CNN 2.1 CNN 3 2 CNN. CNN, 1 (C1), 1 (S1), 2 (C2), 2 (S2),. 1 CNN 2. 2.1.1 32 32 1% 1% (Histogram stretching). 4. C1. 4. (a) (b) Fig. 4. (a) original speed-limited signs and (b) histogram stretched signs 2.1.2 C1 C1 4. C1 3 3. 30 30. 3 3 5. 9 9 LeCun [14]. 0~255 9 3. 2 CNN (a) (b) Fig. 3. The proposed 2-layer CNN achitecture, (a) convolution and subsampling layer for feature extraction, (b) random forest layer for classification
2 : (EunJu Lee et al.: Speed-limit Sign Recognition Using Convolutional Neural Network Based on Random Forest) 9., 300. 5 300 80% 3. 5-(c) 4 1 2. 5-(c) 4. 3 C1 1:1. (1) (Non-linear activation) ReLU(Rectified Linear hidden Unit) [18]. ( ), ReLU. ReLU Gradient vanishing [19]. max 5. C1 Fig. 5. C1 layer filters 2.1.3 S1 CNN.,. C1 2 2 15 15 4 6. Max pooling (a) 8x6, (b) 2x2 max-pooling, (c) max-pooling Fig. 6. Max pooling method (a) feature map of 8x6 pixel size, (b) appling max-pooling after 2x2 division, (c) max-pooling result
(JBE Vol. 20, No. 6, November 2015). Mean-pooling, Max-pooling, Max-pooling.. 6 Max-pooling.. S1 1 50. 7 C2 2. 2.1.4 C2 C2 C1. C1 S1 6. C2 S1 7. C2 Fig. 7. Weight combination method for C2 layer feature map 2.1.5 S2 3 C2 S2 S1. C2 2 2 max-pooling 6 6 6. 2.2 RF (Randomly connected RF) CNN MLP.,. SVM Adaboost. S2 216 ( : 36 : 6). 8 12,550 100. N. N Adaboost (BRF, Boosted Random Forest) [20].
2 : (EunJu Lee et al.: Speed-limit Sign Recognition Using Convolutional Neural Network Based on Random Forest) 80.. BRF C1~S2 8.. 1. 2 CNN GTSRB (German Traffic Sign Recognition Benchmark) [21]. GTSRB 30. 15 15 180 180, 51,883 13,490,,,. GTSRB 20, 30, 50, 60, 70, 80, 100, 120 8. 1. GTSRB Table 1. GTSRB Dataset Specification GTSRB Data Resolution of speed limit signs Number of signs Image Training set 20 200 30 2200 50 2250 60 1400 70 25 26 ~161 168 1900 80 1800 100 1400 8. Fig. 8. Speed-limit sign classification using Random forest Test set 120 1400 20 40 30 150 50 150 60 120 70 29 28 ~121 130 115 80 150 100 115 120 100
(JBE Vol. 20, No. 6, November 2015) CNN GPU CPU CPU. Intel i7 CPU 3.6GHZ 32GB Windows 7 64 R2010a. 1. 1 CNN 32 32. 2. 1.. 2. 2 Table 2 Comparison of speed-limit sign recognition according to the preprocessing Preprocessing method Average precision original image histogram equalization histogram stretching 85.73% 79.75% 91.2% 5.47%, (Histogram equalization) 11.45%... 9. 3. 13,490 12,550, 940, CNN. C1 0~255 9 9. 300. CNN S2. 2 CNN (2CNN), 3. (2CNN+RF) Multi-layer Perceptron (2CNN+MLP) Support Vector Machine (2CNN+SVM) 9. (a) (b) (c) Fig. 9. Comparison of image pre-processing result (a) original image (b) histogram equalization (c) histogram stretching 1. 10 Precision. 91.2% 2CNN+SVM 5.2%, 2CNN+MLP 1.3%., 20, 30,
2 : (EunJu Lee et al.: Speed-limit Sign Recognition Using Convolutional Neural Network Based on Random Forest) 10. Fig. 10, Result of performance comparison 50, 60, 80, 100. 20 100% Precision., 50 75.4% precision. 70 120 20 precision. 2CNN+MLP 70 120 94.7% 100%. Penalty( value)=5, variance( )=1 RBF(Radial Basis Function) 2CNN+SVM 7.. 1 CPU 1 0.24ms, 2CNN+MLP 0.3ms, 2CNN+SVM 2.46ms., SVM,. 2CNN+SVM. IV.,,,,, 2 Randomly-CNN.,. CNN
(JBE Vol. 20, No. 6, November 2015) CNN. CNN.,.,.,. Spatial Pyramid Pooling(SPP), CNN. (References) [1] L. Kwangyoung, K. Seunggyu and B. Hyeran, Real-time Traffic Sign Detection using Color and Shape Feature, Korea Computer Congress 2012, Vol 39, No 1. pp. 504-506, June, 2012. [2] G. J. L. Lawrence, B. J. Hardy, J. A. Carroll, W. M. S. Donaldson, C. Visvikis and D. A. Peel, A study on the feasibility of measures relating to the protection of pedestrians and other vulnerable road users, Final Tech. Report, TRL. Limited, pp. 206, June, 2004. [3] N. Dalal and B. Triggs, Histograms of oriented gradients for human detection, IEEE Con. Computer Vision and Pattern Recognition, vol. 1, pp. 886 893, 2005. [4] N. Barnes, A. Zelinsky and L. Fletcher, Real-time speed sign detection using the radial symmetry detector, IEEE Trans. Intelligent Transportation Systems, Vol. 9, No. 2, pp. 322-332, 2008. [5] S. Maldonado-Bascon, S. Lafuente-Arroyo, P. Gil-Jimenez, H. Gomez-Moreno and F. Lopez -Ferreras, Road-Sign Detection and Recognition Based on Support Vector Machines, IEEE Tran. Intelligent Transportation Systems, Vol. 8, No. 2, pp. 264-278, June, 2007. [6] Y. Aoyagi and T. Asakura, A Study on Traffic Sign Recognition in Scene Image Using Genetic Algorithms and Neural Networks, IEEE Int. Conf. Industrial Electronics, Control, and Instrumentation, Vol. 3, pp. 1838-1843, Aug. 1996. [7] G. JaWon, H. MinCheol, K. Byoung Chul and N. Jae-Yeal, Real-time Speed-Limit Sign Detection and Recognition using Spatial Pyramid Feature and Boosted Random Forest, 12th International Conference on Image Analysis and Recognition, pp.437-445, July, 2015. [8] M. Mathias, R. Timofte, R. Benenson and L.V. Gool, Traffic sign recognition How far are we from the solution?, IEEE Int. Con. Neural Networks, pp. 1-8, 2013. [9] N. Barnes, A Zelinsky and L.S. Fletcher, Real-time speed sign detection using the radial symmetry detector, IEEE Trans. Intelligent Transportation Systems, pp. 322-332, 2008. [10] Y. Aoyagi and T. Asakura. Detection and recognition of traffic sign in scene image using genetic algorithms and neural networks., SICE-ANNUAL CONF. pp. 1343-1348, 1996. [11] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick and T. Darrell, Caffe: Convolutional architecture for fast feature embedding, in Proceedings of the ACM International Conference on Multimedia, pp. 675-678, November, 2014. [12] Y. LeCun, C. Cortes and C.J. Burges, The MNIST database of handwritten digits, 1998. [13] Y. LeCun, L. Bottou, Y. Bengio and P. Haffner, Gradient-based learning applied to document recognition, in Proceedings of the IEEE, pp. 2278-2324, 1998. [14] P. Sermanet, Y. LeCun, Traffic sign recognition with multi-scale convolutional networks, IEEE Int. Conf. Neural Networks, pp. 2809-2813, 2011. [15] A. de la Escalera, J. Armingol, J. Pastor and F. Rodriguez, Visual Sign Information Extraction and Identification by Deformable Models for Intelligent Vehicles, IEEE Tran. Intelligent Transportation Systems, Vol. 5, No. 2, pp. 57-68, June, 2004. [16] D.S. Kang, N.C. Griswold and N. Kehtarnavaz, An invariant traffic sign recognition system based on sequential color processing and geometrical transformation, Proc of IEEE Southwest Symposium on Image Analysis and Interpretation, pp. 88-93, 1994. [17] X. Baro, S. Escalera, J. Vitria, O. Pujol and P. Radeva, Traffic Sign Recognition Using Evolutionary Adaboost Detection and Forest-ECOC Classification, IEEE Trans. Intelligent Transportation Systems, Vol. 10, No. 1, pp. 113-126, Mar, 2009. [18] X. Glorot, A. Bordes and Y. Bengio, Deep sparse rectifier networks, in Proceedings of the 14th International Conference on Artificial
2 : (EunJu Lee et al.: Speed-limit Sign Recognition Using Convolutional Neural Network Based on Random Forest) Intelligence and Statistics, Vol. 15, pp. 315-323, 2011. [19] K. Kyungmin, H. Jungwoo and Z. Byoungtak, Character-based Subtitle Generation by Learning of Multimodal Concept Hierarchy from Cartoon Videos, Journal of korea Intelligent Information System Society, Vol. 42, No. 4, pp. 451-458, 2015. [20] L. Breiman, Random Forests, Machine Learning, vol. 45, pp. 5-32, 2001. [21] J. Stallkamp, M.. Schlipsing, J. Salmen and C. Igel, The German traffic sign recognition benchmark: a multi-class classification competition, IEEE Conf. Neural Networks, pp. 1453-1460, July, 2011. - 2001 : () - 2004 : ( ) - 2006 : - ORCID : http://orcid.org/0000-0002-3225-3788 - : - 1983 년 : 경북대학교전자공학과졸업 ( 공학사 ) - 1985 : ( ) - 1991 : University of Texas at Arlington ( ) - 1985 5 ~ 1987 7 : - 1991 9 ~ 1995 2 : - 1995 3 ~ : - ORCID : http://orcid.org/0000-0001-7288-283x - :,, - 1998 : () - 2000 : ( ) - 2004 : ( ) - 2004 : 3-2005 8-2005 : 9 ~ : - ORCID : http://orcid.org/0000-0002-7284-0768 - :,,