THE JOURNAL OF KOREAN INSTITUTE OF ELECTROMAGNETIC ENGINEERING AND SCIENCE. 2018 Jul.; 29(7), 550 559. http://dx.doi.org/10.5515/kjkiees.2018.29.7.550 ISSN 1226-3133 (Print) ISSN 2288-226X (Online) Human Walking Detection and Background Noise Classification by Deep Neural Networks for Doppler Radars 권지훈 하성재 곽노준 Jihoon Kwon Seoung-Jae Ha* Nojun Kwak** 요약 (deep neural network: DNN)..,,.,., 90.3 %, 86.1 %. 97.3 %, 96.1 %. Abstract The effectiveness of deep neural networks (DNNs) for detection and classification of micro-doppler signals generated by human walking and background noise sources is investigated. Previous research included a complex process for extracting meaningful features that directly affect classifier performance, and this feature extraction is based on experiences and statistical analysis. However, because a DNN gradually reconstructs and generates features through a process of passing layers in a network, the preprocess for feature extraction is not required. Therefore, binary classifiers and multiclass classifiers were designed and analyzed in which multilayer perceptrons (MLPs) and DNNs were applied, and the effectiveness of DNNs for recognizing micro-doppler signals was demonstrated. Experimental results showed that, in the case of MLPs, the classification accuracies of the binary classifier and the multiclass classifier were 90.3% and 86.1%, respectively, for the test dataset. In the case of DNNs, the classification accuracies of the binary classifier and the multiclass classifier were 97.3% and 96.1%, respectively, for the test dataset. Key words: Doppler Radar, Deep Neural Network, Micro-Doppler, Radar Pattern Recognition, Radar Machine Learning. (2017-0-00306) (Radar R&D Center, Hanwha Systems) * (Department of Information and Communication Systems, Korea Polytechnics) ** (Graduate School of Convergence Science and Technology, Seoul National University) Manuscript received April 11, 2018 ; Revised May 15, 2018 ; Accepted July 7, 2018. (ID No. 20180411-047) Corresponding Author: Nojun Kwak (e-mail: Nojunkwak@snu.ac.kr) 550 c Copyright The Korean Institute of Electromagnetic Engineering and Science. All Rights Reserved.
. 서론 LED.,,,.,,.,,.,,.,,...,., RCS(Radar Cross Section).,,,....,., ( 1.4 m/s) [1]... (Window time), DC (Oscillator).., ( )., (Micro-Doppler) [2] [7]. (classifier) [2] [4],[7] [10].,.,. LED,,., (multilayer perceptron) [5]., 0.5 [5]. 551
THE JOURNAL OF KOREAN INSTITUTE OF ELECTROMAGNETIC ENGINEERING AND SCIENCE. vol. 29, no. 7, Jul. 2018.,. [5].,.,.,,.. 1 [2],[3].,, [5].. [5] (Window time), [5]. 그림 1. Fig. 1. The processing flow suggested by previous researches to recognize the micro Doppler signals.,... CFAR(Constant False Alarm Rate).,... 딥뉴럴네트워크 (deep neural network: DNN) (artificial neural networks: ANN) (hidden layer).,. DNN (raw data).,..,,.,,. CNN(Convolution Neural Network), RNN(Recurrent Neural Networks). CNN,, SAR 552
[11]. CNN. CNN,,. CNN. RNN [12]. RNN., CNN. (feedforward) ( ). 2-1 ReLu and Softmax (gradient vanishing), (Sigmoid) (activation function). (hyperbolic tangent) (0, 1). 0,., ReLU(Rectified Linear Unit),. DNN ReLU. ReLU 0 0, 0.,,, Softmax. Softmax. (1) Softmax. : Sample vector, : Weight vector, : possible outcomes. 2-2 Loss Function, Optimizer and Dropout (cross entropy (mean squred error)., Adam. (overfitting) (training dataset) (fitting) (test dataset). (fully connected neural network) (dropout), [13].. The Micro-Doppler Signals, 2.. 그림 2. Fig. 2. Data gathering by Doppler radar. (1) 553
THE JOURNAL OF KOREAN INSTITUTE OF ELECTROMAGNETIC ENGINEERING AND SCIENCE. vol. 29, no. 7, Jul. 2018. LED... 8 (a) (LOS), (b), (c), (d), (e), (f), (g), (h). 3. 4 STFT(Short Time Fourier Transform). 512 (a) (LOS) (a) Outdoor(LOS) (c) (c) Snow (e) (e) Human walking (b) (b) Fan (d) (d) Rain (f) (f) Human walking with fan (a) (LOS) (a) Outdoor (LOS) (b) (b) Fan (g) (g) Human walking wih snow (h) (h) Human walking and rain 그림 4. 8 Fig. 4. The collected signals (8 cases). (c) (c) Snow (e) (e) Human walking (g) (g) Human walking wih snow 그림 3. 8 Fig. 3. The collected signals 8 cases. (d) (d) Rain (f) (f) Human walking with fan (h) (h) Human walking and rain., 320,000.. 160,000 160,000. (a) 40,000, (b) 40,000, (c) 40,000 (d) 40,000., (e) 40,000, (f) 40,000, (g) 40,000, (h) 40,000. (train- 554
ing dataset) 240,000, 8. 40,000 40,000 (test dataset) (cross validation dataset). 8.. Design And Experiment 4-1 Binary Classifier ( 이진분류기 ) [e, f, g, h] [a, b, c, d]. 5, 6. (MLP) 5. (input layer) 512, (hidden layer) 64, (output layer) 1. (Sigmoid)., 8. 512. 6, 64 그림 5. (MLP) Fig. 5. Binary classifier using multi-layer perceptron. 그림 6. Fig. 6. Multi-class classifier using deep neural network. 그림 7. ROC, precision & recall Fig. 7. ROC, precision & recall of MLP binary classifier. ReLU. Sigmoid... (early stopping). patience, patience, (epoch). (MLP), 91.6 %, 90.7 %, 90.3 %. Receiver Operating Characteristic(ROC) AUC(Area Under and ROC curve) 0.979, Precision-Recall Graph AP(Average Precision) 0.960. 555
THE JOURNAL OF KOREAN INSTITUTE OF ELECTROMAGNETIC ENGINEERING AND SCIENCE. vol. 29, no. 7, Jul. 2018. 그림 8. DNN ROC, precision & recall Fig. 8. ROC, precision & recall of DNN binary classifier., (DNN), 97.7 %, 97.5 %, 97.3 %. ROC AUC 0.996, Precision- Recall Graph AP 0.995. 4-2 Multi-Class Classifier( 다중분류기 ),. 4 Sigmoid MLP, 84.2 %. 5, 86.1 %., 6. 5 MPL, DNN (Sigmoid). 9., ReLU DNN(ReLU) 10. 그림 10. DNN(ReLU) Fig. 10. Multi-class classifier using DNN(ReLU). DNN(Sigmoid) 87.2 %, 86.4 %, 86.1 %., ReLU DNN, 96.5 %, 96.3 %, 96.1 %. 1 DNN(Sigmoid) Confusion Matrix, 2 DNN(ReLU) Confusion Matrix.,. 표 1. Sigmoid DNN confusion matrix Table 1. Normalized confusion matrix of multi-class classifier using deep neural network with Sigmoid activation function. 그림 9. DNN(Sigmoid) Fig. 9. Multi-class classifier using DNN(Sigmoid). Actual class Estimated class (Unit : %) (a) (b) (c) (d) (e) (f) (g) (h) (a) 89.2 0.1 4.8 0.2 3.5 0.1 2.2 0.0 (b) 0.0 91.9 0.0 0.0 0.1 8.0 0.0 0.0 (c) 0.7 0.0 93.7 2.5 0.2 0.0 2.8 0.1 (d) 0.1 0.0 1.8 79.6 0.7 0.0 1.5 16.4 (e) 2.5 0.1 5.7 2.0 79.0 0.5 9.6 0.6 (f) 0.0 5.7 0.1 0.1 0.7 92.5 0.1 0.8 (g) 0.8 0.0 5.9 3.5 6.7 0.2 82.9 0.1 (h) 0.0 0.0 0.6 17.1 1.3 0.1 0.2 80.6 556
1, DNN(Sigmoid),. (a) (LOS) 3.5 % (e), 2.2 % (g), (b) 8.0 % (f) +., (c) 2.8 % (g) +, (d) 16.4 % (h) +. 1,,., (e) 2.5 % (a) (LOS), 5.7 % (c), (f) + 5.7 % (b), (g) + 5.9 % (c), 3.5 % (d), (h) + 17.2 % (d). 1,,,. 2, DNN(ReLU). ReLU.. (a) 표 2. ReLU DNN confusion matrix Table 2. Normalized confusion matrix of multi-class classifier using deep neural network with ReLU activation function. Actual class Estimated class (Unit : %) (a) (b) (c) (d) (e) (f) (g) (h) (a) 96.2 0.2 1.6 0.2 1.1 0.2 0.5 0.0 (b) 0.1 98.0 0.0 0.0 0.0 1.9 0.0 0.0 (c) 1.8 0.0 97.7 0.1 0.2 0.0 0.2 0.0 (d) 1.1 0.0 1.2 93.3 0.7 0.1 0.2 3.5 (e) 0.7 0.0 0.3 0.0 93.5 0.3 4.8 0.3 (f) 0.0 1.3 0.0 0.0 0.2 98.0 0.3 0.0 (g) 0.2 0.0 0.1 0.0 1.8 0.1 97.6 0.1 (h) 0.0 0.0 0.2 1.0 2.5 0.1 1.6 94.6 (LOS) 1.1 % (e), (b) 1.9 % (f) +, (d) 3.5 % (h) +.. 결론..,.,,.,,.,., 3 Sigmoid MLP 6 ReLU DNN., MLP 90.3 %, DNN 97.3 %.,, 5 DNN(Sigmoid) 7 DNN(ReLU)., DNN(Sigmoid) 86.1 %, DNN(ReLU) 96.1 %. ReLU DNN. 557
THE JOURNAL OF KOREAN INSTITUTE OF ELECTROMAGNETIC ENGINEERING AND SCIENCE. vol. 29, no. 7, Jul. 2018.,.,,.,.,.. References [1] R. C. Browning, E. A., Baker, J. A. Herron, and R. Kram, "Effects of obesity and sex on the energetic cost and preferred speed of walking," Journal of Applied Physiology, vol. 100, no. 2, pp. 390-398, 2006. [2] Y. Kim, S. Ha, and J. Kwon, "Human detection using Doppler radar based on physical characteristics of targets," IEEE Geoscience and Remote Sensing Letters, vol. 12, no. 2, pp. 289-293, Feb. 2015. [3] Y. Kim, H. Ling, "Human activity classification based on micro-doppler signatures using an artificial neural network," in 2008 IEEE Antennas and Propagation Society International Symposium, 2008. [4] P. Van Dorp, F. C. A. Groen, "Feature-based human motion parameter estimation with radar," IET Radar, Sonar & Navigation, vol. 2 no. 2, pp. 135-145, 2008. [5] J. Kwon, N. Kwak, "Human detection by neural networks using a low-cost short-range Doppler radar sensor," in 2017 IEEE Radar Conference, Seattle, WA, May 2017, pp. 755-760. [6] Y. Kim, T. Moon, "Human detection and activity classification based on micro-doppler signatures using deep convolutional neural networks," IEEE Geoscience and Remote Sensing Letters, vol. 13, no. 1, pp. 8-12, Jan. 2016. [7] J. Li, S. L. Phung, F. H. C. Tivive, and A. Bouzerdoum, "Automatic classification of human motions using Doppler radar," in the 2012 International Joint Conference on Neural Networks, 2012. [8] P. Van Dorp, F. C. A. Groen, "Feature-based human motion parameter estimation with radar," IET Radar, Sonar and Navigation, vol. 2, no. 2, pp. 135-145, 2008. [9] V. C. Chen, "Doppler signatures of radar backscattering from objects with micro-motions," IET Signal Processing, vol. 2, no. 3 pp. 291-300, 2008. [10] T. Thayaparan, S. Abrol, E. Riseborough, L. J. Stankovic, D. Lamothe, and G. Duff, "Analysis of radar micro-doppler signatures from experimental helicopter and human data," IET Radar, Sonar & Navigation, vol. 1, no. 4, pp. 289-299, 2007. [11] J. Ding, B. Chen, H. Liu, and M. Huang, "Convolutional neural network with data augmentation for SAR target recognition," IEEE Geoscience and Remote Sensing Letters, vol. 13, no. 3, pp. 364-368, 2016. [12] H. Sak, A. Senior, and F. Beaufays, "Long short-term memory recurrent neural network architectures for large scale acoustic modeling," in Fifteenth Annual Conference of the International Speech Communication Association, 2014. [13] Y. Gal, Z. Ghahramani, "Dropout as a Bayesian approximation: Representing model uncertainty in deep learning," in International Conference on Machine Learning, 2016, pp. 1050-1059. 558
2005 3 : ( ) 2007 1 : 2015 3 : ( ) [ 주관심분야 ],,,,, 1997 : ( ) 1999 : ( ) 2003 : ( ) 2006 9 2007 2 : BK 2007 3 2013 8 : / 2013 9 : [ 주관심분야 ],, 2001 : ( ) 2006 : ( ) 2006 2009 : 2010 2011 : VMASC( ) 2011 2014 : 2014 : [ 주관심분야 ], M/W 559