THE JOURNAL OF KOREAN INSTITUTE OF ELECTROMAGNETIC ENGINEERING AND SCIENCE. 2017 Jan.; 28(1), 33 41. http://dx.doi.org/10.5515/kjkiees.2017.28.1.33 ISSN 1226-3133 (Print) ISSN 2288-226X (Online) Hand Gesture Classification Using Multiple Doppler Radar and Machine Learning 백경진 장병준 Kyung-Jin Baik Byung-Jun Jang 요약 SVM (Support Vector Machine).,,.,., NI DAQ USB-6008, MATLAB. Push, Pull, Right Slide Left Slide 4, SVM. Abstract This paper suggests a hand gesture recognition technology to control smart devices using multiple Doppler radars and a support vector machine(svm), which is one of the machine learning algorithms. Whereas single Doppler radar can recognize only simple hand gestures, multiple Doppler radar can recognize various and complex hand gestures by using various Doppler patterns as a function of time and each device. In addition, machine learning technology can enhance recognition accuracy. In order to determine the feasibility of the suggested technology, we implemented a test-bed using two Doppler radars, NI DAQ USB-6008, and MATLAB. Using this test-bed, we can successfully classify four hand gestures, which are Push, Pull, Right Slide, and Left Slide. Applying SVM machine learning algorithm, it was confirmed the high accuracy of the hand gesture recognition. Key words: Doppler Radar, Machine Learning, Hand Gesture, Hand Mouse, SVM. 서론 HCI(Human Computer Interaction)., TV. TV 2016 ( ) (B0717-16-0065, 400 1,600nm / (Si)- (Ge) ). (Department of Electronic Engineering, Kookmin University) Manuscript received October 10, 2016 ; Revised December 12, 2016 ; Accepted December 16, 2016. (ID No. 20161010-010S) Corresponding Author: Byung-Jun Jang (e-mail: bjjang@kookmin.ac.kr) c Copyright The Korean Institute of Electromagnetic Engineering and Science. All Rights Reserved. 33
THE JOURNAL OF KOREAN INSTITUTE OF ELECTROMAGNETIC ENGINEERING AND SCIENCE. vol. 28, no. 1, Jan. 2017.. IPTV(Internet Protocol TV). HCI [1]. HCI HCI. HCI MicroSoft Kinect, [2],[3]. HCI LOS (Line Of Sight). HCI,, [4]. HCI. Google 'Soli', Soli 60 GHz [4]., LOS. 60 GHz ISM(Industrial, Scientific, and Medical), 2.4 GHz 5.8 GHz ISM.. 3., FMCW(Frequency Modulated Continuous Wave),, UWB(Ultra-Wideband) [4],[5]. Google Soli, 60 GHz ISM FMCW DSSS (Direct-Sequency Spread Spectrum), [5] 3, 5.56 GHz 7.25 GHz 1.69 GHz FMCW Wi-Track.,,. 60 GHz 77 GHz., Wi-Fi RFID [6],[7]. [6] RFID RF-IDraw, [7] Wi-Fi WiSee.,,. CW(Continuous Wave) [8] [11]. CW,. 2.4 GHz, 5.8 GHz, 10 GHz,,. [8] 2.4 GHz, [9]. CW, [10]. CW., 34
.,. [8] SVM(Support Vector Machine). SVM.. 2 HW SW, 3. 4.. 다중도플러레이다센서 2-1 도플러레이다센서요구사항 TV HCI 1. TV. 1) / / / 2. 2)., push. 3) push push pull., TV TV / / /, 4. TV 4., 그림 1. TV HCI Fig. 1. TV HCI control scenario by hand using multiple doppler radars.. 2-2 다중도플러레이다센서 HW 2-1 2.45 GHz ISM (monostatic).. 2(a). 2.45 GHz VCO (Voltage Controlled Oscillator) PLL(Phased Locked Loop), PLL Atmega128. 2.45 GHz (CP: Circular Polarization) 90 (RHCP: Right- Handed CP). (LHCP: Left-Handed CP). LHCP CP 90, I/Q. [11], 35
THE JOURNAL OF KOREAN INSTITUTE OF ELECTROMAGNETIC ENGINEERING AND SCIENCE. vol. 28, no. 1, Jan. 2017., 1 MHz CW ( 2,450 MHz 2,451 MHz). CW CW. CW, CW. MATLAB NI DAQ USB-6008. 2(a) (push).. (1). cos (1) (a) (a) Push, Pull hand gesture, [Hz], CW [Hz], [m/s], [m/s],. (Push), '+'. (Pull), ' '. Push Pull. 2(b) Right Slide Left Slide. (1) (2) [10]. max min (2) Right Slide Left Slide,., Right Slide, 2(b). Left Slide 2(b) Right Slide., Right Slide, Left Slide. 2-3 다중도플러레이다센서 SW (b), (b) Right & Left Slide hand gesture 그림 2. Fig. 2. Principles of Doppler radar with respect to various hand gestures(push, Pull, Right Slide, and Left Slide). I/Q I/Q PC. 3.., ADC PC. MA 36
그림 3. Fig. 3. Configuration of suggested system. TLAB STFT(Short Time Fourier Transform),., SVM.,. SVM, SVM, 2. SVM (hyperplane), Margin hyperplane. Hyperplane support vector hyperplane. hyperplane,., support vector. SVM STFT 4...,. 그림 4. Fig. 4. The classification algorithm block-diagram of hand gesture recognition.. 4 2 SVM 4. 5 3 SVM, decision tree. Push/Pull Right Slide/Left Slide SVM1, SVM1 Pull Push SVM2, Right Slide Left Slide SVM3 3 SVM. SVM1, Push/Pull, Right Slide/Left Slide, Push/Pull 2., Right Slide/Left Slide 2. Push/Pull, Right Slide/Left Slide. SVM2, Push Pull 2. SVM3, 2 Slide. 2-2, 2 Slide. MATLAB SVM. SVM.. 37
THE JOURNAL OF KOREAN INSTITUTE OF ELECTROMAGNETIC ENGINEERING AND SCIENCE. vol. 28, no. 1, Jan. 2017. 그림 5. SVM Fig. 5. Suggested decision tree using SVM. 3-1 실험구성. 실험및결과고찰. 6. ISM 2.45 GHz Monostatic Doppler Radar 2, NI DAQ USB- 6008,, MATLAB PC. 3 V. CW,. NI DAQ USB-6008 1024 Sample/sec ADC PC. PC I/Q, STFT. FFT 1,024, window 256. 2-3 SVM 4.,, 7 SVM. 3 SVM 7 SVM1. SVM1 4 2, Push/Pull Right Slide/Left Slide SVM. 4 100, 400. 90 %, 10 %., SVM1. SVM2, SVM3. SVM, 4 그림 6. Fig. 6. Experimental configuration. 그림 7. SVM Fig. 7. The verification block-diagram of SVM. 38
(a) (b) (c) 그림 8. Fig. 8. The extraction process spectrogram of Doppler frequency; (a) raw data of pull hand gesture, (b) removed data from the noise signal of (a), (c) extracted Doppler frequency. (a) SVM. 3-2 실험결과 SVM, SVM1, SVM2, SVM3. STFT. 8 pull. 8(a), 8(b). 8(c). SVM1 SVM2 6.13 6.13 Hz. 7 SVM. 3 SVM 4 50 200 9. 4 50, 5 10. 9 (a) SVM1 100 Push/Pull( ) 100 (b) (c) 그림 9. 4 SVM. (a) SVM1, (b) SVM2, (c) SVM3 Fig. 9. SVM model for hand gesture classification. (a) Classification by SVM1, (b) Classification by SVM2, (c) Classification by SVM3. 39
THE JOURNAL OF KOREAN INSTITUTE OF ELECTROMAGNETIC ENGINEERING AND SCIENCE. vol. 28, no. 1, Jan. 2017. Right Slide/Left Slide( ). 9(b) SVM1 Push/Pull( ) 100 SVM2 50 Push( ) 50 Pull( )., 9(c) SVM1 Right Slide/Left Slide ( ) 100 SVM3 50 Left Slide( ) 50 Right Slide ( ). 10, 50. SVM, 80 % SVM. Pull Push, Left Slide Right Slide [12].,., SVM. 4,. 그림 10. Fig. 10. Confusion matrix for hand gestures.. SVM.. 결론,. 2 SVM 4 (Push, Pull, Left Slide Right Slide) 100 %. 4, HCI.. References [1], " ",, 15(11), pp. 1377-1383, 2012 11. [2] Zhengyou Zhang, "Microsoft kinect sensor and its effect", IEEE Multimedia, vol. 19, no. 2, pp. 4-10. 2012 [3] Zhihan Lv, et al. "Finger in air: touch-less interaction on smartphone", Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia, ACM, 2013. [4] Jaime Lien, et al. "Soli: ubiquitous gesture sensing with millimeter wave radar", ACM Transactions on Graphics (TOG), vol. 35, no. 4, pp. 142, 2016. [5] Fadel Adib et. al., "3D tracking via body radio reflections", 11th USENIX Symposium on Networked Systems Design and Implementation(NSDI 14), 2014. [6] Jue Wang, Deepak Vasisht, and Dina Katabi. "RF- IDraw: virtual touch screen in the air using RF signals", ACM SIGCOMM Computer Communication Review 44.4, pp. 235-246, 2015. [7] Qifan Pu, et al. "Whole-home gesture recognition using 40
wireless signals", Proceedings of the 19th Annual International Conference on Mobile Computing & Networking, ACM, 2013. [8] Youngwook Kim, Hao Ling, "Human activity classification based on micro-doppler signatures using a support vector machine", Geoscience and Remote Sensing, IEEE Transactions on 47.5, pp. 1328-1337, May. 2009. [9] Qian Wan, et al. "Gesture recognition for smart home applications using portable radar sensors", 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6414-6417, 2014. [10] Yiran Li et al. "Non-contact multi-radar smart probing of body orientation based on micro-doppler signatures", 20014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 598-601, 2014. [11], " PLL 2.4 GHz ",, 20(12), pp. 1325-1332, 2009 12. [12], " ", WISET, vol. 1 pp. 412-417, 2015. 2017 2 : ( ) 2017 3 : [ 주관심분야 ],, 1990 2 : ( ) 1992 2 : ( ) 1997 2 : ( ) 1995 3 1999 1 : LG ( ) 1999 1 2003 9 : 2003 10 2005 8 : 2013 9 2015 8 : (CP) 2005 9 : [ 주관심분야 ] RF,, 41