Pattern Recognition

Size: px

Start display at page:

Download "Pattern Recognition"

정희 복
5 years ago
Views:

1 SELF-DRIVING CARS AND DEEP LEARNING 아주대학교구형일

2 Course overview Introduction Self Driving Cars/Machine Learning/Deep Learning Machine Learning Artificial Neural Network (ANN,MLP) Convolution Neural Network (CNN) Recurrent Neural Network (RNN) (Deep) Reinforcement Learning

3 Course overview with applications with (some) codes

4 INTRODUCTION

5 교통사고원인 1. Driver distraction 2. Speeding 3. Drunk driving 4. Reckless driving 5. Rain 6. Running red lights 7. Running stop signs 8. Teenage drivers 9. Night driving 10. Design defects 11. Unsafe lane changes 12. Wrong-way driving 13. Improper turns 14. Tailgating 15. Driving under the influence of drugs 16. Ice 17. Snow 18. Road rage 19. Potholes 20. Drowsy driving 21. Tire blowouts 22. Fog 23. Deadly curves 24. Animal crossings 25. Street racing 26. Others

6 TED: Sebastian Thrun

7 SELF-DRIVING CARS

8 DARPA Grand Challenge II (2006)

9 DARPA Urban Challenge (2007)

10 Autonomous-driving is hard The world is complex The world is unpredictable The world is hazardous

11 Moravec s Paradox The main lesson of 35 years of AI research is that the hard problems are easy and the easy problems are hard. The mental abilities of a four-year-old that we take for granted recognizing a face, lifting a pencil, walking across a room, answering a question in fact solve some of the hardest engineering problems ever conceived... As the new generation of intelligent devices appears, it will be the stock analysts and petrochemical engineers and parole board members who are in danger of being replaced by machines. The gardeners, receptionists, and cooks are secure in their jobs for decades to come. Pinker, Steven (September 4, 2007) [1994], The Language Instinct, Perennial Modern Classics, Harper, ISBN

12 Moravec s Paradox

13 Moravec s Paradox

14 Moravec s Paradox RoboCup 2016: NimbRo vs AUTMan

15 Moravec s Paradox A Compilation of Robots Falling Down at the DARPA Robotics Challenge

16 Why? Encoded in the large, highly evolved sensory and motor portions of the human brain is a billion years of experience about the nature of the world and how to survive in it. We are all prodigious Olympians in perceptual and motor areas, so good that we make the difficult look easy. Abstract thought, though, is a new trick, perhaps less than 100 thousand years old. We have not yet mastered it. It is not all that intrinsically difficult; it just seems so when we do it Moravec, Hans (1988), Mind Children, Harvard University Press

17 How hard is driving?

18 Deep learning to the rescue DNN BIG DATA GPU The GPU is the workhorse of modern A.I. POPULAR SCIENCE

19 SELF DRIVING CARS

20 The basic self-driving loop MAP SENSE CONTROL LOCALIZE PLAN PERCEIVE

Autonomous Driving 자율주행레벨 * Level 0 단순경고 Level 1 단순

차간거리유지 Adaptive Cruise Control 현대제네시스 ( 15.

제한적자율주행 Auto Pilot 완전자율주행 상용화기능 차선이탈경고 Lane

Warning System 차선유지 Lane Keeping Assist 자동긴급제동

/ 후방경고 Blind Spot Warning System 주차보조자동주차원격자동주차

21 Autonomous Driving 자율주행레벨 * Level 0 단순경고 Level 1 단순 ADAS Level 2 퓨전 ADAS Level 3 제한적자율주행 Level 4 완전자율주행 차간거리유지 Adaptive Cruise Control 현대제네시스 ( 15.12) 구글카 ( 개발중 ) 차간거리 / 차선유지 Smart Cruise Control 제한적자율주행 Auto Pilot 완전자율주행 상용화기능 차선이탈경고 Lane Departure Warning System 전방추돌경고 Forward Collision Warning System 차선유지 Lane Keeping Assist 자동긴급제동 Advanced Emergency Braking 차선변경 Lane Change Assist 측 / 후방경고 Blind Spot Warning System 주차보조자동주차원격자동주차 무인발렛주차 법제도 허용개선필요불허 *National Highway Traffic Safety Administration (NHTSA- 미국고속도로교통안전청 ) 기준

23 GOOGLE S SELF DRIVING CAR

24 Google self-driving car GPS LiDAR Camera Radar

25 Visualization of LIDAR data

Disengagements Reports Disengagements: deactivations of the autonomous mode when a failure of the autonomous technology is detected (272 cases) when the safe

27 Disengagements Reports Disengagements: deactivations of the autonomous mode when a failure of the autonomous technology is detected (272 cases) when the safe operation of the vehicle requires that the autonomous vehicle test driver disengage the autonomous mode and take immediate manual control of the vehicle. (13+56 cases)

28 California Autonomous Testing Disengagements (2015)

29 TESLA S AUTOPILOT

31 Tesla vs Google

32 Google 과 Tesla 의 자율주행자동차기술차이 Computer Vision LIDAR 사용 ( 높은위치인식능력 ) Camera 사용테슬라의 CEO 앨런머스크는구글의 LIDAR 센서에관해 그렇게비싼센서를사용한자율주행자동차를개발하는것은과하다 (overkill) 라고비판 Car Control 완전자율주행기술목표 2013 년구글은일부직원의출퇴근에자율주행차를타도록했는데차안비디오카메라를모니터링한결과운전자가잠이드는등운전에집중하지않음. 이후완전자율주행기술을목표로개발 Autopilot 기능제공자동차의비행기화 비행기에서돌발상황에만파일럿이개입하듯돌발상황에서운전자의조작이필요

34 How the Accident happened

35 Traffic Fatalities Total miles driven in U.S. in 2014: 3,000,000,000,000 (3 million million) Fatalities: 32,675 (1 in 90 million) Tesla Autopilot mile driven since October 2015: 300,000,000 (300 million) Fatalities: 1

36 UBER

37 Pittsburgh, your self-driving Uber is arriving now

38 UBER/nuTonomy

39 NVIDIA S DRIVERWORKS

40 NVIDIA

41 Perception

42 Visualization

43 Planning

44 COURSE OVERVIEW

45 Methods Traditional Deep-Learning based Non-machine Learning Machine-Learning based method Supervised SVM MLP CNN RNN (LSTM) Localizati on GPS, SLAM Self Driving Perception Pedestrian detection (HOG+SVM) Detection/ Segmentat ion/classif ication Dry/wet road classificati on ADAS Planning/ Control Optimal control End-toend Learning End-toend Learning Tasks Driver state Behavior Prediction/ Driver identificati on Vehicle Diagnosis Smart factory DNN * * Reinforcement * Unsupervised * *

46 ARTIFICIAL NEURON

47 ADAS Tasks Self Driving Localizati on Perception Planning/ Control Driver state Vehicle Diagnosis Smart factory Methods Traditional Deep-Learning based Non-machine Learning Machine-Learning based method Supervised SVM MLP CNN RNN (LSTM) DNN Reinforcement GPS, SLAM Pedestrian detection (HOG+SVM) Detection/ Segmentat ion/classif ication Dry/wet road classificati on Optimal control End-toend Learning End-toend Learning Unsupervised

48 뉴런 : 신경망의기본단위

49 인공뉴런 (Artificial Neuron) 실제뉴런 뉴런의수학적모델

50 예시 : 연어와농어의구별 폭 (w) 7.3l + 3.4w = 100 밝기 (l) l w 농어 연어 7.3l + 3.4w l + 3.4w < 100

51 예시 : 연어와농어의구별 l 7.3 l w 3.4 w Σ 연어 / 농어 100

52 Artificial Neuron w T x g(w T x) Activation function (non-linear)

54 Multi-layer Perceptron

55 TYPES OF MACHINE LEARNING Neural Network 기반방법을중심으로

56 Methods Traditional Deep-Learning based Non-machine Learning Machine-Learning based method Supervised SVM MLP CNN RNN (LSTM) Localizati on GPS, SLAM Self Driving Perception Pedestrian detection (HOG+SVM) Detection/ Segmentat ion/classif ication Dry/wet road classificati on ADAS Planning/ Control Optimal control End-toend Learning End-toend Learning Tasks Driver state Behavior Prediction/ Driver identificati on Vehicle Diagnosis Smart factory DNN * * Reinforcement * Unsupervised * *

57 Why neural networks? Universal function approximator

58 Why neural networks? It can learn

59 Why neural networks? There can be lots of variations (layouts)

60 Types of Machine Learning Supervised Learning Classification/Regression Semi-supervised Learning/Weakly supervised Learning/ Unsupervised Learning Clustering Feature Learning Generative Model Learning Reinforcement Learning Deep Q-Learning Policy Gradient Learning

61 Supervised learning workflow

62 Supervised vs unsupervised Supervised Learning Data: (x, y) x is data, y is label Goal: Learn a function to map x -> y Examples: Classification, regression, object detection, semantic segmentation, image captioning, etc Data: x Unsupervised Learning Just data, no labels! Goal: Learn some structure of the data Examples: Clustering, dimensionality reduction, feature learning, generative models, etc

63 Unsupervised Learning Generative Model (Generative Adversarial Network)

64 Unsupervised Learning Generative Model (Generative Adversarial Network)

65 Unsupervised Learning Dimension Reduction/Feature Learning (Auto-Encoder)

66 Supervised vs Reinforcement Supervised Learning Data: (x, y) x is data, y is label Goal: Learn a function to map x -> y Examples: Reinforcement Learning reward x i, y i = (, ) Game state Joystick control

67 Reinforcement Learning Reinforcement learning is an area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward.

68 Reinforcement Learning

69 DEEP LEARNING

70 Methods Traditional Deep-Learning based Non-machine Learning Machine-Learning based method Supervised SVM MLP CNN RNN (LSTM) Localizati on GPS, SLAM Self Driving Perception Pedestrian detection (HOG+SVM) Detection/ Segmentat ion/classif ication Dry/wet road classificati on ADAS Planning/ Control Optimal control End-toend Learning End-toend Learning Tasks Driver state Behavior Prediction/ Driver identificati on Vehicle Diagnosis Smart factory DNN * * Reinforcement * Unsupervised * *

71 구글트렌드 : 딥러닝

72 Neural network Back propagation, Nature 1986 장점 일반적인문제에적용할수있는학습법 Biological 시스템과관련이깊음 문제점 Training 이쉽지않음 현실적인문제에잘동작하지않음

Neural network Back propagation, Nature Non-linear SVM 1986 1992 다양한시도들 Flat

74 Neural network Back propagation, Nature Non-linear SVM 다양한시도들 Flat structure SVM, Boosting, Biological 시스템과거리가생김 특정한문제를해결하는특정한방법 (SIFT, LBP, HOG, GMM-HMM)

Neural network Back propagation, Nature Deep belief network, Science 1986 1992 2006 비지도학습을이용한

75 Neural network Back propagation, Nature Deep belief network, Science 비지도학습을이용한 pre-training Training 방법향샹 Dropout, RectLinear, Normalization, 컴퓨터구조의발달 GPU Multi-core computer 시스템 빅데이터 +

76 Neural network Back propagation, Nature Deep belief network, Science Speech

77 음성인식성능

79 Neural network Back propagation, Nature Deep belief network, Science Speech Object recognition Submission Method Error rate Supervision Deep CNN ISI XRCE/INRIA OXFORD_VGG FV: SIFT, LBP, GIST, CSIFT FV: SIFT and color 1M-dim features FV: SIFT and color 270K-dim features

Drumstick Mud turtle Output: Scale T-shirt Giant panda

80 ImageNet Large Scale Visual Recognition Competition (ILSVRC) Steel drum Output: Scale T-shirt Steel drum Drumstick Mud turtle Output: Scale T-shirt Giant panda Drumstick Mud turtle

81 Neural network Back propagation, Nature Deep belief network, Science Speech Object recognition IMAGENET 2013: 영상인식 RANK Name Error rate Description 1 NYU Deep Learning 2 NUS Deep Learning 3 OXFORD Deep Learning

82 Neural network Back propagation, Nature Deep belief network, Science Speech Object recognition IMAGENET 2013: 영상인식 RANK Name Error rate Description 1 Google Deep Learning 2 Oxford Deep Learning 3 MSRA Deep Learning

83 Neural network Back propagation, Nature Deep belief network, Science Speech Object recognition The game of GO

84 The AI race is on IBM Watson Achieves Breakthrough In Natural Language Processing Facebook Launches Big Sur Baidu Deep Speech 2 Beats Humans Google Launches TensorFlow Toyota Invests $1B In AI Labs Microsoft & U.Science & Tech, China Beat Humans on IQ

85 Deep Learning: Representation Learning

86 The Mammalian Visual Cortex is Hierarchical

87 Deep Learning: Scalable Machine Learning

88 딥러닝모델의특징 다층구조 (multi layer) 신경망의구조모사 상위층으로갈수록추상화된정보가학습과정에서자동으로생성 문제해결과정자동화 End-to-end learning 사람의개입을배제하고오직 raw input 과 output 사이에모든과정을데이터에서학습하는방향추구 분산표현 Distributed representation 여러뉴런이협력하여정보저장 / 처리

89 DRAWBACKS

90 Current Drawbacks Big data: inefficient at learning from data Supervised data: costly to annotate real-world data Need to manually select network structure Need to hyper-parameter tuning Learning rate Loss function Mini-batch size Number of training iterations Momentum Optimizer selection Defining a good reward function is difficult

91 Faulty Reward Functions in the Wild

94 BACKUPS Clustering

95 AI 역사

96 AI 역사

98 I m sorry Dave.

99 AI 역사

100 딥블루 딥블루 vs 게리카스파로프, 1997 Deep Blue vs Kasparov vs Brute-force search power 6~8 수를내다봄

101 AI 역사

102 IBM Watson 슈퍼컴퓨터 질문예시 ) Kathleen kenyon s excavation of this city mentioned in joshua showed the wall had been repaired 17 times WHAT is Jericho This child star got his first onscreen kiss in MY GIRL WHO is Macaulay Culkin

103 AI 역사 2016

104 CLUSTERING

105 Unsupervised Learning Clustering To group the object having same feature Female group male group

106 Unsupervised Learning Clustering

107 Clustering Example

Pattern Recognition

Pattern Recognition SELF-DRIVING CARS AND DEEP LEARNING 아주대학교구형일 Course overview (keywords) Introduction Self Driving Cars/Machine Learning/Deep Learning Machine Learning Artificial Neural Network (ANN,MLP) Convolution Neural