Pattern Recognition

Size: px

Start display at page:

Download "Pattern Recognition"

민중 진
6 years ago
Views:

1 SELF-DRIVING CARS AND DEEP LEARNING 아주대학교구형일

2 Course overview (keywords) Introduction Self Driving Cars/Machine Learning/Deep Learning Machine Learning Artificial Neural Network (ANN,MLP) Convolution Neural Network (CNN) Recurrent Neural Network (RNN) (Deep) Reinforcement Learning

3 Course overview with applications with (some) codes

4 INTRODUCTION

5 교통사고원인 1. Driver distraction 2. Speeding 3. Drunk driving 4. Reckless driving 5. Rain 6. Running red lights 7. Running stop signs 8. Teenage drivers 9. Night driving 10. Design defects 11. Unsafe lane changes 12. Wrong-way driving 13. Improper turns 14. Tailgating 15. Driving under the influence of drugs 16. Ice 17. Snow 18. Road rage 19. Potholes 20. Drowsy driving 21. Tire blowouts 22. Fog 23. Deadly curves 24. Animal crossings 25. Street racing 26. Others

6 TED: Sebastian Thrun

7 Traditional resource waste problem Car utilization rate Energy utilization rate Driving (2.6%) Looking for parking (0.8%) Sitting in congestion (0.5%) Parked (96%) 86% of fuel never reaches the wheels Engine losses Move a person/thing (4~5%) Inertia, Aerodynamics Rolling resistance, Aux. power, Transmission losses Idling Road utilization rate Idle state (90%) Occupied by car (10%)

8 SELF-DRIVING CARS

9 DARPA Grand Challenge II (2006)

10 DARPA Urban Challenge (2007)

11 Autonomous-driving is hard The world is complex The world is unpredictable The world is hazardous

12 Moravec s Paradox The main lesson of 35 years of AI research is that the hard problems are easy and the easy problems are hard. The mental abilities of a four-year-old that we take for granted recognizing a face, lifting a pencil, walking across a room, answering a question in fact solve some of the hardest engineering problems ever conceived... As the new generation of intelligent devices appears, it will be the stock analysts and petrochemical engineers and parole board members who are in danger of being replaced by machines. The gardeners, receptionists, and cooks are secure in their jobs for decades to come. Pinker, Steven (September 4, 2007) [1994], The Language Instinct, Perennial Modern Classics, Harper, ISBN

13 Moravec s Paradox

14 Moravec s Paradox

15 Moravec s Paradox RoboCup 2016: NimbRo vs AUTMan

16 Moravec s Paradox A Compilation of Robots Falling Down at the DARPA Robotics Challenge

17 Why? Encoded in the large, highly evolved sensory and motor portions of the human brain is a billion years of experience about the nature of the world and how to survive in it. We are all prodigious Olympians in perceptual and motor areas, so good that we make the difficult look easy. Abstract thought, though, is a new trick, perhaps less than 100 thousand years old. We have not yet mastered it. It is not all that intrinsically difficult; it just seems so when we do it Moravec, Hans (1988), Mind Children, Harvard University Press

18 How hard is driving?

19 Deep learning to the rescue DNN BIG DATA GPU The GPU is the workhorse of modern A.I. POPULAR SCIENCE

20 Hard problems for programmers Talk, Read, Walk, Drive, Atari, Easy problems for humans Hard problems for humans Easy problems for programmers Modelbased things (e.g., Physics simulation)

21 Hard problems for programmers Talk, Read, Walk, Drive, Atari, Deep Learning to the Rescue Easy problems for humans Hard problems for humans Easy problems for programmers Modelbased things (e.g., Physics simulation)

22 자율주행소개

23 The basic self-driving loop MAP SENSE CONTROL LOCALIZE PLAN PERCEIVE

Autonomous Driving 자율주행레벨 * Level 0 단순경고 Level 1 단순 ADAS Level 2

Pilot 완전자율주행 상용화기능 차선이탈경고 Lane Departure Warning System 전방추돌경고

Advanced Emergency Braking 차선변경 Lane Change Assist 측 / 후방경고 Blind

24 Autonomous Driving 자율주행레벨 * Level 0 단순경고 Level 1 단순 ADAS Level 2 퓨전 ADAS Level 3 제한적자율주행 Level 4 완전자율주행 차간거리유지 Adaptive Cruise Control 현대제네시스 ( 15.12) 구글카 ( 개발중 ) 차간거리 / 차선유지 Smart Cruise Control 제한적자율주행 Auto Pilot 완전자율주행 상용화기능 차선이탈경고 Lane Departure Warning System 전방추돌경고 Forward Collision Warning System 차선유지 Lane Keeping Assist 자동긴급제동 Advanced Emergency Braking 차선변경 Lane Change Assist 측 / 후방경고 Blind Spot Warning System 주차보조자동주차원격자동주차 무인발렛주차 법제도 허용개선필요불허 *National Highway Traffic Safety Administration (NHTSA- 미국고속도로교통안전청 ) 기준

26 GOOGLE S SELF DRIVING CAR

27 Google self-driving car GPS LiDAR Camera Radar

28 Visualization of LIDAR data

Disengagements Reports Disengagements: deactivations of the autonomous mode when a failure of the autonomous technology is detected (272 cases) when the safe

30 Disengagements Reports Disengagements: deactivations of the autonomous mode when a failure of the autonomous technology is detected (272 cases) when the safe operation of the vehicle requires that the autonomous vehicle test driver disengage the autonomous mode and take immediate manual control of the vehicle. (13+56 cases)

31 California Autonomous Testing Disengagements (2015)

32 TESLA S AUTOPILOT

34 Tesla vs Google

35 Google 과 Tesla 의 자율주행자동차기술차이 Computer Vision LIDAR 사용 ( 높은위치인식능력 ) Camera 사용테슬라의 CEO 앨런머스크는구글의 LIDAR 센서에관해 그렇게비싼센서를사용한자율주행자동차를개발하는것은과하다 (overkill) 라고비판 Car Control 완전자율주행기술목표 2013 년구글은일부직원의출퇴근에자율주행차를타도록했는데차안비디오카메라를모니터링한결과운전자가잠이드는등운전에집중하지않음. 이후완전자율주행기술을목표로개발 Autopilot 기능제공자동차의비행기화 비행기에서돌발상황에만파일럿이개입하듯돌발상황에서운전자의조작이필요

37 How the Accident happened

38 Traffic Fatalities Total miles driven in U.S. in 2014: 3,000,000,000,000 (3 million million) Fatalities: 32,675 (1 in 90 million) Tesla Autopilot mile driven since October 2015: 300,000,000 (300 million) Fatalities: 1

39 UBER

40 Pittsburgh, your self-driving Uber is arriving now

41 UBER/nuTonomy

42 NVIDIA S DRIVERWORKS

43 NVIDIA

44 Perception

45 Visualization

46 Planning

47 COURSE OVERVIEW

48 Methods Traditional Deep-Learning based Non-machine Learning Machine-Learning based method Supervised SVM MLP CNN RNN (LSTM) Localizati on GPS, SLAM Self Driving Perception Pedestrian detection (HOG+SVM) Detection/ Segmentat ion/classif ication Dry/wet road classificati on ADAS Planning/ Control Optimal control End-toend Learning End-toend Learning Tasks Driver state Behavior Prediction/ Driver identificati on Vehicle Diagnosis Smart factory DNN * * Reinforcement * Unsupervised * *

49 ARTIFICIAL NEURON

50 ADAS Tasks Self Driving Localizati on Perception Planning/ Control Driver state Vehicle Diagnosis Smart factory Methods Traditional Deep-Learning based Non-machine Learning Machine-Learning based method Supervised SVM MLP CNN RNN (LSTM) DNN Reinforcement GPS, SLAM Pedestrian detection (HOG+SVM) Detection/ Segmentat ion/classif ication Dry/wet road classificati on Optimal control End-toend Learning End-toend Learning Unsupervised

51 뉴런 : 신경망의기본단위

52 인공뉴런 (Artificial Neuron) 실제뉴런 뉴런의수학적모델

53 예시 : 연어와농어의구별 폭 (w) 7.3l + 3.4w = 100 밝기 (l) l w 농어 연어 7.3l + 3.4w l + 3.4w < 100

54 예시 : 연어와농어의구별 l 7.3 l w 3.4 w Σ 연어 / 농어 100

55 Artificial Neuron w T x g(w T x) Activation function (non-linear)

57 Multi-layer Perceptron

58 TYPES OF MACHINE LEARNING Neural Network 기반방법을중심으로

59 Methods Traditional Deep-Learning based Non-machine Learning Machine-Learning based method Supervised SVM MLP CNN RNN (LSTM) Localizati on GPS, SLAM Self Driving Perception Pedestrian detection (HOG+SVM) Detection/ Segmentat ion/classif ication Dry/wet road classificati on ADAS Planning/ Control Optimal control End-toend Learning End-toend Learning Tasks Driver state Behavior Prediction/ Driver identificati on Vehicle Diagnosis Smart factory DNN * * Reinforcement * Unsupervised * *

60 Why neural networks? Universal function approximator

61 Why neural networks? It can learn from data.

62 Why neural networks? There can be lots of variations (layouts)

63 Types of Machine Learning Supervised Learning Classification/Regression Semi-supervised Learning/Weakly supervised Learning/ Unsupervised Learning Clustering Feature Learning Generative Model Learning Reinforcement Learning Q-Learning Policy Gradient Learning

64 Supervised learning workflow

65 Supervised vs unsupervised Supervised Learning Data: (x, y) x is data, y is label Goal: Learn a function to map x -> y Examples: Classification, regression, object detection, semantic segmentation, image captioning, etc Data: x Unsupervised Learning Just data, no labels! Goal: Learn some structure of the data Examples: Clustering, dimensionality reduction, feature learning, generative models, etc

66 Unsupervised Learning Generative Model (Generative Adversarial Network)

67 Unsupervised Learning Generative Model (Generative Adversarial Network)

68 Unsupervised Learning Dimension Reduction/Feature Learning (Auto-Encoder)

69 Supervised vs Reinforcement Supervised Learning Data: (x, y) x is data, y is label Goal: Learn a function to map x -> y Examples: Reinforcement Learning reward x i, y i = (, ) Game state Joystick control

70 Reinforcement Learning Reinforcement learning is an area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward.

71 Reinforcement Learning

72 DEEP LEARNING

73 Methods Traditional Deep-Learning based Non-machine Learning Machine-Learning based method Supervised SVM MLP CNN RNN (LSTM) Localizati on GPS, SLAM Self Driving Perception Pedestrian detection (HOG+SVM) Detection/ Segmentat ion/classif ication Dry/wet road classificati on ADAS Planning/ Control Optimal control End-toend Learning End-toend Learning Tasks Driver state Behavior Prediction/ Driver identificati on Vehicle Diagnosis Smart factory DNN * * Reinforcement * Unsupervised * *

74 구글트렌드 : 딥러닝

75 Neural network Back propagation, Nature 1986 장점 일반적인문제에적용할수있는학습법 Biological 시스템과관련이깊음 문제점 Training 이쉽지않음 현실적인문제에잘동작하지않음

Neural network Back propagation, Nature Non-linear SVM 1986 1992 다양한시도들 Flat

77 Neural network Back propagation, Nature Non-linear SVM 다양한시도들 Flat structure SVM, Boosting, Biological 시스템과거리가생김 특정한문제를해결하는특정한방법 (SIFT, LBP, HOG, GMM-HMM)

Neural network Back propagation, Nature Deep belief network, Science 1986 1992 2006 비지도학습을이용한

78 Neural network Back propagation, Nature Deep belief network, Science 비지도학습을이용한 pre-training Training 방법향샹 Dropout, RectLinear, Normalization, 컴퓨터구조의발달 GPU Multi-core computer 시스템 빅데이터 +

79 Neural network Back propagation, Nature Deep belief network, Science Speech

80 음성인식성능

82 Neural network Back propagation, Nature Deep belief network, Science Speech Object recognition Submission Method Error rate Supervision Deep CNN ISI XRCE/INRIA OXFORD_VGG FV: SIFT, LBP, GIST, CSIFT FV: SIFT and color 1M-dim features FV: SIFT and color 270K-dim features

Drumstick Mud turtle Output: Scale T-shirt Giant panda

83 ImageNet Large Scale Visual Recognition Competition (ILSVRC) Steel drum Output: Scale T-shirt Steel drum Drumstick Mud turtle Output: Scale T-shirt Giant panda Drumstick Mud turtle

84 Neural network Back propagation, Nature Deep belief network, Science Speech Object recognition IMAGENET 2013: 영상인식 RANK Name Error rate Description 1 NYU Deep Learning 2 NUS Deep Learning 3 OXFORD Deep Learning

85 Neural network Back propagation, Nature Deep belief network, Science Speech Object recognition IMAGENET 2013: 영상인식 RANK Name Error rate Description 1 Google Deep Learning 2 Oxford Deep Learning 3 MSRA Deep Learning

86 Neural network Back propagation, Nature Deep belief network, Science Speech Object recognition The game of GO

87 The AI race is on IBM Watson Achieves Breakthrough In Natural Language Processing Facebook Launches Big Sur Baidu Deep Speech 2 Beats Humans Google Launches TensorFlow Toyota Invests $1B In AI Labs Microsoft & U.Science & Tech, China Beat Humans on IQ

88 Deep Learning: Representation Learning

89 The Mammalian Visual Cortex is Hierarchical

90 Deep Learning: Scalable Machine Learning

91 딥러닝모델의특징 다층구조 (multi layer) 신경망의구조모사 상위층으로갈수록추상화된정보가학습과정에서자동으로생성 문제해결과정자동화 End-to-end learning 사람의개입을배제하고오직 raw input 과 output 사이에모든과정을데이터에서학습하는방향추구 분산표현 Distributed representation 여러뉴런이협력하여정보저장 / 처리

92 DRAWBACKS

93 Current Drawbacks Big data: inefficient at learning from data Supervised data: costly to annotate real-world data Need to manually select network structure Need to hyper-parameter tuning Learning rate Loss function Mini-batch size Number of training iterations Momentum Optimizer selection Defining a good reward function is difficult

94 Faulty Reward Functions in the Wild

Pattern Recognition

Pattern Recognition SELF-DRIVING CARS AND DEEP LEARNING 아주대학교구형일 Course overview Introduction Self Driving Cars/Machine Learning/Deep Learning Machine Learning Artificial Neural Network (ANN,MLP) Convolution Neural Network