ADAS & AD October 31 th, 2017 ADAS : Advanced Driver Assistance System AD : Autonomous Driving
ProSense : www.prosensetek.com
개요 알고리즘 : ADAS & AD 비즈니스모델
그래픽기반자율주행시뮬레이터 자율주행차개발의全영역 ( 데이터획득, 데이터처리, 응용개발, 검증 ) 에시뮬레이션방법론을적용함. 개발 Cycle 실차량 센서데이터획득 센서융합 데이터저장 Dataset 마련 딥러닝학습 지도제작 AD SDK CUDA 알고리즘 H/W : Drive PX2 S/W : DriveWorks TensorRT, cudnn 시뮬레 이션 가상센서데이터 가상센서융합 가상 Dataset 자동생성 딥러닝학습 가상지도자동생성 AD SDK CUDA 알고리즘 H/W : Drive PX2 S/W : DriveWorks TensorRT, cudnn
Test Platform : Real-time Perception, Self-driving An SUV for real-time perception and a self-driving car including control algorithms Multi-Camera Multi-Radar Drive PX2
ADAS Directions : Algorithms Licensing 3D Lane & Road, Radar Fusion High Accuracy AEB : Depth/VD/PD Vision Cruise : High Curvature Robustness
AD (Autonomous Driving) Directions Perception : Real Time High Accuracy Better Control by better perception Service : Last Mile Mobility Simulation based AD Development
Algorithms : VD, PD, LD, 6D, Tracking, CNN-OD, Stixel, SVM
Stereo Matching & Stixel Real-time depth map estimation 20ms (avg.) with GTX 1080 200~300ms with Jetson TX1 STIXEL (Stick + Cell) Used to detect and to represent obstacles INPUT Depth STIXEL 9
Robust Stereo Matching Matching costs Matching costs Challenging outdoor environment 2 1 Random forest d C(, k) Cˆ( p, d ) Q ( p )C( p,d) (1 Q( p )) p D k D Confidence-based cost modulation Overlaid d Disp. map 3 Learning-based confidence measure selection and training 4 Semi-global matching Proposed method Confidence prediction Min-Gyu Park and Kuk-Jin Yoon, Leveraging Stereo Matching with Learning-based Confidence Measures," IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2015 10
Lane Detection Classical lane detection approach (300fps with 3.00GHz CPU) Input image IPM image Filter & Threshold Detect lanes Result Deep learning-based lane detection Dataset : 1,500 images Train dataset : 60,000 images (augmentation) Fully Convolutional Network 10ms (100fps) with Jetson TX1 11
12 Vehicle Detection Real-time vehicle detection Under development HoG + Kernel SVM ACF + Boosting (aggregated channel features) 15ms with Jetson TX1 Feature extraction VGA ROI definition QVGA Expandable to PD (pedestrian detection)
ADAS current project : Vision based Cruise Control When the blue vehicle changes lane, the red vehicle should drive steadily without abrupt acceleration Current target vehicle speed : 60km/h Target vehicle changes lane after sometime. Current subject vehicle speed : 60km/h After target vehicle changes the lane, subject vehicle speed up to 100km/h (Cruise speed : 100km/h) Radar Radius : 150m Camera
Adaptive Cruise Control Implementation details Fusion: Radar + Camera Lane monitoring by Camera Ego-lane detection Medium curvature Proprietary Radar algorithm Calibration Radar + Camera
ADTF Architecture for Surround View Monitoring System
Key competences Photogrammetry algorithms 3D Reconstruction algorithms ADAS algorithms System engineering Mature development processes (Scrum, Kanban, Waterfall) CEVA MM-3101 & XM4 optimization CUDA & OpenCL intensive computation FPGA programming
Stereo vision algorithm High performance 20 fps (Zynq 7020, HD resolution) Low latency 50 ms (Zynq 7020, HD resolution) Sparse disparity map 60000-110000 features (HD resolution)
Forward Collision Avoidance Functions Stereo SGM, AD-Census Stixels & CNN-based obstacle detection Collision estimation (tracking & trajectory & time to collision) Free space to drive
Forward Collision Avoidance CNN Object detection Classes car, pedestrian, cyclist CNN YOLOv2 Training set 90110 images map 0.7-0.8 Performance Platform nvidia Drive PX2 20 fps Resolution 512x288 px
Traffic sign detection Implementation details SSD-M with MobileNet features High performance 20 fps on Jetson TX2 High accuracy 0.74 map European, Russian, South Korean traffic signs ~30000 images Classifying speed limits ~17000 images Low model size 22 Mb Detection performance TX2: 35 ms, PX2: 24 ms Classification performance TX2: 6 ms, PX2: 4 ms Accuracy F1-score 0.89
그래픽기반자율주행시뮬레이터 (1/3) 자율주행을위한그래픽기반가상 Dataset 의효용성이검증됨. 또한실제로구현이어려운상황의경우그래픽기반가상 Dataset 의효용성은더욱증대됨.
그래픽기반자율주행시뮬레이터 (2/3) 그래픽기반가상 Dataset 을통한학습과실영상기반학습을병행한경우더욱우수한인식성능 < 그래픽객체예시및가상카메라배치 > < 학습효과의성능비교 >
AD current project : Last Mile Mobility We are now collaborating with Korean mini-bus makers and Korean local governments.
전기차기반자율주행버스아르메 (ARMA) Lidar 센서 : 환경을매핑하여위치지정, 3D 인식 GPS RTK : GPS 센서와기지국간통신 ( 차량의정확한위치결정 ) Odometer : 변위및바퀴속도측정 차량속도추정, 위치확인 카메라스테레오비전 : 도로환경분석및정보추출 Arma Device Route http://navya.tech/
자율주행버스 EZ10 다중센서 localization 기술, 장애물감지및회피, 탐색, 경로계획및제어, 연결성 (V2V, V2I), 차량관리, 안전및사이버보안 3가지작동모드 (Metro 모드 : 모든역에서정차, Buses 모드 : 요청시에정차, On-demand 모드 : 호출가능 ) 3 modes Hybrid localization http://easymile.com/
Bus Rapid Transit(BRT) : 전용차선에서운영. 속도, 규칙성, 빈도, 다수의수송인원 Buses : 경로명확. 상업적, 경제적, 생태적효율성 Tram-train : 도시지역의경전철트랙과교외지역의철도트랙에서작동 Metro : Roissy-Charles de Gaulle의공항터미널, 기차역및주차장을연결하는전기메트로 CDGVal 운영 BRT Bus Tram-train Metro https://www.transdev.com/en/
전용가이드웨이에서운행하는자율주행셔틀 ( 안전성, 유연성보장 ) 4~6 人승객 PRT 차량 ( 택시 ), 16~24 人승객 GRT 차량 ( 버스 ) 대중교통운영모니터링및감독 (TOMS), 차량제어시스템 (VCS), 안내제어시스템 (GCS) TOMS GRT https://www.2getthere.eu/technology/vehicle-types/
다수의마이크로공장을이용한오픈소스차량디자인의소량생산 자율주행버스 Olli 개발 (IBM Watson의고급인지능력통합 ) 360도센서 원격차량모니터링, 관리및경고시스템 Autonomous vehicle Inside the vehicle https://localmotors.com/
자율주행버스개발 ( 대학캠퍼스, 비즈니스파크및주거지역내여행용 - 2 인승, 6 인승, 12 인승셔틀버스 ) 기후제어, Lidar 및 Camera 로 360 도감지, 장애물탐색및회피등 Autonomous vehicle Running the car https://auro.ai/
물류서비스 모바일어플리케이션을통해승용차와운전자를연결 원하는차량서비스선택가능 ( 이코노미, 프리미엄, 카풀등 ) Utilization the Uber taxi Apps for driver partners https://www.uber.com/ko-kr/
자율주행을위한확장가능 full-stack 소프트웨어솔루션 대규모자치차량을통해 point-to-point 이동성을제공하는솔루션개발中. ( 도시에서의자율주행, 스마트폰기반승차, 차량경로및관리, 원격조작을통해차량을제어하는소프트웨어 ) 미국, 싱가포르및유럽에서테스트完. Autonomous vehicle NuCore software http://www.nutonomy.com/
Thank you very much! Paul Kang paulkang@prosensetek.com