Parking Assist System
주차보조시스템의 다양한 형태 후방 장애물 거리 경보 - 차량 후미에 장착된 초음파 센서를 이용하여 장애물 감지 - Visual, audio warning 제공 Parkmatic Parking Sensor Parking Dynamics PD1 http://www.youtube.com/watch?v=xla9hmkrlsa http://www.youtube.com/watch?v=eqaxuks1ioo
주차보조시스템의 다양한 형태 주변 장애물 거리/방향 표시 - 차량 둘레에 장착된 초음파 센서를 이용하여 장애물 감지 - 화면에 장애물 방향과 거리 표시, 음향 경보 병행 Audi Q7 Parking System BMW PDC (Parking Distance Control) http://www.youtube.com/watch?v=1_y3gycvdou http://www.youtube.com/watch?v=lgjb17udp64
주차보조시스템의 다양한 형태 Rear View Camera (단순 후방 카메라) 후진 기어를 넣으면, 후방 카메라 영상을 HMI로 보여줌. Rear View Camera feature of the 2010 Mercedes-Benz E-Class Sedan and Coupe. http://www.youtube.com/watch?v=ech6dwxl81m
주차보조시스템의 다양한 형태 후방 카메라 + 예상 경로 표시 - 후방 카메라로 주차 상황 제공 - 조향각에 따른 예상 경로 표시 Audi Q7 Advanced Parking System http://www.youtube.com/watch?v=otml8h_zrxu
주차보조시스템의 다양한 형태 Around View Monitor - 전후좌우에 장착된 4개의 어안렌즈 카메라 사용 - 어안렌즈 왜곡 보상 후 모자이크 조감도 제공 2008 Infiniti EX35 http://www.youtube.com/watch?v=h-tdwyk57ci
주차보조시스템의 다양한 형태 Around View Monitor의 주행 중 활용 - 골목길에서 나올 때, 좌우 교통상황 파악 - 좁은 골목길 통과 시, 좌우 측 관찰 Honda Multi-View Camera System http://www.youtube.com/watch?v=dvzftc5drqg
주차보조시스템의 다양한 형태 Self Parking (후진만) - 초음파 기반 측면 빈 주차공간 인식 - GUI 기반 주차위치 설정 - 조향 조작 자동화, 브레이크 페달로 조절 2010 Toyota Prius: Intelligent Parking Assist http://www.youtube.com/watch?v=2eng94_fljw
주차보조시스템의 다양한 형태 Self Parking (후진만) - 초음파 기반 측면 빈 주차공간 인식 - 장애물까지 거리 가시화, 차량 주변 조감도 제공 -조향조작자동화 BMW 5 Series (F10) - Park Assist http://www.youtube.com/watch?v=_a1hnpsffvu http://www.youtube.com/watch?v=9q8zlofjnf0
주차보조시스템의 다양한 형태 Self Parking (전진+후진) - 초기위치에서 후진에 최적인 위치로 전진 후 후진 -조향조작자동화 Honda Park Assist http://www.youtube.com/watch?v=ai7rc9rbqiq http://www.youtube.com/watch?v=zlrxmmojfoy
주차보조시스템의 다양한 형태 Remote Parking Assist - 개인 차고에 주차하는 상황 - 내려서 리모콘으로 진행 조정 - 정면 벽면에 특수 반사판 설치하여 거리/각도 측정 BMW Technology of the Remote Park Assist http://www.youtube.com/watch?v=hil_pl7jxgq
주차보조시스템의 다양한 형태 Remote Parking Assist - 측후방 초음파 센서로 빈 공간 및 후방 장애물 인식 VW Passat CC Park Assist Vision http://www.youtube.com/watch?v=6qrpbzk9sf0
주차보조시스템의 다양한 형태 접근 차량 경보 (주차위치에서 출발할 때) - BSD용 radar 센서 활용 2010 Ford Taurus Cross Traffic Alert http://www.youtube.com/watch?v=qacovrcgc_0
목표위치 설정방법 [1] GUI-Based Markings-Based Free Space-Based Infra-structure-Based
목표위치 설정방법 : 빈 공간 인식 기반 [1] Free Space-Based Ultrasonic sensor-based Short range radar (SRR)-based Single image understanding-based Motion stereo-based Binocular stereo-based Light stripe projection-based Scanning laser radar-based
세가지 좌표계,, Homography One-to-one correspondence between three coordinate systems
GUI 기반: : Drag and Drop Interface Moving by dragging the inside of rectangle Performance comparison with arrow button-type (8 situations, 50 persons) Rotating by dragging the outside of rectangle
GUI 기반: : Drag and Drop Interface Jung, Ho Gi, Kim, Dong Suk*, Yoon, Pal Joo, Kim, Jaihie, NOVEL USER INTERFACE FOR SEMI-AUTOMATIC PARKING ASSISTANCE SYSTEM, 31st FISITA World Automotive Congress, Yokohama, Japan, 22-27 October 2006.
주차구획인식 기반: 전자동 방식 Ho Gi Jung*, Dong Suk Kim, Pal Joo Yoon and Jaihie Kim, Parking Slot Markings Recognition for Automatic Parking Assist System, IEEE Intelligent Vehicles Symposium 2006, Tokyo, Japan, June 13-15, 2006, pages: 106-113.
주차구획인식 기반: 전자동 방식
주차구획인식 기반: 반자동 방식 (one touch) Clicking inside of target parking slot Marking detection Guideline detection Separating line detection Ho Gi Jung*, Dong Suk Kim, Pal Joo Yoon, Jaihie Kim, Structure Analysis Based Parking Slot Marking Recognition for Semi-automatic Parking System, LNCS Vol. 4109 (SSPR 2006 and SPR 2006, Joint IAPR International Workshops on Structural and Syntactic Pattern Recognition and Statistical Techniques in Pattern Recognition, Hong Kong, China, August 17-19, 2006), August 2006, pages:384-393.
주차구획인식 기반: 반자동 방식 (two touch) 목표주차위치 입구 양쪽 끝을 클릭함으로써 다양한 종류의 주차구획표시 인식 Ho Gi Jung, Yun Hee Lee, Jaihie Kim, Uniform User Interface for Semi-automatic Parking Slot Markings Recognition, IEEE Transactions on Vehicular Technology, Vol. 59, No. 2, Feb. 2010, pp. 616-626.
주차구획인식 기반: 반자동 방식 (two touch) 주차구획표시의 종류 및 junction pattern 종류
주차구획인식 기반: 반자동 방식 (two touch) Neural network을 이용한 junction pattern 종류 인식 Intensity histogram에 대한 Clustering으로 구 획표시영역 분리 Skeleton의 distance transform에 대하여, genetic algorithm을 이 용하여 template localization
주차구획인식 기반: 반자동 방식 (two touch) 191 case에 대한 평가에서 90% 이상의 성공률을 확인
Motion Stereo 기반 Jae Kyu Suhr, Ho Gi Jung, Kwanghyuk Bae, Jaihie Kim, Automatic free parking space detection by using motion stereo-based 3D reconstruction, Machine Vision and Applications, Vol. 21, No. 2, Feb. 2010, pp. 163-176.
Motion Stereo 기반 LKT feature tracking을 이용한 point correspondence 검출 Last image of image sequence Feature point tracking result
Motion Stereo 기반 Corresponding points RANSAC with 8-point M-estimator Inliers Fundamental matrix Essential Matrix Calibrated camera intrinsic parameters Camera Matrix (R, t) 3D reconstruction (triangulation)
Motion Stereo 기반 Triangulation에 의한 3D 복원 x cam X x' cam Y Y ' Z X Z ' X ' x PX x ' P cam ' X cam x cam PX ' cam ' 0 0 x P X 3T 1T xp p 3T 2T yp p AX = 0 where A 3T 1T 3T 1T 3T 1T x 'p' p' xp X p X 0 x 'p' X p' X 0 3T 2T 3T 2T 3T 2T yp X p X 0 y 'p' X p' X 0 y 'p' p' 2T 1T 2T 1T xp X yp X 0 x'p' X y'p' X 0
Motion Stereo 기반 Epipole location in image 3D reconstruction with all corresponding points 3D reconstruction after removing the point near the epipole
Motion Stereo 기반 800 800 700 700 600 600 500 500 Z (cm) 400 300 Z (cm) 400 300 200 200 100 100 0 0-400 -300-200 -100 0 100 200 300 400 X (cm) -400-300 -200-100 0 100 200 300 400 X (cm) 800 800 700 700 600 600 500 500 Z (cm) 400 300 Z (cm) 400 300 200 200 100 100 0 0-400 -300-200 -100 0 100 200 300 400 X (cm) -400-300 -200-100 0 100 200 300 400 X (cm) Parking situation With using the proposed method Without using the proposed method
Motion Stereo 기반 Corner Z (cm) 800 700 600 500 400 300 200 100 0-400 -300-200 -100 0 100 200 300 400 X (cm) Orientation Frequency count 45 40 35 30 25 20 15 Mean: 14.9 cm 10 5 0 0 10 20 30 40 50 Corner point error (cm) Corner point error Frequency count 45 40 35 30 25 20 Mean: 1.4 15 10 5 0 0 1 2 3 4 5 6 7 8 Orientation error (degree) Orientation error
LSP 기반: 실내주차장 대응 Coping with dark situation of underground parking lots Image with light stripe Image without light stripe Light stripe by difference Reconstructed 3D information Ho Gi Jung, Dong Suk Kim, Pal Joo Yoon, and Jaihie Kim, Light Stripe Projection based Parking Space Detection for Intelligent Parking Assist System, Proceedings of the 2007 IEEE Intelligent Vehicle Symposium, Istanbul, Turkey, Jun. 13-15, 2007. Ho Gi Jung, Jaihie Kim, Light stripe projection-based Target Position Designation for Intelligent Parking Assist System, IEEE Transaction on Intelligent Transportation Systems, Vol. 11, No. 4, Dec. 2010, pp. 942-953.
LSP 기반: 실내주차장 대응 The equation of light plane point P 0 (0, -b, 0), the normal vector n of light plane nx ( P 0 ) 0 sin sin n sin cos cos The equation of line Op A point on light stripe (X, Y, Z), f: focal length, Q: a point on line Op X Z Y x f y Q kx, ky, k f The solution by cross-point k btan cos f tan xsin ycos One-to-one mapping from (x,y) to (X,Y,Z) X Y xb tan cos f tan xsin ycos yb tan cos f tan xsin ycos Z f btan cos f tan xsin ycos
LSP 기반: 실내주차장 대응 Reconstructed 3D data Orientation estimation Depth map construction Target position establishment Inverse rotation Projection
LSP 기반: 실내주차장 대응 Situation Cases Success No. Recognition rate Vehicle-vehicle 65 63 96.92 Vehicle-pillar 160 150 93.75 Pillar-pillar 18 17 94.44 Total 243 230 94.65
Scanning Laser Radar 기반 Scanning laser radar의 소형화, 저가형개발추세 20 0.11333 0.17902 15 0.1581 y (m) 10 0.1173 5 0.17322 0.16946 0.19663 0.14654 0 0.16769 y x 0.16094 0.012292-5 0 5 10 15 x (m) Ho Gi Jung, Young Ha Cho, Pal Joo Yoon, and Jaihie Kim, Scanning Laser Radar-Based Target Position Designation for Parking Aid System, IEEE Transactions on Intelligent Transportation Systems, VOL. 9, NO. 3, Sep. 2008, pp. 406-424.
Scanning Laser Radar 기반 Rectangular corner detection l : ax byc0 1 l : bx ayd 0 2 x1 y1 1 0 a xn yn 1 0 b 0 yn xn 0 1 c d yn xn 0 1 X n A n 0.8 0.7 [ U, S, V ] SVD( A ) n n n n 0.6 0.5 error 0.4 0.3 Unaffected by orientation and robust to noise 0.2 0.1 10 0.051062 10 0 1 2 3 4 5 6 7 8 9 10 11 n 9 8 7 6 9 8 7 10 9 8 y (m) 5 4 3 2 1 y (m) 6 5 4 3 0.15481 y (m) 7 6 5 4 3 0 0 2 4 6 8 10 x (m) 2 1 0 0 2 4 6 8 10 x (m) 2 1 0 0 2 4 6 8 10 x (m)
Scanning Laser Radar 기반 Round corner detection 4 3 l : pxqyr 0 1 2 2 2 : 0 e ax bxy cy dx ey f e ax bxy cy dx ey f 2 2 1 : 0 l : pxqyr 0 2-3 -4-2 0 2 4 6 8 or x (m) 4 3 2 y (m) y (m) 2 1 0-1 -2 1 0-1 -2 y (m) 4 3 2 1 0-1 -2-3 -4-5 0.17188-2 0 2 4 6 8 x (m) -3-4 -3-2 -1 0 1 2 3 4 5 6 7 x (m) Rectangular corner detection Round corner detection 8 6 4 8 6 4 2 2 y (m) 0-2 0.21687 y (m) 0-2 0.12899-4 -4-6 -6-2 0 2 4 6 x (m) -2 0 2 4 6 x (m)
Scanning Laser Radar 기반 Established target position Corner detection Main reference corner detection (ROI, free space constraint) Subreference corner detection
Scanning Laser Radar 기반 Scanning laser radar-based shows the best performance (112 situations 98.2%, 615.3msec) Against the sun Dark underground pakring lots With various kinds of vehicles Cloudy weather
Infra-structure 기반 주차지원 시스템 [2][3] Parking administration, sensing and driver assistance system으로 구성.
Infra-structure 기반 주차지원 시스템 [2][3] 1. Parking spot assignment and parking-data transmission 2. Rough guidance from the parking lot entrance to the parking spot entrance 3. Find guidance until the completion of parking (주의) 차량에 대한 능동제어는 포함되지 않았음.. GUI를 이용한 안내만 지원.
Infra-structure 기반 주차지원 시스템 [2][3]
Infra-structure 기반 주차지원 시스템 [2][3] 네 가지 HMI mode 1. Navigation mode 2. Handle operation mode 3. Target position mode 4. Gear change mode Rough guidance Fine guidance
자동 발렛파킹시스템 자동 발렛파킹을 위한 센서기반 공간인지 및 자동주행 개념도
Open Problems Parallel Parking Device? http://www.youtube.com/watch?v=uir63vohxeg
References 1. Ho Gi Jung, "Target Position Designation Methods for Parking Assistant System", Ph.D. Dissertation, Yonsei University, Aug. 2008. 2. Massaki Wada, Kang Sup Yoon, Hideki Hashimoto, Development of Advanced parking Assistance System, IEEE Transactions on Industrial Electronics, Vol. 50, No. 1, Feb. 2003, 4-17. 3. Massaki Wada, Mami Mizutani, Masaki Saito, Xuchu Mao, Hideki Hashimoto, ican: Pursuing Technology for Near-Future ITS, IEEE Intelligent Systems, Vol. 19, No. 1, Jan./Feb. 2004, 18-23.