무인이동체란?
무인이동체활용분야및분류
무인이동체응용및활용분야
무인이동체공통기술
무인이동체공통기술개발
4 차산업혁명과무인이동체
인간과무인이동체간인터페이스
인간 - 이동체인터페이스기술의정의및구성 인간 - 이동체인터페이스 무인이동체를운용하는관점에서인간-기계인터페이스와인간-기계상호작용을모두의미하는 HMI 분야는사람이직관적인명령으로무인이동체를조종하도록하는 1인터페이스 ( 정보교류 ) 기술, 인간과기계의상호작용이가능한 2인터랙션 ( 교감 ) 기술을포함 단계 정의 설명 H-Level 1 H-Level 2 Human as operator 사람으로부터이동체로의단방향작용만있고이동체는단순무인이동체에사람이조종명령을내리고이동체는이한상태정보를전달하는것외에영향을미치지못함를단순히수행하기만하는단계 Human as supervisor 무인이동체가자율적으로임무를수행하는과정에서기동방무인이동체가주어진기동과임무를사람의허락하에식의선택지를사람에게제공함으로써수동적인영향을미침스스로수행하고, 사람이이를감독하는단계 H-Level 3 Human machine teaming 인간과무인이동체가서로협력하여임무를수행하는단계 무인이동체가임무달성을위해최적의운용방법을판단하여행동함에있어서, 인간과능동적인상호작용을통해인간과기계모두최고의효율을내는단계
자율수준에따른 HMI 단계구분 Level Level Description Observation Perception/ Situation Awareness Decision Making Capability Example 1 Remote Control Remote camera images viewed by operator None Remote operation in relatively simple stationary environments Basic teleoperation Human as Operator 2 Remote Control with vehicle State Knowledge Local pose, dash-board sensors, and depth image display for operator Basic health and vehicle state reporting Remote operation in relatively complex stationary environments Tele-operate with operator knowledge of geometry of environment 3 Pre-Planned mission or retro-traverse INS/GPS waypoints, collision avoidance ANS commanded steering based on planned path Basic path following with operator help Pre-planned path, retro-traverse, or operator waypoint selection 4 On-board processing of sensory images Perception of simple surfaces and shapes Negotiation of simple environment Robust leader follower with operator help Follow foot soldiers on road march or easy cross-country 5 Simple obstacle detection and avoidance Local perception and map database Real-time path planning based on hazard estimation Basic cross country semi-autonomous navigation Cross country with frequent operator intervention Human as Supervisor 6 Complex obstacle detection and avoidance, terrain analysis Perception and world model representation of local environment Planning and negotiation of complex terrain and objects Cross country with obstacle negotiation with some operator help Cross country in complex terrain with limited intervention 7 Moving object detection and tracking, on/off-road autonomous driving Local Sensor fusion with a priori maps of road network, representation of moving objects Robust Planning and Negotiation of Complex Terrain, Environmental Conditions, hazards and objects Cross country with obstacle avoidance with little operator help Cross country in complex terrain with full mobility speed with limited intervention Human Machine Teaming 8 9 10 Cooperative operations, convoy, intersections, on-coming traffic Collaborative operation, traffic signs and signals, near human levels of driving skill Full autonomy with human levels of performance or better Real-time fusion of data from external sources, broad knowledge of rules of the road Perception in bad weather and difficult environmental conditions Data fusion from all participating battlefield assets Advanced decisions based on shared data from other similar vehicles Collaborative reasoning for cooperative tactical behaviors Total independence to plan and implement to meet defined objectives Rapid effective execution of on-road driving tasks with minimal operator input Accomplish complex collaborativ e missions with some operator oversight Accomplish complex collaborative missions with no operator intervention On-road operations under normal road conditions with little supervision Effective combat mission accomplishment with little supervision Fully autonomous combat missions accomplished with results equal to or better than with human soldiers * 출처 : US Army Unmanned Ground Vehicle Autonomy Level
기술의구성 인간친화적 사용자인터페이스 원격운용기술 사용자의도추론 및대응기술 협력기술 무인이동체를전통적인 원거리에서무인이동체를 무인이동체가운용자로부터 다수의무인이동체와사람간의 인터페이스보다훨씬 운용함에있어발생하는 전달되는명령어를보다 효율적인협업을위한기술 더직접적이고자연스러운 문제점을해결하는기술 명확하고, 운용자의의도에 방법으로통제하기위한기술 맞게해석하여대응하는기술 가상환경 / 가상현실, NUI 직접운용, 감시제어, 의사결정지원시스템 운용자의도인식과대응기술, 피드백인터페이스 상호작용아키텍쳐및성능지표 개발, HMI 상호신뢰성연구
인간 - 이동체인터페이스기술개념도
가상환경 / 가상현실 영화 ( 아이언맨 ) 속 VR 활용 자율비행드론용 VR 시스템 (MIT)
Natural User Interface (NUI)
원격운용기술
사용자의도추론및대응기술 Mind-Controlled Drone Fleets
협력기술
Human Machine Teaming 에서요소별관계도 * 출처 : Steinberg, 2012
HMI 관련논문및특허동향 공통 HMI drone * 드론 * * 자율주행 * * 자율자동차 * * 무인기 * * 무인항공기 * * 무인이동체 * * 무인차 * * 무인선 * * 무인잠수 * * 콥터 * * 무인체 * UAV UGV UMV ROV UUV drone "autonomous vehicle" "autonomous car" "unmanned aerial vehicle" UAV "unmanned ground vehicle" UGV "unmanned surface vehicle" USV "unmanned underwater vehicle" UUV "Unmanned vehicle" "GCS" * 조종 * "VR" Virtual 가상현실 "AR" Augment* * 통제 * 증강현실 동작인식 * 음성인식 * FPV 인터페이스 Interface Interaction HMI * 상호작용 Haptic 햅틱 촉각 핸드폰 스마트폰 노트패드 모바일 NUI Natural 자연어 BCI BMI 뇌파인식 * gcs "Ground control station" "Virtual reality" "Augmented reality" "Hand gesture" "Voice recognition" "Verbal recog nition" "Speech recognition" fpv "first person view" Interface Interaction HMI Haptic "Semantic Gap" "Motion Tracking" Cellular notebook smart* notepad mobile NUI BCI BMI "Brain Computer Interface" "Brain Machin e Interface" Human- Machine Interface ( 드론 or ( 육상 near2 무인 near1 이동체 ) or ( 무인 near2 육상이동체 ) or 무인차 or ( 무인 near2 자동차 ) or 멀티콥터 or ( 무인 near1 이동체 ) or 무인체 or ( 자율 near1 주행 near1 ( 이동체 or 자동차 )) or 무인선 or ( 무인 near1 선박 ) or 콥터 or (( 소형 or 초경량 ) near1 무인기 ) or ( 자율 near1 자동차 ) or 무인기 or ( 무인 near1 항공기 ) or ( 무인 near2 ( 소형 or 초소형 ) near1 비행기 ) or ( 무인 near2 잠수 ) or ( 수중 near2 무인 near1 이동체 ) or ( 무인 near2 잠수함 ) or ( 무인 near2 잠수정 ) or ( 쿼드 near1 콥터 )) and (( 지상 near2 조종 ) or 조종기 or (( 가상 or 증강 ) near1 현실 ) or (( 동작 or 음성 ) near1 인식 ) or (1 인칭 near1 시점 ) or ( 레이싱 near2 드론 ) or (( 인간 - 기계 ) near2 인터페이스 ) or (( 인간 - 로봇 ) near2 인터페이스 ) or (hmi not hmi*) or ((( 인간 - 기계 ) or ( 인간 - 로봇 )) near2 ( 상호 near2 작용 )) or (( 인간 - 기계 ) near2 통신 ) or 핸틱 or 촉각 or ( 동작 near2 인식 ) or ( 감시 near1 제어 ) or 전화기 or 휴대폰 or 핸드폰 or 노트북 or 스마트폰 or ( 노토 near1 패드 ) or ( 모바일 near2 장치 ) or (nui not nui*) or ( 사용자 near2 인터페이스 ) or ( 자연어 near2 처리 ) or (bci not bci*) or (bmi not bmi*) or ( 뇌파 near1 인식 )) (drone or (autonomous near2 vehicle) or (manless near2 vehicle) or (uninhabit* near2 vehicle) or (autonomous near2 car) or (manless near2 car) or (unman* near2 vehicle) or (uninhabit* near2 car) or (unman* near2 aerial near2 vehicle) or (manless near2 aerial near2 vehicle) or (uninhabit* near2 aerial near2 vehicle) or (UAV not UAV?*) or (unman* near2 ground near2 vehicle) (manless near2 ground near2 vehicle) or (uninhabit* near2 ground near2 vehicle) or (UGV not UGV?*) or (unman* near2 surface near2 vehicle) (manless near2 surface near2 vehicle) or (uninhabit* near2 surface near2 vehicle) or (USV not USV?*) or (unman* near2 underwater near2 vehicle) or (manless near2 underwater near2 vehicle) or (uninhabit* near2 underwater near2 vehicle) (UUV not UUV?*) or copter or (quad near1 copter) or (multi near1 copter)) AND ((GCS not GCS*) or (ground near2 control* near2 station) or (human near2 controller) or (person* near2 controller) or (VR not VR*) or (AR not AR*) or (virtual near1 reality) or (hand near2 gesture near2 recogni*) or (voice near2 recogni*) or (speech near2 recogni*) or (FPV not FPV*) or (first near2 person near2 view) or (drone near2 (racing or race)) or (man near2 machin* near2 interface) or (HMI not HMI*) or (human near2 (machin* or robot or comput*) near2 (interface or interaction)) or (human near2 machin* near2 communicat*) or (haptic* near2 device) or (haptic* near2 apparatus) or (semantic* near2 gap) or (motion near2 track*) or (cellular near2 phone) or notebook or (smart near2 phone) or notepad or (mobile near2 device) or (remote* near2 control*) or (natural near3 user near3 interfac*) or (natural near3 language near3 process*) or (brain near3 (comput* or machin*) near3 interfac*) or (nui not nui*) or (bci not bci*) or (bmi not bmi*))
논문건수 특허건수 국내외기술동향 해외연구기술현황 국내연구기술현황 특허동향 시선, 음성, 동작, 뇌파등을인식하여기동모드변경및기본적인움직임을제어하는기술개발중 복합무인시스템에서의운용자감시제어모델링연구진행중 인간-자동화 / 로봇통합연구, 원거리작동으로인한지연제어에관한연구, HCI(Human- Computer Interaction) 등의연구수행중 무인이동체연동 VR 체험시발생하는어지럼증제거연구진행중 운용자의시선정보및심박수를이용하여메타지표인임무과부하추정기술개발중 인간-기계협업시스템의기능분배최적화연구진행중 2007년이후관련분야특허출원수는 264건이며, 이중 143건이상의특허가 14년이후에몰려있음. 주요출원인으로는 Parrot, The Boeing Company, IHI Aerospace, Honeywell, DJI 가있음
국내기술수준및경쟁력분석 국내기술수준 기술개발의시급성 파급효과 시선, 음성, 동작등의인식을위한기술개발및복수무인이동체제어연구는진행중인반면, 운용자의사결정지원연구에서는아직활발히진행되지는않음. 무인이동체운용자를위한 AR/VR의구현기술에대한요구가커지고있음. 복합무인시스템에서의운용자지원기술과운용자상황인식지원기술에대한수요가있음. 기존의운용자와무인이동체의관계를뛰어넘는 NUI, 유무인복합팀으로임무를수행하며기존의방식을뛰어넘는더폭넓은임무수행가능할것으로예상 선진국대비국내기술수준 경쟁국대비국내기술격차
인간 - 이동체인터페이스기술개발로드맵 요소기술 무인이동체운용친화성향상을위한가상환경 / 가상현실 NUI 를적용한무인이동체운용기술 무인이동체직접운용을위한운용자피로도저감인터페이스기술 감시제어를통한합리적인무인이동체운용지원 무인이동체안전운용을위한사용자의도추론및대응기술 무인이동체와운용자간의협력운용을위한신뢰성연구 세부기술운용자를위한무인이동체관점 AR 구현기술다중운용자에게동시지원이가능한 VR/AR 플랫폼기술음성및제스처인식을통한무인이동체운용기술생체정보 ( 뇌파, 시선등 ) 를활용한무인이동체운용기술운용자친화적인필드형멀티모달원격조작및운용시스템개발운용자상황인식지원기술운용자의숙련도, 피로도등현재상태를고려한인터페이스개발감시제어운용자의피로도 / 과부하완화기술복합무인시스템에서의운용자지원기술다수 / 다종무인이동체임무효율적배분기술제한된조건과환경을동시에고려한다기준의사결정기술운용상황을고려한운용자의도인식및추론기술파악된운용자의도에따른대응행동선택기법인간-이동체상호교감을위한멀티모달기반피드백인터페이스다수무인이동체와운용자간의협력성능지표개발다수무인이동체운용및관리인터페이스개발무인이동체와운용자간협업시신뢰성측정기술무인이동체와운용자간신뢰성향상기술 1 단계 2 단계 3 단계 Y Y+1 Y+2 Y+3 Y+4 Y+5 Y+6 Y+7 Y+8 Y+9