VR/AR for Medicine 2018.01.25 한국과학기술연구원의공학연구소 김영준
2
Contents 연구소개 VR/AR 의료응용사례 VR/AR 의료응용분야및전망 3
4
3D 의료영상처리 S/W 3D 가상수술 3D 환자모델링 3D 형상모델처리기술 5
3D Patient Modeling Automatic Orbital Wall Modeling Existing method: manual operation on >100 images Proposed method: ~1 min. (1/40 less time) Resulted 3D orbital wall model Avr. 31m 12s Semi-automatic 3D modeling of orbital wall Avr. 43.7s Comparison test of 3D modeling time (manual vs. proposed method) 6
3D Surgical Planning 7
3D Modeling of Surgical Guide 8
9
AI 기반 3D 의료용 S/W 기술 임상빅데이터 AI 진단 AI 치료계획 AI 치료분석 환자 3D 데이터 Classification 치료방법추천 수술결과분석 10
3D CNN 기반어깨관절진단 학습 Rotator Cuff Tear 진단 Voxception-ResNet 네트워크사용, epoch 110 Train set 3573개 (None-RCT : 1749, RCT : 1824), Test set 200 >95% 정확도 학습진행에따른정확도및 activation map 변화 Voxception-ResNet 11
3D CNN 기반어깨관절 (Rotator Cuff Tear) 진단 12
13
https://github.com/vtk-book/example 14 14
3D simulation, 3D printing, AI, VR/AR Medical application
Contents 연구소개 VR/AR 의료응용사례 VR/AR 의료응용분야및전망 16
Epson MOVERIO https://epson.com/moverio-augmented-reality 17
Epson MOVERIO 18
Evena Eye-On Glasses https://evenamed.com/ 19
Microsoft HoloLens 20
Microsoft HoloLens 21
3D4Medical http://3d4medical.com/ 22
MediSIM http://3d4medical.com/ 23
The Body VR: Anatomy Viewer http://thebodyvr.com/ 24
3D Organon VR Anatomy https://www.oculus.com/experiences/rift/872418872856459/ 25
NEOFECT http://www.neofect.com/ 26
Motek https://www.motekforcelink.com/ 27
Motek https://www.motekforcelink.com/ 28
Dental Simulation - 3D Smile Design (CEREC) 29
NeuroVR 신경외과수술훈련용시뮬레이터 두개골개폐, 뇌내시경수술술기훈련
Osso VR 31
Help Me See Cataract Surgery (SOFA, InSimo) 32
SOFA (Simulation Open Framework Architecture) 33
Virtual Training Simulation Dental simulation Intravenous injection simulator Laparoscopic surgery simulation Arthroscopic surgery training 34
http://www.intuitivesurgical.com/products/skills_simulator/
Surgical simulator http://simbionix.com/ ANGIO Mentor VirtaMed HystSim PELVIC Mentor U/S Mentor
Laparoscopic surgery simulator SimSurgery / Norway Immersion / USA Simbionix / USA Surgical-Science / Sweden Mimic Technologies / USA Mentice / Sweden Haptica / USA
Laparoscopic Surgical Training Simulation Deformable body modeling 3D visualization / haptic rendering / collision detection Gallbladder removal simulation Gallbladder removal simulation Haptic device for laparoscopic surgery simulation 38
Laparoscopic training simulator 39
Arthroscopic surgery simulator Diagnostic and therapeutic knee / shoulder arthroscopy training in orthopedic surgery 관절경수술 ARTHRO MENTOR http://simbionix.com/simulators/arthromentor/
Arthroscopic surgery simulator 3D human body modeling (bone, muscle, ligament, ) Realistic visualization : normal mapping (texture) + lighting Arthroscopic camera simulation 3D human shoulder modeling Arthroscopic camera simulation
Arthroscopic surgery simulator Task: Basic navigation skill, feeling of burr Navigation, tool manipulation and orientation skills Deburring Navigation
Simulation for Craniopagus Surgery 43
Contents 연구소개 VR/AR 의료응용사례 VR/AR 의료응용분야및전망 44
혼합현실기반수술계획및가이드 Tokyo Univ. ( 일본 ) 악안면수술환자 3 차원해부모델 AR 환자치아형태 3D 매칭, makerless 실시간트래킹 Pisa Univ. ( 이탈리아 ) 악안면수술을위한 AR 기반네비게이션시스템 수술계획대로악안면골재배치 AR 수술계획정보제공
혼합현실기반원격치료 Proximie ( 미국 ) AR 기반원격진료솔루션 아이폰 / 아이패드로연결된두지점의전문가수술보조서비스제공 Google, University of Alabama at Birmingham ( 미국 ) AR 기반원격수술가이드솔루션 구글글래스이용 Teaching surgeon의술기를 AR로보면서수술진행
Nadine Hachach-Haram (TEDWomen 2017) How augmented reality could change the future of surgery https://www.ted.com/talks/nadine_hachach_haram_how_augmented_reality_could_ change_the_future_of_surgery 47
Medical Education Visible Human Project (NLM, 1986) Visible Korean ( 한국과학기술정보원, 2000) Anatomage Table Medical education(ms Hololense, 2016) KIST 김래현
Live Surgery Medical Realities, UK KIST 김래현
Surgical Simulation Laparoscopic surgery (CAE, KIST) Arthroscopy surgery (VirtaMed) Eye surgery (HelpMeSee) Intravenous injection (KIST) Dental training (KIST) KIST 김래현
Surgical Planning & Guide Neuro surgery planning (UCLA) Knee & Maxillofacial surgery planning (KIST) AR-based surgical guide (Fraunhofer MEVIS Institute) KIST 김래현
AR Navigated Spine Surgery: Scopis https://navigation.scopis.com/news/scopis-introduces-mixed-reality-surgical-holographic-navigation-platform 52
53
54
LINKFLOW http://linkflow.co.kr/ 55
Vive Pro (HTC) 해상도 : 2880 x 1600 전방카메라 2대, 마이크 2대가상공간 : 10m x 10m 무선어댑터출시예정 56
Future of VR Hi resolution(8k), wide FOV(120), fast updated(120 fps) Multimodal stimuli (tactile, smell,..) Seamless and precise tracking ( <10 ms latency, sub-mm) HTC VIVE (2K resolution, 110 degree FOV, 90 fps, 2 controller, Tracking: sub-millimeter precision and < ~11ms latency) KIST 김래현
Future of VR Mobile VR Multi-view 360 video Social VR 360 Panoramic video (YouTube, Facebook) Social VR(Facebook) KIST 김래현
Future of AR Lighter, Cableless, Brighter H/W Wide FOV (Field of View) Easy Calibration Accurate Registration AI for Improved Function MS HoloLens 3 rd generation (2019) Magic Leap One (2018 spring) MAXST
60
Youngjun Kim E-mail: junekim@kist.re.kr 3D printing, 3D simulation, AI, VR/AR Medical application