Multiple View Geometry in Computer Vision Second Edition Richard Hartley and Andrew Zisserman, Cambridge University Press, March 2004. http://users.cecs.anu.edu.au/~hartley/ http://www.robots.ox.ac.uk/~vgg/index.html http://www.cs.unc.edu/~marc/ Multiple View Geometry in Computer Vision
M. Pollefeys, L. Van Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, R. Koch, Visual modeling with a hand-held camera, International Journal of Computer Vision 59(3), 207-232, 2004. 2
3
4
5
Visual 3D Modeling from Images 6
Projective Transformations Camera Calibration Epipolar Geometry Feature Points Correspondence Search RANSAC Algorithm 3D Reconstruction SIFT&ASIFT 7
Photo Tourism http://phototour.cs.washington.edu/ 8
Building Rome on a cloudless day, Jan-Michael Frahm, Pierre Georgel, David Gallup, Tim Johnson, Rahul Raguram, Changchang Wu, Yi-Hung Jen, Enrique Dunn, Brian Clipp, Svetlana Lazebnik, Marc Pollefeys, ECCV 2010 Building Rome on a cloudless day 9
Lowe, David G. (1999). "Object recognition from local scale-invariant features". Proceedings of the International Conference on Computer Vision. 2. M. Brown and D. G. Lowe. Automatic Panoramic Image Stitching using Invariant Features. International Journal of Computer Vision, 74(1) 2007 AutoStitch http://cs.bath.ac.uk/brown/autostitch/autostitch.html 10
Blaise Agüera y Arcas : Seadragon, Bing Maps, Photosynth, MIT Technology Review TR35 in 2008 Photosynth DeepZoom & HD View http://photosynth.net/ 11
Image Composite Editor http://research.microsoft.com/en-us/um/ redmond/groups/ivm/ice/ 12
PhotoCity http://photocitygame.com/ 13
Depth Camera
Bumblebee http://www.ptgrey.com/products/stereo.asp Sony Bumblebee Stereo Camera 15
Structured Light 3D Surface Imaging 16
17
Kinect for XBox 18
http://www.microsoft.com/en-us/kinectforwindows/discover/features.aspx What is Kinect? Motion sensing input device by Microsoft Depth camera tech. developed by PrimeSense Invented in 2005 Software tech. developed by Rare First announced at E3 2009 as Project Natal Windows SDK Releases 19 2/13/2012
20
21
Kinect for XBox The subpattern is 211 x 165, totalling to 633 x 495 spots, sensor chip with SXGA resolution (1280 x 1024) Both dimensions are odd numbers, so that there can be a central bright spot. The number of (bright or dark) spots inside the subpattern region is 34,815. 3,861 of them are bright 0.1109 = 1 / 9.017094. Therefore, on average every 3 3 region there is one bright spot. No bright spots are 9-connected. No repetitive structures. Always 4 bright spots per 6 6 spots Each bright spot is surrounded by dark spots. the number of spots per pixel is constant, a local thresholding operation could be implemented, that quickly filters out the spots in the sensor image, and converts it in a binary image for easy neighbor extraction and ID computation. The average brightness looks quite constant. A small region of spots, that is unique within the whole pattern Depth or disparity is finding the ID in a reference image. 22
23
Real-Time Human Pose Recognition in Parts from Single Depth Images, Jamie Shotton Andrew Fitzgibbon Mat Cook Toby Sharp Mark Finocchio, Richard Moore Alex Kipman Andrew Blake, Microsoft Research Cambridge & Xbox Incubation 24
http://research.microsoft.com/en-us/projects/surfacerecon/ KinectFusion 25
Z-CUBE@ETRI 26
ToF 3D Camera 27
DepthCamera@Samsung 28
Light Field Camera
http://graphics.stanford.edu/projects/array/ Stanford Multi-Camera Array 640 480 pixels 30 fps 128 cameras synchronized timing continuous streaming flexible arrangement 30
http://lightfield.stanford.edu/lfs.html Stanford Light Field Archive 31
http://lightfield.stanford.edu/lfs.html Stanford Light Field Archive 32
https://www.lytro.com/ Lytro Camera 33
Adaptive Optics Microlens Array Adaptive Optics microlens array 125μ square-sided microlenses 4000 4000 pixels 292 292 lenses = 14 14 pixels per lens 34
http://lightfield-forum.com/light-field-camera-prototypes/adobe-lightfield-camera-protypes/ Adobe LightField Camera 35
3D Display System
3D Display Human Visual System 3D Display Systems Stereoscopic display Autostereoscopic display Holographic display Stereo Disparity 37
Classification of 3D Display Glasses Stereoscopic display - Polarization method - Sutter glasses method Non-glasses Autostereoscopic display - Lenticular-type - Barrier-type Real 3D Floating Integral imaging Holography 38
3D Display Stereoscopic display Multiview display Integral Image display Holographic display Reasons for not buying 3D TV (NPD Group, 2011) 39
3D Display Stereoscopic display Multiview display Integral Image display Holographic display 15 views 50 1999 Cambridge Univ. 40
http://www.uv.es/imaging3/structure.htm 3D Display Stereoscopic display Multiview display Integral Image display Holographic display Super multi-view camera system M x N CCD Camera [1024X768] Resolution : 1024 x 768 x M x N 41
3D Display Stereoscopic display Multiview display Integral Image display Holographic display Ideal 3D Display Technique Based on Wave Optics Real images in space Full parallax Continuous viewing points Coherent illumination Color images Fig. 4 1 Chart of the principle of holographic display operation 1 computer, 2 RF processor, 3 RF signal, 4 acoustic-optical modulator, 5 laser, 6 lens, 7 vertical scanner, 8 beam splitter, 9 horizontally connected scanners, 10 output lens, 11 dispersion window (ground-glass), 12 display area, 13 display plane, 14 drive 42
3D Display Human Visual System 3D Display Systems Stereoscopic display Autostereoscopic display Holographic display Stereo Disparity 43
http://blog.naver.com/postview.nhn?blogid=intel007&logno=50144775536 Lenticular lens 44
http://en.wikipedia.org/wiki/lenticular_lens Lenticular Lens R is the angle between the extreme ray and the normal at the point where it exits the lens, p is the pitch, or width of each lenticular cell, r is the radius of curvature of the lenticule, e is the thickness of the lenticular lens h is the thickness of the substrate below the curved surface of the lens, and n is the lens's index of refraction. 45
Dodgson - Stanford Workshop on 3D Imaging.pdf Slanted Lenses This all changed in the mid 1990s, when Cees van Berkel, at Philips research lab in the UK, discovered that slanting the lenticular array solved both problems. The slanted configuration meant that both horizontal and vertical resolution are divided amongst the multiple horizontally spaced views. We thus get a reasonable resolution in both dimensions In all views. The slanted lenslets also smear out the dark bands so that they are no longer visible. 46
Real 3D Display 2010 방송통신산업전망컨퍼런스 - 완전시차연속시점제공 - 저피로도구현 - 간단한시스템구조 - 칼러 Real 3D 영상구현용이 - 공간영상재생기술 47
Development of high-definition 3D image processing technologies using advanced integral imaging with improved depth range Real 3D Display System Time/Space multiplexing Real 3D image 최적단일렌즈어레이설계기술 3D 콘텐츠생성및영상신호처리핵심기술개발 48 고선명실시간 3D 영상디스플레이핵심원천기술개발및시제품 Viewers 2012 년산업융합원천기술개발사업홈 / 정보가전분야집적영상 (IP) 깊이표현범위를개선한고선명 3D 영상처리기술개발 시스템향상핵심기술
Camera Array Simulation using OpenGL SubImage ElementImage 49
Resolution Improvements 단일렌즈어레이와전자마스크를이용한깊이우선 IP 기반의 3D 영상디스플레이기술 깊이우선 IP 기술에서해상도를복잡한기계적구조물없이증가시킬수있은핵심기술 기존방법 : 렌즈어레이를통하여발생하는레이들의교차점이렌즈어레이의기초렌즈크기와동일하기때문에고해상도구현이힘듬 핵심기술 : 세계최초로깊이우선 IP 방식에전자마스크를결합사용하는방법을접목시분할기법을통하여해상도를급격히증가시킴 < 기존방법의예 > < 개발예정핵심기술 > Sony 960HZ 3D LED TV KDL-55HX850-55" 1080P Samsung 960hz D8000 LED TV - 46, Full HD Samsung Transparent LCD 22 1680x1050 15% Resolution = number of lenslet (n times ) Viewing angle = 2atan(d/2nf) (n times ) Depth range = d 2 /λn 2 (n 2 times ) 50 50
Optical System Setup Specifications of Lenslet Array Type Fresnel, Square Lenslet pitch 5mm No. of Lenslet 15 15 Focal length 10mm Camera Transparent -LC Panel Screen + MLA Elemental Images Projector 51
Experimental Results 52
Experimental Results t = t1 t = t2 t = t3 t = t4 Computational summation Resolution improvement by 2x2 Proposed method Conventional method 53
Different Viewing Angle Conventional method Proposed method 54
Depth Improvements 단일렌즈어레이와전자렌즈를이용한해상도우선 IP 기반의 3D 영상디스플레이기술 해상도우선 IP 기술에서깊이영역를증가시킬수있은핵심기술 기존방법 : 해상도우선 IP 기술에서깊이영역을향상시키기위해서는디스플레이패널과렌즈어레이사이의거리 (g) 를변화시켜야함. 이전의방법들은기계적으로디스플레이패널또는렌즈어레이를움직이는방법을사용하였음. 핵심기술 : 세계최초로해상도우선 IP 방식에렌즈어레이와전자렌즈를결합사용하는방법을접목시분할기법을통하여깊이영역을급격히증가시킴전자렌즈 shifting 효과를추가한렌즈자국을제거 < 기존방법의예 > < 개발예정핵심기술 > Resolution = number of lenslet m Viewing angle = 2atan(d/2g) Depth range (n times ) 55 55
Prof. Yi-Pai Huang, National Chiao Tung Univ. Taiwan Current LC-lens for 3D Display Adaptive Liquid Crystal Lens(LC-Lens) Array for 3D Display, Interaction, and Capturing, Yi-Pai Huang 56
InI Camera & Display Systems IP 카메라및 9 인치프로젝션형 3D 디스플레이시스템 국내특허 : 중간요소영상을이용한입체영상표시장치, 등록번호 100730406, 2007.06.13 국제특허 : US Patent Pending, Appl No. 12/004309, 2007.12.19 기술개요 Display 시스템사양 고해상도 IP 카메라제작 집적영상기술의콘텐츠제작전용 S/W 개발 집적영상전용소프트웨어로서최적화 고해상도영상디스플레이장치 3D 영상표시를위한렌즈어레이부포함 Real/Virtual 영역구동이가능한동영상콘텐츠제공 IP 카메라사양 유효화소수 610 만 (3000*2000) 렌즈배열스펙 폭 :1.08mm 초점 :3.0mm 렌즈릿배열개수 150x150 (9 인치 ) 렌즈릿초점거리 5.02mm 요소영상해상도 1600x1200 프로젝터모델 EPSON EMP-820 57
iphone4 3D Display System iphone4 를이용한깊이우선 IP 기반의 3D 영상디스플레이시제품 국내외특허출원진행중 : 탈착식렌즈어레이기반의 3D 영상디스플레이방법및시스템 기술개요 시제품제작 깊이우선 IP 기술을이용한무안경 3D 영상표시 Retina Display 를이용한컬러모아레제거 탈착식구조의렌즈어레이사용 기술특징 (1) Pickup (2) IP 콘텐츠생성 (3) 3D display 렌즈릿배열개수 75x50 (4 인치 ) 렌즈릿초점거리 요소영상해상도 Display 모델 5.02mm 960*640 iphone 4 58
Lens Array 디스플레이장치 : Barco3420 2048 x 1536 화소크기 : 0.207 mm 렌즈어레이사양 전체크기 : 684mm x 385.3440 mm 전체렌즈개수 : 426x240 개 기초렌즈사양 초점거리 : 8.028 mm 직경 : 1.6056 mm 디스플레이장치 : IBM t221 3840 x 2400 화소크기 : 0.1245 mm 렌즈어레이사양 전체크기 : 679.77 x 380.97 mm 렌즈개수 : 91x51 개 기초렌즈사양 초점거리 : 29.88mm 직경 : 7.47 mm 59
a Real 3D Display System using Integral Imaging Technology Thanks you!! http://kowon.dongseo.ac.kr/~lbg/ 60