Asia-pacific Journal of Multimedia Services Convergent with Art, Humanities, and Sociology Vol.7, No.3, March (2017), pp. 421-430 http://dx.doi.org/10.14257/ajmahs.2017.03.49 이종센서기반지형정보구축프레임워크 김원진김창재정동기박준규 요약 최근에실내외공간정보를근실시간으로취득하여토목건축재난안전등다양한분야에서소규모지역에대한공간정보를빠른시간에저비용으로갱신하고자하는요구가증가하는추세이다유용한차원공간정보를구축하기위해서는다양한센서에서얻은각각의데이터를정확하게정합하고해석해야한다본논문은센서들의상호보완을위해멀티센서광학카메라레이저스캐너를이용하고위치정보와자세정보를활용하기위해를백팩시스템에적용한다각센서의내부표정요소를취득하기위한셀프캘리브레이션을수행하고이종센서간기하학적관계를해석하기위한시스템캘리브레이션이고려되었다우선셀프캘리브레이션은다수의지상기준점과광학카메라의물리적모델을기반으로수행된다더불어시스템캘리브레이션에서는이종센서들의커버리지와센서특성을고려한캘리브레이션설계가필수적이다이러한방법론을기반으로대용량의실내외차원공간정보를효율적으로취득하기위한이동형차원공간정보취득솔루션을제안한다주요어공간정보데이터캘리브레이션멀티센서센서융합데이터융합 Abstract Nowadays, a demand for low cost and up-to-date indoor and outdoor information acquisition is increasing in the fields of civil engineering, construction and emergency management. In order to build useful 3D geo-spatial information, the datasets acquired from various sensors should be registered each other. In this paper, we proposed an approach to mount an optical camera, a laser scanner and a GPS/IMU Received (January 14, 2017), Review Result (January 31, 2017) Accepted (February 7, 2017), Published (March 31, 2017) 본학술발표논문은중소기업청에서지원하는년도산학연협력기술개발사업의연구수행으로인한결과물임을밝힙니다 ISSN: 2383-5281 AJMAHS Copyright 2017 HSST 421
unit on the backpack system for complementary effects among various sensors. Self-calibration procedures to acquire interior orientation parameters of the relevant sensors and the System-calibration to interpret the geometric relationship among the sensors were considered. First, Self-calibration is performed based on a number of ground control points and the physical model of the optical camera. In addition, it is essential to design the System-calibration considering the coverage and characteristic of sensors. Based on this methodology, we proposed a mobile data acquisition solution to obtain indoor and outdoor 3D geo-spatial information efficiently. Keywords : 3D Geo-spatial, Calibration, Multi-sensors, Sensor Fusion, Data Integration 서론 최근에실내외공간정보를근실시간으로취득하여토목건축재난안전등다양한분야에서소규모지역에대한공간정보를빠른시간에저비용으로갱신하고자하는요구가증가하는추세이다특히기술이발전할수록정밀한공간정보가요구됨에따라접근하기어려운지역에대해서도정밀한차원공간정보구축의필요성이제기되고있다국외사례를살펴보면구글는백팩플랫폼을개발하여실시간으로포인트클라우드를획득하여사용자가휴대용장비를이용해서를입력가능하도록하였다에서는카트형식으로실내공간정보를연속적으로취득하고정합하는기술을개발하였다최근에는카트를작게하고계단도오르고내릴수있는를발표했다는광학카메라와레이저스캐너등장비를백팩시스템으로구축한를발표하였다이와같이국외에서는많은연구를기반으로다양한플랫폼을개발하고이를이용하여차원공간정보를구축하고있다국내에서는다양한센서를융합하여이동로봇드론백팩등다양한플랫폼을이용한차원데이터획득및융합에대한연구가활발히진행되고있다하지만경량의백팩형기반으로다양한센서를융합하고이를기반으로차원공간정보를연속적으로얻는플랫폼에대한연구는아직미진한상태에있다이에본연구에서는이종센서간시스템캘리브레이션을수행하고이를기반으로데이터정합알고리즘을수행하여차원공간정보를획득하기위한방법론에대해고찰하고자한다공간정보를효율적으로얻기위해촬영이가능한광학센서와레이저스캐너시스템을기반으로한경량화된플랫폼을고려하였다이를통해실내외에서차원공간정보를효과적으로얻기위해광학영상에서색상정보를추출하고레이저스캐너에서차원점군데이터를추출하여데이터정합을실시한다본연구에서제안한데이터정합알고리즘은이종센서에서추출된내외부표정요소를기반으로마운팅파라메타를추출하고이를기반으로차원공간정보를획득한다본논문의장에서는이종센서기반데이터취득시스템에서고려된센서들에대한소개와이를이용한센서융합알고리즘을제안한다그리고데이터정합을위한각센서들의캘리브래이션과센서간시스템캘리브레이션방법론을제안한다장에서는본연구의의의와이번연구를바탕으로향후연구에대한방향을제시한다 422 Copyright 2017 HSST
Convergence Research Letter Vol.1, No.2, June (2016) 이종센서기반 데이터취득시스템 고려된이종센서들과제시된플랫폼 실내외차원공간정보를구축하기위하여다양한연구에서다양한센서가이용되고있다본연구에서는아래그림에제시된광학렌즈레이저스캐너그리고를사용하였다광학카메라를이용해다각도에서촬영된영상은색상정보를기반으로차원공간을구축할수있다하지만광학카메라와레이저스캐너를이용하면색상정보와차원점군데이터를한번에얻을수있기때문에이종센서간시너지효과를발휘할수있다그리고를이용하여데이터취득시센서의위치및자세정보를동시에얻을수있기때문에보다정밀한데이터정합이가능해진다이장비들을백팩에설치하여이동하면서정보를구축할수있도록설계하였다 GPS/IMU bublecam VLP-16 3D scanner 그림고려된이종센서들 표 의사양 Parameter Range(dynamic) Accelerometers 2 g 4 g Gyroscopes 250 /s 500 /s 2000 /s Magnetometers 2 G 4 G 8 G Pressure 10 to 120 KPa 16 g Bias Instability 20 ug 3 /hr - 10 Pa Initial Bias < 5 mg < 0.2 /s - < 100 Pa Initial Scaling Error < 0.06 % < 0.04 % < 0.07 % - Scale Factor Stability < 0.06 % < 0.05 % < 0.09 % - Non-linearity < 0.05 % < 0.05 % < 0.08 % - Cross-axis Alignment Error < 0.05 < 0.05 < 0.05 - Noise Density 150 ug/ Hz 0.005 /s/ Hz 210 ug/ Hz 0.56 Pa/ Hz Bandwidth 400 Hz 400 Hz 110 Hz 50 Hz ISSN: 2383-5281 AJMAHS Copyright 2017 HSST 423
이종센서기반의이동형 차원공간정보구축방법론고찰 표 광학카메라의사양 Aperture:f /2.0 Optics Filed of View : 4 lenses X 190 FOV (160 used) Focal Length : 1.2mm Minimum Focus Distance : 25cm Sensor : 5MP X 4 Size and Weight Dimension: 80mm (3.14 ) diameter, circular Weight: Bublcam: 280g Storage On board microsd/microsdhc Supports 4GB 32GB Class 10 UHS-1 micro SD card Power Power Source: 3.7V LiPo at 1560mAh Charging Method: mini USB 2.0 port with USB cable + power adapter Full Charging Time: Approx. 2h Video Mode Video File Format : Saved as. MP4, H.264 Microphone: Omni PUI mic, -45dB+-3 Audio Format : MP3 Video File Resoluiton: Multiplex 30 fps: 1440p x 1440p 15 fps: 1920p x 1920p Equirectangular 30 fps: 1984p x 992p 15 fps: 2688p x 1344p Image Mode Photo File Format : Saved as. JPG Approximate Image Resolution : 14MP Image File Resolution : Multiplex : 3840p x 3840p Equirectangular : 5376p x 2688p 표 레이저스캐너의사양 VLP-16 Dual Returns 830 grams 16 Channels 100m Range 300,000 Points/Sec 360 Horizontal FOV ± 15 Vertical FOV Low Power Protective Design 424 Copyright 2016 HSST
Convergence Research Letter Vol.1, No.2, June (2016) 그림 제안된플랫폼 도카메라레이저스캐너센서들과운영시스템및배터리등을기반으로위에제시된그림과같이이동하면서차원공간정보를획득할수있는경량화된백팩시스템플랫폼을제안한다 제안된센서융합알고리즘 앞서언급된센서들로부터획득된데이터를이용하여기하학적으로정밀한차원공간정보를구축하기위해서는시스템캘리브레이션과데이터정합이정밀하게수행되어야한다아래의그림은이와관련한시스템캘리브레이션과데이터융합에대한전체흐름도이다 시스템캘리브레이션기능 데이터정합기능 그림 센서융합알고리즘 ISSN: 2383-5281 AJMAHS Copyright 2017 HSST 425
이종센서기반의이동형 차원공간정보구축방법론고찰 시스템캘리브레이션우선시스템캘리브레이션을통하여각센서들특히광학센서의내부표정요소를파악하고센서들간의기하학적인관계를파악하는시스템마운팅파라미터캘리브레이션을수행해야한다본연구에서는전방위카메라를사용하는관계로모든방향에캘리브레이션타깃이존재하도록테스트베드제작하였다이후공선조건식과내부표정요소파라미터들에기반한을수행하여광학카메라내부의파라미터값을획득하였다더불어전방위카메라와레이저센서전방위카메라와간기하학적관계를해석하여내외부표정요소를추출하였다 광학카메라의캘리브래이션내부표정요소캘리브레이션광학카메라캘리브레이션의목적은영상시스템내부의기하학적인형상을수학적으로설명하기위한것이다영상내부시스템의파라미터들을구하기위하여이들파라미터들을이용한를수행해야한다이를위하여전방위카메라를고려하여모든방향에캘리브레이션타겟들이존재하도록테스트베드를제작하였다이후아래와같은공선조건식과내부표정요소파라미터들을이용하여를수행하여파라미터값을추출한다 카메라투영중심좌표 지상점 의좌표 주점의좌표 카메라의주점거리 점의영상좌표 회전행렬 렌즈왜곡에의해발생하는영상좌표의변위량 시스템마운팅파라미터캘리브레이션센서들간의파라미터캘리브레이션시스템마운팅파라미터캘리브레이션의목적은여러센서들을장착하고있는백팩형실내공간정보획득시스템에서이들센서들간의를파악하는것이다전방위카메라의대표센서와나머지센서들간의기하학적관계해석은아래그림에서보는바와같이기준이되는센서와다른센서들간의관계식을통하여이해된다이를위 426 Copyright 2016 HSST
하여수학적관계식을성립하고 를미지수로하여풀게된다 Convergence Research Letter Vol.1, No.2, June (2016) 좌표계에서 의위치벡터 좌표계에서전방위카메라대표센서의위치벡터 좌표계와전방위카메라대표센서간의회전행렬 전방위카메라대표센서와나머지센서중하나간의 벡터 전방위카메라대표센서와나머지센서중하나간의회전행렬 와전방위카메라간의기하학적관계해석은다음의식을바탕으로한다 의위치벡터 좌표계에서 의위치벡터 좌표계와 간의회전행렬 과전방위카메라내특정센서간의 벡터 과전방위카메라내특정센서간의회전행렬 전방위카메라대표센서간의 벡터 과전방위카메라대표센서간의회전행렬 전방위카메라대표센서와나머지센서중하나간의 벡터 전방위카메라대표센서와나머지센서중하나간의회전행렬 와레이저스캐너간의기하학적관계해석은다음의식를바탕으로한다 ISSN: 2383-5281 AJMAHS Copyright 2017 HSST 427
이종센서기반의이동형 차원공간정보구축방법론고찰 의위치벡터 좌표계에서 의위치벡터 좌표계와 간의회전행렬 레이저스캐너간의 벡터 과레이저스캐너간의회전행렬 데이터정합고밀도의데이터취득에있어효율성을극대화하기위하여이동식스캔방식이점차선호되고있으며그대안으로이공간정보분야에서주목받고있다은이동체이동플랫폼에각종센서를장착하여주변환경에대한정보를취득하고이를바탕으로플랫폼의위치정보와지도를작성하는방법론이다센서관측데이터의불확실성실시간성확보넓은환경에서의등의문제를해결하기위해다양한방법론이연구되었다 앞의시스템캘리브레이션방법을이용하게되면각센서들간의상대표정요소를획득하게된다이는하나의특정에서각센서들로부터획득된자료들을하나의통합된좌표계로정밀하게묶어주는역할을한다그러나이러한데이터들이각마다획득되게되어무수히많은융합데이터들이획득된다무수한데이터들을하나의좌표체계하에두기위하여서는간정밀한데이터정합이필수적이다각간에는개의회전요소 와개의이동요소 가존재한다이들개의파라미터를기법과방법론을이용하여서로다른간의변환식을구한다본연구에서는다양한기법들중가장정밀한방법중의하나로알려진기법을선정하였다기법은정보를누적하여한꺼번에의최적화문제를해결한다정보를누적하는동안에는플랫폼의위치오차가증가하지만수집한모든정보에가장최적인추정치를제공해주므로로작성된지도보다정확한지도를작성할수있다일반적으로기법은라불리는작성단계와라불리는최적화단계로구성되어있다이러한기법을통하여얻어진각간의변환파라미터들을이용하여아래와같은식를통하여취득된모든데이터를하나의좌표계로정밀하게통합시키게된다 428 Copyright 2016 HSST
Convergence Research Letter Vol.1, No.2, June (2016) 변환후좌표값 변환전좌표값 스케일값 일반적으로 간의회전행렬 간의이동치 결론 공간정보를효율적으로얻기위해촬영이가능한광학센서와레이저스캐너시스템을기반으로백팩형차원공간정보획득플랫폼을제안하였다정밀한데이터취득을위해서센서들간캘리브레이션을바탕으로내외부표정요소를해석하고이를이용하여데이터정합에대한방법론을제시하였다본연구는실제환경하에서소규모장비를이용하여효과적이고근실시간적으로데이터의취득을가능하게할수있는데큰의미가있다이렇게취득된차원공간정보를활용하여실내공간관리실내공간시뮬레이션실내공간정보검색실내내비게이션등다양한실내공간응용서비스제공이가능하다또한연구에서제안된시스템은대형장비가접근하기어려운상황에서연속적으로차원공간을획득하는경우에효과적으로활용될수있다향후연구에서는본연구를바탕으로차원데이터취득을위한전과정을자동화하는알고리즘을개발하고이종센서간효과적인기하배치를통하여완성된플랫폼의무게중심을고려한백팩시스템을도입할예정이다또한취득된데이터로부터의미있는정보를효율적으로취득하기위한노이즈처리및시스템최적화에대해서도연구를수행하고자한다 References [1] Al-khawaldah, M., and Nüchter, A. Multi-Robot Exploration and Mapping with a rotating 3D Scanner. In Proceedings of the 10th International IFAC Symposium on Robot Control, Sep (2012), pp.313-318 [2] Eui-Ik Jeon, Kyoungah Choi, Impyeong Lee, Development of a Low-cost Multi-sensor System Based on a Micro-UAV for Rapid Mapping, THE KOREAN SOCIETY FOR GEOSPATIAL INFORMATION SCIENCE, (2014), pp.229-232 [3] K. Lee, N.L. Doh, Linearized measurement for the SLAM application of mobile robot. International Journal ISSN: 2383-5281 AJMAHS Copyright 2017 HSST 429
이종센서기반의이동형 차원공간정보구축방법론고찰 of Precision Engineering and Manufacturing (2010), pp.73-74. [4] Chow, J. C.K., Multi-Sensor Integration for Indoor 3D Reconstruction, UCGE Report Number 20399, May-2, (2014) [5] Derek D. Lichti. Terrestrial laser scanner self-calibration: correlation sources and their mitigation,. (2010), pp.93-102. [6] Jafar Amiri Parian, Armin Gruen. Integrated laser scanner and intensity image calibration and accuracy assessment, (2005), Part3, W19, pp.18-23. [7] Yuriy Reshetyuk. A unified approach to self-calibration of terrestrial laser scanners, (2010), pp.445 456 [8] Jae-Bok Song, Seo-Yeon Hwang, Past and State-of-the-Art SLAM Technologies, Transactions on Control Automation and Systems Engineering, Journal of Institute of Control, Robotics and Systems Engineering, Vol. 20, No.3, (2014), pp. 372-379 [9] ChangHyun Jun, Jaehyeon Kang, Suyong Yeon, Hyunga Choi,, Nakju Lett Doh. Adjusting Point Cloud and Extracting Structure of Indoor Environment for 3D Indoor Mapping. Journal of the Institute of Electronics Engineers of Korea, (2013), pp.694-696. [10] Hyukdoo Choi, Euntae Kim, SLAM based on Probability technology trend analysis, KROS : Korea Robotics Society, Vol. 7 No. 3, (2010), pp. 27-34 [11] https://www.icp.org/ (2016) [12] http://sparpoint.blogspot.kr/2015/10/blog-post_40.html, Oct-20 (2015) [13] https://opensource.googleblog.com/2016/10/introducing-cartographer.html, Oct-5 (2016) [14] http://leica-geosystems.com/products/mobile-sensor-platforms/capture-platforms/leica-pegasus-backpack (2016) 430 Copyright 2016 HSST