(Special Paper) 23 1, 2018 1 (JBE Vol. 23, No. 1, January 2018) https://doi.org/10.5909/jbe.2018.23.1.36 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) VR AR a) Spatio-temporal Data Visualization Survey for VR and AR Environment Hyunjoo Song a) (Virtual Reality) (Augmented Reality). (Internet of Things). VR AR,.,. VR AR. Abstract VR(Virtual Reality) and AR(Augmented Reality) devices are becoming more common, and the need for proper contents presentation techniques in such environments has been growing ever since the popularization of the devices. One of the contents is the spatio-temporal data, which has become more prominent since it could be both generated and consumed by a large number of ordinary users. In this work, the researcher analyzed the characteristics of spatio-temporal data as a source for visualization in VR and AR environment, and categorized prior visualization methods for such data, which were devised for traditional monitors. The researcher also reviewed the hardware specification of state-of-the-art devices, and examined the possibility of adopting the previous visualization approaches. This work is expected to contribute in designing spatio-temporal visualization for VR and AR environment by utilizing their unique characteristics. Keyword : Virtual Reality, Augmented Reality, Spatio-temporal Data, Information Visualization a) (Department of Digital Media, Duksung Women s University) Corresponding Author : (Hyunjoo Song) E-mail: hjsong0001@duksung.ac.kr Tel: +82-2-901-8767 ORCID: http://orcid.org/0000-0002-4931-2940 2017 ( ) (No. NRF-2017R1C1B5017772). Manuscript received November 13, 2017; Revised December 20, 2017; Accepted December 20, 2017. Copyright 2017 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND (http://creativecommons.org/licenses/by-nc-nd/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.
: VR AR (Hyunjoo Song: Spatio-temporal Data Visualization Survey for VR and AR Environment). (VR: Virtual Reality) (AR: Augmented Reality),, [1]., [2], (LAE: Live Actor and Entity) [3].,, (HMD: Head Mounted Display) [4].,.,., [1]. VR AR.,, VR AR., VR AR... VR AR, ( 1). 1.,. (e.g., GPS). (e.g.,, ) (Quantified Self [5] ). 1. Table 1. Classification of spatio-temporal data Spatial Physical location Virtual location Temporal Regularly measured time points Irregularly measured time points Fixed Flexible Fixed Flexible Weather data, Urban traffic flow Spatially aggregated SNS feed Vehicle trajectories GPS logged photos Flow simulation data Movement data in virtual space - -
1.1., [6].,.,,,,.,., ( 1a).. IoT(Internet of Things),.,, GPS,. ( 1b). 1.2,.,,..,.. (Social Network Service), GPS, ip.,. 2.,. 1. (a) (b) Fig. 1. Two types of spatio-temporal data with different acquisition methods (a) acquire data from multiple pre-defined fixed locations for multiple time points (b) acquire data from a trajectory of a certain object
: VR AR (Hyunjoo Song: Spatio-temporal Data Visualization Survey for VR and AR Environment), 3.,..,.,.., (parallel coordinates) (scatterplot matrix). VR AR.., [6][8] ( 2)., (occlusion).. small multiples. (occlusion),. 2 3, 2 (e.g.,, ), (e.g., 1970, 1980 ). 1. Small multiples Tufte small multiples [7],., 2., ( [9]) Fig. 2. Spatio-temporal data visualization by placing temporal visualization on a corresponding spatial location (Map of Seoul[9])
3. small multiples Fig. 3. Small multiples visualization with identical axis configuration for spatio-temporal data 2. - (Space-time cube) Space-time cube [10][11] 2, x y, z.,, 2 [12]. time flattening, Minard [13] 4. cube,, small multiples. 3 3 (rendering) [14]. 3 (occlusion), space-time cube 3 3. 3.. - 3 (rendering) 2,. 3,, [15].. 4. Time flattening Fig. 4. Illustration of Napoleon s march to Moscow by time flattening
송현주: VR 및 AR 환경에서의 시공간 데이터 시각화를 위한 동향 분석 (Hyunjoo Song: Spatio-temporal Data Visualization Survey for VR and AR Environment) 탐색 및 분석을 하는 도중 맥락을 잃을 수 있기에 다수의 뷰에 존재하는 데이터들에 대해서 직관적인 매칭을 위하여 사용자가 특정 뷰에서 상호작용(brushing)한 결과가 다른 뷰에서도 연동(linking)되어 나타내어지는 brushing and linking 기반의 상호작용을 제공하고 있다. [16] 품 (표 3), 이 두 가지 AR 제 의 경우에도 화면 해상도는 각각 1268 x 720, 1280 x 1440으로 일반 PC 모니터에 비해 역 영 을 갖고 있다. 번 장에서는 VR 및 AR기기의 동향을 시각화 관점에 이 서 정리하였다. VR의 경우 CAVE(Cave Automatic Virtual 처럼 다수의 프로젝터와 벽으로 만들어진 Environment)[17] 방 형태의 장비를 사용할 수도 있으나 본 연구에서는 일반 사용자가 상대적으로 쉽게 접할 수 있는 HMD 형태의 장비 만 고려하였다. 표 2는 HMD 형태로 시중에서 장비들을 정리한 결과이다. 판매 중인 거치형이 아닌 HMD 형태의 장비는 일반 PC 모니터에 비해 낮은 해상도를 갖고 있음을 확인할 수 있다. 휴대기기를 연결하여 사용하는 Google이나 삼성 제품의 경우에도 현재 최고 사양의 스마트폰 디스플레이 해상도가 1440 x 2960이므로 양안에 나누어 배치할 경우 1440 x 1480이 되어, 이 역시 일반 PC 디스플레이에 비교 했을 때 상대적으로 시각화에 사용 가능한 면적이 좁다. 롤 위해서 사용가능한 컨트 러들을 살펴보면, VR 기기의 경 까 품 Metavision사의 Meta 지의 2가지의 상용 제 이 있는데 별도의 원격 조정 장치를 사용했으며, AR 기기들은 공통적으로 사용자의 손동작을 사용하였다. 이 롤 외에 컨트롤러로는 HoloLens의 음성 인식 컨트 이 있으나, 장비에 고성능의 마이크를 내장하고 있 변 소음에 영향을 받을 수 있다는 한계 점을 갖고 있다. 아울러 VR과 AR 기기 모두 동작 감지 장 비(motion tracker)를 사용하여 사용자의 머리 움직임을 측 정하고 있었으며, 여기서 나오는 정보를 바탕으로 보고 있 어야 한다는 점과 주 표 3. 시각화를 위한 AR 기기의 사양 Table 3. Specification of AR devices for visualization Microsoft HoloLens Meta2 Headset type Mobile Tethered Resolution (per eye) 1268 x 720 1280 x 1440 Controls Voice, Hand gesture, Handheld remote Hand gesture 위의 표를 살펴보면 한편, AR 기기의 경우 현재 Microsoft사의 HoloLens와 Appearance 표 2. 시각화를 위한 VR 기기의 사양 Table 2. Specification of VR devices for visualization PlayStation VR HTC Vive Oculus Rift Google Daydream View Samsung Gear VR Tethered Tethered Tethered Mobile Mobile 960 x 1080 1080 x 1200 1080 x 1200 Native to phone DualShock 4, PlayStation Move HTC Vive motion controllers Oculus Touch, Xbox One gamepad Handheld remote Native to phone Handheld remote, touchpad on headset Appearance Headset type Resolution (per eye) Controls 좁은 한편, 시각화에서 한 축을 담당하고 있는 상호작용[18]을 우 주로 손으로 들고서 사용하는 Ⅳ. 시각화 관점에서의 VR 및 AR 환경 41
., [19].. VR AR 1. VR (dimension reduction). 1 2, 2. 3, (visual clutter) (occlusion),. VR 3 [20][21], - (node-link graph) VR (stereo graphic). 3 3 2 3. 3 3 2. VR 3 3. (CT) VR [22]. 2. Small multiples,, VR AR. [20],. 3 (immersive),.. 2. AR AR VR. AR 2 VR.., AR,.,.
: VR AR (Hyunjoo Song: Spatio-temporal Data Visualization Survey for VR and AR Environment) 3. VR AR VR.. AR,... VR AR,. VR AR. small multiples space-time cube,. [22]. VR AR., 1. [6]. 2 VR,. AR,. Yelp [23] Monocle. (References) [1] E. Olshannikova, A. Ometov, Y. Koucheryavy, and T. Olsson, Visualizing big data with augmented and virtual reality: challenges and research agenda, Journal of Big Data, Vol. 2, No. 1, pp.22, October 2015. [2] I. Kim, C. Eem, and H. Hong, Hierarchical subdivision of light distribution model for realistic shadow generation in augmented reality, Journal of Broadcast Engineering, Vol. 21, No. 1, pp.24-35, January 2016. [3] K.-H. Yoo, Standard model for live actor and entity representation in mixed and augmented reality, Journal of Broadcast Engineering, Vol. 21, No. 2, pp.192-199, March 2016. [4] E. Kim, J. Kim, E. Yoo, and T. Park, Study on virtual reality (VR) operating system prototype, Journal of Broadcast Engineering, Vol. 22, No. 1, pp.87-94, January 2017. [5] Quantified self, http://quantifiedself.com (accessed Nov. 10, 2017). [6] G. Sun, Y. Liu, W. Wu, R. Liang, and H. Qu, Embedding temporal display into maps for occlusion-free visualization of spatio-temporal data, Proceedings of IEEE Pacific Visualization Symposium, Yokohama, Japan, March 2014. [7] E. R. Tufte, Envisioning information, Optometry and Vision Science, Vol. 68, No. 4, pp.322-324, April 1991. [8] N. Andrienko and G. Andrienko, Interactive visual tools to explore spatio-temporal variation, Proceedings of the working conference on Advanced Visual Interfaces, Gallipoli, Italy, pp.417-420, 2004. [9] Map of Seoul, https://commons.wikimedia.org/wiki/file:image-map_ Seoul-teukbyeolsi-big.png (accessed Nov. 10, 2017). [10] N. Andrienko and G. Andrienko, Visual data exploration using space-time cube, Proceedings of International Cartographic Conference, Durban, South Africa, pp.1981-1983, 2003. [11] A. M. MacEachren, How Maps Work: Representation, Visualization,
and Design, The Guilford Press, New York, USA, pp.252-254, 1995. [12] B. Bach, P. Dragicevic, D. Archambault, C. Hurter, and S. Carpendale, A review of temporal data visualizations based on space-time cube operations, Eurographics Conference on Visualization, Swansea, Wales, United Kingdom, 2014. [13] E. R. Tufte, The visual display of quantitative information, Graphics Press, Cheshire, USA, 1986. [14] P. Gatalsky, N. Andrienko, and G. Andrienko, Interactive analysis of event data using space-time cube, Proceedings of the Information Visualization, Washington DC, USA, pp.145-152, 2004. [15] K. Kurzhals, F. Heimerl, and D. Weiskopf, ISeeCube: visual analysis of gaze data for video, Proceedings of the Symposium on Eye Tracking Research and Applications, Safety Harbor, Florida, USA, pp.43-50, 2014. [16] D. A. Keim, Information visualization and visual data mining, IEEE Transactions on Visualization and Computer Graphics, Vol. 8, No. 1, pp.1-8, January 2002. [17] C. Cruz-Neira, D. J. Sandin, T. A. DeFanti, R. V. Kenyon, and J. C. Hart, The CAVE: audio visual experience automatic virtual environment, Communications of the ACM, Vol. 35, No. 6, pp.64-72, June 1992. [18] J. S. Yi, Y. Kang, and J. T. Stasko, Toward a deeper understanding of the role of interaction in information visualization, IEEE Transactions on Visualization and Computer Graphics, Vol. 13, No. 6, pp.1224-1231 November 2007. [19] E. M. Kolasinski, Simulator sickness in virtual environments, U.S. Army Research Institute for the Behavioral and Social Sciences, Alexandria, USA, 1995. [20] O. Kwon, C. Muelder, K. Lee, and K.-L. Ma, A study of layout, rendering, and interaction methods for immersive graph visualization, IEEE Transactions on Visualization and Computer Graphics, Vol. 22, No. 7, pp.1802-1815, July 2016. [21] F. Barahimi, and S. Wismath, 3D graph visualization with the oculus rift, Proceedings of International Symposium on Graph Drawing, Würzburg, Germany, pp.519-520, 2014. [22] Q. Lin,Z. Xu, B. Li, R. Baucom, B. Poulose, B. A. Landman, and R. E. Bodenheimer, Immersive virtual reality for visualization of abdominal CT, Proceedings of SPIE, 8673, 2013, http://doi.org/10.1117/ 12.2008050 (accessed Nov. 10, 2017). [23] Yelp, https://www.yelp.com (accessed Nov. 10, 2017). - 2009 8 : ( ) - 2011 8 : (, HCI ) - 2016 2 : (, HCI ) - 2017 3 ~ : - ORCID : http://orcid.org/0000-0002-4931-2940 - : -,,,