Copyright of figures and other materials in the paper belongs original authors. Using Visual and Spatial Cues to Represent Social Contacts in AR (Augmented Reality) Alaeddin Nassani et al. Siggraph Asia Symposium on Mobile Graphics & Interactive Application, 2017 Presented by Min-Gyu Kim 2019. 01. 03
VR? AR? VR (Virtual Reality) : experiencing fake world as real world HMD (Head Mounted Display) AR (Augmented reality) Min-Gyu Kim 2019. 01. 03 # 2
VR? AR? Entertainment Game Movie, 360 Video Travel Exhibition Education Design, Study Simulation Medicine Operation Education Simulation Min-Gyu Kim 2019. 01. 03 # 3
Abstract AR(Augmented Reality) System 사용성증가 일반적인 VR System 이가지는한계와복잡성이주원인 AR System : Mobile VR, Google Glass, Microsoft HoloLens Min-Gyu Kim 2019. 01. 03 # 4
Abstract Social System 중심의연구부재와위치기반시각화중심연구 체험하는사용자의사회적관계를고려 X 사용자중심 Social System 설계및 AR 기반 Social Contacts 제공 Social Contacts : 개인및그룹간의사회적상호작용 Min-Gyu Kim 2019. 01. 03 # 5
1. Introduction (1/4) AR(Augmented Reality) 을체험할수있는시스템개발및상용화준비 Microsoft, Facebook, Apple, Samsung? VR 체험하기위한 HMD 수요, 공간적문제로인한 AR 시스템접근이편리 소셜네트워크연결을통해사용자간정보공유및상호작용연구진행 Facebook s Social VR (Oculus Connect 2016) Min-Gyu Kim 2019. 01. 03 # 6
Facebook s Social VR Min-Gyu Kim 2019. 01. 03 # 7
1. Introduction (2/4) HMD 를착용하는것이아닌간단한착용식 AR 디스플레이를활용해 Social VR 을체험하도록연구가진행 이전엔모바일, 휴대용 AR 시스템을사용하여소셜네트워크를사용 Presslite s Twitter 360 [Presslite, 2009] Junaio Glue image recognition technology for augmented reality AR [Junaio 2, 2010] Min-Gyu Kim 2019. 01. 03 # 8
1. Introduction (3/4) Live Video Streaming, 원격협업시스템등 AR 시스템기술사용 Augmented reality annotation for Social video sharing. [Alaeddin Nassani (University of Canterbury) et al. SA, 2016.] Collaborative Augmented Reality. [Mark Billinghurst(University of Washington) and Hirokazu Kato. Communication of ths ACM, 2002.] 모바일을활용한 AR 시스템의한계점은단순한텍스쳐, 이미지전달인터페이스와사용자의시야를고려 X Min-Gyu Kim 2019. 01. 03 # 9
1. Introduction (4/4) 실제공간과사물을활용한 Social AR 협업상호작용시스템 Holoportation: Virtual 3D Teleportation in Real-Time [Sergio Orts-Escolano (Microsoft Research) et al. UIST, 2016.] 이러한시스템과연구들의한계는한정된인원및같은환경 ( 카메라배치, AR 디스플레이 ) 구성 Min-Gyu Kim 2019. 01. 03 # 10
Contribution 사용자의 Social Network 정보를 AR 환경에적용한새로운시스템 사용자의소셜네트워크에서많은양의연락처및사용자정보들을시각적으로표시및관리하기위한연구가필요 Microsoft HoloLens 와같은 wearable AR 디스플레이를착용해사용자 Social Network 정보를항상표현하고상호작용가능 Education, Simulation, Exhibition, Operation 에활용 Min-Gyu Kim 2019. 01. 03 # 11
About Microsoft Hololens 현실화면에실제객체의스캔된 3D 이미지를출력및조작가능한혼합현실 (Mixed Reality, MR) 시스템또는장비 완전한가상화면을보여주는가상현실 (VR) 이나실제화면에덧씌우는증강현실 (AR) 과는다름 Min-Gyu Kim 2019. 01. 03 # 12
About Microsoft Hololens Min-Gyu Kim 2019. 01. 03 # 13
Related Work
2. Related Work 2.1 사용자위치기반 Social Network 연구 가상회의및협업을위한 Remote Proxemics 연구 같은공간이아닌사용자와협업을위해 Eery Space 사용 환경및사용자간상호작용하기위해 Social bubbles 시스템을적용 Remote Proxemics. [Mauricio Sousa et al. Collaboration Meets Interactive Spaces, 2016.] Min-Gyu Kim 2019. 01. 03 # 15
2. Related Work 2.2 3D Avatar 를활용한 Social Network 연구 (1/2) 배경환경및사용자를나타내는 Avatar 표현에따른 Presence 연구 현실적인 3D Avatar 표현이협업및공동작업에효율을나타냄을확인 Effects of Avatar and Background Representation Forms to Co-Presence in Mixed Reality (MR) Tele-conference System. [Dongsik Jo et al. SIGGRAPH ASIA, 2016.] Effects of Virtual Human Appearance Fidelity on Emotion Contagion in Affective Inter-Personal Simulations. [Matias Volante et al. TVCG, 2016.] Min-Gyu Kim 2019. 01. 03 # 16
2. Related Work 2.2 3D Avatar 를활용한 Social Network 연구 (2/2) 다른사용자들과의 Social 환경위한 Face-to-Face 상호작용연구 기존의사용자를 2D 이미지로표현하는원격화상회의시스템인 Skype, Cisco TelePresence TX9000 와달리 3D 표현연구 Kinect 를활용하여다른사용자를 3D 표현 Immersive 3D telepresence. [Henry Fuchs, at al. Computer, 2014.] 이외 High Fidelity, Itsme3D 등의기업들은사용자들이 3D Virtual Avatar 로표현되는 Social VR 환경을구축중 Min-Gyu Kim 2019. 01. 03 # 17
2. Related Work 2.3 AR Interface 연구 (1/2) AR 환경에서이미지, 객체등많은정보를관리하는방법이필요 Information filtering for mobile augmented reality. [Simon Julier et al. ISAR, 2002.] User interface management techniques for collaborative mobile augmented reality. [Tobias Hollere et al. Computer & Graphics, 2001.] Min-Gyu Kim 2019. 01. 03 # 18
2. Related Work 2.3 AR Interface 연구 (2/2) AR 환경에서주로사용되는 UI 인 Label 정보를관리하는연구 Image-driven view management for augmented reality browsers. [Raphael Grasset et al. ISMAR(International Symposium on Mixed and Augmented Reality), 2012.] L : Label, x : Screen position, α : Weight, P : Penalty factor Min-Gyu Kim 2019. 01. 03 # 19
Filtering Using a Social Continuum & Implementation
3.1 Filtering Using a Social Continuum (1/2) 새로운 Social AR Interface 는시각적및가까운단서를사용해사용자의 Social Network 정보를표시 Visual fidelity 사용자의친밀한가족및친구는현실적으로보여줄수있음 Proximity 현실적으로표현하는것이 AR, VR 환경에서몰입, 효율을높임 사회적환경에서사람간의거리는친밀감수준에따라다름을활용 Collaboration Meets Interactive Spaces. [Craig Anslow et al. Collaboration Meets Interactive Spaces 1st, 2017.] Visual fidelity + Proximity 를통해사용자주변의실제세계에사람들의가상표현을배치 Min-Gyu Kim 2019. 01. 03 # 21
3.1 Filtering Using a Social Continuum (2/2) 원하는사람에게효율적으로정보공유와관심을기울이기쉬움 실시간으로변화되는가상의상황과협업에적합 Figure 2 : Proxemic & Visual Fidelity Filtering of Avatars Min-Gyu Kim 2019. 01. 03 # 22
3.2 Implementation (1/3) Social Continuum metaphor 를적용한 AR 환경에대한실험및분석을위해 HoloLens 의프로토타입을개발 Unity 3D, HoloToolkit-Unity 를활용해환경구축, Avatar 는 Morph3D 사용 Unity3D + HoloToolkit SDK Morph3D Min-Gyu Kim 2019. 01. 03 # 23
3.2 Implementation (2/3) 프로토타입은 HoloLens Spatial Mapping 기능을사용해사용자의초기위치를중심으로동심원 (Circle Manager) 을배치 Circle 에서사용자의 Social Network 정보가 3D Avatar, 2D Image, Bust image, emoji 로표시 (Friend Manager) HoloLens Spatial Mapping Min-Gyu Kim 2019. 01. 03 # 24
3.2 Implementation (2/3) Min-Gyu Kim 2019. 01. 03 # 25
3.2 Implementation (3/3) Social Network 를정렬하기위해사용자는손동작 ( 탭, 드래그 ) 를사용해 Avatar 를조작하여실시간으로업데이트가능하게함 이때각 Circle 에알맞은 Avatar 의모습으로변형 Figure 3 : System Components Min-Gyu Kim 2019. 01. 03 # 26
Focus Groups & Results
4.1 Focus Groups Focus Group 8~12 명으로구성된집단과깊이있는상호작용적인인터뷰를수반하는평가기법 해당조직이나그룹에관련되어있는사람들이나논의되는주제에관심있는사람들로구성 본연구에서 11 명의참가자와함께두개의포커스그룹으로진행 첫번째그룹 : AR / VR 연구를담당하는 6 명대학원생 두번째그룹 : AR / VR 경험이없는그래픽디자이너및 UX 디자이너 실험은 User participatory design 과 Usability Test 로 2 번진행 Min-Gyu Kim 2019. 01. 03 # 28
4.1 Focus Groups 4.1.1 User Participatory Design (1/2) Focus Group 은 AR 환경에서 Social Network 를시각화하는방법에대한토론및 Brainstorming 세션으로시작 Brainstorming : 창의적인아이디어생산을위한학습도구및회의기법 프로토타입시스템을시연하지않고, 다음의 3 가지에대한질문을통해피드백을얻기로함 T1 : Imagine the future of social networks in AR T2 : Map social group in terms of physical distance T3 : Map social groups in terms of visual fidelity Min-Gyu Kim 2019. 01. 03 # 29
4.1 Focus Groups 4.1.1 User Participatory Design (1/2) Task 1 : 참가자들에게 AR 에서 Social Network 를어떻게표현할것인지, 설명및그림으로표현후그룹에서발표및피드백교환 Task 2 : 사용자의현재위치에서물리적인거리측면으로네개의집단을놓아보도록지시 Intimate, Friend, Acquaintance, Stranger Task 3 : 참가자들에게 4 가지다른종류의 Visual fidelity 를 Social groups 으로대체하도록함 Visual fidelity : 3D avatar, 2D image, Busy image, Emoji Social Groups : Intimate, Friend, Acquaintance, Stranger Min-Gyu Kim 2019. 01. 03 # 30
4.1 Focus Groups 4.1.2 Usability Test (1/3) 프토로타입구현을시연하고참가자들에게다음네가지설문진행 Baseline (B) Proximity (P) the Intimate group was the closest, then Friends, then Acquaintance, then Strangers Visual Fidelity (F) Combined (C) Figure 4 : Four conditions of representing social contacts as seen through HoloLens; Baseline(B): same distance and visual, Proximity(P): different distance, Visual Fidelity(V): different visual, and Combined(C): different distance and visual Min-Gyu Kim 2019. 01. 03 # 31
4.1 Focus Groups 4.1.2 Usability Test (2/3) 참가자는각조건을무작위로체험하고각조건에대하여 System Usability Scale (SUS) 설문진행 Baseline (B) Proximity (P) Visual Fidelity (F) Combined (C) SUS : Usability 에대한주관적인평가를쉽게확인하기위해 10 가지항목으로설문지를구성 개별항목에대한점수는의미없고 SUS 합계에의미부여 SUS-A quick and dirty usability scale. [John Brooke. Usability evaluation in industry 189, 1996.] Min-Gyu Kim 2019. 01. 03 # 32
4.1 Focus Groups 4.1.2 Usability Test (3/3) 마지막으로각참가자에게주관적설문진행 Likert scale of 1 to 7 (1=Not very natural/easy and 7=Very natural/easy ) SQ 1 : How natural was the mapping of proximity to social relationship? SQ 2 : How natural was the mapping of visual fidelity to social relationship? SQ 3 : How easy was it to distinguish between the different avatar types? Min-Gyu Kim 2019. 01. 03 # 33
4.2 Results 4.2.1 User Participatory Design (1/3) 11 명의참가자 four female, aged between 16 and 41 years old, M=29, SD=5.89 대부분매일 Social Network ( 페북, 잉스타, Snapchat) 사용 (85%) 월간기준으로 AR / VR 헤드셋 (HoloLens, HTC Vive) 사용 (60%) Task 1 번에서 AR 플랫폼에서 Social Network 표현에대하여크게반복되는두가지의주제가존재함 Virtual Avatars : 공간적단서를활용해주변의친구를표현 사용자는다른 Avatar 와상호작용을통해정보공유및 Communication 가능 Miniatures : 사용자중심으로미니어처 Avatar( 구또는 Bubble) 를표시 Min-Gyu Kim 2019. 01. 03 # 34
4.2 Results 4.2.1 User Participatory Design (2/3) Task 2 번에서사용자의현재위치에서물리적인거리측면으로네개의집단을놓아보도록지시에대한결과 해당결과는사용자들이친밀감과 Proximity 을연관시켰다는것을확인 Figure 5 : Visual fidelity categorization for social contacts Min-Gyu Kim 2019. 01. 03 # 35
4.2 Results 4.2.1 User Participatory Design (3/3) Task 3 번인 4 가지다른종류의 Visual fidelity 를 Social groups 으로대체한결과 친밀도에비례하여 Visual 적으로표현하는사례의연구와비교적같음 Figure 6 : Proximity categorization for social contacts Min-Gyu Kim 2019. 01. 03 # 36
4.2 Results 4.2.2 Usability (1/2) SUS Score 결과 B(69), V(69), C(72) 로 Good, B(67) 로 OK Baseline (B) Proximity (P) Visual Fidelity (F) Combined (C) Proximity 와 Visual Fidelity 조건을결합한 Combined 조건이비교적사용하기에편리함을확인 Min-Gyu Kim 2019. 01. 03 # 37
4.2 Results 4.2.2 Usability (2/2) 주관적설문지 SQ1, SQ2, SQ3 각각에대하여 Friedman Test 진행 Friedman Test : 비모수통계기법, 대응되는두집단이상의순위비교 결과적으로유의미한차이가있음을확인 a : 행의수 b : 열의수 R j : j 번째열의순위합 SQ 1: X 2 = 18.407, p < 0.001 SQ 2: X 2 = 21.194, p < 0.001 SQ 3: X 2 = 20.967, p < 0.001 Figure 7 : Subjective questions results by condition by question. Min-Gyu Kim 2019. 01. 03 # 38
Discussion & Conclusion
5. Discussion (1/2) 사용자연구결과 AR 환경에서필터가없는것보다 Social 사용자를표현해줄필터를선호함을확인 순위결과를기반으로가장선호되는 Visual Fidelity 및 Combined 이후 Proximity, Baseline 순 참가자들은 Visual Fidelity 가 Avatar 를구별하는데가장쉽다고판단 Figure 8 : Ranking (4=highest, 1=lowest)) Whiskers indicate standard error. Min-Gyu Kim 2019. 01. 03 # 40
5. Discussion (2/2) 실험을진행한몇참가자의공통된의견에대한 User Interface 개선 Avatar 를구별하기힘들며, 같은거리에존재하는 Avatar 에대한정보까지둘러봐야되는문제가존재 Circle 거리가너무좁아 Avatar 에대한거리감이오히려부작용 표현되는 Avatar 에대한사용자정보가너무부족 전반적으로사용자가자신과의관계에따라 Social Network 필터링하는것을선호 Min-Gyu Kim 2019. 01. 03 # 41
6. Conclusion 본연구는착용가능한 AR 인터페이스에서이전의시각적정보와위치정보만제공하는것이아닌사용자의 Social Network 를표현 Social Network 의사용자정보들을 Avatar 로표현 포커스그룹에서다양한방법에대한피드백받아표현방법을고려 HoloLens 디스플레이를통해 AR Interface 구현및실험분석진행 사용자 Social Network 정보표현은향후상호작용연구로발전되어 AR 과 VR 을결합한 Social VR 환경에유용하게사용됨을확인 Min-Gyu Kim 2019. 01. 03 # 42
7. Limitations 본연구에서제안하는 AR 환경을제공하기위해 HoloLens 를사용 사용된모델은개발자모델 (335 만원이상 ) 실제사용자모델을사용하는것이아닌가상의 Avatar 를사용 3D 캐릭터와실제캡쳐된사용자모델을활용한 VR 연구에서캡쳐된사용자모델을선호 Min-Gyu Kim 2019. 01. 03 # 43
Appendix 1 Overlap on the importance map Overlap on the edge map Leader line length Min-Gyu Kim 2019. 01. 03 # 44
Appendix 2 비모수통계방법을사용하는경우 Transformation으로해결이안될때 표본수가너무적어분포를알수없는경우 집단들의표본수가서로크게다를경우 변수가명칭혹은순서척도일때 중앙값의비교가목적 최소한의가정. 즉등분산성, 정규분포등의가정을만족못할때 Min-Gyu Kim 2019. 01. 03 # 45
Appendix 3 Min-Gyu Kim 2019. 01. 03 # 46