증강현실에서사실적인그림자생성을위한조명분포모델의계층적분할 Hierarchical Subdivision of Light Distribution Model for Realistic Shadow Generation in Augmented Reality 저자 (Authors) 김익수, 임창경, 홍현기 Iksu Kim, Changkyoung Eem, Hyunki Hong 출처 (Source) 방송공학회논문지 21(1), 2016.1, 24-35 (12 pages) JOURNAL OF BROADCAST ENGINEERING 21(1), 2016.1, 24-35 (12 pages) 발행처 (Publisher) 한국방송 미디어공학회 The Korean Society Of Broad Engineers URL http://www.dbpia.co.kr/article/node06598265 APA Style 김익수, 임창경, 홍현기 (2016). 증강현실에서사실적인그림자생성을위한조명분포모델의계층적분할. 방송공학회논문지, 21(1), 24-35. 이용정보 (Accessed) 중앙대학교서울캠퍼스 165.194.69.210 2016/04/12 16:50 (KST) 저작권안내 DBpia 에서제공되는모든저작물의저작권은원저작자에게있으며, 누리미디어는각저작물의내용을보증하거나책임을지지않습니다. 이자료를원저작자와의협의없이무단게재할경우, 저작권법및관련법령에따라민, 형사상의책임을질수있습니다. Copyright Information The copyright of all works provided by DBpia belongs to the original author(s). Nurimedia is not responsible for contents of each work. Nor does it guarantee the contents. You might take civil and criminal liabilities according to copyright and other relevant laws if you publish the contents without consultation with the original author(s).
(JBE Vol. 21, No. 1, January 2016) (Special Paper) 21 1, 2016 1 (JBE Vol. 21, No. 1, January 2016) http://dx.doi.org/10.5909/jbe.2016.21.1.24 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) a), b), b) Hierarchical Subdivision of Light Distribution Model for Realistic Shadow Generation in Augmented Reality Iksu Kim a), Changkyoung Eem b), and Hyunki Hong b).,,. 3D (geodesic dome)... (gradient ray) (ratio). Abstract By estimating environment light distribution, we can generate realistic shadow images in AR(augmented reality). When we estimate light distribution without sensing equipment, environment light model, geometry of virtual object, and surface reflection property are needed. Previous study using 3D marker builds surrounding light environment with a geodesic dome model and analyzes shadow images. Because this method employs candidate shadow maps in initial scene setup, however, it is difficult to estimate precise light information. This paper presents a novel light estimation method based on hierarchical light distribution model subdivision. By using an overlapping area ratio of the segmented shadow and candidate shadow map, we can make hierarchical subdivision of light geodesic dome. Keyword : augmented reality, illumination distribution estimation, shadow segmentation, hierarchical subdivision a) (GSAIM, Chung-Ang University) b) (School of Integrative Engineering, Chung-Ang University) Corresponding Author : (Hyunki Hong) E-mail: honghk@cau.ac.kr Tel: +82-2-820-5417 ORCID: http://orcid.org/0000-0002-0815-9492. [2014(10045289), ] Manuscript received November 16, 2015; Revised January 4, 2016; Accepted January 4, 2016.
2 : (Iksu Kim et al.: Hierarchical Subdivision of Light Distribution Model for Realistic Shadow Generation in Augmented Reality). (AR: augmented reality) CG [1]... (environment mapping) [2].. (omni-directional) [3]. (geodesic dome) [4]. 3D. (gradient ray) 1. Fig. 1. Proposed system
(JBE Vol. 21, No. 1, January 2016).,... 1. 3D.. 1..., Agusanto (probe) [5]. (mirror ball) 3D [6]. [7]. Zhang [8], Feng [9]. Okatani, [10]. Peter,. [11]. [12]. 2.. Frahm (markerless). 3 [13]. Haller [14]. Sato (linear least squares algorithm) [4]. 3D [15]. 3. (Canny) (edge), [16].
2 : (Iksu Kim et al.: Hierarchical Subdivision of Light Distribution Model for Realistic Shadow Generation in Augmented Reality) 2. [15] Fig. 2 Light sources on virtual hemisphere. 3D,. < 3>. (a) 3D (b). (d) (e)., (projection) (e). 3D (c). (e). (f). 3. (a):, (b):, (c):, (d):, (e):, (f): [16] Fig. 3 (a): Input image, (b): mask image for shadow surfaces, (c): candidate shadow map (d): edge image, (e): segmented regions with gradient-based ray, (f): rendering images by estimated light sources
(JBE Vol. 21, No. 1, January 2016).,.,.,.. 3D. 4 40 3D. 40 4. 40, x-y( ); 3( ) Fig. 4. 40 polygons on Geodesic dome, x-y plane(left); three-dimensional space(right) 5., C1( ) C2( ) Fig 5. Average accuracy of threshold, C1(left) C2(right)
2 : (Iksu Kim et al.: Hierarchical Subdivision of Light Distribution Model for Realistic Shadow Generation in Augmented Reality). (1). C1 C2., C1 C2.. 5%. (1) Mi, N. O S. 5 C1 C2 (ground truth).. 1 8. 6. Fig. 6. Hierarchical subdivision of triangle Mi O C1. S C2.., 4. 4 16, 64. 6. [17]. 7 (mapping) 7. Fig. 7. Subdivided and new point generating method by projection
(JBE Vol. 21, No. 1, January 2016) 1. Table 1. Pseudo code for hierarchical subdivision of light distribution Pseudo code Initialize T, Ii (i = 1,,,,,40) funtion DetectLight (Ii) // Detect light sources funtion ComputeRatio (IN) // Compute relative area ratio of detected light sources of geodesic dome. While(C1 < 0.95 && C2 > 0.05) // Check accuracy of shadow regions and shadow maps. { } For each light sources, Subdivide Mi // Subdivide Mi light source into four region. About Subdivided Mij (j = 1... 4) funtion DetectLight (Mij) // Repeat funtion DetectLight Determine light sources. void funtion DetectLight (light 1,...,N) // function detecting light { while (S > 0) // Continue this process until there remains no shadow regions. { For each i th light source (i = 1,...,N) { Compute 1. intersection region Oi of S and Mi 2. relative area ratio of i th light source } Choose m th lights source with the maximum area ratio value among Ri. } } For each i th light source (i = 1,...,N) { Mi = Mi Om // Remove shadow by detected light source from the candidate shadow map. } S = S Om // Remove shadow by detected light source from the shadow. void function ComputeRatio(light 1,...,N) // Compute C1 and C2. { } Detected light sources Li(i = 1... N) // N : the number of detected light sources. 1. Compute error ratio between Oi and union Mi 2. Compute error ratio between S-O and union S Compute relative area ratio C1, C2, // O is union set of all Oi
2 : (Iksu Kim et al.: Hierarchical Subdivision of Light Distribution Model for Realistic Shadow Generation in Augmented Reality) < 8> Fig. 8 Diagram of proposed system
(JBE Vol. 21, No. 1, January 2016). InTel Core i7 3.40GHz CPU MS LiteCam Cinema HD.. 1, 2, 3. 1 9. (a):, (b):, (c): (ground truth), (d): 40, (e): 160, (f): Fig. 9 (a): Input image, (b): segmented regions with gradient-based ray, (c): ground truth, (d): 40 candidate lights, (e): 160 candidate lights, (f): candidate lights using Hierarchical subdivision
2 : (Iksu Kim et al.: Hierarchical Subdivision of Light Distribution Model for Realistic Shadow Generation in Augmented Reality) 40, 160.. 3D (a) (b) 9. (c). 40 160 (d) (e). (f), (ground truth) (c) 10., (a):, (b): Fig. 10. Various object rendering, (a): Input image, (b): result image 2 Table 2 Comparison of the previous and proposed method Light Method computational time(ms) method of candidate shadow maps in initial setup 40 lights 160 lights accuracy(%) computational time(ms) accuracy(%) computational time(ms) proposed method accuracy(%) light1 4,392 73.3 7,944 97.2 4,644 97.2 light2 4,646 79.4 8,246 91.1 5,202 95.1 light3 4,675 72.6 8,449 93.8 5,535 95.4
(JBE Vol. 21, No. 1, January 2016). 10 2........ (References) [1] R. Azuma, A survey of augmented reality, Presence: Teleoperators Virtual Environ. Vol. 6, No. 4, 355~385, 1997. [2] S. Gibson and A. Murta, "Interactive rendering with real-world illumination", Proc. of 11th Eurographics Workshop on Rendering, pp. 365-376, 2000. [3] W. Son and H. Hong, "Dynamic relighting in augmented reality for mobile game", Journal of Korean Society for Computer Game, Vol.2, No.23, pp. 159-164, 2010. [4] I. Satoi, Y. Sato, and K. Ikeuchi. "Illumination from shadows." IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, Vol. 3, pp: 290-300, 2003. [5] K. Agusanto et al., Photorealistic rendering for augmented reality using environment illumination, Proc. of IEEE Int. Symp. on Mixed and Augmented Reality, pp. 208-216, 2003. [6] P. Debevec, Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography, Proc. of ACM Siggraph, pp. 189-198, 1998. [7] I. Sato, Y. Sato, and K. Ikeuchi, Acquiring a radiance distribution to superimpose virtual objects onto a real scene, IEEE Trans. on Visualization and Computer Graphics, Vol.5, No.1, pp. 1-12, 1999. [8] Q. H. Jin-tao Ma, Ya Zhou and Y. Zhang, Marker cube and probe sphere based illuminant detection algorithm, Proc. of Int. Symp. Photoelectronic Detection and Imaging, 2007. [9] Y. Feng, Estimation of light source environment for illumination consistency of augmented reality, Proc. of IEEE the Congress on Image and Signal Processing, vol. 3. pp. 771-775, 2008. [10] T. Okatani and K. Deguchi, Estimation of illumination distribution using a specular sphere, Proc. of 15th Int. Conf. on Pattern Recognition, vol. 3, pp. 592-595, 2000. [11] S. Peter, S. Ines, and H. Michael, "Image Based Shadowing in Real-Time Augmented Reality," International Journal of Virtual Reality, Vol. 5, No.3, pp. 1-7, 2006. [12] Y. Jung, T. Kim, and H. Hong. "Moblie AR Image Rendering Method using Environmental Light Sources.", Korean Society For Computer Game, Vol. 2, No. 26, pp. 9-17, 2013. [13] J. M. Frahm, K. Koeser, D. Grest, and R. Koch, Markerless augmented reality with light source estimation for direct illumination, Proc. of Conf. on Visual Media Production, 2005. [14] M. Haller, S. Drab, and W. Hartmann, A real-time shadow approach for an augmented reality application using shadow volumes, Proc. of the ACM Symp. on Virtual Reality Software and Technology, pp. 56-65, 2003. [15] C. Eem, I. Kim, Y. Jung, H. Hong, "Estimating Illumination Distribution to Generate Realistic Shadows in Augmented Reality", Proc. of KSII Trans. Internet Inf. Syst, VOL. 9, NO. 6, pp. 2289-2301, Jun. 2015. [16] C. Eem I. Kim, H. Hong, "Using gradient-based ray and candidate shadow maps for environmental illumination distribution estimation." Optical Engineering, Vol. 54, No. 7 : 073112-073112, 2015. [17] S. Lee, J. Lee, "A Comparative Study of Genetic Algorithm and Mathematical Programming Technique applied in Design Optimization of Geodesic Dome", Proceeding of KASS Symposium, pp. 101-106, 2008
2 : (Iksu Kim et al.: Hierarchical Subdivision of Light Distribution Model for Realistic Shadow Generation in Augmented Reality) - 2014 : - 2014 : - ORCID : http://orcid.org/0000-0003-1219-3340 - :,, - 1999 : - 1995 2000 : R&D - 2000 2006 : - 2006 2009 : BZweb Technologies CTO - 2009 2011 : KAIST - 2013 : - ORCID : http://orcid.org/0000-0002-2209-6133 - :,, - 1998 : - 1998 1999 : - 2002 2003 : Post-doc., Univ. of Colorado at Denver - 2000 2013 : - 2014 : - ORCID : http://orcid.org/0000-0002-0815-9492 - :,,