(Regular Paper) 23 6, 2018 11 (JBE Vol. 23, No. 6, November 2018) https://doi.org/10.5909/jbe.2018.23.6.914 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) a), a), a), a) Mobile Augmented Reality Application for Early Childhood Language Education Sanghoon Kang a), Minwoo Shin a), Minji Kim a), and Hanhoon Park a). ( ), / ( ), ( ),. /,..,. Abstract In this paper, we implement an Android application for infant language education using marker-based augmented reality. Combining animal word markers (noun), size/color word markers (adjective), and action word markers (verb) in puzzle form to make a simple sentence, the application shows virtual contents related to the content of the sentence. For example, when an animal marker is showed up on a camera, the corresponding animal appears. Additionally, when the motion markers are combined, the animal's appearance changes into an animation in which it acts. When a user touched a marker, user can hear the sound of the word, which gives an auditory effect, and by adding the rotation function, user can see the animation in any direction. Our goal is to increase infants interest in learning language and also increase the effectiveness of education on the meaning of words and the structure of simple sentences, by encouraging them to actively participate in language learning through visual and auditory stimuli. Keyword : Augmented reality, marker-based, language education, Android application, Unity a) (Department of Electronic Engineering, Pukyong National University) Corresponding Author : (Hanhoon Park) E-mail: hanhoon_park@pknu.ac.kr Tel: +82-51-629-6225 ORCID: http://orcid.org/0000-0002-6968-4565 This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2018R1D1A1B07045650). Manuscript received September 14, 2018; Revised November 7, 2018; Accepted November 7, 2018. Copyright 2016 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND (http://creativecommons.org/licenses/by-nc-nd/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.
3 : (Sanghoon Kang et al.: Mobile Augmented Reality Application for Early Childhood Language Education).,,,,,, [1]., [2].,. (Language Acquisition Device: LAD). 5~6. [8]. [3]...,..,, [4].,,.., 4, 5, [5]. (,, ),... Unity.,,. [6]... Unity, PC, Unity Vuforia SDK. Vuforia SDK 3,
( 1 ). 1. Vuforia SDK - Fig. 1. Marker-based augment reality using Vuforia SDK,, ( 2 ). 1..,, [7].,. (3 ) ( ) ( 3 ). 2.,, Fig. 2. Examples of word markers (for adjective, noun, verb) used in the proposed application 3. Fig. 3. Combination of verb word cards and noun word cards
3 : (Sanghoon Kang et al.: Mobile Augmented Reality Application for Early Childhood Language Education). ( ) ( ).. ( 1 ). ( ) ( ) ( ).., 2. 2. ( ) (, ) 1. Pseudo code 1. Function for measuring the distance between two markers and changing the state of virtual contents accordingly function: model_selection measure Distance_run between Animal_card and Run_card; measure Distance_sleep between Animal_card and Sleep_card; measure Distance_talk between Animal_card and Talk_card; measure Distance_roll between Animal_card and Roll_card; if Distance_run < 2 inactivate Model_Base; activate Model_Run; else if Distance_sleep < 2 inactivate Model_Base; activate Model_Sleep; else if Distance_talk < 2 inactivate Model_Base; activate Model_Talk; else if Distance_roll < 2 inactivate Model_Base; activate Model_Roll; else activate Model_Base; inactivate Model_Run; inactivate Model_Sleep; inactivate Model_Talk; inactivate Model_Roll; 4. Fig. 4. Scaling the rendered cat model by adding/combining adjective word markers
(,, ),,. 3 ( + + ), ( 4 ).., Bool.., 2. Pseudo code 2. Function for scaling a rendered model function: scale_change // measuring the distances between markers measure Distance_Big between Animal_card and Big_card; measure Distance_Small between Animal_card and Small_card; if Distance_Big < 1.9 scale up Animal_model to 1.5; else if Distance_Small < 1.9 scale down Animal_model to 0.5; else return Animal_model to the original size; 5. Fig. 5. Changing the color of rendered models by adding/combining adjective word markers
3 : (Sanghoon Kang et al.: Mobile Augmented Reality Application for Early Childhood Language Education) X, Y, Z 1.5, 0.5. ( 2 ).,, ( 5 ) (,, ). 3.. 3. Unity. + + 3. Unity 2 1. 4.., + + ( 6 ). 3. Pseudo code 3. Function for coloring a rendered model function: color_change // measuring the distances between markers measure Distance_Red between Animal_card and Red_card; measure Distance_Blue between Animal_card and Blue_card; measure Distance_Yellow between Animal_card and Yellow_card; // changing the texture image to that with a different color in part if Distance_Red < 2 change Animal_color to red; else if Distance_Blue < 2 change Animal_color to blue; else if Distance_Yellow < 2 change Animal_color to yellow; else change Animal_color to the original; 6. Fig. 6. Markers created like jigsaw puzzle pieces
4.,.. Unity Raycast. Raycast (z- ) ray. ray ( 4 ). 5. Unity.,..,. 360 ( 7 ). 5. 4. Raycast Pseudo code 4. Function for sensing the touched object using Raycast to play sound function: sound_effect generate Ray from Touched_Point; if Ray is collided // sensing whether the ray collided with something switch(object) { // identifying the collided object and playing the related sound case Cat: Play(Souund_cat) break; case Dog: Play(Souund_dog) break;... } 7. Fig. 7. Rotating the rendered model by clicking on a button
3 : (Sanghoon Kang et al.: Mobile Augmented Reality Application for Early Childhood Language Education) 5. Pseudo code 5. Function for rotating a rendered model function: model_rotation // determine whether to rotate by clicking the button on/off if button is pushed rotatestate =!rotatestate; if rotatestate is true rotate Cat; // rotating the model in proportional to the elapsed time between frames (rotatespeed), 50f. rotatestate true false.. 10 (,,,,,,,,, ), 5 / (,,,, ), 7 (,,,,,, ) 350., 7 ( ), / ( 8 ).. + +... x -20. 9 8. Fig. 8. Making multiple models do the same action
. 25~30cm 30~45... 9. x Fig. 9. Rotating the rendered model around the x-axis for a better view 10.,,,. TV..,, 3D ( ),,,.,,.,, 350. (,, ),., 10. Fig. 10. Running our application on a smart phone and a tablet and testing its functionalities
3 : (Sanghoon Kang et al.: Mobile Augmented Reality Application for Early Childhood Language Education),. /,..., (, ).,.,,,,.. (References) [1] I. S. Lee, An Influence of Forest Experience Activity on Child's Language Expression and Nature-friendly Attitude, MS Thesis, Sungshin Women s University, p. 26, 2008. [2] J. H. Kim, The Effects of Story Making Through Traditional Folk Tales on Young Children's Language Expression and Pro-social Behaviors, MS Thesis, Keimyoung University, p. 16, 2007. [3] M. K. Kim, A Study on the Response of Young Children to Famous Picture Book, MS Thesis, Woosuk University, p. 1, 2001. [4] W. J. Lim, A Study on the Effects of Imagery Training Using Visual and Auditory Tool on Kindergarten Children s Ability of Sentence Understanding, MS Thesis, Woosuk University, p. 2, 2010. [5] S. I. Hong, S. I. Jeon, S. Pae, and I. Lee, Development of phonological awareness in Korean children, Communication Sciences & Disorders, vol. 7, no. 1, pp. 49-64, 2002. [6] Y. Choi, J. Kim, and H. Park, Face augmented reality using OpenCV library, Proceeding of KIPS Conference, pp. 48-49, 2017. [7] S. H. Jeong and K. B. Kyoung, Educational application of augmented reality contents, KCA Journal, vol. 5, no. 2, pp. 79-85, 2007. [8] C. Chomsky, The Acquisition of Syntax in Children from 5 to 10, MIT Press, 1969. - 2018 : - 2018 ~ : - ORCID : https://orcid.org/0000-0001-7268-1448 - :, - 2011 ~ : - ORCID : https://orcid.org/0000-0001-5226-8808 - :
- 2014 ~ : - ORCID : https://orcid.org/0000-0001-6610-1072 - : - 2000 : - 2002 : - 2007 : - 2008 ~ 2011 : NHK - 2012 ~ : - ORCID : http://orcid.org/0000-0002-6968-4565 - :,, 3 /