Recent Research Trend of Gesture-based User Interfaces In this paper, we review recent research trends in gesture-based user interfaces to support natural user interactions with media contents. With the developments of information and communication technology, users are able to enjoy varous media contents through various devices anytime and anywhere. From the perspective of user interactions, however, we still suffer from unnatural user interfaces and learning phase of new user interfaces on the various types of media contents and devices. Thus, to support more natural and comfortable user interfaces, there have been many research activities on the development of gesture-based user interfaces. The gesture-based user interface is relatively intuitive and simple in comparison with other user interfaces, such as speech, haptics and eye-gaze, so that it can support more natural user interactions. In particular, the gesturebased user interface is more effective on mobile devices due to its intuitiveness and simplicity. Therefore, we believe personalized gesture-based user interface is required to support user-centered multi-modal interactions in ubiquitous computing environment. Keywords: User Interfcae, Gesture-based Interface, Context-awareness, UbiComp, Augemented Reality
I II
III
IV
V [1] M. Weiser, ''The Computer for the 21st Century,'' Scientific American, Vol. 265, No. 3, 1991, pp. 94-104. [2] B. Schilit, N. Adams, and R. Want, ''Context-Aware Computing Applications,'' in Proceedings of IEEE Workshop on Mobile Computing Systems and Applications, 1994, pp. 85-90. [3] H. Lieberman and T. Selker, ''Out of context: Computer systems that adapt to, and learn from, context,'' IBM Systems Journal, Vol. 39, No. 3-4, 2000, pp. 617-631. [4] A. K. Dey and G. D. Abowd, ''Towards a Better Understanding of Context and Context-awareness,'' in the Workshop on The What, Who, Where, When, and How of Context-Awareness, as part of the 2000 Conference on Human Factors in Computing Systems (CHI 2000), 2000.
[5] A. Schmidt, M. Beigl, and H.-W. Gellersen, ''There is more to Context than Location,'' Computers and Graphics, Vol. 23, No. 6, 1999, pp. 893-901. [6] B. J. Rhodes, ''The Wearable Remembrance Agent: A System for Augmented Memory,'' in Proceedings of The First International Symposium on Wearable Computers, 1997, pp. 123-128. [7] A. Sears, J. Feng, K. Oseitutu, and C.-M. Karat, ''Hands-Free, Speech-Based Navigation During Dictation: Difficulties, Consequences, and Solutions,'' Human-Computer Interaction (HCI) Journal, Vol. 18, No. 3, 2003, pp. 229-257. [8] V. F. M. Salvador, J. S. de Oliveira Neto, and A. S. Kawamoto, ''Requirement Engineering Contributions to Voice User Interface,'' in Proceedings of First International Conference on Advances in Computer- Human Interaction, 2008, pp. 309-314. [9] K. Kyung and J. Park,,, Vol. 21, No. 5, 2006, pp. 93-108. [10] L. Kim and S. Park,,:, Vol. 31, No. 11-12, 2004, pp. 682-691. [11] K. R. Park and J. Kim, ''Real-time Facial and Eye Gaze Tracking System,'' IEICE Transactions on Information and Systems, Vol. E88-D, No. 6, 2005, pp. 1231-1238. [12] M. R. M. Mimica and C. H. Morimoto, ''A Computer Vision Framework for Eye Gaze Tracking,'' in Proceedings of the XVI Brazilian Symposium on Computer Graphics and Image Processing, 2003, pp. 406-412. [13] D. J. Sturman and D. Zeltzer, "A Survey of Glove-based Input," IEEE Computer Graphics and Applications, Vol. 14, No. 1, 1994, pp. 30-39. [14] G. B. Guerra-Filho, ''Optical Motion Capture: Theory and Implementation,'' Journal of Theoretical and Applied Informatics (RITA), Vol. 12, No. 2, 2005, pp. 61-89. [15] T. B. Moeslund and E. Granum, ''A Survey of Computer Vision-Based Human Motion Capture,'' Computer Vision and Image Understanding, Vol. 81, No. 3, 2001, pp. 231-268. [16] M. Turk, ''Computer Vision in the Interface,'' Communications of the ACM, Vol. 47, No. 1, 2004, pp. 60-67. [17] W. T. Freeman, D. B. Anderson, P. A. Beardsley, C. N. Dodge, M. Roth, C. D. Weissman, W. S. Yerazunis, H. Kage, K. Kyuma, Y. Miyake, and K. ichi Tanaka, ''Computer Vision for Interactive Computer Graphics,'' IEEE Computer Graphics and Applications, Vol. 18, No. 3, 1998, pp. 42-53. [18] P. Maes, T. Darrell, B. Blumberg, and A. Pentland, ''The ALIVE system: Full-body Interaction with Autonomous Agents,'' in Proceedings of Computer Animation, 1995, pp. 11-18. [19] K. Hinckley, J. Pierce, M. Sinclair, and E. Horvitz, ''Sensing Techniques for Mobile Interaction,'' in Proceedings of the 13th annual ACM symposium on User Interface Software and Technology, 2000, pp. 91-100. [20] H. Witt, T. Nicolai, and H. Kenn, ''Designing a Wearable User Interface for Hands-free Interaction in Maintenance Applications,'' in Proceedings of the Fourth Annual IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOMW 06), 2006. [21] T. G. Zimmerman, J. Lanier, C. Blanchard, S. Bryson, and Y. Harvill, ''A Hand Gesture Interface Device,'' in Proceedings of the SIGCHI/GI conference on Human factors in computing systems and graphics interface, 1987, pp. 189-192. [22] K. Kurihara, S. Hoshino, K. Yamane, and Y. Nakamura, ''Optical Motion Capture System with Pan-Tilt Camera Tracking and Realtime Data Processing,'' in Proceedings of the 2002 IEEE International Conference on Robotics and Automation, Vol. 2, 2002, pp. 1241-1248. [23] H. Chen, G. Qian, and J. James, ''An Autonomous Dance Scoring System using Marker-based Motion Capture,'' in Proceedings of IEEE 7th Workshop on Multimedia Signal Processing, 2005, pp. 1-4. [24] C. Keskin, A. Erkan, and L. Akarun, ''Real Time Hand Tracking and 3D Gesture Recognition for Interactive Interfaces using HMM,'' in Proceedings of the Joint International Conference ICANN/ICONIP 2003, 2003. [25] Y. Tao and H. Hu, ''Colour Based Human Motion Tracking for Home-based Rehabilitation,'' in Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, Vol. 1, 2004, pp. 773-778. [26] I. Haritaoglu, D. Harwood, and L. S. Davis, ''W4: Who? When? Where? What? A Real Time System for Detecting and Tracking People,'' in Proceedings of International Conference on Face and Gesture Recognition, 1998, pp. 222-227. [27] T. Horprasert, D. Harwood, and L. S. Davis, ''A Statistical Approach for Real-time Robust Background Subtraction and Shadow Detection,'' in Proceedings of the 7th IEEE International Conference on Computer Vision, Frame Rate
Workshop (ICCV 99), 1999, pp. 1-9. [28] D. Hong and W. Woo, ''A Background Subtraction for a Vision-based User Interface,'' in Proceedings of the Fourth International Conference on Pacific Rim Conference on Multimedia, Vol. 1, 2003, pp. 263-267. [29] A. F. Bobick and J. W. Davis, ''The Recognition of Human Movement Using Temporal Templates,'' IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 3, 2001, pp. 257-267. [30] G. R. Bradski and J. W. Davis, ''Motion Segmentation and Pose Recognition with Motion History Gradients,'' Machine Vision and Applications, Vol. 13, No. 3, 2002, pp. 174-184. [31] T. Horprasert, I. Haritaoglu, C. Wren, D. Harwood, L. Davis, and A. Pentland, Real-time 3D Motion Capture, in Proceedings of 2nd Workshop on Perceptual User Interfaces, 1998. [32] G. K. Cheung, S. Baker, J. Hodgins, and T. Kanade, ''Markerless Human Motion Transfer,'' in Proceedings of 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004, pp. 373-378. [33] D. Hong and W. Woo, ''A 3D Vision-based Ambient User Interface,'' International Journal of Human Computer Interaction, Vol. 20, No. 3, 2006, pp. 271-284. [34] K. Oka, Y. Sato, and H. Koike, ''Real-Time Fingertip Tracking and Gesture Recognition,'' IEEE Computer Graphics and Applications, Vol. 22, No. 6, 2002, pp. 64-71. [35] T. Starner, B. Leibe, D. Minnen, T. Westyn, A. Hurst, and J. Weeks, ''The Perceptive Workbench: Computer Vision-Based Gesture Tracking, Object Tracking, and 3D Reconstruction for Augmented Desks,'' Machine Vision and Applications, Vol. 14, No. 1, 2003, pp. 59-71. [36] A. Bottino and A. Laurentini, ''How to Make a Simple and Robust 3D Hand Tracking Device using a Single Camera,'' in Proceedings of the 11th WSEAS International Conference on Computers, 2007, pp. 414-419. [37] S.-J. Cho, C. Choi, Y. Sung, K. Lee, Y.-B. Kim, and R. Murray-Smith, ''Dynamics of Tilt-based Browsing on Mobile Devices,'' in CHI 07 Extended Abstracts on Human Factors in Computing Systems, 2007, pp. 1947-1952. [38] E.-S. Choi, W.-C. Bang, S.-J. Cho, J. Yang, D.-Y. Kim, and S.-R. Kim, ''Beatbox Music Phone: Gesture-based Interactive Mobile Phone using a Tri-axis Accelerometer,'' in IEEE International Conference on Industrial Technology (ICIT 2005), 2005, pp. 97-102. [39] D. Hong and W. Woo,,, Vol. 26, No. 1, 2008, pp. 88-97.