Gesture Recognition

Teachers: Denis Lalanne

Introduction

In recent years, there has been a tremendous effort in research toward novel devices and techniques to improve the naturalness of interaction with computers. Gesture recognition is among the most promising attempt in this direction. Gestures are expressive, meaningful body motions involving physical movements of the fingers, hands, arms, etc., with the intent of conveying meaningful information or interacting with the environment.

First attempts to solve this problem resulted in mechanical devices that directly measure hand and/or arm joint angles and spatial position. This group is best represented by the so-called glove-based devices. Glove-based gestural interfaces require wearing a cumbersome device, and generally carrying a load of cables. This hinders the ease and naturalness with which the user can interact.

The problem of having to wear gloves can be overcome by using video-based interaction techniques. Numerous approaches have been applied to the problem of visual interpretation of gestures for HCI. Images of hands, geometric moments, contours, silhouettes, and 3D hand skeleton models are a few examples. In recent years, however, there has been an interest in incorporating the dynamic characteristics of gestures, which carry as many information as postures.

The major goal of this seminar is to brush a state-of-the-art in gesture recognition, both at the recognition, dialog and application levels.

Goal

Read articles corresponding to the select theme (defined with the organizer), synthesize them in an article and present them to the participants of the seminar. The article must be authored in Latex following ACM Strict.

Participants

If you are willing to participate to this seminar, please contact the organizer. Max 6 students will be accepted.

Agenda

  • October 2nd, 14h: Seminar Kick-off

    • Introduction to Gesture, acquisition and recognition (Denis Lalanne)

  • October 30th, 10h15:

    • Présentations d'un chercheur invité (Frédéric Kaplan);

  • December 4th, 10h15:

    • « how to make a good scientific presentation » (Rolf Ingold) [PDF]

  • December 18th, 9h15:

  • February 19th, 9h15:

Themes & References

  1. Surveys

Sushmita Mitra, Tinku Acharya, “Gesture Recognition: A Survey”, IEEE Transactions on systems, man, and cybernetics - part c: applications and reviews, vol. 37, no. 3, May 2007.

Ying Wu, Thomas S. Huang: Vision-Based Gesture Recognition: A Review. Book on Gesture-Based Communication in Human-Computer Interaction, Springer, 103-115 (1999).

  1. Finger tracking & Pointing Gesture

J.L. Crowley, F. Berard, and J. Coutaz, “Finger Tracking As an Input Device for Augmented Reality,” Proc. Int’l Workshop on Automatic Face and Gesture Recognition, Zurich, Switzerland, pp. 195-200, June 1995.

M. Fukumoto, Y. Suenaga, and K. Mase, “Finger-Pointer”: Pointing Interface by Image Processing,” Computers and Graphics, vol. 18, no. 5, pp. 633-642, 1994.

Yepeng Guan; Mingen Zheng, "Real-time 3D pointing gesture recognition for natural HCI," Intelligent Control and Automation, 2008. WCICA 2008. 7th World Congress on , vol., no., pp.2433-2436, 25-27 June 2008.

Kehl, R.; Van Gool, L., "Real-time pointing gesture recognition for an immersive environment," Automatic Face and Gesture Recognition, 2004. Proceedings. Sixth IEEE International Conference on , vol., no., pp. 577-582, 17-19 May 2004.

  1. Static hand gesture (posture)

V. Athitsos and S. Sclaroff. Estimating 3D hand pose from a cluttered image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 2003.

T. Starner and A. Pentland, “Visual Recognition of American Sign Language using Hidden Markov Models,” Proceedings of the International Workshop on Automatic Face and Gesture Recognition, Zurich, Switzerland, 1995.

  1. Dynamic hand gesture

J. Davis and M. Shah, “Recognizing Hand Gestures,” Proc. European Conf. Computer Vision, Stockholm, Sweden, pp. 331-340, 1994.

K. Ishibuchi, H. Takemura, and F. Kishino, “Real Time Hand Gesture Recognition Using 3D Prediction Model,” Proc. 1993 Int’l Conf. Systems, Man, and Cybernetics, Le Touquet, France, pp. 324- 328, Oct. 17-20, 1993.

Vladimir Pavlovic, Rajeev Sharma, Thomas S. Huang: Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review. IEEE Trans. Pattern Anal. Mach. Intell. 19(7): 677-695 (1997).

  1. Arm and body gesture recognition

Y. Azoz, L. Devi, R. Sharma, "Reliable Tracking of Human Arm Dynamics by Multiple Cue Integration and Constraint Fusion," Computer Vision and Pattern Recognition, IEEE Computer Society Conference on, pp. 905, 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'98), 1998.

Wren, C.R.; Azarbayejani, A.; Darrell, T.; Pentland, A.P., "Pfinder: real-time tracking of the human body," Pattern Analysis and Machine Intelligence, IEEE Transactions on , vol.19, no.7, pp.780-785, Jul 1997.

A.D. Wilson and A.F. Bobick, “Recovering the Temporal Structure of Natural Gestures,” Proc. Int’l Conf. Automatic Face and Gesture Recognition, Killington, Vt., pp. 66-71, Oct. 1996.

  1. Applications

T. Baudel and M. Baudouin-Lafon, “Charade: Remote Control of Objects Using Free-Hand Gestures,” Comm. ACM, vol. 36, no. 7, pp. 28-35, 1993.

W.T. Freeman and C.D. Weissman, “Television Control by Hand Gestures,” Proc. Int’l Workshop on Automatic Face and Gesture Recognition, Zurich, Switzerland, pp. 179-183, June 1995.

R. Kjeldsen, J. Kender, "Toward the use of gesture in traditional user interfaces," Automatic Face and Gesture Recognition, IEEE International Conference on, pp. 151, Second IEEE International Conference on Automatic Face and Gesture Recognition (FG '96), 1996.

Yang Liu, Yunde Jia, "A Robust Hand Tracking and Gesture Recognition Method for Wearable Visual Interfaces and Its Applications", Proceedings of the Third International Conference on Image and Graphics (ICIG’04), 2004.

  1. Devices based gesture recognition

Thomas Schlomer, Benjamin Poppinga, Niels Henze, Susanne Boll, "Gesture Recognition with a Wii Controller", Proceedings of the 2nd international Conference on Tangible and Embedded interaction, 2008.

Deller, M.; Ebert, A.; Bender, M.; Hagen, H., "Flexible Gesture Recognition for Immersive Virtual Environments," Information Visualization, 2006. IV 2006. Tenth International Conference on , vol., no., pp.563-568, 5-7 July 2006.

Fröhlich, M. & I. Wachsmuth (1998): Gesture recognition of the upper limbs - from signal to symbol. In I. Wachsmuth and M. Fröhlich (eds.): Gesture and Sign Language in Human-Computer Interaction (pp. 173-184). Berlin: Springer-Verlag (LNAI 1371).

Wei Du, Hua Li, "Vision based gesture recognition system with single camera", 5th International Conference on Signal Processing Proceedings, 2000.

  1. Multimodal gestures

Koons, D. B., Sparrell, C. J., and Thorisson, K. R. 1993. Integrating simultaneous input from speech, gaze, and hand gestures. In intelligent Multimedia interfaces, M. T. Maybury, Ed. American Association for Artificial Intelligence, Menlo Park, CA, 257-276. (ask Bruno Dumas in room B440 for the book)

Cohen, P. R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L., and Clow, J. 1997. QuickSet: multimodal interaction for distributed applications. In Proceedings of the Fifth ACM international Conference on Multimedia (Seattle, Washington, United States, November 09 - 13, 1997). MULTIMEDIA '97. ACM, New York, NY, 31-40.

Holzapfel, H., Nickel, K., and Stiefelhagen, R. 2004. Implementation and evaluation of a constraint-based multimodal fusion system for speech and 3D pointing gestures. In Proceedings of the 6th international Conference on Multimodal interfaces (State College, PA, USA, October 13 - 15, 2004). ICMI '04. ACM, New York, NY, 175-18.

Bolt, R. A. 1980. “Put-that-there”: Voice and gesture at the graphics interface. In Proceedings of the 7th Annual Conference on Computer Graphics and interactive Techniques (Seattle, Washington, United States, July 14 - 18, 1980). SIGGRAPH '80. ACM, New York, NY, 262-270.

Date: 2009, Fall