HCI 2015
International

Sunday, 2 August 2015, 14:00 - 17:30

Lucio Davide Spano (short bio)
Department of Mathematics and Computer Science, University of Cagliari, Italy

Objective:

The tutorial objective is to provide a brief yet complete introduction to the main techinques and problems for designing and implementing gesture interfaces.
During the tutorial, the attendees will:

• Understand the interaction capabilities of the different standard devices and APIs used for tracking the user’s gestures
• Identify the main difficulties in the design of gestural UIs
• Understand the main design patterns for gestural interaction
• Understand the pro and cons of the heuristic gesture recognition
• Understand the pro and cons of the machine learning recognition

Content:

The tutorial provides an introduction to the methodologies and techniques for creating gesture interfaces. We start from the foundation of gestural communication between humans, describing briefly the different gesture categories (e.g. iconic, metaphoric, deitic etc.) and the cultural dependencies in their interpretation. Such discussion sets the ground for understanding how this type of communication can be used for creating interfaces.

After that, we describe the enabling recognition technologies, explaining how they work, the tracked information and which kind of gestures can be recognized with each class of device: multitouch screens, pressure boards, remotes and computer-vision (both bare hands and full body). We provide examples analysing the programming APIs of different de-facto standard devices (e.g. iOS, Kinect SDK, Leap Motion SDK, WiimoteLib etc.)

Having set the technological background, we analyse technical difficulties in creating gesture interfaces. We discuss the following problems: the Midas touch (tracking only the user’s movements that are intended for the interaction), the fat finger (hand and/or limbs occlusion while interacting with the user interface), the intermediate feedback (user guidance for gestures having a long duration in time) and the spaghetti code (the gesture recognition code and the definition of the user interface behaviour are usually mixed), together with a set of user interface design guidelines for gesture interfaces (vocabulary design and evaluation). In particular, we discuss the exploitation of the 3D space, super-real metaphors (real physical objects enhanced with impossible but plausible actions), and the scaffolding principle (learning by doing).

Finally, we discuss two different approaches for gesture tracking and recognition (heuristic and machine learning) and the possible future evolutions in this research field.

Target Audience:

The tutorial is aimed to developers, researchers and students interested in exploiting gestural interaction for creating interactive applications. UI designers can benefit from a systematic description of the problems related to gesture recognition with different devices and their possible solutions.

Bio Sketch of Presenter:

Lucio Davide Spano is an assistant professor at the University of Cagliari, Italy. He got his PhD at the University of Pisa, Italy, with a thesis on declarative models for gestural interaction. He worked at the Italian National Research Council and collaborated to different projects founded by the European Commission (Sereona, ServFace, Smarcos). He is the author of different research papers on gestural interface modelling and development.

Links

• Institution webpage
• Department webpage
• List of papers
• Other CV info