Gaze-supported multi-modal interactions

Gaze-supported multi-modal interactions


While eye tracking is becoming more and more relevant as a promising input channel, diverse applications using gaze control in a more natural way are still rather limited. Though several researchers have indicated the particular high potential of gaze-based interaction for pointing tasks, often gaze- only approaches are investigated.


This paper presented a detailed description of a user-centred design process for gaze-supported interaction techniques for the exploration of large image collections. For this purpose, gaze input was combined with additional in- put modalities: (1) a keyboard and (2) a mobile tilt-enabled multi-touch screen. The integration of user feedback at such an early stage of the design process allowed for the development of novel and more natural gaze-supported interaction techniques. While gaze acted as a pointing modality, the touch and tilt actions complemented the interaction for a multifaceted interaction. Based on user-elicited interaction techniques we developed an extended multimedia retrieval system, Gaze Galaxy, that can be controlled via gaze and touch-and-tilt input to explore large image collections. First user impressions on the implemented interaction techniques

were gathered and discussed. Results indicate that gaze in- put may serve as a natural input channel as long as certain design considerations are taken into account. First, gaze data is inherently inaccurate and thus interaction should not rely on precise positions. Using the gaze positions for setting a fish-eye lens and zooming in at the point-of-regard were described as intuitive. Secondly, users should be able to confirm actions with additional explicit commands to pre- vent unintentional actions.

My thoughts

This research could be easily adapted for use in other media applications and situations. Browsing and selecting sound files in a DAW; editing parameters in a DAW; selecting from a list of users or contacts and many other possibilities. Infancy technologies like Gaze will require lots of research and development to bring to market (so to speak) but high risk brings high reward in such cases. This is definitely an area of research that interests me and I will be actively looking at some demo’s or applications that can be brought to a small control group.