Designing for Low-Latency Direct-Touch Input

Designing for Low-Latency Direct-Touch Input

Abstract

Software designed for direct-touch interfaces often utilize a metaphor of direct physical manipulation of pseudo “real- world” objects. However, current touch systems typically take 50-200ms to update the display in response to a physical touch action. Utilizing a high performance touch demonstrator, subjects were able to experience touch latencies ranging from current levels down to about 1ms. Our tests show that users greatly prefer lower latencies, and noticeable improvement continued well below 10ms. This level of performance is difficult to achieve in commercial computing systems using current technologies. As an alternative, we propose a hybrid system that provides low-fidelity visual feedback immediately, followed by high-fidelity visuals at standard levels of latency.

Conclusion

In this paper, we have described sources of latency, and demonstrated how several of these can be eliminated in building a demonstrator system capable of 1ms touch latency. Further, we have described the results of tests which showed that users were able to perceive order-of magnitude improvements in latency over current-generation hardware. Our results suggest that performance beyond 1ms may still yield improvement that is perceptible to users.

We have constructed a prototype Accelerated Touch system, wherein a traditional direct-touch layer is paired with a low-latency layer that displays nearly immediate visual feedback on user interaction, independent of application logic, but visually tied to the underlying UI widget. We have further described the design of this visual language to satisfy the various constraints of a dedicated low-latency touch processor, and we have described a potential architecture for a direct-touch system that pairs Accelerated Touch with more traditional touch interaction.

A common complaint that we heard from people who used our system extensively was that it “broke” them – that they now find the latency of current generation devices completely unacceptable. The implication is that improving latency might be an effective competitive strategy for de- vice vendors. It is our hope that this paper will spark innovation in the design of hardware and software capable of lower latency of response to user input.

We see a wealth of future work in further investigating the limits of human perception of touch-screen computer systems, and better understanding the effect of performance parameters such as latency on the usage of touch-screens. For example, are there performance benefits for input under reduced latency? Further, we have conflated latency and frame rate, future devices may decouple these two parameters, and could optimize one or the other; further investigation is needed into the effects of such a change.

My thoughts

To gain increase in latency requires that all three elements in a system (the sensor, the software and the display) are addressed. Obviously only the software can be looked at in the case of user design as attempted to redesign capacitive screens and operating systems is way beyond the scope of my research. Interestingly this research does touch upon Card, Mackinlay and Robertsons earlier work that suggests 10Mhz and 100ms are the limits of acceptability. With advances in technology and with my focus of designing interfaces for use with audio applications then I would suggest that the 100ms limit be at least halved to be seen as acceptable. Definitely an area which requires more research and is of interest to me.