Patterns for the design of musical interaction with everyday mobile devices

Patterns for the design of musical interaction with everyday mobile devices

Abstract

The growing popularity of mobile devices gave birth to a still emergent research field, called Mobile Music (music with mobile devices). Our particular research investigates such re-purposing of ordinary mobile devices for use in musical activities. In this paper we propose the use of patterns in the design of musical interaction with these devices. We introduce the musical interaction patterns that came out of our investigation so far, and describe the exploratory prototypes which served as inspiration and, at the same time, as test-bed for these proposed interaction patterns.

Conclusion

In our work we identified four musical interaction patterns that can be implemented in common mobile devices. This small, initial set of patterns obviously does not mean to be a thorough taxonomy of musical interaction in general. We are also still on the process of compiling other related pattern sets: for interactions made possible by musical ubiquitous computing environments (i.e., involving cooperation, emergence, location awareness, awareness of contextual sound/music resources, etc.) and for musical interfaces (which instantiate musical interaction patterns, possibly using existing UIDPs). Nevertheless, the four patterns listed here already account for musical interaction in ubiquitous environments when a single mobile device is the user interface, plus they suit designs that need to ensure that music can still be made with a mobile device even with no access to pervasive musical resources (in case those are not available or are unreachable, e.g., due to connectivity limitations).

We have also been conducting preliminary tests on patterns comprehensibility, to observe if the proposed patterns can be learned quickly by designers from outside the CM area. Some other tests are being made to confirm the independence of patterns in relation to different types of musical activities, e.g. by comparing user performance and quality of use when carrying out the same musical activity following two different interaction patterns. These tests and their results will be the subject of forthcoming papers.

A pattern-oriented approach for interaction design in mobile music is an effort towards a necessary switch from the current technology-oriented perspective to a more user-centred perspective of CM as a whole, and this paper is just a step towards this goal. However, much work is still needed in order to extend the scope of current CM research to cope with many well-known HCI concerns. We are convinced that a better understanding of HCI issues in CM research and development is a good starting point, not only to identify the capabilities and limitations of future work, but mainly to establish a common ground for discussing several interesting questions that are still open.

My thoughts

Main Argument

Interaction patterns should be developed to allow conceptual designs to be implemented on mobile devices in the field of mobile music. These patterns allow for metaphor construction better suited for musical interaction in the context of ubiquitous musical activities. An interaction pattern language can demonstrate how design problems may be solved according to sound user-centred design principles.

Secondary arguments

Interfaces for musical performance or interaction should be designed with human input centred design rather than traditional computer centred design. Interfaces for musical performance should fit to our physical, embodied natures, rather than only operating in the realm of symbolic processing.

Interim conclusion

The need exists to apply HCI methods for interaction design of these solutions or as a way for improving user experience.

Main conclusion

Four main patterns of musical interaction for musical interaction in ubiquitous environments when a single mobile device is the user interface have been established as taxonomy of musical interaction. Further work to observe patterns outside of these areas have been established as well as comparing user performance and quality of use when carrying out the same musical activity following two different interaction patterns.

My research aims to further Fores by establishing which situations specific interaction patterns can be used in when used as a human input centred design element (ergo the traditional mixer element of a midi setup).