CHI 2008 Proceedings ´ Workshops April 5-10, 2008 ´ Florence, Italy

Sonic Interaction Design: Sound, Information and Experience

Davide Rocchesso Gerhard Eckel Abstract IUAV University of Venice IEM - Graz, Austria Sonic Interaction Design (SID) is an emerging field Venezia, Italy [email protected] that is positioned at the intersection of auditory display, [email protected] ubiquitous computing, interaction design, and Karmen Franinovic interactive arts. SID can be used to describe practice Stefania Serafin ZHDK - Zürich, Switzerland and inquiry into any of various roles that sound may Aalborg University Copenhagen [email protected] play in the interaction loop between users and artifacts, Ballerup, Denmark services, or environments, in applications that range [email protected] Thomas Hermann from the critical functionality of an alarm, to the artistic University of Bielefeld significance of a musical creation. This field is devoted Frauke Behrendt Bielefeld, Germany to the privileged role the auditory channel can assume University of Sussex [email protected] in exploiting the convergence of computing, Brighton, UK communication, and interactive technologies. An over- [email protected] Sandra Pauletto emphasis on visual displays has constrained the University of York, UK development of interactive systems that are capable of Nicola Bernardini [email protected] making more appropriate use of the auditory modality. Conservatorio “C. Pollini” Today the ubiquity of computing and communication Padova, Italy Patrick Susini resources allows us to think about sounds in a pro- [email protected] IRCAM active way. This workshop puts a spotlight on such Paris, France issues in the context of the emerging domain of SID. Roberto Bresin [email protected] KTH Keywords Stockholm, Sweden Yon Visell Sound Design, Auditory Display, Multimodal Interaction, [email protected] McGill University Product Design, Interactive Arts and Music, Sound Montreal, Canada Perception and Cognition, Sound Modelling [email protected] ACM Classification Keywords H.5.5. Sound and Music Computing Copyright is held by the authors. H.5.2. User Interfaces CHI 2008, April 5 –10, 2008, Florence, Italy

ACM 978-1-60558-012-8/08/04.

3969 CHI 2008 Proceedings ´ Workshops April 5-10, 2008 ´ Florence, Italy

Introduction Goals of the Workshop Thinking back to Edison's phonograph or, more recently, The SID workshop intends to foster discussions around to the mp3 craze, one may well say that music has and promote the integration of scattered research been a driving factor for technology [1]. But, can the efforts in sound modeling, design, art, perception and same be said for sound in general? To date, non- cognition into a coherent movement capable of musical sounds have been accepted as by-products of improving the interactive acoustic appearance of future technologies rather than being exploited for their environments. It aims to achieve this by providing a intrinsic value. As a result, an acoustically polluted forum to present and discuss relevant current work world is what most people have been experiencing within and spanning these areas, organized around since the industrial revolution. Times are mature to several key themes. think about sound as one of the main design dimensions of the environments in which we live and Perceptual, Cognitive, and Emotional Study work. That means overcoming the sound-as-noise of Sonic Interactions cultural barrier and promoting a sound-as-information This strand of the workshop showcases current attitude. This tendency is already visible in the market research surrounding the ways humans perceive, where new products (e.g., the Nintendo Wii) exploit the understand, and interact through sound – knowledge tight coupling between sound and gesture in that is basic for the design of new systems that utilize it interaction. The same trends have manifested in the as a primary modality. From low-level psychoacoustic performing arts for many years, and have been recently features to more complex attributes motivated by exposed as main issues in the New Interfaces for auditory modeling, work in this area has positively Musical Expression (www.nime.org) series of impacted numerous application areas. This has conferences, which are also a spin-off of a CHI included the identification of sound features salient to workshop. Designing the sonic appearance of the function of an artifact or system, as in the design of interactive systems is thus becoming a competitive alerting or signaling sounds [2], to perceptual features issue as well as a lively playground, which nevertheless of importance for the design of musical sounds or depends on knowledge that has grown in several control spaces, or those that are matched to more interrelated fields of research and practice. The holistic subjective quality descriptors for a product’s relevant areas of research have grown in significance, identity, interaction and task context. Knowledge about both as a result of the design needs associated to certain classes of functional sounds is of particular improving the sonic jungles we are increasingly interest for our everyday life. Perception and confronted with, and as economies of scale and classification of everyday sounds is also an emerging miniaturization have contributed to a widening array of theme which is featured in one of the contributions to interactive artifacts and systems that are embedded the workshop. Among other desirable outcomes of with ever more sophisticated computing, sensing and research in this area, one may identify the development actuating capabilities. of new sound assessment and predictive tools that may be useful for sound designers, and the creation of

3970 CHI 2008 Proceedings ´ Workshops April 5-10, 2008 ´ Florence, Italy

psychologically founded frameworks that are useful for role of enactive engagement with sound–augmented the evaluation of the quality of a particular sound interactive objects. The work of such artists is design [2]. increasingly being recognized as a driving factor for technology and sometimes for science as well. New Product Sound Design fields such as that of “aesthetic computing” [5] are Sound design already plays a significant role in many emerging from the joint efforts and mutual interests of areas of product design, especially those which create scientists and artists. Artists and designers have been products with high functional densities, strong design engaged for some time in questioning the ways that identities, or which address demanding markets such as sound is used in everyday life and in creatively car design. Prominent industries that have benefited exploring future possibilities for interactive products from it include the automobile and cosmetics industries, and systems, including those with embedded sound [6] but lower profile applications have arisen in diverse or in a mobile context [7]. This workshop strand other areas, from kitchen appliances to toys and office critically explores the present use of sound as integral equipment. This strand of the workshop presents the to such artworks, and reveals related future state-of the art in product design and discusses future opportunities and trends for interactive sound artefacts opportunities. The interactive use of sound should be through presentations from artist-researchers in the seen as distinct from applications that serve primarily sonic and performing arts. Some associated goals of to signal or alert a user to an event, process, or state. this area of contact are to inspire SID research in The workshop showcases ways in which sound can take complementary disciplines, to reveal interactive product on a more prominent role [2]: Sound can create or opportunities, and expose social and ethical issues. reveal new functionalities in a product, when these functionalities may not be apparent through other Auditory Display and Sonification modalities, such as the physical form or other visible Auditory display research to date has gravitated around indicators. Sound design is also crucial in mobile a number of topics such as the sonification of datasets devices, as shown in one of the contributions to the (interactive or otherwise), auditory feedback in workshop. Sound can also shape the sonic appearance computing displays, auditory icons, earcons, signaling, of an artifact, where appearance is thought of as and mobile communication and computing applications. primarily referring to the qualities of the sound that is Some of this work has been motivated by an increase in produced through interaction. Sound can also be used awareness of the need for multimodal human computer to provide feedback to aid users' performance with or interfaces, as the contexts in which computing takes control over an interface, tool, device, or physical place today include many in which an additional visual activity. [4] display may not be effective, or may not be an option. Sonification is beginning to impact many fields such as Interactive Art and Music information engineering, data mining, biomedicine, and A major strand of the workshop deals with recent work rehabilitation. By contrast with visualization, in the interactive and performing arts that exploits the sonification inherently develops in time, is critically

3971 CHI 2008 Proceedings ´ Workshops April 5-10, 2008 ´ Florence, Italy

dependent on interaction, and exploits the fastest of References human senses. Research in interactive sonification has [1] The S2S² Consortium. A Roadmap for Sound as a key objective that of empowering the designers of and Music Computing, 2007, www.soundandmusiccomputing.org/roadmap such data sonifications [8]. The role of interactivity is crucial in applications exploiting the auditory channel in [2] C. Suied, L. Pruvost, P. Susini, N. Misdariis, S. Langlois, S. McAdams, "Speeded detection of sound the exploration of complex information spaces. signals based on temporal differences", J. Ac. Soc. Am. Research is required to devise and test new interaction 119(5):3333, 2006. methods and new mapping strategies for sonic [3] P. Susini, N. Misdariis, G. Lemaitre, D. exploration in order to improve the effectiveness of Rocchesso, P. Polotti, K. Franinovic, Y. Visell, K. information foraging, browsing, and surveying with Obermayer, "Closing the Loop of Sound Evaluation and sound. Design,” Proc. of the 2nd ISCA/DEGA Workshop on Perceptual Quality of Systems, 2006. Sonic Interaction Design [4] D.Rocchesso, R. Bresin, M. Fernstrom, These key themes provide a solid framework for the “Sounding Objects”, IEEE Multimedia, 10(2):42-52, workshop's aim of advancing the discussion 2003. surrounding the varied roles of sound in interactive [5] P. A. Fishwick, editor. Aesthetic Computing. MIT objects, systems and environments. The themes are Press, Cambridge, Massachusetts, 2006. well represented in the accepted contributions in the [6] K. Franinovic, D. Hug, Y. Visell, “Sound form of theoretical and aesthetical aspects of SID, novel Embodied: Explorations of sonic interaction design for tangible musical instruments, algorithms for synthesis everyday objects in a workshop setting”, Proc. of the of everyday sounds and sonification. Transversal to the Intl. Conf. on Auditory Display (ICAD), 2007. four themes, other areas of discussion emerge from the [7] L. Gaye, L. Holmquist, F. Behrendt, A. Tanaka. submitted contributions, such as the need for a new “Mobile Music Technology. Report on an Emerging pedagogy of product sound design, or the role of sound Community”. Proc. of the Intl. Conf. on New Interfaces for Musical Expression (NIME), 2006. in enhancing presence in virtual environments. Sonic Interaction Design is emerging as a discipline where all [8] T. Hermann and A. Hunt. An introduction to interactive sonification. IEEE Multimedia, 12(2):20–24, these areas contribute valuable elements for closing the 2005. Editorial, special issue on interactive sonification. design loop in sound-based interaction. [9] Informational website of the COST IC0601 Action on Sonic Interaction Design: http://www.cost- Acknowledgements sid.org/ This workshop is sponsored by the European COST Action IC0601 on Sonic Interaction Design [9].

3972