Sonic Interaction Design Sonic Interaction Design
Total Page:16
File Type:pdf, Size:1020Kb
Sonic Interaction Design Sonic Interaction Design edited by Karmen Franinovi ć and Stefania Serafin The MIT Press Cambridge, Massachusetts London, England © 2013 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher. MIT Press books may be purchased at special quantity discounts for business or sales promotional use. For information, please email [email protected] or write to Special Sales Depart- ment, The MIT Press, 55 Hayward Street, Cambridge, MA 02142. This book was set in Stone Sans and Stone Serif by Toppan Best-set Premedia Limited, Hong Kong. Printed and bound in the United States of America. Library of Congress Cataloging-in-Publication Data Sonic interaction design / edited by Karmen Franinovi ć and Stefania Serafin. pages cm Includes bibliographical references and index. ISBN 978-0-262-01868-5 (hardcover : alk. paper) 1. Sonic interaction design. 2. Product design. 3. Sound in design. 4. Human-computer interaction. I. Franinović , Karmen, 1975 — editor of compilation. II. Serafin, Stefania, 1973 — editor of compilation. TS171.S624 2013 004.01 ' 9 — dc23 2012028482 10 9 8 7 6 5 4 3 2 1 Contents Introduction vii I Emergent Topics 1 1 Listening to the Sounding Objects of the Past: The Case of the Car 3 Karin Bijsterveld and Stefan Krebs 2 The Experience of Sonic Interaction 39 Karmen Franinovi ć and Christopher Salter 3 Continuous Auditory and Tactile Interaction Design 77 Yon Visell, Roderick Murray-Smith, Stephen A. Brewster, and John Williamson 4 Pedagogical Approaches and Methods 125 Davide Rocchesso, Stefania Serafin, and Michal Rinott 5 Perceptual Evaluation of Sound-Producing Objects 151 Bruno L. Giordano, Patrick Susini, and Roberto Bresin II Case Studies 199 Audio and Touch 201 6 Perceptual Integration of Audio and Touch: A Case Study of PebbleBox 203 Sile O’ Modhrain and Georg Essl 7 Semiacoustic Sound Exploration with the Sound of Touch 213 David Merrill and Hayes Raffle 8 The Gamelunch: Basic SID Exploration of a Dining Scenario 225 Stefano Delle Monache, Pietro Polotti, and Davide Rocchesso vi Contents 9 ZiZi: The Affectionate Couch and the Interactive Affect Design Diagram 235 Stephen Barrass 10 SonicTexting 245 Michal Rinott 11 The A20: Interactive Instrument Techniques for Sonic Design Exploration 255 Atau Tanaka, Olivier Bau, and Wendy Mackay Sonification of Human Activities 271 12 Designing Interactive Sound for Motor Rehabilitation Tasks 273 Federico Avanzini, Simone Spagnol, Antonio Rod à , and Amalia De G ö tzen 13 Sonification of the Human EEG 285 Thomas Hermann and Gerold Baier 14 High-Density Sonification: Overview Information in Auditory Data Explorations 299 Johan Kildal and Stephen A. Brewster Sound in Virtual Reality 307 15 Simulating Contacts between Objects in Virtual Reality with Auditory, Visual, and Haptic Feedback 309 Jean Sreng and Anatole L é cuyer 16 Sonic Interaction via Spatial Arrangement in Mixed-Reality Environments 329 Mike Wozniewski, Zack Settel, and Jeremy R. Cooperstock 17 Heigh Ho: Rhythmicity in Sonic Interaction 341 Cumhur Erkut, Antti Jylh ä , and Davide Rocchesso 18 Barking Wallets and Poetic Flasks: Exploring Sound Design for Interactive Commodities 351 Daniel Hug Contributors 369 Index 373 Introduction Sound is an integral part of every performative and aesthetic experience with an arti- fact. Yet, in design disciplines, sound has been a neglected medium, with designers rarely aware of the extent to which sound can change the overall user experience. The shaping of the auditory character of products has been dominated by functional pri- orities such as noise reduction, the representational uses of sounds as auditory icons, and sound as emergency signal or alarm. These roles that sound has played as func- tional, iconic, and signaling element during the interaction with an artifact have arguably limited our everyday experiences with sound. Sonic interaction design (SID) emerged from the desire to challenge these prevalent design approaches by consider- ing sound as an active medium that can enable novel phenomenological and social experiences with and through interactive technology. By using sound in an embodied and performative way, this burgeoning new field works with emergent research topics related to multisensory, performative, and tactile aspects of sonic experience. Yet, it builds on existing knowledge and themes that have been at the heart of sound discourses for decades, such as the relationship between sound and location, issues of privacy and power, and the ability of a sounding artifact to communicate information. A deep examination of such topics requires the involve- ment of diverse perspectives and methods belonging to a range of different disciplines including interactive arts, electronic music, cultural studies, psychology, cognitive sciences, acoustics, interaction design, and communication studies. Thus, SID can be defined as an interdisciplinary field of research and practice that explores ways in which sound can be used to convey information, meaning, and aesthetic and emo- tional qualities in interactive contexts. As a design practice, SID is the creative activity of shaping the relationships between artifacts, services, or environments and their users by means of interactive sound. In this context, the goal of SID is not only to extend the existing research on interactive sound but also to foster new application viii Introduction areas and new domains of practice for sound designers, architects, interaction design- ers, media artists, product designers, and urban planners. Product Design: Reduction versus Quality Since its origins, sound design for products has focused on the elimination of undesir- able noise that is produced through physical interaction. In the early 1950s, several companies begun to apply evaluation to their products through precise measurements of the level of sound produced during their use. Such tests allowed designers and engineers to accordingly modify the mechanical design of their products in order to reduce sonic annoyance for the user. Today, similar measurements are combined with psychophysical methods well suited to characterize the acoustic annoyance of the user for products such as vacuum cleaners, coffee machines, and air conditioners [ 14 ]. However, this type of evaluation often fails to account for the users’ emotional and cognitive responses to the functional and aesthetic aspects of a product [3 ]. Thus, more recently, psychologists have begun to explore emotional responses to sound in prod- ucts beyond so-called user preference tests [ 15 ], reflecting a more general trend in design research of exploring the emotional impact of products [ 11 ]. Although in practice the trend toward reducing annoying sound, rather than designing its quality, still dominates in product design, there are a few cases in which sound is used to affect the overall perceived quality of a product. For example, in order to provide strong product identities, sonic branding professionals have begun to use sound to enhance the use of a product rather than to deploy sound solely as an addition to the visual advertisements. Various luxury goods are designed in a way that action performed with the product generates desirable sounds, as for example in opening of a perfume bottle, starting a motorcycle, or closing the door of a car [12 ]. Meanwhile, scientific researchers have demonstrated that other senses such as the sense of taste can be affected through sound, leading them to conclude that the auditory cues elicited by our contact or interaction with different surfaces (such as abrasive sandpapers or even our own skin) and products (including electric toothbrushes, aerosol sprays, food mixers, and cars) can dramatically change the way in which they are perceived, despite the fact that we are often unaware of the influence of such auditory cues on our perception. [ 13 ] Although designers could benefit from applying such perceptual phenomena to prod- ucts, the ethical issues related to the use of multisensory effects without the awareness of the user still remain to be discussed. Introduction ix Human-Computer Interaction: Metaphors and Sonification In digital products, sound has been predominantly used to accompany screen-based interaction or to present information in the form of sound. Auditory displays can help extract meaning from complex data, alert to a particular event, or signal the accom- plishment of an action performed with a graphical icon on the screen [ 8 ]. Such rep- resentational uses of sound have defined a number of functional roles for computer sound as well as contributed to development of new design tools. Extensive research was conducted on earcons — abstract sounds that accompany different computer events [ 2 ] — and auditory icons — everyday sounds associated with interactions on the computer desktop [ 5 ]. In this context, everyday sounds have proved to be efficient interactive metaphors because users could easily associate such sounds with physical events that cause them. A typical example is the sound of crushing paper when moving a file to a trash can on a computer desktop [ 6 ]. This research also presented a designerly way of thinking about sound, suggesting that its qualities should be exaggerated in order to improve the perception of the sound event and thus facilitate