Creative In- Teractions – the Mobile Music Work- Shops 2004 –

2 3

Creative In- teractions – The Mobile Music Work- shops 2004 –

2008 Nicolaj Kirisits • Frauke Behrendt • Lalya Gaye • Atau Tanaka 4 Acknowledgements 5

Creative Interactions –

First and foremost we are deeply grateful to all contributors to and participants of the workshops as well as the reviewers; especially those who have been involved with the event over several years. We would like to thank all Institutions that supported and hosted the workshop in the previous years, namely the Viktoria Institute, The Mobile Music Work- NIME, University of Sussex, Futuresonic, STEIM, Waag and the University of Applied Arts Vienna.

Thank you, Lars-Erik Holmquist for initiating the first workshop and thank you, Kristina Andersen for joining the Steering Committee of the workshop series. We also wish to thank Bernhard Faiss for layouting this publication. shops 2004 – 2008 Many thanks to Dr. Gerald Bast and the University of applied arts, Vienna for sponsoring this catalogue. 5th International Mobile Music Workshop The cover picture was taken by Bernhard Faiss May 13 – 15, 2008 University of applied arts, Vienna Unless stated otherwise all rights remain with the author. http://www.mobilemusicworkshop.org 6 VIENNA MAP Time Schedule 7

Spat_Lab

Day 2 – MAY 14

Poster Presentation

10:00 – 12:00 Mobile Tangible Interfaces as Gestural Instruments F. Kayali / M. Pichlmair / P. Kotik 19:00 • Community - dinner at Xpedit An Augmented Reality Framework for Wireless Mo- Day 3 – MAY 15 bile Performance M. Wozniewski / N. Bouillot / Z. Settel / J.R. Cooper- Hands-on Sessions stock 10:00 – 12:00 undersound and the Above Ground R. Widerberg / Y. Harris / S. Symons B Department for Digital Arts A. Bassoli / J. Brewer / K. Martin / I. Carreras / D. Tac- coni 1010 Vienna, Sterngasse 13 12:00 – 13:30 Lunch soundFishing • C. Midolo Rudolfs- park Day 1 – MAY 13 Paper3 Praterstraße 12:00-13:30 • Lunch

Taborstraße 13:30 • Developments and Challenges turning Mobile Donaukanal 10:00 • Welcome Paper2 Phones into Generic music Performance Platforms Wipplingerstraße 10:30 • Keynote G. Essl / G. Wang / M. Rohs

Sterng.B 13:30 • Some Challenges Related to Music and Move- Schwedenplatz 14:00 • A Typology for Listening in Place U1 U4 12:00 – 13:30 • Lunch ment in Mobile Music Technology P. Rebelo / M. Green / F. Hollerweger Franz-Josefs-Kai A. R. Jensenius Marc-Aurel-Str. Paper1 14:30-15:00 • Coffee break Fleischmarkt 14:00 • Real-Time Synaesthetic Sonification of Travel- Tuchlauben 13:30 • Caressing the Skin: ing Landscapes Mobile devices and bodily engagement T. Pohle / P. Knees closing session - panel debate Rotenturmstraße F. Schröder Graben 14:30-15:00 • Coffee break 15:00 – 17:00 Stephans- Wollzeile platz A 14:00 • MoGMI: Mobile Gesture Music Instrument A. Dekel / G. Dekel Performances2 closing Party U1 U3

Stubenring U4 14:30 – 15:00 • Coffee break 19:00 – … Dr. KarlU3 15:00 • Framework Lueger Platz A. Haberl / K.Filip / S. Faessler / N. Kirisits Vordere Zollamts-Straße Performance1 concerts U3 15:30 • Tango Intervention Vienna Kärntner Straße 15:00 • Transit L. Robert 20:00 • springfield RVL-003 Stadtpark Spat_Lab J. Perschy / R. Mathy / M. Wyschka 16:00 • IMPROVe - mobile Phone sound improvisation Parkring U4 Burggarten 15:30 • Craving R. Widerberg 20:30 • taus B. Garnicnig / G. Haider T. Blechman / K. Filip 16:30 • Collaborative Musical Games with PhonePlay A Main Building University of Applied Arts Installations J. Knowles 21:00 • Institute for transacoustic research 1010 Vienna, Oskar Kokoschka-Platz 2 (Stubenring 3) 16:00 – 17:00 • Digital Claiming N.Gansterer / M.Meinharter / J.Piringer / E.Reitermayer 8 foreword 9

Looking Back at Five Years of the MMW by 5 days! Downloadable music was as yet far from into narratives spread across geographical space. these sessions, uncompleted yet on-going projects newer research and development institute active in ed Tactical Sound Garden at the 3rd MMW. Frauke being a commercially viable reality. Napster had al- Other presentations reflected on artistic approaches received critical review and support by peers includ- the fields of networked art, education and creative made the trip to Brooklyn to give the presentation, Frauke Behrendt ready been shut down, but Kazaa was still the pre- to mobile music and socio-cultural impacts of mo- ing Atau, Lars Erik, and Maria Håkansson. Projects industries. make contact with the New York scene, and to raise Lalya Gaye dominant peer-to-peer file sharing system. It was in bility, music consumption and urban culture. included mobile music systems and enabling tech- Keynote presentations at the 4th MMW under- interest in bringing the MMW back to North Ameri- Atau Tanaka this context that the first workshop sought to situate nologies, interface design, art pieces, and philo- scored this breadth. American artist Teri Rueb re- ca for a future edition. the place of music in the landscape of rapidly devel- This edition of the workshop included hands-on ac- sophical approaches to mobile music. The projects traced a cultural approach to mapping out a loca- Introduction oping mobile computing technologies. tivities that focused on “bodystorming” of mobile were grouped according to three key themes: Mo- tion-sensitive audio work. STEIM’s Michel Waisvisz 5th MMW – Vienna music applications and scenarios. With this method bile collaborative music making, Soundscapes & recounted a personal musical history that included It started with two conferences in 2003. At Ubicomp From Lalya’s notes of the first workshop: of structured brainstorming, participants explored mobile music listening, HCI in mobile music and dismantling pianos in childhood, to creating ana- The workshops have drawn an interdisciplinary 2003 in Nottingham, UK, in the same demo room Mobile music is a new media relying on the use of various mobile music themes, developed simple Uses of music in mobile setting. logue live samplers with pedal-controlled Revox group of participants including artists, musicians, were two music demos: Arianna Bassoli’s now leg- mobile computing. This media is characterised by application ideas, and physically enacted scenarios open reel recorders, to conceiving one of the first designers, industry representatives and academics endary TunA, and Atau Tanaka’s Malleable Mobile a tension between place and music creation, listen- of use in order to get an embodied understanding During a hands-on session, participants got direct truly mobile musical instruments, the CrackleBox. Fi- from ubiquitous computing, media studies and so- Music. They looked at the implications of ubiqui- ing or sharing. This tension is created by its mobile of design challenges and opportunities specific to experience with cutting-edge mobile music and nally, Regine Debatty, the indefatigable art blogger, cial sciences. They have featured a range of activi- tous computing technology on music consumption nature and results from the fact that mobile music mobile music. Enacting scenarios stimulated dis- locative audio technology. The tutorials were fol- shared her insights on her continuously evolving vi- ties that go beyond the typical conference set-up, and creation. Around the same time in Paris, Lalya devices can be used anywhere but at the same time cussions on mobile ways of sharing and projecting lowed by bodystorming sessions where participants sions of the creative potential of technological, bio- most notably with hands-on, critique, and structured Gaye and Ramia Mazé’s Sonic City was presented have an awareness of place. When becoming mo- music in public space, and of social meaning and sketched application ideas for mobile music using logical, and cultural forms, and put theory into prac- brainstorming sessions. This year’s workshop is host- at the UIST conference, looking at ways that the ur- bile, the media is displaced from a particular loca- adequacy. the presented technology as well as simple audio tice through active coverage of the workshop on her ed by the University of Applied Arts Vienna, and ex- ban environment could become an input to a por- tion to the movements of a particular user, provid- devices as props. blog, We Make Money Not Art. tends the creative emphasis of the MMW from the table generative music system. A kind of community ing a soundtrack to her life as the Sony Walkman™ Siggraph 2005 musical focus of STEIM towards the visual arts and emerged in email discussion afterwards sharing the once did, but with additional properties such as NIME 2006 Holding the workshop across two locations in the design disciplines. The activities of the Spat_Lab, idea that the technologies presented in scientific networking capabilities (enabling music sharing with The Mobile Music community broadened its hori- vibrant city of Amsterdam permitted a real engage- introduced by Nicolaj Kirisits and described in this conferences could be harnessed for musical means. strangers in public space), or context-awareness (en- zons in August of that year by organizing a panel Like it had at Siggraph, the MMW in 2006 reached ment with the urban environment. Cathy van Eck’s volume, are consistent with the spirit of the MMW, This all took place before the commercial intro- abling certain forms of mobile music making). discussion on Ubiquitous Music at ACM SIGGRAPH out beyond the discipline to present a report on Hearing Sirens led workshop participants like a Pied and created the context for the 2008 partnership. duction of Walkman-branded mobile phones and in Los Angeles. As the largest international confer- the emerging field at the New Interfaces for Musical Piper through the streets and canals of Amsterdam Apple’s iPhone put mobiles and music in the public The first workshop had an open call, and attracted ence on digital media and emerging technologies, Expression (NIME) conference. The connection to in a sonic walk from Waag to STEIM. With Crav- Creative Mobile Music consciousness. In the midst of rapid commercial and fifteen participants. Aside from the projects already the SIGGRAPH panel underscored the pertinence NIME is particularly relevant to MMW - the notion of ing, Bernhard Garnicnig and Gottfried Haider from consumer takeup, it is the focus on creativity that mentioned, notable contributions came from Mat- of mobility and musical interaction to a wider field. real time interaction in music through sensors, net- Spat_Lab used location and orientation sensing to Key themes on mobile music that have emerged makes the Mobile Music Workshop (MMW) special, tias Östergren in what is to this day the only working Presentations included those by participants of the works, and other inputs are a core concern of NIME situate a theatre work of Sarah Kane in the plaza over the five years of the workshop include creat- and gives it continuing relevance. car-oriented system presented at the workshops, first two workshops, including Gideon D’Arcangelo, that MMW draws heavily upon to extend the no- outside of Niewmarkt. The Handy Dandy captured ing, sharing and design. Mobile music is concerned Chris Salter’s contribution on notions of play, and a a well known American radio journalist. tion of mobile music beyond the simple download- the attention of a local newspaper by staging a rock with the urban environment as musical interface, First MMW - Göteborg poster on mobile sound art by Frauke Behrendt who play metaphor. This was underlined by holding the band of air guitars on Bluetooth mobile phones. It for location-aware sound art, audio annotation of was to go on to organize the 3rd workshop. Sum- With the title “ubiquitous music,” we sought to second workshop during NIME’04 in Vancouver, was this group of six, spread across two projects and physical space, and other creative applications. New The initial contact made at Ubicomp and UIST led maries from these and other projects appear later in maintain the open-ended creativity and research much in the way that the first NIME took place as a led by Nicolaj Kirisits, that would form the organiz- form-factors and content formats explore how mu- to continued exchange and discussions about po- this catalogue. orientation of the community, in a time when the workshop within the ACM conference on Computer ing team of this year’s MMW. sic and mobility can be addressed in design meth- tential collaborative projects, culminating in the idea term “mobile music” was being rapidly co-opted Human Interaction (CHI’01). Meanwhile research ods. Wider discussions have addressed relations by Lalya’s doctoral supervisor, Lars Erik Holmquist, 2nd MMW - Vancouver by industry to promote use of mobile phones as ba- questions that the MMW was confronting, includ- Conflux 2007 between the body, space and sound, synchronicity, to organize a workshop. The idea was to bring to- sic MP3 players. Siggraph was an environment that ing issues of creative engagement as implemented foreground/background activities, and the social ac- gether the research Arianna was conducting at Me- The second workshop was organised in May 2005 represented the intersection of conference, festival, and deployed on consumer devices, were increas- In September 2007, the workshop series was pre- ceptance of new behaviors in public space. diaLab Europe, that Atau was doing at Sony CSL as part of the NIME 2005 conference in Vancouver, and industry trade show, at once creative, academic, ingly pertinent to NIME as students and researchers sented to the North American community at the Paris, and that Lalya was doing at the Viktoria Insti- Canada. By now, the community was better defined, and commercial. increasingly became interested in miniaturized, por- Conflux festival. Conflux is a New York festival of This catalogue summarizes the projects, presenta- tute, and to open things up to the research commu- and the workshop time was shared between presen- table electronics. contemporary psychogeography where projects tions, and performances from the first five years of nity at large, and to make ties with industry. Under tations of new projects, demos, in depth-discussions 3rd MMW - Brighton investigate “everyday urban life through emerg- the Mobile Music Workshop. In this time we have Lars Erik’s initiative, the first Workshop on Mobile and hands-on brainstorming activities. It attracted 4th MMW – Amsterdam ing artistic, technological and social practice.” The defined a field of research and artistic practice. A Music Technologies took place on the 10th and 11th 18 external participants. The third edition of the workshop took place in urban/public/guerrilla art context proved to be a community has formed around this field and the of June, 2004 in Goteborg, Sweden. March 2006 at the University of Sussex, Brighton, The MMW went to Amsterdam in May 2007, co- very fruitful one for mobile music. It was a gathering workshop. We hope that this document will serve Presented projects ranged from Solarcoustics, a de- UK. It gathered nearly 30 participants and focused organized by STEIM (Studio for Electro-Instrumen- of the U.S. locative media scene that included local to inform and inspire future work in the area, and The first workshop was a scoping event, an event to vice that enabled its user to manipulate sounds and on the locative media aspect of mobile music. The tal Music) and the Waag Society for Old and New grassroots projects, and presentations by curators at look forward to the next five years of creative mo- bring together the beginnings of a young commu- create music based on ambient lighting conditions, program consisted of a keynote by “Professor iPod” Media. Kristina Andersen and Ronald Lenz were the Rhizome and the Whitney Museum of American Art. bile music! nity and to define a new research field. It is interest- to a mobile user-interface platform for mobile music Michael Bull, critical discussions, hands-on activities, organizers of this edition of the workshop, and rep- The psychogeography concept was not unfamiliar ing to note that the date of the first MMW preceded making based on a personal area network (PAN), to and break-out sessions where participants received resented the collaboration between one of the orig- to the MMW community, as one of the artists using the European launch of Apple’s iTunes Music Store Location33, a project that transformed music albums feedback on their work-in-progress projects. In inal studios in experimental electronic music and a the banner was Mark Shepard, who had present- 10 2008 University of applied arts, VIENNA 11

5th Mobile Music Workshop 12 2008 papers 2008 papers 13

MoGMI: Mobile Gesture Music Instrument Developments and Challenges turning Mo- bile Phones into Generic Music Performance Amnon Dekel, Gilly Dekel Platforms

Abstract Georg Essl, Ge Wang, Michael Rohs

The MoGMI project that explores ways of enabling Abstract the mobile phone to become a musical instrument University, and is currently an assistant professor at for naïve users. Using the 3 dimensional acceler- There has been an ongoing effort to turn mobile Stanford University in the Center for Computer Re- ometer on the Nokia N95 users can record musi- phones into generic platforms or musical expression. search in Music and Acoustics (CCRMA). His research cal pieces using physical gestures. We developed By generic we mean useable in a wide range of ex- interests include interactive software systems (of all an application that allowed them to select one to pressive settings, where the enabling technology has sizes) for computer music, programming languag- three musical instruments and create music with minimal influence on the core artistic expression itself. es, sound synthesis and analysis, music information them. An additional application lets users play a We describe what has been achieved so far and out- retrieval, new performance ensembles (e.g., laptop drum-set. This initial study explores which acceler- line a number of open challenges. orchestra) and paradigms (e.g., live coding), visual- ometer axis mapping model is preferred by users. ization, interfaces for human-computer interaction, in- Do they prefer a model in which each of the motion bio teractive audio over networks, and methodologies for axes are mapped to a different instrument or one in education at the intersection of computer science and which the motion affects volume, pitch and attack Georg Essl is a Senior Research Scientist at the music. Ge is the chief architect of the ChucK audio of a single instrument. Results show that subjects Deutsche Telekom Laboratories at the Technical Uni- programming language and the Audicle environment. preferred the three axis model in which every axis is versity of Berlin in Germany doing research in human He was a founding developer and co-director of the mapped to a different dimension of music genera- computer interaction (mobile interactions, tangible Princeton Laptop Orchestra (PLOrk), the founder and tion (attack, amplitude, and pitch). This mapping & physical interactions) and sound synthesis (physical director of the Stanford Laptop Orchestra (SLOrk), was deemed better by subjects over simpler or and mathematical models). Before coming to Ber- and a co-creator of the TAPESTREA sound design more complicated mapping models in three of five lin, he worked as a post-doctoral researcher at MIT’s environment. Ge composes and performs via vari- dimensions (easier to learn, produces ‘’nicer’’ mu- MediaLab Europe with Sile O’Modhrain on tangible ous electro-acoustic and computer-mediated means, sic, and in how easy it is to understand the relation- interactions. PebbleBox, a tactile interface for sonic including with PLOrk/SLOrk, with Perry as a live cod- ship between gestures performed and the music performance joint with Sile O’Modhrain and Andy ing duo, and with Princeton graduate student and that is subsequently generated). Brady was invited to the Touch Me exhibition at Vic- comrade Rebecca Fiebrink in a duo exploring new toria and Albert Museum, London in 2005. While at performance paradigms, cool audio software, and bio Media Lab Europe, he participated in the Enactive Eu- great food. ropean Network of Excellence, which studies the role Amnon Dekel thrives at the intersection point of of action in interaction design. Between 2002-2003 I Michael Rohs is a senior research scientist with four disciplines: research in novel human computer was Assistant Professor in Computer and Information Deutsche Telekom Laboratories at TU Berlin. His re- interaction, information technology, digital media Science and Engineering at the University of Florida, search interests are in mobile and pervasive interac- and university level lecturing. The twenty five years where he taught signal processing and synthesis of tion and comprise interfaces at different scales, rang- since Amnon programmed his first computer (a ZX- sound and digital production. He got my Ph.D. in ing from handheld device screens to large public 80) have been spent by him in studying computer Computer Science from Princeton University in 2002 displays, the integration of physical and virtual aspects science, being an Air Traffic Controller, getting a working with Perry Cook on physical simulation of of the user’s environment, and sensor-based mobile BA and an MA degree in cognitive psychology, an musical instruments. He is a member of the IEEE, interaction. His research currently focuses on small- MPS degree in Interactive Telecommunications, the Acoustical Society of America (ASA), the Interna- display interaction, in particular navigation and visu- producing award winning and Pulitzer nominated tional Computer Music Association (ICMA), and the alization techniques for spatially aware displays. An web sites, designing and developing desktop and American Mathematical Society (AMS). He has been example is using camera phones as magic lenses for web based applications, developing interactive dig- technical chair for the International Computer Music large-scale paper maps in order to overlay personal- ital art installations, teaching, industry consulting, Conference in 2004 and 2006. Currently he serve as ized, up-to-date information. As part of his doctoral co-founding an AOL funded internet startup (Earth- Research Coordinator of the International Computer dissertation he developed camera-based interaction noise.com) which was a pioneer in Video Sharing Music Association. techniques for mobile devices, like optical flow con- (5 years too early!), and as of 2005, working on his trol for large public displays and a marker recognition PhD in Computer Science (focusing on merging the Ge Wang received his B.S. in Computer Science in system for camera phones that uses device orienta- physical world with the world of digital information 2000 from Duke University, PhD (soon) in Computer tion as an input parameter. His homepage is available networks). Science (advisor Perry Cook) in 2008 from Princeton at http://www.deutsche-telekom-laboratories.de/~rohs 14 2008 papers 2008 papers 15

bio

Pedro Rebelo is a composer/digital artist working in electroacoustic music, digital media and installation. His approach to music making is informed by the use of improvisation and interdisciplinary structures. He has been involved in several collaborative projects with visual artists and has created a large body of work exploring the relationships between architecture and music in creating interactive performance and installation environments. This includes a series of commissioned pieces for soloists and live-electronics which take as a basis the interpretation of specific acoustic spaces. In the duo laut with saxophonist Franziska Schroeder he investigates the extension of interfaces and control in interactive performance practices. His electroacoustic music is featured in various CD sets (Sonic Circuits IV, Discontact III, Exploratory Music from Portugal, ARiADA). Pedro conducts research in the field of digital media, Some Challenges Related to Music and interactive sound and composition. His writings re- Movement in Mobile Music Technology flect his approach to design and composition by ar- ticulating creative practice in a wider understanding Alexander Refsum Jensenius of cultural theory. Pedro has been awarded a PhD in composition from Abstract the University of Edinburgh and is currently Direc- tor of Research at the Sonic Arts Research Centre, Mobile music technology opens many new oppor- Queen’s University Belfast. tunities in terms of location-aware systems, social interaction etc., but we should not forget that many A typology for Listening Place challenges faced in ”immobile” music technology At present Matt Green is partaking in a PhD Stu- research are also apparent in mobile computing. Pedro Rebelo, Matt Green, Florian Hollerweger dentship at the Sonic Arts Research Centre (SARC), This paper presents an overview of some challeng- Queen’s University, Belfast, UK. This research rep- es related to the design of action-sound relation- Abstract resents a partnership between the school and the ships and music-movement correspondences, and Hewlett-Packard Media labs, Bristol, UK. The PhD suggests how these can be studied and tested in Sound technologies, particularly mobile and locative research focuses on the situated sound sensitive sys- mobile devices. media technologies, can provide unique listening tems; that is pervasive technologies positioned in experiences within situations that are not themselves space that sympathize with the surrounding sound bio exclusive zones for sonic projection, meditation or field as a means to understand, preserve, develop exploration. This paper seeks to contribute to the or distort our sonic experiences within certain places. Florian Hollerweger was born in 1980 in Linz, Aus- Alexander Refsum Jensenius (BA, MA, MSc, PhD) understanding of locative sound design by present- As part of, or separate to, his studies Matt Green has tria. He works as a sound artist, programmer, sound is a music researcher and research musician working a framework consisting of three spatial arche- partaken in several Sound art commissions. These in- engineer, and performer and has performed his own ing in the fields of embodied music cognition and types: the Theatre, the Museum and the City. These clude a permanent interactive sonic entrance space pieces as well as collaborative works with ‘Pd~graz’ new interfaces for musical expression (NIME) at the serve as metaphors through which we can articulate in the Perth Concert Hall, Scotland, UK (2005) and a and others in the United States, Canada, and various University of Oslo and at the Norwegian Academy different types of relations between listener, sound pervasive networked exploration at the Future Sonic countries across Europe. Florian has studied elec- of Music. He studied physics, informatics, math- and place. The Mobile Music Player has been cho- Festival, Manchester, UK (2006) titled ‘Bump!’. Very tronic and computer music in Austria, California, and ematics, musicology, music performance and music sen as an example of a listening condition that both recently his proposal “In hear, Out there” was se- currently at the Sonic Arts Research Centre in Belfast, technology at the University of Oslo and Chalmers characterises and traverses the Theatre, the Museum lected for the Inclusiva.net Locative media workshop Northern Ireland, where he is investigating strategies Institute of Technology, and has been a visiting re- and the City listening archetypes. (2008) held at the MediaLab-Prado in Madrid, Spain. for the design of social listening environments. searcher at UC Berkeley and McGill. 16 2008 papers 17

Real-Time Synaesthetic Sonification of Traveling Landscapes

Tim Pohle, Peter Knees

Abstract

When traveling on a train, many people enjoy looking out of the window at the landscape passing by. We present an application that translates the perceived movement of the landscape and other oc- curring events such as passing trains into music. The continuously changing view outside the window is captured with a camera and translated into midi events that are replayed instantaneously. This allows for a reflection of the visual impression, add- ing a sound dimension to the visual experience and deepening the state of contemplation. The application can both be run on mobile phones (with built- in camera) and on laptops (with a connected Web- cam). Several techniques to transfer the captured images to audio are possible. Most interesting and pleasuring results were achieved by an approach bio that utilizes a tone-to-color mapping like the one of the “Clavier à lumières” by Russian composer Tim Pohle was born in Linz, Germany. He has stud- and pianist and self-claimed synaesthete Alexander ied musicology and graduated in computer sci- Nikolajewitsch Skrjabin. Due to a steady capturing ence. Over the years, he played a number of clas- rate of seven frames per second, there is a clearly sical and modern instruments, and was member in noticeable basic rhythm pattern in the music, which various indie-pop bands. Currently, he is writing his the listener may associate with the steady progres- Ph.D. thesis in Linz, Austria in the field of Music In- sion of the train. Depending on the landscape, formation Retrieval. notes in some bands are played in fast repetition or http://www.cp.jku.at/people/pohle movements, while in other bands they sound only sporadic. Also, a changing landscape is reflected Peter Knees was born in Vienna, Austria. In January in the resulting music, while the overall feeling re- 2005, he graduated in computer science from Vien- mains the same. na University of Technology. Since February 2005 he Generating panoramic pictures from the captured has been working as a Project Assistant at the De- landscapes exhibits some interesting effects caused partment of Computational Perception at the Jo- by the movement of the train. Since frame rate and hannes Kepler University Linz where he performs position of the camera are both static, proximity of research towards a doctoral thesis with a focus on objects and slope and velocity of the train result in Music Information Retrieval. Other research inter- characteristic visual effects. For example, objects ests include Multimedia and Artificial Intelligence. that “move” at high speeds are displayed very nar- Since 2004 he has been studying Psychology at the row, whereas objects filmed at low speeds appear University of Vienna. stretched. http://www.cp.jku.at/people/knees 18 2008 papers 2008 POSTERS 19

Caressing the Skin: Mobile devices and undersound and the Above Ground bodily engagement Arianna Bassoli, Johanna Brewer, Karen Martin, Franziska Schroeder Iacopo Carreras & David Tacconi bio Abstract Abstract Arianna Bassoli holds an MSc in Communication This text examines mobile devices by looking at the undersound is a new type of experience, an appli- Sciences from the University of Siena, Italy, where tactile interaction of the human body with the tech- cation for mobile phones designed for a specific she specialized in mass media. She then worked nological device. I show that the body is rendered situation, traveling by the London Underground. as a research fellow at Media Lab Europe for three performative by engaging with a device and I draw undersound is a way of listening to, distributing years, mainly focusing on the application side of on a performer’s interaction with a musical instru- and affecting the flow of music in the Underground mobile peer-to-peer and ad-hoc networks. She is ment to support this argument. This tactile interac- that goes beyond just the music itself. It is meant currently a PhD candidate at the London School of tion also exposes the tension between the bodily to allow people to see their journeys, the people Economics and Political Science, UK, in the Depart- intimate, as experienced through the skin, and the around them, and the Underground in a new light. ment of Information Systems and Innovation Group. more distant, as represented by the technologi- bio undersound is designed to be spatially distributed She is interested in interaction design, urban com- cal device. I argue that recent design aesthetics are throughout all the Underground network. Musi- puting, and the design of mobile proximity-based driven by the urge to bring the device closer to the Franziska Schroeder is a performer of saxophone cians can add Creative Commons-licensed songs to applications, technologies that support commu- bodily intimate, closer to the skin. and live-electronic music, an improviser and theo- the system at upload points in the ticket halls, and nication and data sharing among co-located peo- rist. She is the founder of the digital media collec- commuters can download songs on the platforms. ple. Arianna is also a research assistant at the LSE, I show that the complexities of the human touch as tive l a u t with composer/pianist Pedro Rebelo. Each song is then tagged with its place of origin working on the EU-funded project BIONETS, which fronted by the skin that allow the human body to Franziska plays in the free improvisation trio “Faint” and this information is visible as the track is being looks into the future of wireless networks. navigate the world in intricate ways become cen- with percussionist Steve Davis and pianist Pedro played. This may trigger memories and musings http://www.karmanet-design.com/ tral to these design aesthetics. For this argument Rebelo. The trio has recently released their first re- around people’s personal relationship to that place. I examine touch by looking closely at the skin and cording on the Creative Source Recordings label. While in the carriages of the Underground, people Johanna Brewer is a PhD candidate in the Informat- at the ways that the skin has been understood over In 2006 Franziska was awarded her PhD by the can browse undersound music of other people in ics department at the University of California, Irvine several centuries. The skin will be examined with School of Arts, Culture and Environment at the Uni- range. Because the system is meant to keep track working with Paul Dourish. She also holds an MA view to its essential position to the perception of versity of Edinburgh, UK. Her current research in- of songs’ spread within the network and the num- in Computer Science as well as BA’s in Computer self, aided by the psycho-analytical interpretation terests include the intersection of philosophy and ber of times they have been played, people can see Science and Philosophy, all from Boston Universi- developed by Daniel Anzieu. It will become clear performance in technology-informed environments, all this information when they look at each other’s ty. She is interested is urban computing, particu- that, historically, the skin was mainly seen as a cov- in particular the role of the body in the age of tech- music. People can then download music from oth- larly in the design of technologies which can forge ering that kept the body together. It was then ex- nological change. Franziska has written for many in- ers in proximity; when this happens, an alert mes- new types of connections between people and posed in the Medieval period as an organ of inter- ternational journals. She has guest-edited a double sage tells users that someone has grabbed a song transform or reinforce old ones. Her dissertation re- change, more akin to a permeable membrane. With issue for the Contemporary Music Review Journal from them. This constitutes a subtle form of com- search centers around how an examination of mo- the release of the taboo of cutting the skin in the (Routledge) and is on the editorial board for the munication able to provide social awareness but bility in urban spaces, specifically the London Un- European Renaissance the perception of the skin ARiADA (Advanced Research in Aesthetics in the not to disclose people’s identity or location. Each derground and the Orange County Transit systems, alters immensely, and the skin is finally exposed as Digital Arts) online journal, UK. of users’ interactions then contribute to a broader might help to inform these designs. an entire environment, as a meeting place for the Franziska performs with improvisers from the UK trend; every time people listen to a song, drop one other senses. and Europe in actual and virtual worlds. She leads off at transfer point or download music from some- Karen Martin is an EngD candidate at University In this paper I highlight that the multi-touch inter- an avatarist existence in SecondLife. one else, there is an effect on the overall state of College London, currently investigating the articu- faces of recent mobile devices allow for the multi- Since April 2007 Franziska has been based at the the system. This information is incorporated into lation between social, spatial and telecommunica- plying functions of the skin to come into being by Sonic Arts Research Centre in Belfast studying Real- public displays that are meant to be installed in tions networks in urban environments, and devel- engaging the body in various gestural moves, by Time Performance in Virtual Worlds. She is funded each of the stations. These displays serve to convey oping methods for designing for mobility in the city. providing conditions for participation, rather than by the UK’s Arts and Humanities Research Council the most recent state of the undersound network, Her background is initially in interactive arts and by simply presenting functions of control that are Fellowship scheme. and function as visual representations of the sum of she obtained an MSc Virtual Environments from the still highly characteristic of many design aesthetics. http://www.lautnet.net all the individual actions shaping that network. Bartlett School of Architecture in 2003. 20 2008 POSTERS 2008 POSTERS 21

Mobile Tangible Interfaces as Gestural Instruments An Augmented Reality Framework for Wireless Mobile Performance Fares Kayali, Martin Pichlmair, Petr Kotik

Mike Wozniewski & Nicolas Bouillot, Zack Settel, In this paper we describe gestures for the inter- Jeremy R. Cooperstock action with tangible mobile interfaces. From the strumming of a guitar’s strings to the beating of Abstract a drum’s decks, traditional musical instruments are played by performing gestures shaped by the We demonstrate that musical performance can take physical representation of the instrument. Since the place in a large-scale augmented reality setting. musical ouput of digital instruments is not defined With the use of mobile computers equipped with by their physical appearance, their interface can be GPS receivers, we allow a performer to navigate structured more freely. Tangible interfaces put this through an outdoor space while interacting with kind of flexibility into practice. an overlaid virtual audio environment. The scene is In order to explore gestures for musical interac- segregated into zones, with attractive forces that tion we proceeded exploratively. The described keep the virtual representation of the performer gestures were derived from three prototype instru- bio locked in place, thus overcoming the inaccuracies ments featuring distinct musical environments we of GPS technology. Each zone is designed with par- developed over the last year. They were implement- Fares Kayali is a viennese artist and researcher fo- ticular musical potential, provided by a spatial ar- ed for the Nintendo DS platform and offer different cusing on game studies and interactivity with mu- rangement of interactive audio elements that sur- approaches to gestural interaction with music. sic. He used his audio-visual performance software round the user in that location. A subjective 3-D The first prototype is a very simplified guitar. Strum- Sonic~Image for various live performances in Aus- audio rendering is provided via headphones, and ming and grabbing chords are abstracted to a tria and abroad. Further projects include interac- users are able to input audio at their locations, single gesture. The player strums the individual tive media installations and video art. His scientific steering their sound towards sound effects of inter- frets of the guitar with the DS’ stylus, triggering work as a project associate and Ph.D. student at the est. An objective 3-D rendering of the entire scene pre-recorded chords. The second prototype is a Institute of Design and Assessment of Technology can be provided to an audience in a concert hall or synthesiser instrument that is almost solely played at the Vienna University of Technology centers on gallery space nearby. with the stylus. The touchscreen is used as playing playful musical interaction in video games. field. The player plays the instrument by either tap- bio ping the screen for individual tones or by sweeping Martin Pichlmair is a media artist living and working across it to produce continuous sounds. In the third in Vienna, Austria. Since he received his doctoral Mike Wozniewski is a freelance researcher, with a prototype, Thumbtack, the player acts in a playful degree in informatics he works as assistant profes- focus on real-time interactive systems, immersive musical environment. Four moving widgets (sound sor at the Institute of Design and Assessment of environments, human motion tracking, sensor inter- agents) can be played with using the stylus to hold, Technology at the Vienna University of Technol- faces, and 3D audio/graphics. Currently, he works drag and throw them around. The widgets obey ogy. His art pieces were shown at various media with institutions such as the Centre for Intelligent simple physical rules. Each of them has a unique art festivals and exhibitions. Recent shows includ- Machines at McGill University, and the Society for sonic characteristic. Every collision among the wid- ing the Ars Electronica Festival, ISEA, Transmediale Arts and Technology [SAT] in Montreal. Recent proj- gets or with the border of the playing field triggers and the Microwave International Festival for New ects (see www.mikewoz.com) involve methods for a distinct sound. The player is thereby enticed into Media Art. In his research and publications, he fo- modelling and controlling 3D audio in virtual envi- playfully creating lasting rhythmical patterns. cuses on theory and practise of interactive art and ronments, as well as research in large-scale mobile Our research resulted in a number of suitable ges- design - from game design and physical interfaces audio applications and multi-user sound installa- tures for musical expression with mobile tangible to open source development models and commu- tions. interfaces. nity media. 22 2008 POSTERS 23

soundFishing

Claudio Midolo

Abstract

The soundFishing interface is a portable digital device able to analyze the sonic environment sur- bio rounding the user and, based on certain rules, au- tonomously capture sound snapshots out of it. Thinking about the visual and sonic human sam- Education pling activity it is clear that a huge gap exists be- 2007/2008 tween the two practices as the first is immensely Graduate MFA Design and Technology student at more popular and inflactionated compared to the Parsons, The New School, second. This situation probably emerges from the New York City. sensorial prominence of Sight over Hearing and the Academic honor - Graduate Dean’s Scholarship. first consequence that takes place is a severe loss in the form of sonic memories. The soundFishing proj- 2004/2007 ect tries to suggest a possible solution to this is- Undergraduate Digital Design student at Istituto sue, in the form of a device able to save these son- Europeo di Design in Milan. ic fragments from oblivion and present them to the Top marks 100/100 cum laude. user, stimulating its curiosity towards these ,other- Thesis: wiiwiiwiiwii - audio/visual musical instrument. wise lost, perceptions. The proposed user scenario Academic honor - Creative Lab Teaching Assistantship. sees the interface being first configured by the user who sets a rule which will control the recording ac- 1998/2003 tivity. Then the device is carried around by the user High school diploma in Sciences from Liceo G.B. for the rest of the day, left alone listening to the au- Grassi in Saronno ral stream of the user’s life. Once back home again Top marks 100/100. the user would listen to the collected sounds which matched the rule set at the beginning. The first, Summer 2006 simpliest output would be an unconsciously filled Summer courses student at New York University sonic diary, illustrating various aural events which Tisch I.T.P. took place during the course of the day. This sound collection would also stimulate curiosity as it cap- Works tures and shows the richness of variety of possibili- Creative laboratory assistant at Istituto Europeo di ties that live within the sonic layer usally is taken for Design. granted. These captured fragments of sound can 2007 then be valuable also to other people such as musi- Freelance media design. cians and audio producers, who can use and share 2005/2007 them as creative assets. In conclusion the key to re- Graphic design at Genesio institute in Milan. ally grab the essence of this project is held by the 2005 concept of curiosity, a virtue that can turn something usual and useless into something unique and Exhibitions meaningful, a powerful entity that can open the April 2008 Invited and performed at Node08 festi- door of knowledge to all of us. val with wiiwiiwiiwii project , Frankfurt. 24 2008 performances 25

bio noid /aka Arnold Haberl, *1970, living in Vienna http://noid.klingt.org/ Klaus Filip http://lloopp.klingt.org/plone/lloopp/ working as a composer, performer and improvisor in various constellations “rahmenbedingung”- sonification of the Almost all of Klaus Filip’s art projects have been poetic act of cycling driven by technological possibilities and the social with his music he tries to understand the reality of need to change structures. Among them subVoice sound we live in. of course this reality is including klaus filip_nicolaj kirisits_noid_silvia faessler (an underground tapemagazine), Sigis Bruder (ear- imaginations, wishes, dreams and acoustical halluzi- ly electronic songs together with singer Sigi Ecker), nations as well as the sound of the fan of his laptop Abstract Christof Kurzmann’s Orchester 33 1/3, Zentrifuge, or the wolf-tone of his cello. music for short films, theatre, dance, sound-instal- his sensual approach, once in a while assisted by a bicycle equipped with sensors and connect- lations. He is the musical and electro-mechanic fa- structural concepts, can have a wide range of con- ed to a sound-computer via wireless device forms ther of BigBaby, an outstanding intermedial project tradictory outcome, that is always to be understood the personal musical interface of each performer. around a sculpture build by Red White and brought as a concentrate (essence), leaving out irrelevant the organic data-stream of parameters like direc- to life by the movements of Cynthia Schwertsik. points. It’s up to the listener to extract a digestable tion, speed, acceleration, pedal-speed, pedal rota- Filip is the inventor and never sleeping developer dose. tion etc. is bound to the inner logic of riding a bike of the open-source software lloopp (http://lloopp. - hence we don’t want to fall down. the outer logic is klingt.org), a musical instrument on the computer to the architectural determination of the space, the bi- provide open structures for live-improvisation, used Silvia Fässler cycle path, as well as the position of the audience. by many well-known electronic musicians. http://gnu.klingt.org/03_releases/00_skylla.html these general conditions structure the music, on the other hand we will ride the bike in a musical way Appearances among others: Zeitfluss Festival/Salz- which defines our movements in space and evokes nicolaj kirisits burg, Unlimited/Wels, Porgy&Bess/Vienna, solo / a choreography. architect composer digital artist duo with “silly” (billy roisz). lives and works in vienna / casablanca Cooperations among others: Cordula Bösze, Klaus http://to.sonance.net/rahmenbedingung teaches at the university of applied arts vienna Filip, Otomo Yoshihide, Arnold Haberl. 26 2008 performances 27

Collaborative Musical Games with PhonePlay

Josh Knowles

Abstract

PhonePlay is a system of software designed and developed by Josh Knowles which allows many people to interact with a single screen and sound Two musical games have been developed using system at the same time. Users call a phone num- this system and will be displayed at the Mobile Mu- ber using any telephone and push numbers on their sic Workshop. phone to interact with the PhonePlay game in real “3001” was designed by Josh Knowles and Joo time. A series of musical games have been devel- Youn Paek at NYU. It is a paddle-style game. Balls oped using PhonePlay which allow the audience to drop from the top of the screen and each player interact with a performance using simple game-like gets their own paddle to control when they call controls. This has proven to be a very fun and entic- in. Depending on how the paddles and the balls ing project that has been displayed and performed bounce and interact, different sounds are made. around the world. This piece was first performed at the New Inter- PhonePlay (http://gophoneplay.com) was origi- faces for Musical Expression conference in New nally developed by Josh Knowles as a part of his York City in 2007. For more information and video, thesis at New York University’s Interactive Telecom- please see: http://gophoneplay.com/nime/ bio munications Program in 2007. Built entirely on open “Blocks” was designed and developed by Josh source software, PhonePlay runs on any Mac, Win- Knowles for a public installation on the front of Josh Knowles is a recent graduate of New York Uni- dows, or Linux computer and numerous people at the world headquarters of Digium, the company versity’s Interactive Telecommunications Program. once to call into a phone number and interact with that develops Asterisk, the open source telephony Before earning his Master’s at ITP, Josh worked as the system in real time using the buttons on their platform at the heart of PhonePlay. In this musical a software developer and ran Frescher-Southern, phones. No special mobile phone software is re- game, each player calls in and controls a “hand” a live electronic music and video events organiza- quired and it works on 100% of phones. which can drop different sorts of blocks down to tion in Austin, Texas. He is also a former director at Josh has a background in electronic musical per- the bottom of the screen in piles. Depending on the Austin Museum of Digital Art and holds a Plan formance stretching back over a decade. Finding how these blocks fall and are arranged, different II Honors (literature, philosophy, creative writing) new ways to interact with and involve the audience musical events occur. For example, a tall stack of undergraduate degree from the University of Texas. in live musical performance has been one of his blocks would cause an arpeggiation of notes to oc- Currently Josh works as a social software developer long-standing goals. PhonePlay has made truly di- cur. A shorter pile would cause chords and other and game designer with a variety of organizations rect group audience interaction with electronic mu- short sequences to occur. For more information, including NYC-based Area/Code Games. Josh cur- sic possible. please see: http://gophoneplay.com/digium/ rently lives in Brooklyn, New York. 28 2008 performances 29

Tango Intervention, Vienna

Robert Lawrence

Abstract

Every city has its hidden histories. „Tango Interven- tion, Vienna“ uses locative technology, GPS cell phones, the Internet, aggressively remixed argentine tango music and public dance interventions to reveal hidden stories and histories along a narrow path through Vienna. The beautiful spectacle of couples dancing to argentine tango in unexpected public places is the public‘s entry into this layered mediation on the meaning of musical tradition, place, history, migration and identity. First encoun- tered by the public as a romantic, and somewhat absurd gesture, this musical intervention takes on a very different meaning when people go to the „Tango Intervention“ website or call the phone bio number and listen to the GPS triggered messages there. The phone messages and the website Lawrence‘s interdisciplinary work combines ele- critically recontextualize the seemingly timeless ments in the physical world and virtual elements on dance performance in very specific histories of the the Internet to examine the way life is now lived in locations in Vienna in which the Tango Intervention two realms of the real and the virtual. He received is taking place. The colonial, post-colonial and his MFA from the University of California at San neo-colonial history of Argentine Tango music is Diego. His work has been exhibited internation- remixed and used as a lens to examine hidden his- ally, and he has received numerous fellowships and tories in the streets of Vienna. By making a public awards including: Fulbright 10 Month Research and spectacle of the intimate social dance Tango, and Teaching Fellowship, NEA/Rockefeller Grant for combining this with specific local histories, all in a Interdisciplinary Projects, Bush Foundation Artists context in which people can contribute their own Fellowship, Intermedia Arts Mcknight Foundation stories and histories, the work creates an interactive Fellowship, Jerome Foundation Grants for Book meditation on private and public, the historical and Arts and for Media Arts Installations, and Film in the the timeless, and on the meaning of musical tradi- Cities Regional Grants for Film/Video. Lawrence is tion and ‚place‘ in a geo-mapped age. Associate Professor and MFA Coordinator in the School of Art And Art History at the University of http://www.TangoIntervention.org South Florida. 30 2008 PERFORMANCES 31

IMPROVe bio

Richard Widerberg, Zeenath Hasan Richard Widerberg Zeenath Hasan http://www.riwid.net PhD candidate Abstract http://webzone.k3.mah.se/K3ZEHA Selected Activities The everyday sounds that we experience are pro- Dånk! Collective. Göteborg, 2008. Member. ethnographic studies duced outside of our own volition. The capacity to GEIGER (Gothenburg Electro-acoustic Institute for Mediating between the technology developer, the capture sounds, however, was not possible till the Grants, Events and Research), 2008. Board Member. working team, domain experts and the technology invention of electro-magnetic recording devices in Pixelache Festival of Electronic Subcultures, Helsin- actor at the research site. the early twentieth century. Since then, the sepa- ki, April 2007, Performing. . Audio technologies among rural populations . ration of sound from its source, and the capabil- Ars Electronica Campus Exhibition, Linz, 2006, Ex- Nokia Mobile Entry Product . India ity to play it back, has made it possible to listen to hibiting and performing. . ICT initiatives for rural connectivity . Hewlett Pack- sounds outside of its original context. The mobile Interferenze New Arts Festival, San Martino Valle ard e-Inclusion . India phone is also a medium through which sounds are Caudina, 2006, Workshop and performance. . Smart textiles for urban youth . Philips Research . heard outside of their original context. However, Futuresonic Urban Festival for Art, Music and Ideas, Finland the normative definition of the mobile phone as a Manchester, 2006, Presentation. medium for communication has restricted its po- Digital Art Weeks Symposium, ETH University, Zur- productions (select) tential as a medium for sounds that exist outside of ich, July 2006, Performance. Enabling platforms for creativity and cross practice the immediate tele-communication. IMPROVe is a The Nursery, Stockholm, 2000-2004. Board Member. collaboration. design and research project that explores the po- The Interactive Art of Öyvind Fahlström, Baltic, . curator . Netfilmmakers 13th Edition . netgallery tential of the mobile phone as a medium of com- Gateshead, England. 2002. Exhibiting. for netfilm, netvideoart and netart . Copenhagen munication beyond its currently dominant role as Education & Selected Workshops . 2008 a transmitter of sounds. The project proposes the Master of Arts in New Media, Medialab, University . co initiator . Mediawala Festival . celebration of design of the mobile phone as a medium for the of Art and Design Helsinki, 2004-2007. technology hack . Doors of exchange of everyday sounds within communities Interaktiva Medier, Dramatiska Institutet, National Perception 9 and CKS . 2007 and across socio-cultural contexts by mobilizing the School for Film, TV and Radio, Stockholm, 2001- . conceptualiser . Deja Vu . public art project with potential of the mobile phone as a tool for the pro- 2002. schoolchildren . Helsinki . 2005 duction of everyday sounds. To listen carefully to Electroacoustic composition, EMS (Electroacoustic . producer . Doors East 2003 . Doors of Perception . the environment is something we want to empha- Music in Sweden), Stockholm, 2001. Amsterdam, Bangalore . 2003 size in our design. We believe that when the pos- Teaching sibility to record and work creatively with the sonic C:Art:Media Master’s programme at Valand School qualification environment exists, then a higher awareness of our of Fine Arts - IT University of Göteborg, 2008. Tutor- . PhD candidate . Media and Communications . environment is achieved. Needless to say, the playing and teaching. School of Arts and Communication . Malmö . Swe- back of the recorded sonic environment is only a Mobile Sound Workshop at Stadia polytechnic den . 1st semester representation of it. But to work consciously with school in Helsinki as part of the Hearing Helsinki . MA New Media . University of Art and Design . this representation is what, we believe, heightens project, 2007. Organizing and teaching. Helsinki . Finland . 2007 our awareness of our sonic environment. Sound and new media courses at University of Art . MSc Communication . Manipal Insitute of Commu- and Design Helsinki, 2006. Teaching. nication . Manipal . Karnataka . India . 2000 The IMPROVe project was initiated as the joint mas- Publications ters thesis work of Richard Widerberg and Zeenath “The mobile phone as a medium for heightened grants Hasan at the MA New Media programme of the sonic perception”. Published in ACM International . Wahlgren Foundation Scholarship for Doctoral University of Art and Design Helsinki. Conference Proceeding Series; Vol. 159: Proceed- Studies . 2007 - 2012 ings of the 8th conference on Human-computer in- . Finnish Arts Council Travel Grant . 2006 http://www.riwid.net/improve/ teraction with mobile devices and services . University Grant for Masters Thesis Project . 2005 32 2008 Concerts 33

bio

Tim Blechmann http://tim.klingt.org

Tim’s music is focused on static noise textures, that are digitally generated and spacial projected in realtime. His pieces are very slow paced, having a low Klaus Filip volume close to the background ambience. For live http://lloopp.klingt.org/ performances, his preferred lineup is the duo with another improvising musicians. (see also 2008 performances) In 2004 he founded ‘plattform fuer freie music’, a concert series for improvised music in stuttgart, cur- Collaborations with Radu Malfatti, Werner Dafel- rently he is co-organizing the concert series v’elak- decker, Dieb13, Christof Kurzmann, Boris Hauf, gala. Christian Fennesz, Jason Kahn, John Butcher, Sa- After studying physics in Tübingen and Stuttgart, bine Marte, Gilles Aubry, noid, Cordula Bösze, Sil- he moved to Vienna in 2005, in order to study com- via Faessler, Taku Unami, Taku Sugimoto,Toshimaru puter sciences, digital arts and electroacoustic mu- Nakamura, Arnold “noid” Haberl, Tim Blechmann, sic (with Wolfgang Musil). Ivan Palacky.

projects: current projects: - duo with Goh Lee Kwang (prepared mixer) ~ los glissandinos (with kai fagaschinski) - taus - duo with Klaus Filip (lloopp) ~ lloopp (open source free- & software for musi- - duo with Manuel Knapp (analog electronics) cians) - pdt - trio with Daniel Lercher & Peter Lutin (lap- ~ tripple duo (so far with oblaat, noid, boris hauf) tops) ~ bigbaby (with red white and cynthia schwertsik) taus ~ ease (with noid) compositions: ~ duo with cordula bösze - sound track for ‘La Voix et le phenomene I’ by ma- ~ taus (with tim blechmann) Klaus Filip, lloopp laysian film-maker lau mun leng (2005) ~ solo Tim Blechmann, nova/Supercollider - vinyl playback, music for vinyl based on a duo improvisation with mattin (2007) recordings: Abstract - rr, computer music (2006-2007) ~ sigis bruder “leftovers” (klaus filip / sigi ecker) trost - mattin/malfatti remix, tape music (2007) 1994 taus is the duo collaboration between Tim Blech- - rrrr (2008) ~ orchester 33 1/3, plag dich nicht 1996 mann and Klaus Filip. Tim’s music is based on algo- ~ “maschine brennt” orchester 33 1/3, charhizma 1999 rithmically generated discography: ~ “building excess” (Klaus Filip / Radu Malfatti / Mat- - solo: s_n, moka bar (2004) tin / Dean Roberts) Grob 651, noise textures, which offer an amorphous funda- - duo with goh lee kwang: drone, no label (2005) ~ los glissandinos: stand clear (creative sources, lisboa ment for Klaus’s carefully woven sine waves. Sound- - solo: M, herbal records (2005) 06/2005) scapes are evolving, - solo: re-reading, freesoftwareseries (2007) ~ “aluk” (Klaus Filip / Toshimaru Nakamura): aluk (ja- - taus: The Organ of Corti, l’innomable (2007) panimprov/tokyo/2006) which are continuously redefined. - solo: rrrr, moka bar (2008) ~ taus: “The Organ of Corti” (l’innomable, ljubljana 2007) 34 2008 Concerts 35

iftaf - institute for transacoustic research translecture

Nikolaus Gansterer, Matthias Meinharter, Jörg Piringer, Ernst Reitermaier

Abstract

A fastforward sonic/audio/visual crash course into transacoustics by the translinguistic theory jockey, with live mindmapping drawings accompanied by experimental electronic music. The aim is to act within the crossover of linguistic, acoustic and graphic intersections. The transacoustic answer to old-school scientific lectures.

Transacoustic research carries out science by means of art and art by means of science. The antiquated differentiation of these two areas is rejected and methods and settings from both areas are com- bined to arrive at unique lines of connection and division. Transacoustic research is concerned with the pe- ripheral effects and tangential areas of acoustics, with their borders to other areas of research. The bio contours and definitional borders are necessarily blurred and vague. Transacoustics presents some- Nikolaus Gansterer thing which is basically, nothing. It can and should born in 1974. studied experimental media-design not be defined in the sense of something laid down at the university of applied arts /vienna. works with in writing. various materials, sound-installation, video-produc- Transacoustics as such, does not exist; there is only tions, installation-art, graphics,... transacoustic research, which constantly circles its imaginary core and thereby arrives at the most diverse results and realizations. Matthias Meinharter The question of the essence of transacoustics is as born 1971, studied ethnology and design at the impossible to answer as the question of art or phi- university for applied arts in vienna, involved in vari- losophy’s essence. The success, productivity and ous projects (including experimental music, design, efficiency of transacoustic research do not depend fashion a.o.) on finding an answer to this question. The institute for transacoustic research was founded in 1998 in Vienna to define and research transacoustics. Iftaf Jörg Piringer uses structures which correlate with those of a sci- born in 1974. student at the schule für dichtung in entific institute. It is divided into several depart- wien (curd duca, sainkho namtchylak, etc). master ments working with various thematic emphases in- degree in computer science. radio artist. sound cluding: auditive phenomenology, social acoustics, poet. musician. vegetable sound research, translinguistics, visual music, experimental instrument skills, bio-acoustics, demography, and klepto-acoustics. Ernst Reitermaier born in 1974. studied philosophy, music and cultural www.iftaf.org management in vienna. various projects in the field www.transacoustic-research.com of experimental music and radio art. 36 2008 Concerts 2008 Hands-on session 37

muio interface

Steve Symons

Abstract

Beyond the mouse: pain free alternative computer interfaces

Many artists and musicians want to explore alternatives to the mouse/keyboard paradigm.

Alternatives include sensors that detect light, heat or distance; or even the accelerometer popularised by the Wii.

There are a growing number of tools available for interfacing computers to the real world, especially at the open-source, low technological level end of the spectrum and despite the efforts of the groups involved the aspiring interface hacker must learn an added layer of electronics and programming in addition to the media and conceptual skills required. The Springfield RVL-003 The muio interface offers a radical alternative. The Jan Perschy, Robert Mathy, Merlin Wyschka muio’s basic minimal parts (one chip, three components, a usb cable and, if you really have to have Abstract the security of knowing the interface is drawing power from the USB port then, a LED) are all readily The Springfield RVL-003 is a band founded in the buyable online and do not need programming, just year 2007, based on a soundinstrument named plugging together. „breath control“ by Jan Perschy. By removing the pain from creating alternative in- This instrument, consisting of a Wii-Remote, that is terfaces the muio seeks to focus attention away placed in front of a speaker, mounted with springs. from the technology of building, to the important Everything put on a microphone tripod and up it issue of how does an interface relate to the user’s goes. experience?

By hitting, shifting, and variing the position of the This hands-on session will remote the bandmember can modify the for him - explore a range of sensors, how they work and individual sound. This played sound will be played what action they afford. back of the resonance body, the speaker. - demonstrate (in a practical way) how to build and Workshop leader: Steve Symons customise a muio interface http://muio.org The tripod will be used as a mounting, the joints - discuss (in a practical way) software that artists are a simulated coordinate system in the real envi- and musicians might like to use the muio with (such As well as being an experienced workshop leader, ronment, and so the position of the remote and the as MAX/MSP, processing, SuperCollider (raw code, Steve Symons is an artist exploring sound and tech- position of the speaker and out of this the „field of ixiQuarks) and c++/openframeworks) nologically mediated interaction. As an active Owl sound“ can be manipulated. So the player can win - allow participants to explore their creative sides, Project member ( http://www.owlproject.com) he the room and fill it with sound. possibly ending in an improvised sonic experience also performs, does his own programming, solder- (equipment and participants allowing!) ing and woodwork. Every member chooses his own soundsample. 38 2008 Hands-on session 2008 Hands-on session 39

Sun Run Sun

Yolande Harris

Abstract

Sun Run Sun’ charts a path between environmental awareness and technological development, using sound as the medium to enhance both. The project investigates the split between the embodied experience of location and the calculated data of position, exploring the individual experience of current location technologies through a personal experience of sound. It seeks to (re)establish a sense of connectedness to one’s environment, and to (re)negotiate this through an investigation into old, new, future and animal navigation using sound.

This project consists of two different parts, a sound installation and a series of portable instruments to take on a walk through the city. In the installation ‘Dead Reckoning’ Yolande Harris reveals the patterns of orbiting satellites coming in and out of range and inconsistencies in how GPS technology locates the self in a longitude/latitude grid. The mobile ‘Satellite Sounders’ transform the live satellite data directly into a sonic composition listened to on headphones as one walks through the city. IMPROVe Live signals from satellites in orbit, together with the performer’s coordinates on earth, generate a Richard Widerberg continuously transforming electronic soundscape. Yolande Harris’s soundscape questions what is in- Abstract side and what is outside, what it means to be located and what it means to be lost. http://www.riwid.net/ 40 Spat_lab - university of applied arts 41

SPAT_LAB 42 Spat_lab - university of applied arts 43

Spat_Lab the body of the in-between4, and architectural space5, there are the time-based “data bodies,” whose mate- A decisive impulse for this development toward ar- Nicolaj Kirisits riality comprises in visible digital data. chitecture came from the breakdown of the virtual, or rather from the failure of the virtuality hype to attain its The Mobile Music Workshop’s collaboration with the One of the aims of the Spat_Lab projects is to find main aim of replacing geographic space as the sphere University of Applied Arts began with Spat_Lab’s re- new ways of configuring these data bodies with the of social activity. Mobility, as the consequence of mini- cent projects. Spat_Lab was founded by me at the help of new insights gained from investigating sound- aturization, only makes sense when geographic space university’s Department of Digital Art. Since then, it ing bodies. However, this approach can also be de- is intentionally seen as not merely an abstract Euclid- has organized research-oriented artistic projects (con- scribed as a process in which media-related contents ian space but also as a field of social activity. Mobility is cepts and ideas: Klaus Filip and Nicolaj Kirisits). The are placed in geographic space as sound that is inau- therefore to be clearly distinguished from virtuality— artists developed and implemented their projects by dible. This deliberate misinterpretation aims to make even when the geospatial network of mobile end user the following two basic guidelines: the use of tech- the potentials for configuring these new architectural devices lets a form of mixed reality emerge. nological artifacts for purposes foreign to them, and elements better comprehensible. the expansion of the concept of body in architecture. Our own body has always been a part of geographic Combining both aspects with sound design is among It can be said that deliberate misinterpretation is the space. Virtuality has not succeeded in dissolving the Spat_Lab’s main interests. Technological artifacts are leitmotiv of the lab, especially when working with tech- significance of this space for human action. Music that converted into musical interfaces and the characteris- nical equipment. Sound is once again the common employs mobile technologies automatically turns the tics of sounding bodies used to expand the definition denominator in all projects realized thus far. Interfaces focus on human body design and, consequently, also of corporeality. What is meant here by sounding body for making music were created by intentional wrong on the geographic space in which the human being are resonances found in architectural space, that is, usage. In the course of their investigations, the partici- exists. Digital art can no longer deny the ubiquitari- spectromorphological content restricted1 to a volume pants removed devices from their social context and ness of technological developments and is becom- in the Euclidian system which may not be visible and placed them in different contexts after making the ing “body art” in geographic space. The performative yet contain all other defining qualities of corporeal- smallest possible changes to them. This opened a vast character of geographic space must therefore always ity (“a body with length, breadth and depth”2) in a potential for new meanings that would either remain remain part of the artistic code. Also, and especially, geometric sense. This definition of the sounding body totally invisible or become only partially visible should digital art takes mobile technology seriously as a mar- must be distinguished from that of a vibrating body, or the devices be used as intended by the manufacturers. ginal condition of human achievement. Such tech- resonator. The resonator is a visible tectonic body in nological limits influence production and reception, geographical space whose physicality does not owe to Currently, these two guiding principles of Spat_Lab are both of which have always been part of an expanded sound but to wood, or metal, for instance. In contrast, interdigitated because we are mainly concerned with definition of art. The recipient who always carries mu- the sounding body, however, is invisible, its material the ubiquitous computer and tracking technologies sic around with him, who deposits it in certain places, is sound itself. Although architecture is comprised of (GPS) as well as with diverse sensory interfaces (mobile collects it or passes it on, becomes an agent, and bodies, yet each of these bodies is not necessarily an telephones, Wii remote controllers, etc). Our long- designer in this case, not merely of the musical con- architectural element. Spat_Lab defines architecture term aim, however, is to gain enduring insights. Spon- text with all its “mobile” aspects such as spontane- as a spatial notation of socially relevant processes. A taneous absurdities and conscious attempts at getting ous network music, music distribution, etc. as investi- material body in geographical space, therefore, be- things wrong serve us to probe limits, whereas sound gated in the MMW series, but also—and this seems comes an architectural element the very moment it as- (in the sense of musique concrete), electronic music to be the direction from which Spat_Lab approaches sumes social relevance. Sound that is naturally located and configurating new forms in geographic space, or these theme—of the architectural body in geographic in space, i.e., already present without any technical in architecture, can always be seen as contractors for space. Mobility is that technology which allows virtual- aid and is in itself a natural body, is barely effective ar- the individual projects. ity to be understood as the materiality of a new archi- chitecturally, apart from a few exceptions, like church tectural body. bells. From an aesthetic or formal point of view, natu- The above mentioned approaches have led to artistic 1 Davis Smalley, Spectomorphology: Explaining Sound-Shapes in Organised ral sound is a precursor (or exceptional case) of Local- projects dealing with sound and mobile technologies, Sound 2 (2), Cambridge: Cambridge University Press, 1997, 107 Based Services (LBS), which certainly bear the potential albeit the emphasis is not so much on their societal, 2 Euclid quoted in Ostwalds Klassiker der Exakten Wissenschaften, Band of being architectural elements. In the case of these sociological aspects, nor with new fields of activity con- 235—Die Elemente, Verlag Harri Deutsch, 2005, XI, 314

LBS, digital information is placed at selected spots cerned with the reception of music but rather on the 3 Vitruvius, transl. Fensterbusch in Dr. Curt, Zehn Bücher über Architektur, in geographical space with the help of GPS devices. time- and process-based aspects of the corporeality Wissenschaftliche Buchgesellschaft Darmstadt, 1976 This information is just as invisible and process-based and architecture that thus emerge. To us this appears 4 Gottfried Semper, Der Stil in den technischen und tektonischen Künste, 1860, reprinted in Fritz Neumeyer, Quellentexte zur Architekturtheorie, as sounding bodies but, due to its significance in hu- to be a major outcome of mobility. Mobility in technol- Prestel 2002 man communication, this very act of placing turns it ogy is the logical consequence of a development in 5 August Schmarsow, Das Wesen der architektonischen Schöpfung, in Fritz into architectural bodies. In architecture, in addition to which more and more functions are packed into ever- Neumeyer Quellentexte zur Architekturtheorie, Prestel 2002. three basic forms of agency, that is, the tectonic body3, tinier bodies, which people can carry about on them. 44 2005 – Spat_Lab Projects 2007– Spat_Lab Projects 45

The Handydandy

The bluetooth rock ‘n’ roll band The Handydandy (B. Bauch, L. J. Gross, N. Kirisits, G. Savcis, J. Stau- dach, F. Waldner), founded in 20056, is an example for the way this principle was applied. Here, their starting point for artistic action was to use mobile phones as musical instruments. The mobile phone was reinvented as interface for a music perform- The Springfield RVL-003 ance whereby its bare technical structure (Bluetooth, keys, etc) and not the sound generation options im- Yet another example of Spat_Lab’s activity is the plied by the producers were used. Questions about band Springfield RVL-003, founded in the course application, which were never raised originally, now of a workshop in 20077. Jan Perschy developed an surfaced. A new genre of art, Bluetooth Rock ‘n’ instrument with a Wii remote controller during the Roll, was born, triggering a debate about wireless workshop and later evolved a musical concept for it music interfaces, the social significance of mobile with Robert Mathy and Merlin Wyschka. Springfield telephones, and forms of performance. RVL-003 will be part of this year’s MMW 2008 con- The Handydandy was part of NIME 06 (Ircam, paris) cert program. and MMW 2007 (Steim), amongst others. 7 Ferngesteuert [Remotecontrolled], Kleylehof, 2007 (Klaus Filip & Ni- 6 Speakers, Kleylehof, 2005 (Klaus Filip & Nicolaj Kirisits). colaj Kirisits).

46 2006 Spat_Lab Projects 47

Transit

Klaus Filip, Nicolaj Kirisits Bernhard Bauch, Philipp Lammer

Abstract The project was made possible through the financial Transit8 was an attempt at creating a sculpture support of Podspot (Prof. Tom Fürstner). with nothing but the bodies of data previously described. Transit employs strategies for the speedy Presentation of the project at the Mobile Music settlement of new land used since the days of Hip- Workshop 2008. podamus von Milet; this settlement of geographic space with bodies of data can be compared with The following artists were invited as settlers: the development of new land. A basic element (a Alev Özdemir cube measuring 1m x 1m x 1m) was used to create Andreas Haider a 20m long, 20 m wide and 20 m high grid. There Bernhard Bauch was, however, no master plan; only the construction Bernhard Lutz rules were predefined —the sculpture was based Bernhard Garnicnig on these rules, the volume available and the con- Daniel Kauer tent produced by the settlers. Each settler was given Ella Krampl eight cubes and had the freedom to select a site for Georg Novotny them. Twenty percent of the volume could be filled Gordan Savicic communally or altered as desired by the settler. The Gottfried Haider rest of the volume had to remain free; the aim was Julian Palacz to leave 60% of the area undeveloped. Each cube Julia Staudach could be filled with text, sound, video material or Kathrin Dörfler simply program codes. Neighboring cubes could Leo Peschta network with each other, exchange data or forward Lucas Czjzek it. Filling the cubes worked as follows: each digital Luc Gross artist, equipped with a computer and a GPS device, Mario Fischer could go to the site of the cube and “load” his / Mischan Gholizadeh Toosarani her content into it. In this way, the artists worked on Milos Paripovic their digital concepts in geographic space the way Nina Kataeva a sculptor would. Recipients could experience these Perktold Martin digital sculptures produced with the help of GPS Peter Scharmüller devices in the course of settlement; their playback Peter Tilg devices were filled with digital content at one of the Philipp Lammer many points where the content had been previously Sophie Wagner positioned. In contrast to the settlers, the recipients 8 were not permitted to alter the sculpture. Transit, Kleylehof, 2006 (Klaus Filip & Nicolaj Kirisits). 48 2007 Spat_Lab Projects 49

B Digital Claiming Andreas Haider Tim Blechmann Klangbild [Sound Pattern] (Fig. B) Media rush in 20 geo-tagged claims on Plocica Is- Untitled land (Croatia) Klangbild is the attempt at depicting sounding The primary source material for my Claim was the bodies visually. In acoustic space, it is only possible Digital Claiming9 is Spat_Lab’s most recent project. rushing sound of water, which I recorded at different to hear bodies that either produce sound or reflect The Croatian isle of Plocica, with no other building places and at different times. These field recordings it. Sound is the prerequisite for acoustic perception on it but a lighthouse, was rented for the project were made to undergo several transformations in just as light is for visual perception. for a week. Twenty artists were invited to mark their order to dissolve temporal structures. When sound is absent, no acoustic image or sound Claims on the island with the help of GPS devices. This altered material is the basis for a three-channel pattern of a space can emerge. The fact that we “A Mining Claim is the claim of the right to extract video installation with a sounding floor. An acoustic describe even acoustic experiences in images minerals from a tract of public land. In the United environment was to be created via indirect sound, proves how visually dominated our perception is, States, the practice began with the California gold which resembled the shores of my Claim on Plocica. a phenomenon that Klangbild examines and ques- rush of 1849. In the absence of effective govern- This piece is an acoustic sketch of the installation. tions. If visual perception is absent and we can only ment, the miners in each new mining camp made perceive our world acoustically, the mind conjures up their own rules, and chose to essentially adopt corresponding homogeneous images of sound, Mexican mining law then in effect in California. The Kathrin Dörfler body and space. These are visual representations of Mexican law gave the right to mine to the first one Untitled (Fig. A) acoustic perception. to discover the mineral deposit and begin mining it. The sound patterns of waves on a stony shore are The area that could be claimed by one person was “The composer becomes a cartographer if he lets examined with the help of an audiovisual composi- limited to that which could be mined by a single in- himself be guided. If one wants to allow tones tion for which hydrophones were installed on rocks dividual or a small group.”10 and stillness time then the task of the composer at the key points of wave refraction and recorded The material found in each Claim (acoustic, visual, no longer lies in searching for their expression but synchronously with the image. In the composition, haptic, body-time, etc) served as raw material for rather in allowing them “to be” what they are […] the levels of the recorded sounds determine the linear/non-linear and/or algorithmic compositions. This is why I mean that stillness is a state that is free level of visibility of each sound image (waves, rocks). Pieces composed with audio, codes, or film mate- of intentions.” (Daniel Charles on John Cage) The higher the sound energy produced by a sound- C E rial were “tagged” with the site. In these Claims the ing body, the more concrete and clear its visual task was not about digging for tangible spatial bod- A taut cord runs along the interstices between the representation. ies in the sense of gold or other metals but rather blocks of stone at the shores of Plocica. The crevices The visual part of the work is a composition and for objects that become corporeal because they form various resonance spaces in which binoral interpretation of the visual data. represent a site (length, breadth, height). microphones are placed at several points for record- The perceivable acoustic space is variable and What emerged in the process was a geography ing the sounds. dependent on the user, who can “sound surf” in of transformations, land surveyed by producing The different resonance spaces correspond with the via self-navigated positioning. This can be done by its time-based representatives. The perception of different filtered sounds. The soundscapes in these selecting various sound tracks —three positions are space based on data collected by precisely meas- in-between spaces ultimately become sounding possible within the setting and three outside it. uring and recording within the Cartesian system is bodies, which produce a multi-perspectival projec- hence replaced by a typology of interpretations. tion of the incessantly breaking waves. The map of the island drawn out in the course of Laura Skocek this project comprised a series of individual installa- Dusk Dawn / Island Maps (Fig. C, D, E) tions and unveils the discrepancy between a seem- Klaus Filip Series of photographs D ing objectivity and the unquestionable supremacy Sonoplocica Movements on 3 plateaus on Plocica were observed of earth survey via mobile Global Positioning Sys- and snap-shots of the island were recomposed into tems (GPS), and process and time-based entities of impressions of the stony surface on a/my “claim” a sequence. body and space. on plocica, a couple of plants, a couple of animals, some water. the photos were arranged in the chron- The series Island-Maps was taken from three dif- The works will be presented at the Mobile Music ological order of the time when they were taken. ferent positions. These photographs were pieced Workshop 2008. the analysis of the image produces the sound together into three maps, the photographer being emerges, whereby the mappings of the frequen- the center. A “false” image of Plocica’s geography 9 Digital Claiming, Plocica, Croatia 2007 (Klaus Filip & Nicolaj Kirisits). cies of a sinus bank are encoded over the image thus emerges from the subjective representations 10 Wikipedia, HYPERLINK “http://en.wikipedia.org/wiki/Land_claims” matrix. the volume of each partial tone is defined by of the brief visitor on the island and her arbitrary http://en.wikipedia.org/wiki/Land_claims the brightness of a pixel; play at medium volume. division of zones. 50 2007 Spat_Lab Projects 51

B C D Peter Scharmüller plocica – grenzgang//uferdiskussion [walking the line//shore discussion]

This representational video by the media artist Peter Scharmüller raises the omnipresent question about the switch between media. Try to imagine Gottfried Haider the masses of water and rocks incessantly crashing Im Nebelmeer über Plocica / against each other. Each wave is a reason for remov- Sea of Fog over Plocica ing every rock that protrudes from the water. The solidity of the rock is the antithesis, corrosion Starting point for the investigation was the striking being the manifestation of synthesis. absence of the island we were about to set foot on, When the gaze switches from one medium to the from Google Earth satellite imaginary. This defi- other, the mass of air solidifies into a quicksilver mir- ciency was initially met by flying a camera-equipped ror that hurls the spindrift back into the water. helium balloon over this remote isle. The balloon, attached by a string to the artist, formed a prosthetic extension of his body in a Jan Perschy physical as well as sensory sense, as the camera im- random walk ages were also instantly transmitted to the ground. Walking a preconceived path this way, the tiny strip Interpreting the structure and characteristics of the of land gained an unexpected orbital dimension. ground through one’s own environment inspired But also the joining with the balloon itself, the me to press this record. The piece of wood swirling rhythm of its tumbling motion and inert shift of gaze through the stones and thereby altering the ground developed a surprising dynamics in the course of Sophie Wagner of the claim is a basic measure; it is also the unit of the performance. brzina hodanja (Fig. E) measure in my survey. The record of a moment in E A time made in this manner was reinterpreted and Back on land, the video footage obtained this way equipped with my actioncam i moved along the given form through my own subjective impressions. is being used to claim land by means of a projection borders of my claim, capturing images and sounds The ground as data medium is stored on disc with device. This is archived by translating the images above and under the water. the help of a personal codec. into a spherical coordinate system, thus connecting the rhythm of the images was determined by the Values inscribed on the record, layered over the back to the logic of Google Earth. conditions of the claim, such as the movements of sound produced by it, allow it to interpret itself. my float or of a stunt kite. each sequence is an instrument for the music band Claudia Larcher for which the visitor can compose new pieces of Robert Mathy 472 (Fig. B, C, D) music using the sounds i gathered on my walks PLING! (Fig. A) Temporary installation along borders. White balloons A found stone/picked up on the shore/ was dropped every 20 meters from the same height The number 472, also the title of the installation Florian Waldner above sea level. This procedure was repeated until reflects the number of balloons used by the artist Wave Lands the area was covered completely. The grid of sound as material for her intervention in the landscape. samples resulting from the acoustic recordings The artist filled the balloons with either air or water The stones on the Plocica island show patterns made during each step of the process served as a and carefully placed them in the crevices and holes that are formed by the tide over a long period of positioning map for a motor-operated loudspeaker, in the rocks. In order to ensure that every gap was time. Photographs of these patterns are reduced to which reflected each sound at its respective posi- filled she had to treat each balloon differently so two-dimensional graphical lines. Audio recordings tion. that it fitted perfectly. taken at the same place are analyzed concerning The terrain’s topology led to variations in the length This seamless lining of cracks and crevices in the their spectral composition. The data resulting from of the recordings while acoustic differences were rocks with balloons formed a white line, like a fine that analysis is used to animate the graphical lines caused by the surface of the ground. The aim was drawing in the landscape that was reminiscent of in three dimensional space. Such that movement of to inspire the recipient to imagine the conditions at the contours or traces of an unknown, unidentifiable the abstracted patterns is dependent on the sound the recording site. creature. of the sea. 52 2007 Spat_Lab Projects 53

A B Nina Tommasi Imaginary fusion of acoustic location (Fig. D, E, F)

The main idea was to make a site comprehensible as a dynamic network of sound objects. (The definition of this fusion between interdependent corporealities can only be comprehended as process-based; it defies a priori total representation because of its complexity. The survey, the time of day when this is carried out and the process itself of surveying are used for deliberately reducing and manipulating the site’s C complexity. In this way, the conditions of the site are reorganized in order to perceive the whole site in a different light altogether. This reduction aimed at producing added compositional value, which would make the site perceivable by means of audiovisual impressions of it and processual changes to it and thereby generate new possibilities of representation. The survey procedure is not be seen as the record of geographic-tectonic data in a precise Cartesian system of coordinates but rather as a kind of “nest- ling” against the form, so to speak. Leo Peschta While cords marked the rocks and produced new Wavesynth 1.0 (Fig. A, B, C) architectural spaces/points of reference for the sound recordings, this geometric expansion of the The coastal area is a constantly changing environ- site and its subjective sensory perception lent the ment. Every moment is the arrangement of stones site an immanently changeable individuality. and water in it differently, never the same. By the force of the impact of water on the banks a wide set The tectonic, visible body, along with all its pecu- of different sounds is generated. liarities, was variously linked with other surrounding “bodies” which, analogous to the cords, “nestled” For the installation “Wavesynth” relevant areas of against it, formed and defined it and its in-between the bay (optical: where is water when; acoustical: spaces. where does the it sound interesting) have been Important constants that lent the site its individual- equipped with sensors. In the water there was a ity and generated the possibilities of recognizing it, small current initiated (5V, 20mA). Every time it got such as sounding bodies or “wind bodies,” origi- in touch with one of the sensors, the current level of nated from places outside the staked out claim. D E F the sensors output was changed. As sensors tense This body/site/object can only be seen in the con- resistance wires have been used, which produce text of its geographic surroundings and the point at different values depending on the position in which which each respective body is linked with the other. they get in contact with electrical charged water. Sound recordings were made along the fixed line, representing the “points of reference” for the other A computer continuously measured this values and “corporealities” surrounding the rocks. routed this data to an audio application, which used Although the content of these sound files served as it to reinterpreted the real time sound of the bay. reference to the geometric space and for the mo- The transformed Sounds have been played back ment in time when it was surveyed, the morphing on-site vis speakers and therefore mixed for the spectrum of sound turned the site into a constantly listener with the ambient sound of the island. changing “setting.” 54 2007 Spat_Lab Projects 55

Craving stage directions has been rearranged and expanded using pieces of everyday conversations to work with given sound input (parameterized as frequency and Bernhard Garnicnig, Gottfried Haider individual clusters according to the demands of cer- source location) is filtered by the diffraction and re- tain places. flection properties of the torso, head and pinna before reaching the eardrum and inner ear. These Craving11 is in fact a special case because it was not Method location-specific filter effects provide the human made during the Spat_Lab workshop (but at the De- neural system with enough cues to properly locate partment of Digital Art, University of Applied Arts The selection and spatial and temporal distribution a sound’s source. Through the realistic simulation of Vienna), but it must be seen as a part of it because of of sound elements require a detailed study of text these effects it is now possible to place sound emit- the theme it addressed. and conditions of the space such as architecture, ting “props” into the listener’s environment. The final version of Craving can be heard at its flow of movements and rhythms. The technology original site at MMW 2008. (GPS, etc.) framing the production obviously plays Site another, very important role. Introduction As environmental influences such as weather or Craving was envisioned for production in Vienna DC, social interaction surrounding the participants or a modern complex of commercial and residential In Craving Bernhard Garnicnig and Gottfried Haider their personal movement patterns cannot be fore- buildings in the city’s Donaustadt district. This most aurally stage a text inspired by the late Sarah Kane’s seen, the sound design is not geared towards con- preeminent area is defined by a branch of the river play Crave in public space. It unfolds while mem- structing a linear narrative. It aims, rather, to create Danube in the south and the United Nations build- bers of the audience individually wander a high-rise individual, but loosely-connected scenes. To archive ing in the north. Vienna DC was conceived entirely area, wearing headphones and a mobile comput- this, acoustic elements are placed on street corners, on the drawing board after plans for a World Fair ing device. on wide, open spaces or in lively passage ways as in this location had been vetoed in a referendum in they relate to a sensation and meaning created by that same year of 1991. Nevertheless, ten years after Process their architecture or the human beings inhabiting it. its opening, the area is still urbanity in progress, as In order to do this the artists have developed a soft- various vacant lots create a layered surface, whose The audience is escorted from the Mobile Music ware, which enables a composition of temporally heaps of dirt contrast with the spotless facades oth- Workshop venue in Vienna’s city center to the site and spatially dynamic acoustic scenes. erwise dominating the view. Vienna DC houses nu- of the production. Once arrived, they have the op- Sound fragments such as spoken language or merous multinational corporations and information portunity to explore the location two at a time. music are grouped together, following an internal technology firms in office skyscrapers, but there are Equipped with a Wearable Computer and head- temporal logic. These groups are distributed all over also vivid residential zones in between. One can lit- phones the recipient is immersed in sound surround- the area and linked through the recipient’s percep- erally walk around a corner to see the number of ings he can physically navigate. The path they chose tion as he moves through the space. suits diminished and people leading their lives in is in no way - auditory or visually - predetermined, Applying their other senses and their feeling for a slower and more informal way. There is a bizarre thereby allowing the audience to let themselves be the specific place the participants then put the per- city within, whose 4.000 inhabitants have adopted guided by aspects of the place itself such as its archi- ceived sensations into a larger context. This ability to to the given system of open spaces and the spatial tecture while experiencing the production. freely associate intentional design elements through logic of the complex. For them the architects envi- reflection accepts the spectator in the temporal and sioned a church, a museum exhibiting works of an Text spatial complexity of his cognition. Austrian sculptor, a bilingual school and kindergar- ten, a supermarket, a number of cafés located in The text used in Craving draws on Crave, a play by Technology the lobbies of skyscrapers, and a restaurant. Other British dramatist Sarah Kane (1971 - 1999). In it, four unique architectural features also strongly influence sparsely drawn characters weave a tapestry made up The participant is equipped with a wearable comput- the way in which the space is perceived: a wide flight of quotations and fragments, the cloth of which are er and headphones. Custom software determines his of stairs leading up to nothing, surveillance cameras their individual traumas, loves, grieves and resigna- position via GPS, tracks his head- and body move- places at eye level, deserted children’s playgrounds, tions. Plot and signs indicating temporal developments through a magnetometer. Based on their re- a vast empty space whose floor is covered in glaring ments are reduced to a minimum. It is in repetition sults the computer renders the audio composition white paint. This microcosm allows the artists to use and the final defeat of communication of internal in real-time. Through a simulation of binaural hear- the space’s emotional tectonics and possible asso- landscapes that we come full circle to the urbane ing, sounds previously affiliated to certain places ciations while breaking with the normal patterns of Wüste (the urban desert) in between the towers of now become audible from their specific direction. movement, perception and interaction with the envi- the Donaustadt. The software incorporates a real-time virtual acoustic ronment and other people. Kane’s text, which is filled with elements of subjective environment rendering engine. It is based on head- 11 meditations on urban surroundings, but devoid of related transfer function (HRTF), describing how a Craving, Bernhard Garnicnig & Gottfried Haider 56 2007 Amsterdam, STEIM Institute and Waag Society, Netherlands 57

4th Mobile Music Workshop 58 2007 KEYNOTES 59

Keynote Address 07 May

Michel Waisvisz

Excerpts from Régine Debatty’s blog entry on http://www.we-make-money-not-art.com

One of the directors of STEIM, co-host of MMW 2007, Michel Waisvisz is a composer/performer of live electronic music, who has invented new ways to In 1973, Waisvisz arrived at STEIM and worked on achieve physical touch with electronic music instru- the CrackleBox, a handheld instrument based on ments, for example by literally touching the elec- the same principle of body conductivity… tricity inside the instruments… All sorts of Crackle objects could be manipulated He illustrated his quest to find and develop physical by children… Among the artefacts (some of which relationships with electronic musical instruments by are re-invented by young interaction designers and performing a short improvisation on The Hands, an shown today at events such as the Milan Furniture interface he conceived in the early 1980s… fair) shown were phones that distort your voice according to the strength you use to squeeze the In the ‘60s, when he was a teenager he would do receiver; a musical bike where the generator was musical experiments with his brother: putting a pia- connected to speakers instead of a headlight; a se- no upside down and playing the instrument just by ries of connected CrackleBoxes that makes melo- touching his strings… dies when you pour some tea in the cups; a cuckoo clock producing scratchy sounds, etc. He showed us a fantastic picture of him becoming literally a tape reading machine using “The Ta- Another of Waisvisz famous projects is the Web pePuller” instrument (image on the right). He was where each thread in the spiderweb-like instru- live sampling, scratching 2 tape heads using foot- ment is a sensor. People can play it and manipu- pedals. He’d pull one forward with a foot to create late the timbre in a very intuitive way by grabbing music while rewinding the other tape with the other the strings. foot, unheard of the public. He mentioned several projects that investigate this He discussed his fascination for the VC3, a syn- (still under-developed) physical relationship with thesizer that can be used without a keyboard. He musical instruments: Jon Rose’s Hyperstring bow; bought a VC3, opened its back and put his fingers Nicolas Collins’ “trombone-propelled electronics”, inside. He thus used the body to extend the cir- the Lady’s Glove by Laetitia Sonami. cuitry and modified the sound in ways he found interesting. The manipulations gave him the feeling A last work he mentioned is Kristina Andersen’s en- that the sound was floating in the room and that he semble, a suitcase full of sounds and clothes. Sen- could grab it. He decided that instead of opening sors are fitted on the garments in such a way that the instrument back he should better customize it. the function of the sensor is conceptually support- This was the inspiration for what later became the ed by the form-factor of the garment. CrackleBox. He was fascinated by the idea of a human being who is turned into a variable electrical Licensed under Creative Commons 1.0 (www.cre- conductor/resistor, and a thinking [wet] element of ativecommons.org) the musical instrument. 60 2007 KEYNOTES 61

Network Landscapes: Landscape, Public Space & Mobile Music… Molecules? If we brush against the grain of mobile media forms, what might we discover as the underside of Teri Rueb this condition and how will we respond? As a soci- bio ety we have become atomized, but the question re- mains “Can we form molecules?”1, and what might Rueb’s large-scale responsive spaces and location- When I arrived for the workshop yesterday the they look like? aware installations explore issues of architecture neighborhood was filled with the sounds of a public and urbanism, landscape and the body, and sonic and acoustic space. In 1999 she pioneered gps- outdoor concert. Meanwhile, indoors my acous- As individuals linked through mobile technolo- based interactive sound walks with “Trace”, set tic landscape shifted from Buddy Holly to Roy Or- gies we have become mobile nodes in a complex along a network of hiking trails in the Canadian bison to Johnny Cash as a beautifully preserved network in flux. Beyond the classical figure in a Rockies (funded by the Banff Centre for the Arts). 1950s jukebox pulsed with the last nostalgic selec- landscape, we have become the very material from tion made by a stranger. Regardless of my musical which this “network landscape”2 emerges as the She lectures and exhibits world wide at venues in- preference, these moments held meaning for me interaction of natural, social, technological and bio- cluding Transmediale (Berlin, 2004), SIGGRAPH on a deeper level as they signaled something im- logical networks. (San Antonio, 2002), The International Symposium portant about sound and public space. The mes- on Electronic Arts (Nagoya, 2002; Paris, 2000; Hel- sage was in the medium, not just the content of J.B. Jackson, the great historian of the vernacular sinki, 2004), Consciousness Reframed (Perth, 2002), the sounds. The act of social gathering for shared landscape, argued that we must see landscape ulti- The New Museum of Contemporary Art (New York), listening, individuals suspended in the connective mately as a “shared three-dimensional reality” and the Corcoran Gallery of Art (Washington D.C.), The tissue of sound–whether individually or collectively therefore a question of public space. This appeal Banff Centre for the Arts (Banff), Bell Laborato- programmed–serves as a powerful catalyst in the must also be made in light of how we understand ries (Holmdel), Interval Research Corporation (Palo formation of political identity and the claiming of the “network landscape” of mobile network society. Alto), and The Fraunhoefer Institute/GMD (IRCAM, public space. A public live performance offered as As actors in this landscape, how can we resist being Paris, 2002; Glasgow, 2001). a free concert constitutes a mobile location-based framed as passive consumers or controlled and sur- networked technology quite different from the mo- veilled subjects, and instead embrace our agency She has received grants and commissions from the bile sound platforms that have become the default as creators and participants in the shaping of this ICA Boston / Vita Brevis, LEF Foundation, Artslink, technology referred to and used in artistic practice new public sphere? Turbulence, and various state arts councils. Rueb’s and cultural studies in mobile sound. work has been featured and reviewed in diverse Rather than choosing to create network landscapes publications including “Second Person: Storytell- Where does this shared public space go when we that foster escape or deferral of this challenge, we ing and Games in Playable Media” (edited by Pat adopt the personalized space of mobile music in- must seek to create evermore charged spaces of Harrigan and Noah Wardrip-Fuin, MIT Press, 2006) terfaces to the city? What are the consequences socially, technologically and ecologically mediated and “Information Arts: Intersections of Art, Science of the spread of mobile music devices that would encounter. The various works I will share in this talk and Technology”, (edited by Stephen Wilson, MIT inscribe us within personalized bubbles of sound? trace a ten-year exploration around the question Press, 2001). She holds a B.F.A. in Art and Literary Is this kind of sharing a form of personal or collec- of landscape, public space, identity and network– and Cultural Studies from Carnegie Mellon Univer- tive expression, or are we merely conforming to a in particular, the cybernetic landscape as produced sity and a master’s degree in Interactive Telecom- system of social interaction and exchange that has through and framed by sound. munications from the Tisch School of the Arts, New become an even more intensified interiorization of York University. Rueb is an associate professor in 1 John R. Stilgoe posed the question “Can we make mol- control space? What is the space of compromise ecules?” in reference to the atomization of the public sphere in the con- the Graduate Department of Digital Media at the and negotiation of meaning akin to “public space” text of his seminar “Modernization of the North American Built Environ- Rhode Island School of Design. Rueb is also pursu- in this moment of dual movement between global ment” taught at Harvard University in Spring 2007. ing doctoral research at Harvard Graduate School homogenization and expanded cosmopolitanism? of Design and is founder and principal of Open Air Have we abandoned the constantly shifting land- 2 Since 2006 I have advanced the term “network land- Studio, Cambridge, Massachusetts. scapes” to indicate the extended landscape of natural, social and tech- scape that would take us outside or beyond the nological networks that combine to form the ground of a new landscape condition that I argue is peculiar to mobile network society. This con- comfort zone of our factionalized cohort? As cul- ception builds on J.B. Jackson’s etymology of the word “landscape” as Figure tural producers, critics and consumers – as citizens linked to the medieval German landschaft, from which he proposes the definition “man-made systems overlaid upon the land” (Discovering the - we have an obligation to question “off the shelf” Vernacular Landscape, 1986). Accordingly, I argue that wireless networks Teri Rueb’s “Core Sample”, part of the exhibition constitute a component of the magnetosphere that is shaped by man as technologies that appear as “natural” or “liberat- a system overlaid upon the land. “Art on the Harbor Islands” with the Boston Insti- ing”. tute of Contemporary Art, 2007 62 2007 KEYNOTES 63

Keynote Address 08 May

Régine Debatty

Régine Debatty, blogger from We Make Money Not to bio-art, focusing on artists, curators, and exhibi- Art, gave the closing keynote address at MMW2007. tions broaching aesthetic and ethical questions on As part of her keynote preparations, she had been biotechnology and physiology. She has since kept covering the workshop live on her blog, reporting moving, covering the contemporary art world in her on the works and talks as they were being present- charismatic and personable style. ed at STEIM and Waag. In her talk at Waag’s The- atrum Anatomicum at Nieuwmarkt, Reg changed Throughout this shifting landscape of interests, Ré- roles, from that of observer, to take the podium to gine maintains an interest in art and technology, her describe her present interests and preoccupations. focus driven by the personal and human efforts be- hind projects. While her itinerary continues to be set She retraced her steps to describe how she got into by her interests, she admitted with the success of her full time activity of blogging the art, cultural, and her blog being solicited to cover events, chuckling design worlds. Ironically she embarked on this no- at the apparent misunderstanding of certain orga- madic, very mobile work at a desk job for the Euro- nizers who thought WMMNA was a giant machine pean Commission in Turin. Bored with her job, but or whole office. Reg is a free agent, booking her inspired by the artwork she saw around her, includ- own travel, deciding her destinations. ing that of her current partner, she found that her desire to describe what she perceived, in her own In the area of locative media, one of those destina- non-specialist terms, could interest others. tions has been the Conflux festival of psycho-geography in Brooklyn, NY. She was not paid, sponsored, This approach of following one’s nose, being guided nor subsidized to cover this artist run event – it was by one’s own interests, and writing in an disarming, the topic, the people, and the feeling that motivate accessible way was a perfect match for the blog as her to go. It is this professional spirit, this broad view medium. Initially interested by the area of media art of the scene, and incisive knowledge of specific – works tackling questions of technology and cul- grassroots initiatives, that made Reg the perfect set ture, she went on to cover design events worldwide. of eyes and ears to follow, interact with, and be part Her interests at the time of the keynote had shifted of the MMW2007. 64 2007 performances 65 1 2

Hearing Sirens A performance for mp3 players and portable hornloudspeakers

Cathy van Eck

Abstract

Reversing the Philosophy of Headphones A usual fashion to hear music nowadays is through headphones. The mp3-player made more music transportable than ever before and public spaces are crowded nowadays with people, living in their own acoustic world. My project is about reversing this situation. I am walking around the city, playing music from an mp3-player, this time not for creating private music, but for diffusing it out of two big yellow hornloudspeakers, radiating the sound to the environment.

The Acoustic and Visual Design: Greek Siren and Emergency Siren The siren is both a mythological woman, having the body of a bird and the head of a woman as a noise maker, used to warn in emergency cases. The sirens as bird-women were known in Antiquity for their beautyful singing. It was unable to resist them and most of the men who heard them did not survive. The emergency siren is a noisemaker and can be seen as a survival tool. I used both as an acoustic, visual and conceptual starting-point for the project Hearing Sirens. Construction The portable hornloudspeakers consist of a small bio mp3 player, a box with an amplifier and battery, and two loudspeakers in two big yellow horns. The Cathy van Eck (1979 The Netherlands/Belgium) is construction is made to be worn on the back of the a composer and sound artist. Her work includes performer. compositions for instruments and live- electron- ics as well as performances with (selfmade) sound Acoustical Characteristics objects. The specific construction of the horns and the fact Besides working at her different artistic projects and that they are portable give them special acoustic collaborations, she is currently teaching at the Mu- possibilities. Due to the big horns, the sound is dif- sic and Media Art Faculty in Bern. fused very directional. Therefore the audience can often hear the early reflections before the direct http://www.cathyvaneck.net/ sound. In this way, the hornloudspeakers reveals the acoustical characteristics of the environment. By Figures a small movement of the performer, the pattern of the reflections can change enormously. The sounds 1. Attic red figure vase depicting sirens . diffused by the sirens are made with physical mod- 2. A siren as an outdoor warning noise-maker. els of sirens. 66 2007 performances 67

The Handydandy

„ In French it is called le portable. In Arabic it is sometimes called sayaar or makhmul. In Thailand it is a moto. But here in Nickelsdorf there’s only one reference, which is no other than “the handydandy”...... a handy is a hendi is a handy“

Ing. Zapfl, Nickelsdorf 2006

Abstract

The Handydandy were a bluetooth-rock and noise group from Vienna, Austria / Los Angeles, Califor- nia. They are one of the most commercially success- ful and critically acclaimed bands in the history of popular music. The band’s principal members were Bauch Bernhard, Gross Luc, Kirisits Nicolaj, Savicic Gordan, Waldner Florian and Tschuli Staudach-Jef- ferson.

In Austria, The Handydandy released more than 40 different singles, albums, and EPs that reached number one. This commercial success was repeated in many other countries; their record company, EMI, estimated that by 20055 they had sold over one billion records worldwide. The Handydandy are the best-selling musical act of all time in the United States, according to the Recording Industry Asso- ciation of America. The HandyDandy is a wireless rhizom. A real-time rock axiom based on asymmetrical network mu- In 2006, Rolling Stone magazine ranked The Handy- sic synthesis. “the handydandy” are first of all dandy #1 on its list of 100 Greatest Artists of All stars within an upcoming and never emerging mu- Time. According to that same magazine, their in- sic scene, namely the bluetooth rock’n’roll. their novative music and cultural impact helped define body performance implies powerful and energet- the 2000s and their influence on pop culture is still ic electro-acoustic computer music. each of their evident today. concerts are site-specific artworks themselves. the main concept is an human-opposed computer net- The Handydandy led the mid-2000s musical “Blu- work based on real-time patches. the group itself etooth Invasion” into the United States and world- performs on misused hybrid-media artefacts, also wide. Although their initial musical style was rooted known as mobile phones in the latter 20th century. in 1950s rock and roll and homegrown skiffle, the Cardboard Flying-Vs, pimped violins and sonified group explored genres ranging from Johnny Cage GTI-trubadors are just brief examples of their never- to psychedelic bluetooth-rock. Their clothes, styles, ending musical interface repertoire. the handydan- and statements made them trend-setters, while dy is a slap in your face, after which you still wanna their growing social awareness saw their influence smile. they live and work in miami/US. extend into the social and cultural revolutions of the 1990s. http://thehandydandy.yugo.at/ 68 2007 performances 2007 papers 69

TRATTI - A Noise Maker for Children

Martin Pichlmair, Laura Beloff

Abstract

In this paper, we describe TRATTI, a characteristic piece of Device Art. It is a funnel shaped bull- horn to be worn in front of the belly. Children can TokTek walk around with the TRATTI. First, they record their voice into the device. Then, they can point the Tom Verbruggen TRATTI anywhere they want. The TRATTI constantly snaps images from its surroundings and plays back Abstract the recorded voice samples manipulated through the image, through the environment. TRATTI is TokTek (Tom Verbruggen) structures the unbridled technologically based on mobile phone technology clicks and cuts of his circuit bend gadgets to a frag- and it reflects a number of key features of mobile ile disturbance. Sampling with a joystick Tom cre- phone technology. TRATTI is a loud and disturbing ates unlogic dynamic compositions. piece of real-time art, a very personal musical instrument playing the voice of the musician, accord- Tom’s work is about the communication and non- ing to her standpoint in the world. communication between electronic devices and hu- mans, focusing particularly on his relationship with bio such devices. Drawing on his fine art background, his work explores the relationships between, human Martin Pichlmair (1977) is a media artist living and touch, memory and everyday electronic objects. working in Vienna, Austria. Since he received his For example his work “Moederkoek”, which literally doctoral degree in informatics he works as assistant translated is mother-cake but refers in English to professor at the Institute of Design and Assessment the placebo, Tom performs with his mother and she of Technology at the Vienna University of Technol- bakes a cake, like she used to when he was a young ogy. His art pieces were shown at various media boy. In the contemporary version, in a self-assem- art festivals and exhibitions. Recent shows includ- bled kitchen, Tom performers with his mother, sam- ing the Ars Electronica Festival, ISEA, Transmediale pling her baking and the sounds its produces in and the Microwave International Festival for New realtime. These sounds are arranged and manipu- Media Art. In his research and publications, he fo- lated on the fly and form an ongoing, improvised cuses on theory and practise of interactive art and composition. The performance ends, with the cake design - from game design and physical interfaces going into the oven and the smell of baking filling to open source development models and commu- the room. Once it is baked, the cake is served to nity media. the audience. Laura Beloff’s (1964) interests deal with individuals Tom’s latest invention is the Crackle-Canvas. Us- in the global society adapting to highly complex ing STEIM crackle box hardware, Tom has created bio technologically enhanced world. Her mobile, wear- paintings that produce sound. Each painting can able objects are exhibited internationally in muse- produce sound by itself but when connected with Tom’s work is about the communication and non- ums, galleries, and major media-festivals. She is fre- other paintings forms a ‘painting orchestra’. By communication between electronic devices and hu- quently lecturing about her research and practice in connecting cables between the paintings, the mans, focusing particularly on his relationship with universities and conferences. sound changes, while the cables length, colour and such devices. Drawing on his fine art background, 1999: visiting Professor, Linz Art University, Austria. form, form a drawing on the wall or in the space the his work explores the relationships between, human 2002-2006: Professor for media arts, Art Academy paintings are hanging. touch, memory and everyday electronic objects. of Oslo, Norway. 2007-2011: 5-year Artist-grant by Tom Verbruggen is part of the New Interfaces for the Finnish state. 2007>: lecturing at The University http://www.sonidogris.com/ http://www.lomechanik.com Performances programme. of Art and Design Helsinki, Finland 70 2007 papers 2007 papers 71

Mosomuso: Mobile Social Music Software

Atau Tanaka, Guillaume Valadon, Alex Kummerman Floating Fabulousness: Representation, Performativity and Identity in Musical Abstract Ringtones

MOSOMUSO (Mobile Social Music Software) was a Isabella van Elferen, Imar de Vries collaborative research project funded by the French Ministry of Research. It brought together mobile Abstract startup, Clicmobile, with research partners Sony Computer Science Laboratory Paris, and the LIP6 In this paper, we consider musical ringtones of mo- network lab at Université de Paris 6. The presenta- bile phones to act as virtual, communicative and tion at MMW’07 covered two aspects of Mosomuso, cultural performances. They appear unpredictably, “Social Mobile Music Navigation Using The Com- they communicate signs which are interpreted by pass” and the locative media work, “Net_Dérive”, a variegated and dynamic audience, and establish realized on the Mosomuso infrastructure. stages upon which cultural meanings are portrayed. We will argue that the musical ringtone functions There is an increasing tendency to converge func- as a musical madeleine in Marcel Proust’s sense, tions of several consumer electronics devices (a per- an involuntary mnemonic trigger of a complex web sonal music player, mobile phone, satellite navi- of individual and collective memories. Having this gation, digital camera) o into a single device. The quality, the ringtone lends itself perfectly for the Compass uses mass-market mobile phones in an in- performative manifestation and display of (sub)cul- tegrated location-aware, networked musical naviga- tural identities in the public sphere. tion and exchange application. The Compass is a As virtual, communicative and cultural performanc- tool to study and experiment mobile music naviga- es, musical ringtones have the inherent capacity tion. We use a single interaction metaphor, that of a to function as publicly disseminated madeleines, compass, to guide the user to search, find, and navi- which suddenly announce themselves and disrupt gate closer to friends, styles of music, or places of in- everyday social situations. For this reason, ring- terest. Using the location information retrieved from tones can be seen as a means to actively display the server with the phone’s data link, users once in and communicate a loyalty to floating subcultures, proxmitiy are able to bootstrap ad-hoc networks to as well as triggers for cultural performances with- allow spontaneous music exchange. in the spatial sphere of the ringtone’s carrier. The flaunting character of these performances lends it- Net_Dérive was premiered at the Maison Rouge in self perfectly for the display of fabulousness: hear- Paris in 2006. To perform the work, participants wear ing a ringtone will induce mnemonic reflections. scarf containing two mobile phones and a GPS unit Our findings concern musical ringtones primar- to explore the neighborhood surrounding the gal- ily. Sound effects or recorded speech can equally lery. One phones takes pictures every 20 seconds invoke communicative and cultural performances, uploading geotagged images and upstreams audio but we consider the vast array of individual and bio to the server. The other phone serves as display re- shared musical memories to be more powerful in ceiving audio/visual streams from the gallery space. invoking ‘madeleine trails’ and in manifesting (sub) Isabella van Elferen is a researcher in the area of This creates an interplay of sound and image, an ex- cultural identities. This does not mean that we think Media Studies and Musicology at the University of change between participants in the streets, and the that the functioning of ringtones as communicative Utrecht. Her current research focusses on cyber- creation of an abstract narrative from sonification and cultural performances is only established when gothic subcultures and the gothic aspects of cy- and visualization of locative information. The abstract complete songs are played; even the smallest mu- berspace. visuals and soundscapes seen in the gallery and sical unit such as, say, a bass line or a vocal timbre streamed to the mobile users recreate a Situationist can open up a whole archive of other songs—and Imar de Vries is part of the Institute of Media & Re/ derive using mobile technology, a city-as-instrument. unpack their (sub)cultural libraries. presentation at Utrecht University, Netherlands. 72 2007 papers 2007 Posters 73

Taking Soundings – Investigating Coastal Navigations and Orientations in Sound

Yolande Harris

Taking Soundings is a series of sound art works emerging from an investigation into landscape and navigation. The full paper describes the processes and results of research undertaken during a fellow- bio ship at the Academy of Media Arts (KHM) in Co- logne (2006), and is based on practical and histori- Understanding the relations between sound, image cal research into coastal navigation techniques and and space through technologies of communication the potential relationship to sound. By exploring and navigation, has been the central focus of Yolan- the technologies of lighthouse and satellite naviga- de’s work over the last ten years. She explores the tion the work lays out some artistic strategies for intermediary role of the score, both as practical and mobile music composition by looking at physical conceptual tool, and as an open imaginary situation Egotone | generative ringtone engine. motion, notation in maps charts and scores, spa- for communication. Her Score Spaces project em- tialisation and orientation during navigation and its ploys a spatial approach to composition and has re- Irad Lee realisation in a sound installation, and the mapping sulted in numerous audio-visual performances and of navigation data to sound. installations, including the Meta-Orchestra, theoret- Abstract ical texts, such as Inside Out Instrument, and work- The project stems from my previous mobile artwork shops for composers, sound artists, architects and Egotone is a Generative Ringtone Engine software in extreme locations for an absent or individual au- designers. Her most recent works, Taking Sound- currently in development, that is designed to gen- dience. The Sargasso Sail (1997) explored sounds ings and Sun Run Sun, employ intuitive and scientif- erate interactive music compositions using mobile and the psychological impact of a sailing journey ic modes of knowing and join ancient and contem- device data. Egotone transforms statistical data re- away from land, navigating across the emptiness of porary navigation and orientation techniques from trieved from a mobile device into digital sound and the Bermuda Triangle. The Video-Walker (2002-3), a sextants to GPS, to explore our apparently chang- arranges it into a musical composition to be used portable projector with sensors to control changes ing relation to land and sea environments in the as a ringtone. in video, played with the interface between real age of satellite and mobile technologies. and projected image during the act of walking, as a Yolande has a degree in music from Dartington The application is based on an algorithm that con- powerful experience in hybrid reality. College of Arts and a Master of Philosophy from verts statistical information from a mobile device the University of Cambridge in architecture and the into musical parameters, resulting in an automatic The Taking Soundings installation and perfor- moving image. She has been resident researcher generation of a custom-made, copyright-free, per- mance turn data from lighthouse signals and GPS at the Jan van Eyck Academy in Maastricht, artistic sonalized ringtone with into sound placed in space. The technical set-up fellow at the Academy of Media Arts Cologne and a unique musical form and sound. consists of a handheld GPS receiver read continu- artist in residence at STEIM and the Netherlands ously by Max/MSP+Jitter software. This converts Institute for Media Arts in Amsterdam. She has The concept of Egotone is based on the assump- the data into electronic sound, defines the spatiali- taught interaction design at the Technical Univer- tion that similar people, as users of information sation over sixteen speakers, and controls video sity of Eindhoven, is guest lecturer at the Rietveld storage devices, tend to share similar contents, in playback. The paper describes different choices of Academy Design Lab, and lectures on her work in- which interesting patterns of behaviors can be re- bio sound spatialisation and data mapping to sound, ternationally. Her writings have been published in vealed, such as the mobile device’s owner identity and shows visual traces of GPS error from a fixed the Contemporary Music Review and Journal of Or- and users’ usage patterns. Egotone is able to give Irad Lee is a Tel Aviv born, Amsterdam-based cross- receiver. Experiments with the same set-up whilst ganised Sound an audible representation to these relations, and to media designer working with mobile audio systems driving in a car are described, which suggests the somewhat function as an audible mirror of the mo- and experimental media design. subsequent project Sun Run Sun and the Satellite http://www.yolandeharris.net bile device’s owner, a group of people, or a social Sounders (2008). http://sunrunsun.nimk.nl cross-section. http://iradlee.net 74 2007 Posters 75

Audio Bombing: Magnetic Cassette Tape Graffiti

Mike Fleming, Kang Chang , Kyle Millns

Abstract

Audio Bombing is an alternative form of graffiti that uses magnetic audiotape as its medium. Draw- ing from hip hop and graffiti culture Audio Bomb- ing starts with a basic cassette tape. Using a tape recorder you can record any information you want on to a cassette (music, poems, philosophy, subversive literature, etc.). After recording you remove the tape and cut out the segments that you want to use. Then take your tape segments and go tag whatever you want (buildings, benches, posters, buses, etc.). Using the augmented playhead spray can you can listen to the tags by running the playhead over the tape. The intention of this project was to create a new form of underground expression from a medium that is falling out of use; Reinventing graffiti with cassette tapes which have a long history in hip hop culture. It is open to anyone who has a cassette tape to “audiobomb”. It just takes recording what you want on cassette, cutting the tape up and then tagging it up. Manifesting audio samples into a physical and more visible form while allowing for manipulation of its playback echoes some of the re-appropriation of funk or disco beats seen in productions by DJs in early Hip Hop. It does not require any skill in drawing or traditional graffiti and functions under the radar of most suspicious au- thorities. Since this medium is less visually obtrusive, being only a thin black line, it has an undercover versatility bio which normal graffiti does not. This specifically allows it to infiltrate spaces traditional graffiti can not, Mike Fleming is based at the School of Art + De- such as office buildings, under tables, in elevators, sign at the University of Illinois at Urbana-Cham- coffee shops, schools, and tight spaces. paign, USA. The need to physically run a playhead over the magnetic tape in order to hear the audio tag makes Kang Chang is based at the Department of Natural the scenario of reading someone’s tag mirror the Resources and Environmental Sciences at the Uni- act of writing that tag. This project questions the versity of Illinois at Urbana-Champaign, USA role of the reader when taking part in subversive communication. In reading an audiobomb tag the Kyle Millns is based at the Department of Com- reader is put in the same position as a traditional puter Science at the University of Illinois at Urbana- graffiti writer instead of a traditional (passive) audi- Champaign, USA ence. The reader needs to actively engage with the content to receive the content of the tag. http://audiobombing.blogspot.com/ 76 2007 Posters 2007 Posters 77

Sonic Graffiti:S praying and Remixing Music on the Street

Chia-Ying Lee Mobile Music Creation using PDAs and Smartphones Abstract Ashley Elsdon Sonic Graffiti is a concept for people to spray and remix music on the street. It contains a system of Abstract devices for creators, including the sound cap, the controller and the boom box. The sound cap is de- This paper reviews the current state of available signed to be put on the real spray can, and makes mobile music creating application software for the sound sample spray out with the paint. The PDAs and Smartphones. The paper explores de- controller and the boom box serve as assist devic- veloper’s motivations and thoughts on the future of es during the creating process. Music is composed mobile music, and the responses of a few users to by overlaying paints. Hence the graffiti is the visu- questions about how they use mobile music tech- alization of the music, and the music is the sound- nology, and draws conclusions regarding the future track of the graffiti. Passengers can listen to the mu- of mobile music making. sic with a software player installed in their mobile Palm Sounds (http://the-palm-sound.blogspot. devices. Music is streamed to the device when the com/) is a blog devoted to all forms of mobile mu- passenger come close to the graffiti. sic making and technology. This paper is the first Graffiti is an urban phenomenon with its own prom- attempt at understanding user and developer inent subculture. With the rising of locative media, views and suggesting possible futures in the mo- the invisible audio is able to be tagged in the physi- bile music space. The field of mobile music making cal space. Here tagging technically means geo-tag- on PDAs and smartphones is still relatively small. ging, as a process of depositing digital content in There are only a handful of applications available a physical location. This project explores a concept in a market for PDA software that is comparatively of enabling people spray music on the street and very large. Although the desktop market for music make graffiti with both visual and audio elements. software is huge the handheld market has never ex- A system of physical devices is designed for art- perienced the same level of interest. This paper exists to create and tag music in the urban space with plores some of the current issues. real spray cans. For general viewers/audiences Son- Virtually all of the applications discussed in this pa- ic Graffiti provides a listening experience giving a per have their root functionality in desktop origins. sense of connection with the environment. They are in effect a translation of desktop technolo- Music is abstract to express visually. Some graffiti gies into a handheld environment. However, if you artists distort letters, design patterns to make ab- take the emergence of applications like SpinPad stract works; others do picturesque pieces. I am in- and AxisPad as proof that there musical uses that terested in investigating what new form of expres- could be unique to the handheld environment, then sion would evolve from the blending of music and where does this naturally lead to? Has handheld graffiti. The current design of Sonic Graffiti leaves hardware technology got to the point where input much freedom to artists for developing their own devices can be used in new ways to enable the de- formation and visual languages of music. They can vice to become much more of an instrument rather adopt a more improvisational attitude or sketch out than just a translation of a desktop application? their work before painting. The results may be short The iPhone’s sensor technology could lead to the sound signatures or epic compositions. use of sensors in music handheld music technology to provide a more directly manipulated and sensi- bio tive interface with the possibility of gesture control and recognition. It is perhaps a device of this na- Chia-Ying Lee ture which could extend the usage of mobile music Interaction Design Institute Ivrea in Milano technologies and encourage more users to explore [email protected] the field. 78 2007 Posters 2007 Posters 79

Creative Uses of Virtual Sticky Notes in Art - A Critical Interrogation of The “Bio-tracking” Smart Phone Based Exhibition

Anna Dumitriu An Interactive Musical Installation through Abstract Spatial Sensing

Anna Dumitriu, lead Artist on the Bio-tracking proj- Takuya Yamauchi and Toru Iwatake ect was introduced to the possibilities of using So- cialight during a presentation at the 2006 Mobile Abstract Music Workshop. Socialight is a leading-edge smart phone software, which enables users to post and This paper describes a sound installation work sup- access location specific ‘virtual sticky notes’ in the ported by spatial sensing system with a Personal form of picture, sound or text files. Area Network (PAN), which may be applicable for such areas as dance performances and mobile mu- In September 2006 visitors to the Internation- sic. The introduction section surveys some of spatial al Brighton Photo Biennial Fringe were invited to technologies that track moving objects and identify download Socialight and view “Bio-tracking” an these agents. The system design section describes exhibition of digital photographs, sound works and the architecture that makes use of MAX/MSP ap- text messages at noted cultural locations around plication. The actual installation is supported by the the city. Anna Dumitriu sampled and cultured mi- system. When we consider emerging human behav- crobes from the locations, revealing an incredible, iors in a new audiovisual space with PAN or WAN, it unseen and sublime world to us through a series becomes important to consider the social contexts. of enhanced digital micrographs. Luciana Haill, Ian In the “Sound Jewelry” installation, people with Helliwell and Juliet Kac created a series of sound “senders” walked around the floor in a location works in response to the images, scientific data and system zone and they recognized the changes of locations. Microbiologist Dr John Paul wrote a se- sounds according to the relative distance between ries of text messages to describe the microbes sci- them. The actual “Sound Jewelry” turned out to entifically. be an environment that consists of two layers of sounds. In the foreground, “melodies” are dynami- The photographic images created a dialectic, fus- cally generated by measuring the distances being the pure emotion of the sound responses and tween the participants. In the background, ambient the scientific analytical texts. The philosopher Scho- sounds are automatically generated using the dis- penhauer wrote about music’s ability to capture and tance data. When many people move in a 4X6 m2 express emotion “as an immediate objectification space, the sound only changes based on the near- and copy of the whole will as the world itself”. By est relative distance. However, as the number of juxtaposing these responses the project sought to people increases, the sound changes become more create a synthesis between art and science. complex. Sound complexity was used as part of the installation. Users recognized sound changes as The use of GPS, in mapping the locations where they moved in real time in the space. the microbiological samples were taken, fuses the microscopic and the macroscopic, drawing a thread bio between satellites orbiting the earth and the bacte- ria at our feet. Takuya Yamauchi is part of the “Media Design Pro- gram” at the “Graduate School of Media and Gov- ‘Virtual sticky notes’ are a powerful means of dissemi- ernance” at Keio University SFC, Japan. nating sonic and visual artworks. The Bio-tracking exhibition is a demonstration of the creative uses of the Toru Iwatake is Professor at the “Media Design Pro- medium and successfully engaged artistically, con- gram” at the “Graduate School of Media and Gov- ceptually and philosophically with the technology. ernance” at Keio University SFC, Japan. 80 2007 Posters 2007 Posters 81

Pocket Gamelan: swinging phones and ad Extended Enviro-Guitar hoc standards

Colin Black Greg Schiemer, Mark Havryliv

Abstract Abstract

In this paper, I explore the initial research and de- In this paper, we discuss how mobile phones have velopment regarding my mobile experimental been used as devices for active music making, how “Extended Enviro-Guitar” (XEG) instrument/s as a mobility affects sound and how communication be- type of resonating acoustic profiling device. It also tween phones has been integrated into the fabric of explores the possibilities of using multiple XEGs a new genre of interactive performance by groups within site specific physically spatialised multi-in- of musicians. We identify some of the issues that strument installations and the deconstruction of this stood in the way of developing two new musical ap- abstract sonic terrain via emerging mobile tech- plications for mobile phones, discuss aspects of per- nologies. formance works developed so far using this technology and point the way to future development. bio Computer music has had two persistent technologi- Black’s sound installations include the ‘Butter cal legacies. One is its dependence on performance Composer/sound artist Colin Black has been inter- Churn’ Sound sculpture in Lismore’s Heritage Park, interfaces designed around 12 equal divisions of nationally recognized with the prestigious Prix Italia the Parramatta Heritage Centre’s ‘Parramatta: Peo- the octave. The second is the desktop computing Award (2003) in the category ‘Best Music Radio - ple & Place’ exhibition which is to run for seven environment where musical resources are concen- Composed Work’ for composing and producing his years. In 2002 Black created a dynamic multi-site trated in the hands of a single user. As technological major work ‘The Ears Outside My Listening Room’. soundscape and the Starcourt Theatre Space sound development shifts away from this towards mobile BBC radio 3 has described this winning work as “a installation “Floodscape” for the NORPA produc- computing, new computer performance paradigms haunting evocation of Australia” while the National tion of ‘The Flood’. A finalist in the Australian Na- have begun to emerge. The Pocket Gamelan proj- Radio Company of Ukraine invited Black to help tional Digital Arts Awards ‘98, his experimental ect is motivated by a desire to explore the features adapt this work for a Ukrainian audience in 2004. composition “118, 120 122” was exhibited at Bris- of microtonal intervals found in many non-Western bane’s Institute of Modern Art. musical traditions and seeks to develop applica- Colin Black’s credits include the Australia Council tions that allow microtonal music to be composed for the Arts New Media arts Residency with ‘The He has presented research papers on radio art and and performed using mobile phone technology. The Listening Room’, ABC Radio in 2002, Musical Direc- sound art practice at RMIT University’s, School of combination of flying sound sources and remote tor for live TV variety shows, soundtracks for feature Creative Media (Melbourne, Australia) and the 4th controlled sound using hand-held technology has and short films, video, TV and digital media presen- International Mobile Music Workshop hosted by resulted in a new kind of interactive performance tations and further industry awards for Best Experi- STEIM (the Studio for Electro-Instrumental Music) genre in which microtonal instruments are easy to mental Song and Best Instrumental Composition. and Waag Society in Amsterdam, The Netherlands. play, quick to learn and readily accessible to large Black has also been invited to talk about radio arts numbers of people. Musical interaction involves us- As a Composer, he has been featured on and his work on London’s Resonance 104.4FM, at ing sound sources that are physically relocatable Deutschlandradio’s Kultur’s ‘Klangkunst’ program, the Sibelius Academy in Helsinki, Technische Uni- and wireless communication that allows moving ABC radio (Australia), YLE Radio (Finland) and versität Berlin and the Victoria University of Wel- sound sources to be controlled using hand-held bio his works have been selected for performance at lington. devices. events including ‘En Red O 2000’ music festival In performance Bluetooth-enabled mobile phones Greg Schiemer is Associate Professor at the Faculty Barcelona Spain, the Festival Synthese Bourges Black’s work has been selected for broadcast and attached to a cord are swung in a circular fashion to of Fine Arts, University of Wollongong. Much of his France, Rencontres Musiques Nouvelles, Lunel podcast on stations including Deutschlandradio produce Doppler shift. Phones are used either as a work is associated with musical applications of new France, 60x60 Pacific Basin Regional Concert Los Kultur, the BBC, YLE Radio, Sweden’s Sveriges Ra- flying sound source or as wireless hand-held control- technology since the early 1970s. Greag studied Angeles USA, Zèppelin 2004-Festival de Arte Sono- dio as well as many other European stations and ler to alter the behaviour of other sound sources. By composition with Peter Sculthorpe graduating from ro Barcelona, Spain, Hipersonica 2004 in Sao Paulo, in South Africa, New Zealand and numerous times appropriating mobile phone technology as a me- Sydney University with Bachelor of Music in 1972. Brazil, The Literature Sound Barrier 2002 in Wien, across Australia. dium for interactive musical performance interface After a period working for Digital Equipment Aus- Austria, Sydney University’s Live Wires concerts ‘97, we hope to allow communities of non-expert per- tralia, he lectured in electronic composition at the ‘98 and Melbourne’s Extatic Concert for the Next For more information see http://users.tpg.com.au/ formers to experience the richness and diversity of Canberra School of Music (1983-85) and the Sydney Wave festival ’98. users/cydonian/c_black.html microtonal music. Conservatorium of Music (1986-2002). 82 2007 Demos 2007 Demos 83

robotcowboy. A Human-Computer Perfor- mance System

Dan Wilcox

Abstract

This article presents the human-computer mobile performance project entitled “robotcowboy”. robotcowboy consists of a “one-man band” wearable computer system dubbed “unit” composed of a mobile computer and various input devices such bio as midi controllers, game controllers, and environmental sensors. Gottfried Haider, born 1985 in Vienna, is interested in urban sound scape theory and currently research- robotcowboy is a performance project centered ing on the interdependencies of the algorithmic around using the power of the computer for active, acquisition of space, its coexistent manipulation mobile expression. The main goal of the project is and numerology. He lives and works in Vienna and mobility : performers can use the system as an in- Rotterdam. strument - an extension of themselves. They are free to roam the stage, the street, and the world Exhibitions (Selection): performing computer-based music, becoming Work In Progress Demonstration of Craving: 2008 Participation transmediale08, Berlin / DE “more then an extra” to the machine. In the vein of A Spatial Audio Narrative 2007 Exhibition (mis)used Media, Vienna / AT Terre Theimlitz smashing a laptop on stage, it is an 2007 Exhibition etc17, Vienna / AT attempt to challenge the nature of live computer Bernhard Garnicnig, Gottfried Haider 2007 Participation Mobile Music Workshop, Am- music performance. sterdam / NL Abstract 2004 Award of Distinction Prix Ars Electronica - u19, robotcowboy aims to be a human-computer per- Linz / AT formance system allowing the user to produce a While the auditory composition Craving (see also dynamic audio-visual experience for the audience. spat_lab Projects) was specifically de- There is a history of one-man band acts and per- signed for Vienna DC, a site in the Austrian capi- Bernhard Garnicnig, born 1983 in Bregenz, Austria, formance troupes producing music in the course of tal, the demonstration in Amsterdam provided a studies Digital Arts at the University for Applied the exhibitions, why not attempt to combine both glimpse into the possibilities the technology offers Arts in Vienna, Austria. He is a founding member of using wearable computer technology? to create audiovisual experiences by linking specific the Atelier c17 offspace in Vienna and active in the places with sound structured into space and time. field of improvised turntable détournement. bio Simple acoustic scenes were established in the workshop’s venues, a small park next to STEIM and Selected Exhibitions and Performances Dan Wilcox holds a B.S. in Computer Engineer- the Nieuwmarkt around the Waag building. Sound 2008 albért bernàrd - das mysterieux project, c17, ing from Iowa State University, USA and an M.S. fragments and structures specific to the production Vienna / AT Art & Technology from the IT University / Chalmers, in Vienna were left out in favour of larger areas of 2008 Exhibition Kunstraum Engländerbau, Vaduz / LI Göteborg Sweden, where he also worked on the atmospheric sound and brief temporally-structured 2007 Exhibition (mis)used Media, Vienna / AT “Interactive Installations Course” and as Teaching voice scenes. It was the first public presentation at 2007 Participation Mobile Music Workshop, Am- Assistant. In October 2007 he did a 2 Week Artis- this stage in the life of the project and resulted in sterdam / NL tic Residency at the Studio for Electro-Instrumental new insights into the perception and interpretation 2005 Performance 22. Kasseler Dokumentar-, Film- Music (STEIM) , Amsterdam, NL. of the work by the workshop participants. und Videofest, Kassel / DE 84 2006 University of Sussex, United kingdom 85

3rd Mobile Music Workshop 86 2006 KEYNOTE 2006 INVITED SPEAKERS AND TUTORIALS 87

Create with miniMIXA

Tim Cole

Abstract bio Sound Moves, iPod Culture and Urban BAFTA-winning miniMIXA was a powerful mobile Experience music mixing and performance application de- intermorphic Ltd, established in 2007, is the latest veloped by Tim and Pete Cole (now founders of company from multimedia artist and songwriter Tim Michael Bull intermorphic.com, developer of tools for nurturing Cole (aka colartz) and his brother Pete Cole, with ideas). whom he has worked for over 18 years on developing generative music systems and products and In this lecture I present work on the social na- The presentation discussed the wider market technologies for mobile music making. intermor- ture and meaning of iPod use. The material de- opportunities for mobile creativity applications, phic creates tools for nurturing ideas, such as the rives from a large scale international survey of on-device created social media (user generated trans-generative music system, hyper-instrument users conducted in 2004. content) and how record companies needed to and fluxer “/noatikl/“, and cut-up lyric generator “/ The iPod is used as a prism through which to un- be open in considering new opportunities. Tim liptikl/”. derstand the nature of the public world in which outlined his firm belief that mobile devices have an we live. Sound plays a central role in how the urban important future as creative tools, hence presenting In 1990 the Coles founded SSEYO to develop the citizen becomes rooted in mobile urban space, of the concept musiKfone as “the mobile phone is generative music system, “Koan”, which went on to how they acquire their ‘being in the world’ through the next electric guitar”. He presaged a time when be used by Brian Eno in 1995/6 to create “Genera- the creation of privatized sound atmospheres. iPod many more highly capable mobile phones would tive Music 1”. By 1996 Koan had become the first culture possesses its own processes of auditory be capable of running advanced creativity applica- European developed plugin for Netscape and in gating and filtering that embodies the urban world tions such as miniMIXA. 1998 Tim created the first online collaborative vir- that most of us inhabit. I argue that the role of tual generative composition, Koan^oasis, and gave sound based technologies require investigation 12-track miniMIXA V2 was demonstrated, showing a keynote speech at ISEA. SSEYO raised several in order for us to reach an understanding of how how easy it was for a DJ-like performance to be rounds of funding and went on to win a BAFTA for we come to share social space with others in ur- created and given from a mobile device. In this Technical Innovation in 2001. ban culture. case, the phone we being used more like an stereo iPod users appear to live in a world of mediated instrument where live studio-like FX settings such as By 2002 SSEYO had become a wholly owned sub- we-ness in which sound provides both the dreams reverb, chorus, delay, filter and EQ etc could all be sidiary of Tao Group, where the Coles developed and the chains for the urban subject. Mediated selected and changed in real time on a per-track or the 2005 BAFTA-winning mobile music making aural proximity constituted states of ‘we-ness’ global basis. This was in addition to being able to application “miniMIXA”. whereby ‘direct’ experience is either substituted or on-the-fly select and add in multi-bar audio loops transformed by a mediated, technological form of (including user’s own), MIDI riffs, user microphone In 2007, and prior to the imminent release of aural experience. A dialectical analysis of iPod use bio recordings etc. miniMIXA v3, the Coles parted with Tao Group and points to a disjunction between the objective and founded intermorphic, where they are using their the subjective moment of culture in which iPod us- Michael Bull is a Reader in Media and Film at the miniMIXA also contained a powerful modular many years of experience to develop new, fun and ers attempt to transcend the social precisely by University of Sussex and has written widely on software synthesiser with a host of capabilities, interesting creativity tools for desktop and mobile. immersing themselves in it. It is a world in which sound, music and technology. He is the author and it was shown how sounds generated from this the auditory self of iPod culture perceives the world of Sounding Out the City. Personal Stereos and could also be worked into the mix. Both miniMIXA See: http://www.intermorphic.com and http://www.colartz.com move to sound and are simultaneously moved by Sound Moves:iPod Culture and Urban Experience simpler mode being where key-presses (or screen- sound. Sound itself is normative, mediating and re- (Routledge 2007) and is co-editor of The Auditory tap on touch screen devices) could e.g turn loops flecting the cultural predispositions of the listener Culture Reader (Berg 2003). He is also the found- on/off and the other “cell mix” mode where there Live mix made with miniMIXA V3: http://youtube. who ‘gates’ experience. The manner of this audito- ing editor of The Senses and Society Journal pub- were 4 difference mix sections. com/watch?v=uBP97NwG4nU themselves. Not only do we get the technologies New Trends Forum, a European Thinktank funded ed with devices where they could try out miniMIXA SYPC Mobile Mashup with miniMIXA: http://you- that we deserve, we also get the ones we desire. by Bankinter, Spain. for themselves. tube.com/watch?v=q-nqTef3j30 88 2006 INVITED SPEAKERS AND TUTORIALS 2006 INVITED SPEAKERS AND TUTORIALS 89

Socialight

Michael Sharon

Socialight is a content platform that lets any pub- lisher or person create and discover media and information placed in physical locations such as schools, shops, and parks around the world. Any- one can find the content, whether on their mobile phone when nearby or by browsing a map on the IMPROVe web. With Socialight, we’ve created an entirely new media channel that’s place-based. Richard Widerberg, Zeenath Hasan The Sticky Note The basic building block of Socialight is the Sticky Abstract Note- similar to a yellow Post-it note that you find at the office, except that it can contain text, images, (see also p. 40) audio and even video! Sticky Notes can be stuck to any location in the world and you can choose who The everyday sounds that we experience are pro- can see yours... duced outside of our own volition. The capacity to Socialight Mobile capture sounds, however, was not possible till the The best part about Socialight is using it on your invention of electro-magnetic recording devices in cell phone. This lets you discover all kinds of things the early twentieth century. Since then, the sepa- that are actually near to you! You can also make ration of sound from its source, and the capabil- comments and rate the things you find as well as ity to play it back, has made it possible to listen to stick your own notes. We can also notify you about sounds outside of its original context. The mobile the things that interest you so you never walk past phone is also a medium through which sounds are something cool again! heard outside of their original context. However, What you see is relevant to: the normative definition of the mobile phone as a * where you are medium for communication has restricted its po- * who your friends are tential as a medium for sounds that exist outside of * the channels you subscribe to the immediate tele-communication. IMPROVe is a design and research project that explores the po- It’s like having a guidebook written by your friends tential of the mobile phone as a medium of com- and the people you trust. You can also rate, tag and munication beyond its currently dominant role as leave comments on the things you find. You never a transmitter of sounds. The project proposes the get SPAM because we only notify you about the design of the mobile phone as a medium for the things you want. You can set your notification pref- exchange of everyday sounds within communities erences here. and across socio-cultural contexts by mobilizing the We provide a WAP version that works on almost ev- bio potential of the mobile phone as a tool for the pro- ery phone as well as a JAVA version that works on duction of everyday sounds. To listen carefully to certain handsets and has cool features like GPS in- Michael is a media artist, writer and programmer the environment is something we want to empha- tegration and a slicker interface. whose work runs the gamut from mobile social soft- size in our design. We believe that when the pos- ware to gestural music interfaces to big games and sibility to record and work creatively with the sonic Socialight Online everything in between. He is the co-founder and environment exists, then a higher awareness of our You can explore Socialight with nice big maps CTO of Socialight, a New York-based company de- environment is achieved. Needless to say, the play- and broadband juiciness not possible on mobile veloping social media tools for mobile devices. He back of the recorded sonic environment is only a phones. So we encourage you to look around, join is an Adjunct Assistant Professor at Columbia Uni- representation of it. But to work consciously with some channels, make friends and create some versity’s GSAPP co-teaching a class in Big Urban this representation is what, we believe, heightens Sticky Notes! Then check your mobile preferenc- Games. He is an Adjunct Professor at New York Uni- our awareness of our sonic environment. es are set up correctly and start enjoying the same versity’s Interactive Telecommunications Program, great stuff on your phone! teaching a class called Mobile Application Design. http://www.riwid.net/improve/ 90 2006 Mobile Collaborative Music Making 2006 Mobile Collaborative Music Making 91

Sequencer404

David Jimison, Travis Thatcher

Abstract CELLPHONIA: In The News

Sequencer404 is a mobile phone software applica- Steve Bull, Scot Gresham-Lancaster & Tim Perkis tion that enables multiple users to participate in music generation. Users engage in the orchestra- Abstract tion of a 16 step 4-measure continual composition, similar to those found in older synthesis/sequenc- Cellphonia: In The News is an open source cell ing systems, such as Roland’s TR-808. Up to four us- phone karaoke opera with a mixed final perfor- ers select instruments (percussion, piano, orchestral mance delivered to the participant as a podcast hit, etc.) which determine the samples available. and online as a web based mp3. The ever-changing Connections between phones are made either via current state of the opera will be continuously avail- central server, or, in 2 person mode, through Blu- able as an online stream-cast. etooth. Cellphonia: In The News is an open source location-based karaoke cell phone opera that uses a Once connections are established, users trigger libretto generated from RSS news feeds. The music sound samples by pressing the numeric key pad on is both pre-composed and algorithmically gener- their phone. Keys ‘1-9’ trigger either notes in the ated by news feeds. The full opera is comprised Cm scale, or different timber percussion. Users re- of many callers’ voices mixed with audioenhance- Bio place existing notes, or erase them by pressing the Bio ment tools and delivered by continuous Internet ‘0’ key. Pressing the ‘*’ and ‘#’ keys toggle octaves audio stream-cast. The fresh addition of new caller Steve Bull offers a portfolio of assets from his last in the instrumentation. David Jimison is a PhD candidate in the Digital Me- voices and evolving music creates a never-repeat- seven years as founder of Cutlass, a company which dia program at Georgia Institute of Technology. ed streaming 24/7 opera. Individual songs from the specializes in mobile locative media and art with Key presses are registered by the central server, His research interests are in urban computing. He opera are available as mp3 files that can be down- applications running on 02, Verizon Wireless, TE- which combines the data into a notational structure is currently a Fellow at Eyebeam Art & Technology loaded or retrieved as a podcast. With Cellphonia, LUS Mobility, and Orange. In 2006 his Hot-n-Cold containing all users’ music, which in turn, is read Center. a potential worldwide group of users is provided was a NAVTEQ Lbs Challenge finalist, he launched by each mobile device. This structure enables indi- with a means of unique social interaction that refers his New-York Historical Society Slavery Tour on vidual users to be out of sequence with each other, Travis Thatcher has been involved in research in to a centuries old tradition, opera, in a new con- vodcast, podcast and VOIP, and the NY Times re- while facilitating a near real-time collaboration. Ev- human computer interaction for live performance text that leverages the broadening wireless tech- viewed the premiere of his Cellphonia: San Jose, an ery 16 steps, the new notational structure is loaded and interactive sonification.He has performed as an nological base in a simple, familiar, and accessible opera at ISEA ZeroOne Festival. After completing from the server, and used to trigger the appropriate electronic composer and musician for the last seven manner. In Cellphonia, the artist is both coder and NYU’s Interactive Telecommunications Program, he sounds on the phone. New notes are added upon years and as a saxophonist for the last twelve years. composer, while the caller is both performer and worked as a senior information architect and proto- their event to the server notation, enabling them to Thatcher received a Bachelors in Computer Science audience. type designer/developer for Interval Research. He be heard by other users during the next measure. from Georgia Tech in Spring 2005, and a Masters¬† also won many awards for his media productions As a repetitive musical structure, users can antici- degree in Science, with concentration in music http://cellphone.el.net/NIME/ both national and international and is a member of pate the variations from the other users. technology in May 2007. http://cellphone.el.net/listen/ the Directors Guild of America. 92 2006 Mobile Collaborative Music Making 93

China Gates (A Digital Art Weeks Mobile Music Project)

Art Clay

Abstract

The work China Gates is technically based on possibilities of synchronizing a group of performers using the clock pulse emitted from GPS satellites. Aesthetically, China Gates is rooted in works for open public space and belongs to a genre of works, which celebrate the use of innovative mobile technologies to explore public space and public audience. The performance takes place in a limited city area such as a city square, a park and open courtyard.

A series of tuned gongs is used. The number of gongs is greater than the number of performers participating. Tuned to an Eastern musical scale, these gongs give the piece a touch of the orient on the horizontal, melodic side and a western type dis- sonance on the vertical, chord structure side. The gongs are circulated amongst the players by an exchange process so that an on going change in har- monies can be achieved.

Each of the players wanders through the perfor- bio mance space freely. A custom built GPS interface on the wrist registers the player’s position and de- Sound Artist Art Clay (born in New York, lives in Ba- termines to geographical coordinates when to play sel, Switzerland) has worked in Music, Video & per- the gong. By using a delay between the satellite formance. He is a specialist in the performance of clock pulse and the LED that indicates when to self created works with the use of intermedia. Ap- strike the gong, a harmolodic effect is obtained as pearances at international festivals, on radio and the players gradually shift from a chordal to a me- television in Europe, USA and Japan. Extensive lodic structure dependent on geographical coordi- compositions for acoustic and electronic mediums nates. In general, each player tries to move when in many genre including dance, performance and another is not, so that a “choreographic counter- theater. Art Clay also directs the ‘Digital Art Weeks’ point” results that allows for a rhythmic-melodic Program held yearly at the ETH in Zurich. Recently, coloring caused by the vertical to horizontal un- his work has focused on large-scale performative folding of the struck gong chord. The performance music-theater works and public art spectacles us- ends for each player at the return to the start point. ing mobile devices. He has won awards for music The interface therefore acts as a “conductor”, in- composition, performance, and new media art. He dicating when the gongs are to be hit and how the teaches at various art institutes in Europe including music as a whole will sound in the end. the Zurich University of the Arts. 94 2006 Soundscapes & Mobile Listening 95

Tactical Sound Garden [TSG] Toolkit

Mark Shepard

Abstract

The Tactical Sound Garden [TSG] Toolkit is an open source software platform for cultivating public “sound gardens” within contemporary cities. It draws on the culture of urban community garden- ing to posit an infrastructure for new spatial prac- bio tices for social interaction within technologically mediated environments. Addressing the impact of Mark Shepard is an artist, architect and researcher mobile audio devices like the iPod, the project ex- whose cross-disciplinary practice explores new so- amines gradations of privacy and publicity within cial spaces and signifying structures of contempo- contemporary public space. rary network cultures. His research investigates the implications of mobile and pervasive media, com- The Toolkit enables anyone living within dense munication and information technologies for archi- 802.11 wireless (WiFi) “hot zones” to install a tecture and urbanism. “sound garden” for public use. Using a WiFi enabled mobile device (PDA, laptop, mobile phone), His recent project, the Tactical Sound Garden participants “plant” sounds within a positional au- [TSG] Toolkit, is an open source software platform dio environment. These plantings are mapped onto for cultivating virtual sound gardens in urban pub- the coordinates of a physical location by a 3D audio lic space. It has been presented at museums, fes- engine common to gaming environments - overlay- tivals and arts events internationally, including the ing a publicly constructed soundscape onto a spe- Contemporary Museum, Baltimore, Maryland; Con- cific urban place. Wearing headphones connected flux 2006; Brooklyn, New York; ISEA 2006, San Jose, to a WiFi enabled device, participants drift though California; SIGGRAPH 2007, San Diego, California; virtual sound gardens as they move throughout Futuresonic, Manchester, UK; Sonar Festival, Bar- the city. celona, Spain; The Electronic Language Interna- tional Festival – FILE 2007, São Paolo, Brazil; and The Toolkit is a parasitic technology. It feeds on the the Arte.Mov Festival for Mobile Media, Belo Hori- propagation of WiFi access points in dense urban zonte, Brazil. environments as a free, ready-made, locative infrastructure for cultivating community sound gardens He is co-editor of the Situated Technologies Pam- in contemporary public space. Access points pro- phlet Series, published by the Architectural League ducing the WiFi signals used to determine the lo- of New York and co-author of “Urban Computing cation of a participant may be open or encrypted, and its Discontents” with Adam Greenfield. Oth- and need not be “owned” by those deploying the er publications include “Situating the Device” and TSG system. As the hardware component of the in- “working title: Industrian Pilz”, in Shark, a Journal frastructure is tied to the propagation of WiFi net- of Poetics and Art Criticism, v.1 & 2; and “Tactical works, the extent of the gardens is cast in a par- Sound Garden [TSG] Toolkit”, in 306090 v.9 - Re- asitical relationship to that of a specific wireless garding Public Space, published by Princeton Ar- protocol. Where the presence of access nodes is chitectural Press. minimal, gardens consist of plantings along a side- walk. Where a local density of nodes exist, gardens He is currently an Assistant Professor of Architec- potentially take the scale of a neighborhood. In cit- ture and Media Study at the University at Buffalo, ies where wireless networks are ubiquitous, gardens State University of New York, where he co-directs extend throughout the entire city. the Center for Virtual Architecture. 96 2006 Soundscapes & Mobile Listening 2006 Soundscapes & Mobile Listening 97

Composing the soundscape: Re-engaging with place

Anthony Phillips

Abstract

How does sound shape the everyday experience of our environment? Before audio technology and the now ubiquitous use of mobile devices that incorpo- rate sound our natural or acoustic soundscape provided us with meaningful interaction. Sounds held both personal and collective meanings, articulat- BluetunA ing a sense of community, place and aesthetic value to the individual. Acoustic ecology has shown how Arianna Bassoli soundscapes have changed over time, from the well defined acoustic profiles of the rural environ- Abstract ment to the mechanical and media rich environments of the modern day city. In the city significant BluetunA is an application running on Bluetooth- sounds are increasingly hidden in a homogeneous enabled mobile phones that allows users to share soundscape of mediated sound and urban noise in information about their favourite music with others which meaningful interaction with the auditory envi- nearby. With BluetunA people first create a list of ronment is replaced by a ‘tuning out’. Mobile audio favourite artists or songs, which can be done manu- technologies further perpetuate this sense of de- ally or automatically based on the MP3s already tachment through the creation of multiple spaces uploaded on their mobile phone. Then they are both virtual and physical that the user has to occu- able to see who else in proximity has similar taste in py and negotiate. These technologies encourage a general, or they can search for people who share a type of distracted listening that I refer to as ‘mobile common interest in a specific artist. This search can bio mediated listening’. automatically be repeated periodically if users pre- Drawing on 20th century compositional prac- select a keyword search list, or custom searches can Arianna Bassoli holds an MSc in Communication tices and in particular soundscape composition be made at any time a user likes. When a user en- Sciences from the University of Siena, Italy, where and acousmatic music my research extends exist- counters someone with similar taste, they are able she specialized in mass media. She then worked ing work on meaning and representation in musi- to exchange messages with each other over Blu- as a research fellow at Media Lab Europe for three cal composition. Within auditory design there has etooth. Further, BluetunA is integrated with Last.fm, years, mainly focusing on the application side of been a preoccupation with ‘sound as information’ allowing users to automatically download their Last. mobile peer-to-peer and ad-hoc networks. She is rather than sound as ‘an aesthetic experience’. fm profile to the BluetunA system, and obtain ad- currently a PhD candidate at the London School of Music provides an alternative in which aesthetic ditional music recommendations. To further enrich Economics and Political Science, UK, in the Depart- response determines the personal significance of the BluetunA experience, people can interact with ment of Information Systems and Innovation Group. our experience. Going forward in my research two bio each other through their mobile phones while sit- She is interested in interaction design, urban com- questions are key: what affect do different types of ting in cafes by accessing BluetunA hotspots which puting, and the design of mobile proximity-based sounds have on the ësignificance of our experienceí At the time of this research, Anthony Phillips was provide a wider range of music sharing options. applications, technologies that support communi- and is it possible to categorize aesthetic response working as part of the Equator project at the Inter- With BluetunA we have investigated the opportu- cation and data sharing among co-located peao- based on different types of sounds? The latter act Lab, University of Sussex, Brighton, UK. Anthony nity to create a lightweight application for exist- ple. Arianna is also a research assistant at the LSE, question raises an issue that I would like to discuss is interested in an interdisciplinary approach to ‘me- ing and commonly used technologies (e.g. mobile working on the EU-funded project BIONETS, which at the workshop i.e. methods of measuring aesthet- diated space’ with particular reference to coexistent phones and Bluetooth) able to provide an aware- looks into the future of wireless networks. ic response to sound in contexts that are both tran- interactions. Prior to starting his DPhil Anthony’s ness of the surrounding population and a fun way sitory and public. background was in Music and Multimedia Systems. to get music recommendations. http://www.karmanet-design.com/ 98 2006 HCI in Mobile Music & Uses of Music in Mobile Settings 2006 HCI in Mobile Music & Uses of Music in Mobile Settings 99

Minimal Attention Navigation via Adapted Music

Rachael Hunt, Mark Apperley, Sally Jo Cunningham, Bill Rogers & Matt Jones

Abstract

Navigating using subtle cues from the audio track you are listening to may make your journey as en- SonicPulse – exploring a shared music joyable as the destination. In this project, we are space investigating enjoyable ways of providing pedestri- ans with navigation support, specifically by allowing Akseli Anttila them to navigate to music. Many of the navigation aides available to pedestri- Abstract ans require their full attention; for example, to use a map you must stop and study it closely, reducing In this paper we present a design for a social music engagement with the surroundings. Walking with application for mobile devices. The design allows a guide is much better — you pay almost no atten- users to passively monitor a shared music space, or tion to the task of navigation, but are still directed actively look for other users of the system. The user to your destination. Audio-based interfaces have can furthermore engage in shared music use. The some of these advantages; specifically, they leave proposed design can be used to investigate the the visual sense unimpaired. question of user willingness to engage in playful Currently, systems have been developed using music sharing, and methods which allow both local three audio approaches: spoken cues, audio icons and remote experience sharing. (non-speech cues), and adapted stereo signals. In this paper we present a design for a social music Our work focuses on further investigating minimal application for mobile devices. The design allows attention audio user interfaces. There are a range users to passively monitor a shared music space, or of alternatives for adapting music. How much al- actively look for other users of the system. The user teration (of favored music) will listeners allow? Does can furthermore engage in shared music use. The the type of navigation cue affect the user’s mental proposed design can be used to investigate the load? Will this type of subtle navigation system be question of user willingness to engage in playful as effective as other more traditional navigation music sharing, and methods which allow both local aides? What type of direction do users prefer, and and remote experience sharing. which is the most efficient? Do the user’s objectives In active scanning the user can send out a ping, a alter the style of guidance that they require? personal audio snippet. The ping is reflected from An initial experiment was carried out to measure other users as an echo, a sound describing the na- the cognitive burden of different cue types. We ture of the discovered user. The users can send compared speech cues, audio icons, and adapted bio specific music files or distinct sounds to others, de- bio stereo against walking with a friend. Users listened noting e.g. a willingness to start a shared session. to a spoken audio track while navigating, engaging Rachael Hunt works at the HCI Group at the Uni- Thus abstract dialogue can lead to musical sharing, Akseli Anttila is a doctoral student (music meets them and leaving navigation as a secondary (low at- versity of Waikato, New Zealand. Rachael is inves- colistening and taking turns as DJs. The proposed mediated communication) and a designer at Nokia, tention) activity. Cognitive burden was measured by tigating low cognitive weight navigation systems. system and the results of a field evaluation will pro- working mostly with media applications. He works evaluating users’ memory of the audio track, and Specifically, she is examining how auditory changes vide insight to the value of a musical communica- at the Nokia Research Centre in Finland. He started through questionnaires. The next step is to further in music may be used to guide both tourists and tion system in a mobile context. The main research at UIAH film department (cinematography), moved investigate adapted music navigation cues. When locals to places of particular interest. The two main questions concern the musical and social aspects of to pre-medialab IMI (3D animation and CGI), and listening to a music track of choice, what types of goals of this research are assessing whether music the design. A third question concerns the feasibility holds a MA from Media Lab Helsinki (online com- cues are noticeable, and do they affect the listening can be used for navigation, and whether this is a of the design for prototyping and implementation. munities). experience? lightweight, fun way to navigate. 100 2006 Soundscapes & Mobile Listening 101

Music Mood Wheel – Ear-based Interfaces for Mobile Music Devices

Andreja Andric, Pierre-Louis Xech

Abstract

When we browse our music collection on any mobile device, we actually move inside a tree of tex- tual options that refer to the musical metadata. We search music by title, author, genre, artist, year, etc. But what if we cannot recall the name of the song that exactly matches that particular melancholic mood we are in today? What if, additionally, we have wrong or incomplete metadata, which happens often in private music collections? With thou- sands of songs on our iPod or iPhone, music search becomes a real challenge. In addition, many everyday situations in which mobile music devices are used, for example driving a car, or working out, do not permit wasting too much time and attention on choosing music. In our approach, we shift from the “tree of options” paradigm to a “search by ear” browsing bio experience, inspired by the frequency wheel on old-fashioned radios. A series of prototypes, based Andreja Andric on many low level features automatically extracted Born in Zajecar, Serbia in 1973. He obtained a B.Sc from the audio, was implemented and tested out- in Electrical Engineering and M.Sc. in AI appli- doors with a group of 40 participants. The interfac- cations in education from the School of Electri- Pierre-Louis Xech es performed well compared to two “off the shelf” cal Engineering, Belgrade University, Serbia (for- Joined Microsoft in 1997 as a Project Manage- references: Apple iPod and Samsung Portable Me- mer Yugoslavia). By the end of 2002 he won a PhD ment Consultant, and moved to Microsoft Research dia Center. scholarship from Milan State University in Italy, and Cambridge in 2002 as a Research Program Man- Our first prototype was without display and the subsequently he moved there. He obtained a PhD ager in the External Research Office group. Since song selection was controlled by a trackball. The fi- in computer music applications in 2006 and con- then, he has been involved in various collaborative nal prototype was developed on Windows Mobile tinued to work as a research assistant on the same and interdisciplinary research projects with univer- 5.0 powered Smartphone, and exhibited a 2D map University. His research project Music Mood Wheel sities across Europe in the area of Ambient Intel- of songs. has won a research grant from Microsoft Research ligence. In 2007 he moved to Microsoft France in In the workshop we exposed the lessons learned Cambridge, on their open concourse “Create, Play Paris where he heads the research partnerships pro- from our first experimental study and confronted and Learn” for research in computer applications gramme. Pierre-Louis Xech has a special interest in our “search by ear only” design pattern and related with social and cultural value. From 2007 onwards, investigating the ways in which the research in Intel- research issues with other participant experiences he is employed as software engineer in Parvis, ligent Environments can stimulate the human intel- and contributions. where he works on computer-vision systems for au- ligence in unleashing one’s own creative potential The Music Mood Wheel project evolved from mid tomatic inspection of banknote print and security and skills. In particular, he is interested in exploring 2005 until the end of 2006. It was a collaboration features. how the recent advances in computing and audio, project between the State University of Milan, Com- His research interests chiefly lie with interaction de- ubiquitous computing, vision and machine learning puter Science and Communications department, sign, user evaluation methodologies for multimedia and mobile technology can provide the basic bricks and Microsoft Research Cambridge, External Re- applications, and psychology of music preferences. and blocks for building the new “sound machines” search Office, Intelligent Environments Group. Lives in Milan, Italy. of our everyday life. 102 2005 Vancouver, Canada in Association with NIME 2005 103

2nd Mobile Music Workshop 104 2005 Papers 2005 Interactive Poster 105

Mobile User-Interface For Music

Takuya Yamauchi,Toru Iwatake From calling a cloud to finding the missing track: Artistic approaches to mobile Abstract music The present study considers the interface design Frauke Behrendt for music and graphical content in ubiquitous space. Herein, we present a mobile user-interface Abstract system for music and visual collaboration in a personal area network (PAN). The system, which is con- This paper is challenging the common understand- nected to a local area network (LAN; IEEE802.11b), ing of mobile music as ‘ringtones and i-pods’ by is composed of an agent on a mobile device con- analyzing artistic approaches to it and by offering troller (PDA), a sound engine, and a graphical lan- new categories to contextualize these projects in a guage environment. We considered the middle- move towards a taxonomy of mobile music. Eight ware agent that processes the context awareness in artworks from the rapidly expanding field of Mobile media art and game development. The prototype Art will be described and set into context. Most system “Sound Pad” demonstrated here is a musi- projects do not label themselves as mobile music, cal instrument and the graphical controller of a mo- but analysing these artworks as mobile music bile user-interface. provides a fruitful context for discussing these Ubiquitous space consists of numerous micro ma- works. “Sky Ear”, “Track-The-Trackers”, “BubL chines and host computers that are connected by Space”, “Telenono”, “Schminky”, “Simpletext”, sensors and other devices to electric household “Surface patterns” and “Urban tapestries” illustrate appliances and wearable computers. The ubiqui- the variety of sounds in mobile music: spoken text tous space in a PAN contains context information messages, missing tracks that need to be identified from other agents. Thus, the user-interface of this on a mobile platform, the crackers and whistlers context data must present information without con- of the electromagnetic sphere, knocking sounds fusing the user. A considerable number of papers ‘attached’ to surveillance cameras, other peoples have been published regarding this technology. favourite songs fixed to a specific urban place – “Sound Pad” is a handheld controller for producing and on the far other end of possibilities: silence bio score files and graphical content. The user is able produced by a radiation-proof box or by jamming to move through ubiquitous space while controlling phone signals in close proximity. Frauke Behrendt conducts research into the expe- the Sound Pad and enjoying the artwork made by The analyzed artworks are presented in two rience of urban space via mobile media; focussing media artists. categories: The first part of this paper focuses on on interactive art, music and sound projects that We propose here the Sound Pad mobile device the social context of mobile music exploring new experiment with this experience. She is currently user-interface as a mobile device (PDA) interface, a forms of audience participation and collaborative finalising her PhD (DAAD funded) at the Depart- sound instrument and a visual interface. The Sound mobile music. In the second part the focus shifts ment of Media and Film Studies at the University Pad user is able to manipulate sounds using the to the technological context of mobile music by of Sussex, (UK), is on the steering committee of sound engine (Pure Data) as well as the visual con- “Listening to the invisible”. Overall, the artist offer the International Mobile Music Workshop and Ger- tents of the graphical language (Processing) and a new and unexpected view of the urban space man delegate for the European Action on Sonic use sound composers and graphic designers in where peoples’ movements and the collaborative Interaction Design (SID). Her book “Handymusik. ubiquitous space. bio soundtrack they choose or produce for their urban Klangkunst und ‘mobile devices’” (“Mobile Phone Composers and artists are able to produce sound journeys represent the city in as much as physical Music. Sound Art and Mobile Devices”) has been designs by creating score files in Pure Data. Inter- Takuya Yamauchi is part of the “Media Design Pro- buildings or the grid of the streets. Analysing published in 2004. Frauke’s research is published face designers can also develop new user-interfaces gram” at the “Graduate School of Media and Gov- these examples by focussing on the relationship in English and German, and has been presented at using this mobile interface. In addition, graphic de- ernance” at Keio University SFC, Japan. of geographical, social and technological context various international conferences such as NIME and signers can produce graphic content in Processing. of mobility might prove a helpful framework for ISEA. She is a member of the “Centre for Material The system enables the collaboration of these con- Toru Iwatake is Professor at the “Media Design Pro- understanding the artworks, a first move towards a Digital Culture” and of Richard Sennett’s “NYLON tents and makes it possible to design context for gram” at the “Graduate School of Media and Gov- taxonomy of mobile art and music. Culture and Society” Seminar. networked sensors and agents in ubiquitous space. ernance” at Keio University SFC, Japan. 106 2005 Interactive Poster 107

bio

EDUCATION Interactive Telecommunications Program NYU, New York, NY - MPS, 2005 California College of the Arts, Oakland, CA - Film/ Video/Performance BFA, 1996

SHOWS AND AWARDS 2007 Wellington Urban Design Week. IntensCity. INSite: Solarcoustics:connect Sentimental Plastics 2005 Morgan Barnard Awarded the Production Fellowship at Eyebeam NYC. 2005 Abstract December-Still Here in the Bright and Shining group show at Safe-T Gallery. DUMBO NYC Solarcoustics:CONNECT is a solar-powered personal November-December The Queensbridge Wind electronic device. CONNECT maps light energy to Power Project in Mind in Matter audio oscillations using a solar panel and basic audio Open Source Art. Champaign, IL circuitry. CONNECT is responsive to light either October Disappear/Reappear in Cinematheque’s from the sun, or artificial light sources. By utilizing Promiscuous Cinema series, photovoltaic energy, CONNECT needs no batteries Evidence is Everywhere. San Francisco, CA to operate. It harvests and stores light energy from June-August The Queensbridge Wind Power Project the environment and converts the energy to audio at The Wavehill Public Garden oscillations. By using a large capacitor to store light and Cultural Center. Bronx, NY energy as electricity, CONNECT can be charged April-May The Queensbridge Wind Power Project at over time store the harvested energy. Chicago Contemporary and Classic: Redefining the 21st Century Art Fair at Navy Pier. Chicago, IL The user of Soalracuostics:CONNECT can create MFA Thesis Exhibition, ITP Tisch School of the Arts NYU changes in the audio oscillations by altering the po- 2004 sition of the solar panel to the sun, or by using their September The Queensbridge Wind Power Project hands to control the amount of light hitting the solar in ASCI’s Digital 2004, TOMORROW, The New York panel. As the amount of light on the panel changes, Hall of Science, New York, NY the corresponding audio oscillations change in Ask The Robot: Floatpotion. Premiere live audio and real time. The amount of light is in a proportional video performance. relationship to the voltage being generated, as more 2003 voltage is is applied to the circuit the oscillations Sonnet Subterfuge. Networked performance, NYC, reduces in frequency. This process of manipulation Amsterdam. the amount of light hitting the panel creates a gestural mode of sonic exploration. Rhythm and change CURRENT POSITION in pitch can be controlled by performing repetitive Lecturer: Victoria University of Wellington School of movements with CONNECT in and out of light and Design shadow. By connecting several devices together, Digital Media Design analog collaborative networked audio environments +Coordinating Student project for the Digital Broad- are created. CONNECT gives the user a new aware- casting Conference at Te Papa ness of their surroundings and allows them to “jam” with their environment. http://morganbarnard.com Paper

108 2005 Papers 109

Location33: Envisioning Post iPodalyptic Mobile Music

William Carter and Leslie S. Liu

Abstract

This paper describes a course of research investigating the potential for new types of music made 1 possible by location tracking and wireless technologies. Listeners walk around downtown Culver City, California and explore a new type of musical album by mixing together songs and stories based on their movement. By using mobile devices as an interface, we can create new types of musical experiences that allow listeners to take a more interactive approach to an album. Location33 is an example of one new type of musical system that is made possible through the development of mobile technology. By using tracking systems and wireless technology, the idea of what constitutes a music album can be fundamentally al- 2 3 4 tered and made more consistent with the developing acceptance of the consumer as an active player in the creative production cycle. By using movement and interactivity as a means for navigation though a collection of songs, location33 tries to reinvent the traditional musical album and make it a more interactive experience. In addition, the proj- bio ect explores the potential for a new type of recorded music that is authored not only for a consumer’s At the time of developing Location33, William CD player, but also for a physical space. By bringing Carter was part of the „Interactive Media Division“ people together in a space to listen to music, the at the School of Cinema Television, University of idea of the album also becomes more social. The Southern California. Leslie S. Liu was part of the „In- player becomes not only a part of the musical world tegrated Media Systems Center“ at the University of location33, but also the community of players of Southern California. 5 6 7 who are listening to the album in the space. Ultimately, location33 is still recorded music, and Figures therefore the creation process for the player di- verges from the real-time compositional quality of 1 – Narrator Nodes in Space research projects such as Sonic City where users 2 – The GPS PDA Explorer develop literally new music as they interact with an 3 – The Map environment. However, location33 approaches the 4 – a Mobile Code Embedded in Physical space idea of music production from the emerging sam- 5 – A Web Checklist for mp3 Artifacts pling and DJ culture, respecting the idea that as- 6 – A Song Authoring Map sembling discrete musical fragments can produce 7 – PDA Explorer Components novel and engaging music. 110 2004 Viktoria Institute Göteborg, Sweden 111

1st Mobile Music Workshop 112 2004 INVITED PRESENTATIONS 2004 INVITED PRESENTATIONS 113

Walkman Busting

Gideon D’Arcangelo tunA Abstract Arianna Bassoli “Walkman Busting” is a radio documentary series Abstract created by Gideon D’Arcangelo. The idea of the program is to puncture the private bubble of the At the Mobile Music Workshop I have presented personal listening device and engage listeners in a a series of projects exploring the theme of mobile social experience. Interviews are conducted with music sharing. The first project, tunA, is an applica- people who respond to the question: “Can I listen tion that allows users to share their music locally to what you are listening to?” through handheld devices. Users can “tune in” to other nearby tunA music players and experience, A surprisingly high percentage of people asked simultaneously, what other people are listening to. agree to be interviewed. Music is essentially social, 2) AIRDATE: February 22, 2003 - Metro North Hud- Developed on PDAs and connected via WiFi in ad- and when we listen to music, even in headphones, son River commuter train hoc mode, the application displays a list of people it seems we are predisposed to social interaction. Busts on the train include Sister with her nephew in proximity who are using tunA, gives access to For millennia, music has been by its nature a com- and niece, who were returning from Sing-Sing Prison their profiles and playlist information, and enables munal experience, a way of gluing people together where they visited Sister’s son in jail. She is listening synchronized peer-to-peer audio streaming. The in a shared moment. The advent of portable pri- to Jaheim’s “Keep your H-E-A-D U-P.” Also, Elijah, second project, an extension of this previous work, vate listening devices has interrupted the communal a college radio DJ listening to “emo” punk, who BluetunA, is an application for Bluetooth-enabled function of music in ways we are just beginning to tells tales of the mosh pit. Finally, a man who has mobile phones that allows users to connect to comprehend. Walkman Busting re-wires the social just unleashed his 70s LP collection from its slumber other BluetunA users in range and share music rec- function of music - hijacking a one-way communica- and transferred it to his MP3 player contends with all ommendations. With this application, we sought to tion and making it two-way again. the memories. use technologies that already have a mass penetra- tion (Bluetooth enables mobile phones) to develop The portable listening device enables people of di- 3) AIRDATE: October 4, 2003 - Union Square, New a lightweight version of tunA, able to make users verse cultural backgrounds to coexist in tight quar- York City aware of the musical interests of people nearby ters. Modern people go about in public, each tuned In this episode, we first encounter a young woman and to thereby foster a subtle form of proximal into their own cultural frequency, each connecting listening to the one-stringed berimbao of capoeira social interaction. The third project, undersound, to a group in another place. They share the same music. She is an avid practitioner of the Brazilian is an example of situated design, attempting to bio space but are not really being in the same space to- martial art dance. Fifty feet away, we meet a couple address three different aspects of life in the London gether. Walkman Busting uncovers the cultural jux- of rocker girls in high-school who complain about Underground: situated understanding of the space, Arianna Bassoli holds an MSc in Communication tapositions that are hidden beneath the surface - in the “weird cult-like music” of the Brazilian capoeri- bio localized interpersonal interactions, and emergent Sciences from the University of Siena, Italy, where the headphones and earbuds of the listening public. stas. They revere Korn and System of a Down. We large-scale flows which people constitute and she specialized in mass media. She then worked The CD played at the first MMW contained the fol- also here from a bonafide groupie of Frank Zappa, Gideon D’Arcangelo is an interactive media design- participate in. In order to achieve a unique way by as a research fellow at Media Lab Europe for three lowing four sample episodes: who introduced him to the found sound of John er with a special focus on the impact of new technol- which people can use music to interact with one an- years, mainly focusing on the application side of Cage and Edgar Varese while in a dressing room on ogy on musical experience. He currently produces other and the space around them, undersound uses mobile peer-to-peer and ad-hoc networks. She is 1) AIRDATE: November 23, 2002 - Union Square, the road. the Weekend America series, “Listening In” (http:// three distinct, but deeply interrelated, technologi- currently a PhD candidate at the London School of New York City listeningin.org), which explores the changing ways cal pieces; Bluetooth transfer points located in each Economics and Political Science, UK, in the Depart- These busts include a man deeply involved in his 4) AIRDATE: January 9, 2004 - Union Square, New people use recorded music in their day to day lives. Underground station are to be used for upload- ment of Information Systems and Innovation Group. “Disney Greatest Hits” compilations who is espe- York City He created the “Walkman Busting” radio documen- ing and downloading music in the undersound She is interested in interaction design, urban com- cially adept at interpreting the lyrics from “The Little These busts include some goth-rave kids still out tary series on Public Radio International’s “The Next network, while Bluetooth-enabled mobile phones puting, and the design of mobile proximity-based Mermaid.” Another self-described “dinosaur” lis- from the night before, still in party gear. Morgana Big Thing” which ran from 2002-2005. Since the first are meant to be used for storing, playback and applications, technologies that support communi- tens to the likes of Bing Crosby and the Andrew Sis- interprets the tough and bleak lyrics of R&B singer “New Interfaces for Musical Expression” workshop exchange of music and finally situated visualizations cation and data sharing among co-located people. ters. He calls Rosemary Clooney’s “Mambo Italiano” Jo, while the ever-upbeat “Seven” hips us to some in 2001, he has remained an active member of the providing a station-specific overview of activity Arianna is also a research assistant at the LSE, rock and roll. Lastly, a man into Tony, Tone, Toni harsh underground music that hasn’t “surfaced” yet. NIME community. He is Director of Creative Strate- within the undersound network are to be located at working on the EU-funded project BIONETS, which and Earth, Wind and Fire reminisces about the days Fifty feet away from this scene, a jazz musician born gy at ESI Design and teaches at the Interactive Tele- each station. looks into the future of wireless networks. when the city streets were filled with the sounds of in the same month and city as altoist Stan Getz pays communications Program in the Tisch School of the boomboxes. tribute to his household god. Arts at New York University. 114 2004 INVITED PRESENTATIONS 115

SONIC CITY

Lalya Gaye

Abstract

Nocturnal dub ambiances, pollution as echo chambers, drumming traffic noises, singing street lights… Scratching tramway bells by approaching walls, grabbing metallic railing as guitar strings, turning corners towards a chorus... With Sonic City, the urban environment became a musical interface. At the crossroad between urban exploration and experimental music making, Sonic City enabled its user to create live electronic music by simply walking through a city and interacting with their everyday urban environments. Sonic City was a wearable system that gathered bio sensor-based information about the user’s actions and her environment, and mapped it to the Lalya Gaye is a Swedish-Senegalese HCI researcher sound processing of live urban sounds collected based in Göteborg, Sweden, who works in mul- by a microphone. The resulting music was output tidisciplinary projects at the convergence of art, through headphones in real time and in context, technology, and design. Her prototyping-based as you were walking, which created a tight link research explores potentials of ubiquitous comput- between the user and the city, and emphasised ing for everyday life aesthetic activities, and focuses their interplay. in particular on locative media and mobile music Sonic City was tested by a variety of people in their technology. She also works in various art projects own everyday environments. When wearing this centred on urban public space and audio experi- system, they engaged into a musical duet with the mentations, as well as organises sound-oriented city: urban atmospheres, random encounters and workshops and small festivals. She received a B.Sc. everyday activities all participated in creating new in Physics at the University of Geneva, a M.Sc.Eng. live music. Sonic City turned paths into musical in Electroacoustics at KTH in Stockholm, worked compositions and mobility through the shifting con- several years at the Future Applications Lab, Vikto- texts of the city into a large-scale musical gesture. ria Institute, and is currently finishing a Ph.D. thesis By presenting this project, this talk showed how in Applied Information Technology at the University mobile and ubiquitous computing can enable the of Göteborg. Besides being a permanent member emergence of new forms of music that interface of the steering committee for the international with everyday settings and practices. It meant to workshop series on Mobile Music Technology, she illustrate the potential and opportunities of mobile is a member of the PLAN network for pervasive music making, in terms of creative act embedded in and locative arts and is actively involved in the the everyday. NIME research community. She has presented her The Sonic City project was realised in 2002-04, as a work at various international conferences, festivals collaboration between the Viktoria Institute and the and journals and regularly gives talks, workshops Interactive Institute. More information are available and lectures at universities, institutions and events at http://www.viktoria.se/fal/projects/soniccity/ worldwide. 116 2004 INVITED PRESENTATIONS 117

TGarden: Wearable Instruments, Embodied Interac- tion and Augmented Physicality

Chris Salter, Joel Ryan

Abstract

We report on work done for TGarden, an experimental responsive media environment where small groups of participants from the general public can control and play with real time generated sound and image through improvised movement and gesture. Development on Phase1 of the project bio took place during 2000-2001 with support from the Daniel Langlois Foundation and was shown as a Christopher Salter received his PhD in the areas of work in progress at the Ars Electronica Festival and theater and computer-generated sound at Stan- at V2/Las Palmas in Rotterdam for the European ford University. He has been visiting professor in Cultural Capital of the Year in the fall of 2001. The music, graduate studies and digital media at Brown focus of this presentation lies on issues arising in University and the Rhode Island School of Design the process of designing a physically responsive (RISD) and is currently Assistant Professor in the mobile musical system activated by the motion and Department of Design and Computation Arts at gestures of non-experts, where no predetermined, Concordia University where he also is a researcher a priori representation of gesture can be said to in the Interactive Performance and Sound axis of exist. While so-called “audience participation” in- the Hexagram Institute for Research/Creation in stallations are beginning to take these issues into the Media Arts. His research and artistic practice Joel Ryan was spawned in the first generation of account, there has still been little work to date, at investigates the role of real time sound, image and computer music hackers in San Francisco’s silicon either the conceptual or technical- implementation technologies of interaction within the context of valley, Joel Ryan is a composer who has long cham- levels on how to build a responsive system that is responsive environments and new forms of theatri- pioned the idea of performance-based electronic physically engaging and learnable within a short cal performance and he is widely acknowledged music. Drawing on his scientific background, he pi- period of time while being musically rich and coher- as one of the experts in this growing field. He was oneered the application of digital signal processing ent for the casual, non-expert participant. awarded the Fulbright and Alexander von Hum- to acoustic instruments. At STEIM in Amsterdam boldt Chancellor grants for research/work in Ger- since 1984, he has collaborated extensively with While literature in the field of gesture-activated mu- many between 1993-1995. After collaborating with artists and musicians including Evan Parker, William sical interaction is well established and, most of this Peter Sellars and William Forsythe/Ballett Frankfurt, Forsyth, George Lewis, Steina Vasulka and Jerry work has focused on systems designed for trained he co-founded the art and research organization Hunt. Formerly a Research Associate in physics at and expert performers, dancers and musicians, Sponge. Salter’s work has been shown internation- the Lawrence Berkeley Laboratories of the Univer- where issues of musical (and movement) nuance, ally at venues such as Ars Electronica (Linz), Venice sity of California, he has taught philosophy, physics, control and expression are assumed from the start. Biennale (Architecture), Villette Numerique (Paris), and mathematics. He is a researcher at STEIM in Furthermore, much of this literature assumes tradi- Transmediale (Berlin), EXIT Festival (Maison des Amsterdam, tours with the Frankfurt Ballet and is tional performer/spectator relationships, where the Arts, Creteil-Paris), Place des Arts (Montreal), Elek- Docent in Sonology at the Royal Conservatory in behavior of an interactive system is experienced tra (Montreal), Shanghai Dance Festival (Shanghai), The Hague. He has performed at the Theater Chat- passively by a viewer/listener at a distance. The Yerba Buena Center for the Arts (San Francisco), elet in Paris, the Concertgebau Amsterdam, the Pit work described from TGarden focuses less on the the Banff Center (Banff), Dance Theater Workshop Inn in Tokyo, Brooklyn Academy of Music and The specifics of the hardware and software layers in but (New York), V2 (Rotterdam), SIGGRAPH 2001 (New Kitchen in New York. Recent work includes a series rather suggests a novel approach to the total de- Orleans), Mediaterra (Athens) and the Explorato- of duets with Evan Parker,Frances Marie Uitti and sign of a responsive musical system. This system is rium (San Francisco), and has been featured in Joelle Leandre, EIDOS/TELOS, with William For- architected to create a coherent and felt resonance publications such as The New York Times, ID and syth and Roberto Zucco with the Royal Shakespear between multiple layers: a participant’s improvised Leonardo magazines. He is currently completing Company. Other works include Or Air, The Number movement, sensor input, software and the resulting Entangled: Technology and the Transformation of Readers, Hat Moon Joy, and The Effect of Noise on musical response. Performance to be published by MIT Press in 2009. the Sleep of Children. 118 2004 INVITED PRESENTATIONS 2004 INVITED PRESENTATIONS 119

Sound Pryer: truly mobile joint music listening

Mattias Östergren, Oskar Juhlin Malleable Mobile Music Abstract Atau Tanaka Following the widespread adoption of music me- Abstract dia sharing applications for the Internet a growing number of research projects have explored shar- Malleable Mobile Music takes mobility as input to ing in a mobile context. Insofar these projects have an audio re-mix engine enabling listeners to experi- mainly addressed face-to-face copresence situ- ence familiar music in new ways. This transforms ations. The Sound Pryer prototype, on the other music from a fixed entertainment medium into a hand, is designed to provide joint music listening malleable content form that enables shared experi- experiences among drivers in traffic. Through field ences. trials with a prototype application we have learned the importance of including awareness informa- A group of listeners distributed about town listen tion but not necessarily distributing complete mu- collectively to well known pop song by Bjork. Each sic media content in order to provide meaningful listener selects a part in the music to be his musi- experiences. cal avatar. One person might become the drums, In the Sound Pryer project we set out to explore another the synths, another the voice of Bjork. Lo- truly mobile activities and the impact it has on cation data drives a continuing, evolving live re-mix design of music sharing applications. In essence – as people become closer, their parts are heard Sound Pryer provides joint listening experiences in more clearly. This creates a social re-mix. traffic encounters. It works like a shared car-stereo, you can hear your own music, but also overhear Users interact with the music through subconscious what other people currently play as long as they actions. Rather asking the consumer to become stay within proximity. Sound Pryer also gives a shal- creator, the act of listening is captured by a respon- low graphical impression of other users. It is not sive system. A sensor subsystem on the mobile obvious that joint music listening while driving is terminal captures the user’s reaction while listening. beneficial or even possible technically. However, This includes intensity of gripping the device, as we show through restricted field trials with 13 users well as tapping a rhythm in time with the music. A that it is both doable and enjoyable. Particularly, al- localization algorithm simulates geographic data as though only hearing snippets of music, users were the listeners move about in urban space. These two amused when they could interpret the awareness types of data, personal bodily gesture and commu- information and determine from where the mu- nity geographic distribution, drive the evolution of sic was coming. Thus, we argue that mobile music a familiar song. sharing applications should be designed to reflect the social context and particularly illustrate aware- The music engine takes data from the listener ness of other copresent users and be less focused bio group and creates a single live audio stream. Time on distributing music media files in their whole. domain re-sequencing allows structural reorganiza- Despite a non-exhaustive field trial evaluation of Mattias Östergren is a Ph.D. in Applied Informa- tion of the music from the high level of song form the Sound Pryer prototype we have collected con- tion Technology, at the IT-University of Göteborg. to the low level of rhythm and melody. Frequency vincing evidence that joint music listening in traffic Mattias holds a M. Sc. in computer Science from domain signal processing allows time stretching at is an interesting and promising application of mo- Uppsala Univeristy, Sweden. Mattias research inter- pitch, allowing the song tempo to follow the tempo bile music sharing technology. Particularly, we argue est lies in applications of mobile wireless ad hoc of tapped rhythm. The source is a familiar hit song, that music sharing has the potential of being more networking. He has mainly been working with pro- that is no longer a fixed length and structure, but than the mere exchange of complete music media gramming the Hocman, Sound Pryer prototypes. can be molded to fit the length of a train ride, or content. Sharing snippets of content in conjunction Mattias has also been involved in various other can be shaped to respond to the movement of with awareness of co-located users is and enjoyable projects such as Placememo, Backseat gaming and friends about town. experience in its own right. Road Rager. Currently he is working with RoadTalk. 120 2004 POSTERS 121

Mobile phone music. Sound Art and ‘mobile devices’

Frauke Behrendt

Abstract

There has been quite a lot of research on the mobile phone recently but the visual paradigm has been all dominant once again: there has been no sound-based research. Which sound-based effect does this device have on every day life, on the urban soundscape, the personal auditory lifestyle? And how do artists and musicians use this new medium in their works of sound art? For a research project in 2002/2003, I found more than 100 artistic projects using the mobile phone – but only about a tenth of these projects worked with sound or music. From these, I chose four examples for a more detailed analysis: “Dialtones. A Telesymphony” by Golan Levin Levin, “Wählt die Signale!” (“Dial the Signals!”, 2003) by the artist group Ligna, “Kadoum” by Wagenaar (2000), “Text- FM” (2001) by Fuller and Harwood, and “Nanoloop i-mode” (2002) by Wittchow. I focus on the sound of each project, and also ask which social changes bio within society are reflected in the pieces. With the increasing popularity of the mobile phone, private Frauke Behrendt conducts research into the expe- conversations (calls) are more and more made in rience of urban space via mobile media; focussing a public environment, for example. This indicates on interactive art, music and sound projects that the blurring of the boundaries between public and experiment with this experience. She is currently private spheres. Levin’s “Telesymphony” plays with finalising her PhD (DAAD funded) at the Depart- this social change, as private ringtones are orches- ment of Media and Film Studies at the University trated in a public concert hall. In addition to the of Sussex, (UK), is on the steering committee of social change, I also discuss the technology of the the International Mobile Music Workshop and Ger- mobile phone itself, with its four key features: it is man delegate for the European Action on Sonic mobile, always switched on, potentially always con- Interaction Design (SID). Her book “Handymusik. nected and digital. The spreading of the mobile Klangkunst und ‘mobile devices’” (“Mobile Phone phone changes the production and distribution Music. Sound Art and Mobile Devices”) has been of music, from the desktop to the streets. Finally, published in 2004. Frauke’s research is published mobile phone music is discussed as Sound Art, by in English and German, and has been presented at looking at aspects such as intermediality, interactivi- various international conferences such as NIME and Figure ty and space; considering how mobile phone music ISEA. She is a member of the “Centre for Material is linking and superimposing real and virtual spaces Digital Culture” and of Richard Sennett’s “NYLON “Wählt die Signale!” (“Dial the Signals!”) by Ligna, in new ways. Culture and Society” Seminar. at Hamburger Kunsthalle, 2003. 122 2004 POSTERS 2004 POSTERS 123

The Intelligent Street To Listen to China for One Month Without Speaking Henrik Lörstad, Mark d’Inverno, John Eacott Davide Di Saro, Kristy Trinier Abstract Abstract The Intelligent Street is a music installation that is able to respond intelligently to the collective re- The psychoacoustics performance of the artists quests of users interacting together. The perfor- Kristy Trinier and Davide Di Saro’, consisting of mance it creates is largely influenced by the col- aforementioned performers listening for one entire lective set of text commands from users’ mobile month of the sonic environment in China, without phones. In this way, users in shared environments, speaking, as to perceive the surrounding sound subjugated for so long to uncontrollable and often without the sonic pollution of the voice; to advo- undesired ‘muzak’, can now directly influence their cate the use of conceptual immaterial processing sonic environment and collectively create the aural systems. In a temporal context, the performance allowed soundscape that they desire. We see our project as one to direct the process of listening to long-term enabling inhabitants of any given space from pas- The technological innovations in the field of wear- sound sources uninterrupted by producing a mind sive consumers to active creators, and anticipate it able sound and new acoustic digital networks, status whereby the combining or focused isolation has significant commercial, social and educational generated in the last 20 years, have developed new of a natural sound stimulus was achieved. potential. ways to understand and interpret the sonic environ- The method for systematically treating the listening The use of music, or muzak, in public places is ex- ment based on the contribution of external artificial of noise as art, was to select one type of noise, that tremely common. However, the individual within tools. As reaction, it is proposed with the perform- of environmental noise, and eliminate another type that space has no control over the performance of ance of listening without speaking, to re-examine of noise, that of the voice. By eradicating noise of that music, and it can often become intrusive and the definitions of technology: [Greek tekhnologi, the voice, [composition], resulted in the byprod- unwelcome. In our work we are interested in build- teks + -logi, -logy: systematic treatment of an art or uct of participants using alternative methods of ing interactive and responsive sound installations, craft : tekhn, skill; Logos, the word or form which communication, namely that of writing text to those where the performance results directly from the expresses a thought, also, the thought.] i, within its who wished to engage in conversation, and also interaction of the users within a given space. This original roots, to systematically process using skills gesture, eye contact, and other non-verbal types of project, known as the Intelligent Street, has been for which would form a thought, and address the communication. The primary use of written speech developed collaboratively in Sweden and the UK. concept of the thought itself. for social purposes did not interrupt the processing The mobile phone has been selected as the con- To listen without speaking permits for the absorp- of sound information. trolling medium for this project because of its tion of noise in an objective and pure form of The traditional perspective of sound processing widespread use as a tool for communicating today, natural sound dynamics mediated by psychoacous- technological devices is that of externally artificial available to practically everyone. By making use of tics. The ear is a form of technology that mediates tools, which allow for the duplication, repetition, this easily accessible device we offer almost every the flow of sound waves into nerve impulses which and alteration of specific sounds. The composition passerby the opportunity to actively engage in af- are translated into thoughts of sound: “while other of Davide Di Saro and Kristy Trinier was specific fecting the sonic environment. Intelligent Street bio people hear a person’s voice carried through only to themselves and therefore original in its seeks to explore new possibilities and unexpected vibrations in the air, the person speaking also hears inability to be extracted, copied, or replicated for applications for the mobile phone. It is a project in Henrik Lörstad is a composer. When he developed their own voice as it is conducted from the throat multiple use, and currently exists only in residual which we wish to investigate alternate ways of com- “The Intelligent Street” with Mark d’Inverno and and mouth through bone to the inner regions of documentation of the concept. The sounds heard posing music for non-linear media and attracting John Eacott, he worked at the “Sonic Studio Inter- the ear. Thus, the voice in its production in various by the participants remain fixed in the original participation in a creative process through interac- active Institute” in Piteå, Sweden. The Sonic Studio regions of the body is propelled through the body, context of time and space domain in which they tion. Another ambition we had was to demonstrate is “Interactive Institute’s” research group for sound its resonance is sensed intracranially. A fuller sense were composed. the social need for aural as well as visual stimulation and music in digital media. The research focuses on of presence is experienced as the body becomes within a well designed environment that could im- attached to thought as much as the generation of the intersections where sound and music meet nar- i Longman Dictionary of Contemporary English. pact on future architectural designs. rative, gaming and interactivity. speech is attached to thought.”ii ii Kahn, Douglas. Noise, Water, Meat. Page 7.