ADAPTIVE MUSIC ON THE WEB Wayne Siegel, professor Ole Caprani, associate professor DIEM Department of Computer Science Royal Academy of Music University of Aarhus Skovgaardsgade 2C Aabogade 34 Aarhus, Denmark Aarhus, Denmark ABSTRACT project was to further develop musical forms specifically created for web publication. The project Adaptive Music on the Web was initiated by DIEM in 2004 and realized in collaboration with 2. DEFINITION AND DOGMA the Department of Computer Science at the University of Aarhus with the support of a generous grant from The Heritage Agency of Denmark under 2.1. Definition auspices of the Danish Ministry of Culture. The goal of the project was to develop new musical paradigms Adaptive music can be defined as music that can be that allow the listener to influence and alter a piece of controlled and influenced by a non-expert end user or music while listening within the context of a common listener, either consciously or subconsciously. web browser. Adaptive Music might be considered a subcategory of A set of artistic and technical definitions and interactive music [1]. But while interactive music guidelines for creating adaptive music for web generally refers to a musical situation involving a publication were developed. The technical trained musician interacting with a computer program, possibilities and limitations of presenting adaptive adaptive music involves a non-expert user interacting music on the web were studied. A programming with a computer program. The term adaptive music is environment was selected for the implementation of sometimes used in the context of computer games, adaptive musical works. A group of electronic music referring to music that adapts as a result of game play. composers was established and each composer was In this context, controlling the music is generally not a commissioned to create an adaptive musical work for primary concern for the player (not “part of the web publication in accordance with the established game”, so to speak) [2]. guidelines. A prototype was created for each work. Specialized programming tools were developed for For this project the following definition of implementing these prototypes in the form of adaptive music was adopted: adaptive musical works capable of running within a standard web browser. A web site was set up for “Adaptive music is music that can be influenced presenting these works. and affected by the listener. The listener can control one or more significant musical parameters in the 1. INTRODUCTION music. The end user is not only the ”audience,” but an active participant as well.” The presentation of art on the web today is largely limited to the distribution and presentation of works of 2.2. Dogma art created for other media. Typically these include digitized photographs of paintings, digital text One important motivation for initiating this project documents of literary works, digitized films and was to develop new musical forms for the web as a digital audio recordings. But for composers and media in its own right. Often sound is of secondary musicians, the artistic potential of the web as a media importance in the world of multi-media. Here the in its own right is not limited to the distribution of intention was to make sound the primary concern, digital recordings. Adaptive music is music that can be suppressing the tendency of human perception to influenced by the listener. Adaptive music is focus on vision. A list of requirements, or an artistic constructed using flexible structures, which allow the and technical dogma, was presented to the composers music to unfold and develop in various ways under from the onset. varying degrees of control on the part of the individual listener. The distribution of audio by means of radio 2.2.1. Criteria or CD distribution is a one-way street. The listener’s role is that of a passive ”receiver” of a musical work. For inclusion in the project works were expected to The web is an interactive media. There are numerous meet the following criteria: technical limitations and challenges in regard to bandwidth, compatibility, response time and • Works should have open form. dependability. Yet an increasing number of artists are • Works should be rule-based and flexible. attempting to utilize and incorporate the web’s • Significant musical parameters should be potential for interaction in their art. The goal of this controlled by input. • Software must be downloaded to the user’s “There is no higher audio-centric consciousness at computer. work to facilitate a consistent set of audio capabilities. • Software must run locally on the user’s It will always be up to manufacturers to distinguish computer without streaming data during use. themselves through their own technologies, so standardized implementation will never be the answer.” • Functionality should be independent of the user’s internet bandwidth. The following programming platforms were • Musical works should be experienced by the considered: user as if he/she is using a browser-based service. • Max/MSP • Response between action and reaction must • Java be experienced as fast and agile. • JSyn • Any sound files used must have a duration of • Flash maximum 2 seconds each and 20 seconds total. • Graphics can be used as a user interface but 3.1. MAX/MSP must not become an important artistic Max/MSP is a graphical programming environment element. designed for music, audio, and multimedia. Over the past 15 years Max/MSP has developed into a leading 2.2.2. The Ten Commandments software environment for composers creating interactive music. Marketed by Cycling74, Max/MSP For the sake of illustration, a somewhat tongue-in- is compatible with both Mac OS X and Windows XP cheek, but not entirely ironical, list of “ten [4]. commandments” was created: Max/MSP was considered to be an ideal platform for this project, since many of the composers involved 1. Thou shalt not covet long sound files were familiar with it and had previously used it to 2. Thou shalt love thy listener as a participant create interactive music. A run-time version allows 3. Thou shalt see possibilities in limitations end users to run applications created in Max/MSP 4. Thou shalt not mock modem connections without purchasing the software. Karlheinz Essl uses 5. Thou shalt not reproduce (music) this approach to distribute Max/MSP patches [5]. 6. Thou shalt not give thy programmer There were, however two crucial drawbacks: 1) ambiguous messages the end user is required to download a runtime version 7. Thou shalt respect thy programmers of the software, a process that can be somewhat limitations as thine own complicated, especially for Windows users, and 2) it is 8. Thou shalt not kill thy programmer not possible to create Max/MSP applications that run 9. Thou shalt observe all deadlines within a web browser. 10. Thou shalt not create multimedia 3.2. JAVA 3. PLATFORM Java is an open source and platform independent The next step was to adopt a standardized software programming language compatible with Windows, platform for the implementation of the adaptive Mac OS X and Unix. Java applets are applications or compositions. The choice was by no means an easy programs that are embedded in other applications [6]. one. At the time the project was initiated, there were The use of Java applets in web pages is widespread. no consistent, flexible and compatible industry The Java Sound API specification provides low-level standards for interactive web audio. support for audio operations including audio playback Our priority was to employ a platform that met the and recording, mixing, synthesis and signal processing following criteria: 1) widespread, free and simple within a flexible framework. The Java applets of J- distribution, 2) functional on most commonly used DAFX [7] and Jass [8], show that advanced digital operating systems and browsers, 3) real-time control audio applications can indeed be written in pure Java of sample playback, sound synthesis and signal based on low-level operations of the Java Sound API. processing, 4) precise control of timing. Several options were available, each with it’s own 3.3. JSYN advantages and drawbacks. In general the latter included limitation to specific browser support, JSyn is a freeware plug-in and object library for Java, incompatible architectures, lack of widespread which provides real-time unit-generator based distribution and complexity in installation and use. As synthesis for stand-alone Java applets in a web page stated in a report by the Interactive Audio Special [9]. We considered JSyn to be a powerful and flexible Interest Group [3]: platform in reference to this project. Our main concern was that JSyn was created and supported by a sympathetic but very small company [10]. The JSyn website indicated that the programming environment Adaptive Music Java Applets had not been maintained on a regular basis. We were not entirely convinced that JSyn would be supported Java library of MSP objects or even available in the distant or near future. (groove~, cycle~, reson~, etc.) Java Sound API version 1.5 or higher 3.4. FLASH Java Virtual Machine Flash is a commercial product, marketed by Adobe Systems, which includes a professional authoring Windows XP Mac OS X program and a player provided free for end users. Fig. 1. Java implementation of Max/MSP patches. Since its introduction by Macromedia in 1996, Flash has become a popular platform for adding animation and interactivity to web pages. Flash is commonly 4. COMPOSITIONS used to create animation, advertisements, web art and to integrate video into web pages [11]. Originally Six composers were involved in the project: designed for animation, Flash has a number of limitations and drawbacks in relation to audio in • Jens Hørsving general, and regarding interactive audio in particular.