The Jam-O-Drum Interactive Music System: A Study in Interaction Design Tina Blaine Tim Perkis RhythMix Cultural Works 1050 Santa Fe Avenue 638 Viona Avenue Albany, CA 94706 Oakland, CA 94610 +1.510.528.7241 +1.510.645.1321 [email protected] [email protected] ABSTRACT Perkis came to the project with an extensive background of This paper will describe the multi-user interactive music work in computer-mediated collaborative music making. His system known as the Jam-O-Drum developed at Interval group The Hub connected six electronic musicians in a Research Corporation.1 By combining velocity sensitive input computer network, and performed experiments in new forms devices and computer graphics imagery into an integrated of collective musical tabletop surface, up to six simultaneous players are able to improvisation throughout the '80s and '90s [3]. Both were participate in a collaborative approach to musical interested in further experimentation with spontaneous improvisation. We demonstrate that this interactive music improvisation and interactive audiovisual collaboration. By system embraces both the novice and musically trained changing the context of making music to a casual group participants by taking advantage of their intuitive abilities and experience, we particularly hoped to provide novice players social interaction skills. In this paper and accompanying the experience of musical interaction in an ensemble setting. video, we present conclusions from user testing of this device Several basic goals of the project had been established before along with examples of interaction design methods and the particular configuration of a drum table was conceived. prototypes of interpretive musical and game-like Our intention was: development schemes. Our research was conducted in two phases with two different development teams and will • To explore music and motion in graphics accordingly be addressed herein as Phase One and Phase • To make collective musical experiences available to Two development. novices • Keywords: collaborative, musical improvisation, computer To experiment with different musical and visual styles graphics, interactive music system, input device, interaction • To bring a group of people together for a collaborative design, multi-user, novice, social interaction, velocity approach to music-making sensitive. • To inspire physical movement and non-self conscious behavior in the players INTRODUCTION This project began in July 1998 to explore new ways for From these objectives, the community drum circle emerged people to make music collaboratively. Integrating cross- as a metaphor to guide the form and content of our work. The cultural music was inspired by the authors' experiences of drum circle is a format in which every person has an equal communal music making, both in the context of non-western position as observer and participant; and we saw this as an cultures and contemporary computer music [1]. Prior to this excellent context for creating “in the moment music” [4]. undertaking, Blaine built electronic midi percussion Another important element of our work included the instruments and interactive “show-toys” that integrated music integration of universal design concepts for accessibility and and computer graphics to create unusual opportunities for usability. By creating an interface with a very simple control audience participation with the ensemble D’CuCKOO [2]. gesture — hitting a flat surface — we sought to create an experience which would engage people with a wide range of 1 capabilities. Although most of the sound design was built IRC #1999-039 using percussive soundsby integrating melodic long tones in more ambient compositions we hoped to provide a role for Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies those undisposed or unable to engage in vigorous percussive are not made or distributed for profit or commercial advantage and playing. that copies bear this notice and the full citation on the first page. To We began thinking of creating an immersive musical copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. environment with a projection display on a wall as inspired DIS ’00, Brooklyn, New York. by Myron Kruger’s augmented reality systems designed for Copyright 2000 ACM 1-58113-219-0/00/0008…$5.00. 1 full body interaction, [5], [6]. Eventually we envisioned a messages gave a 7bit (0-127) number telling how hard the shared physical object around which players could gather, in pad was hit. Each pad contained piezoelectric sensors, which essence, turning the immersive space “inside-out.” The Jam- required an audible strike to output a signal but were not O-Drum prototype design incorporated six drum pads in a pressure sensitive. seven foot diameter circular table used as an integrated video The graphic subsystem was straightforward: a second projection surface. The design supports face-to-face audio Macintosh G3 computer, equipped with a simple MIDI and visual collaboration by enabling users to play on drum interface to communicate with the musical subsystem, and an pads embedded in the surface, simultaneously creating LCD video projector receiving the Mac video output (and rhythmical musical events and visual effects. looping it back to a monitor for programmer use). The Over a six month period in 1998, we conducted dozens of projector was mounted near a 15' ceiling, and a large, 4.5 ft. participatory demonstrations with people in the Interval square mirror, tilted at a 45˚angle, reflected the projected community and outside visitors using this graphical user image down onto the table [Fig. 1]. interface. Although time and budget constraints kept us from In the final phases of the project, several additional game formal user testing, on the basis of user feedback from these prototypes were developed on a Windows NT machine, experiences we were able to explore and refine several which temporarily replaced the G3 graphics machine in the different interaction schemes. These iterative approaches to system. interaction design will be referred to herein as Phase One and Phase Two interaction design experiments The two phases The Jam-O-Drum table was built on a seven foot diameter correspond to major changes in the software prototyping welded steel framework with 10” drum pads mounted directly environment used: the Phase One system relied on Opcode onto the frame. The composition board surface of the tabletop MAX for MIDI programming and Macromedia Director for was designed to leave extra space around the pads to avoid the visuals; our streamlined Phase Two platform used MIDI acoustic coupling with the table. An additional six holes were sequencers and custom C programs for MIDI and visual cut in the table surface to allow the mounting of separate processing. speakers in front of each player’s station. The entire table- top is wrapped with a custom fitted cover which feels like TIMELINE: a drumhead. Polyfoam padding was used to soften the 7/98 Concept and Physical Design drum pad surface to allow comfortable hand drumming. 7/98-9/98 Hardware Design and Development 7/98-9/98 Phase One Interaction Design Audio Software Design Rhythmic Quantization Rhythmic Emphasis Weighting Volume Envelope Velocity to Filter and Pitch Bend Mappings Sequence Cycle Sound Distribution Experiments Graphics Design Dancers Techno Rhythm Monitor 10/98-1/99 Phase Two Interaction Design Bliss Paint Call and Response Bouncing Balls (“HexaPong”) IMPLEMENTATION Fig 1: System Design Overview Hardware Environment Software Environment The basic system architecture consisted of two separate In our early prototypes, Opcode’s MAX was used to process Power Macintosh G3/266 computers; one for music and the MIDI information from pads, control playback of backing other for graphics. The machines wereconnected to each tracks, and forward control information to the graphical other using the MIDI serial data protocol, as well as to an subsystem [Figure 2]. MAX is a visual programming system Akai S2000 sampler to play the musical sounds, and an designed to implement real-time control networks, primarily Alesis D4 Drum Trigger Module, which received signals used forMIDI-based musical composition. Akai's MESA II from the drum pads embedded in the table. These velocity 2 software was used to program the Akai S2000 sampler and Sound Distribution Experiments sampler library. Emagic Logic, a combined digital audio and We explored several different methods of routing the custom MIDI sequencer program, was used in the phase two musical sounds and special effects via the sampler’s eight prototypes to play back MIDI sequences which served to channel output board. These sound distribution experiments "script" both the audio and graphic events. were conducted over several different graphical prototypes For the graphics software, Macromedia Director 6.0, with that will be discussed individually in the Interaction Design HyperMIDI XCMD extension, was used for all prototypes in sections. phase one. Metrowerks CodeWarrior was selected as the C The following sound distribution methods were tested: development environment and the Opcode OMS developers • Global Mix: Sources were mixed down to a stereo pair kit was integrated for phase two prototypes. The freeware of signals played back via speakers in the room. library Spriteworld was utilized in the "bouncing ball" • Distributed Sound Sources: A matrix of audio prototypes of phase two, which were written in C. distribution amplifiers and line mixers were used to Other non-dedicated software—Bias Peak, Steinberg’s provide a separate headphone mix for each station [Fig. ReCycle and ReBirth—were used to edit and shape the 1]. Each headphone mix placed the player’s own signal custom library of individual hit and looping samples utilized in the center foreground, with a mix of the other player’s in the project demos. Microsoft Visual C++ was used for the signals spatialized in the headphone image to match the card game prototype which ran on a Windows NT machine.