Kjetil Falkenberg Hansen and Roberto Bresin The Skipproof Virtual Kungliga Tekniska Hogskolan¨ (KTH Turntable for High-Level Royal Institute of Technology) Computer Science and Communication Control of Scratching Lindstedsvagen¨ 24, 10044 Stockholm, Sweden {kjetil, roberto}@kth.se

Skipproof is an application that emulates a typical teaching material (e.g., DJ Q-bert 2003; Sloly and disc jockey (DJ) setup of turntable plus mixer and Frederikse 2004; Webber 2007). also allows high-level control of the playing style Earlier studies (Hansen 2002; Hansen and Bresin known as scratching. High-level control in this 2003) have analyzed and described DJ-performed case means performing with modeled, complex DJ scratch techniques, and some of the most popular gestures through simplified actions: For instance, techniques have been modeled and implemented letting a single movement produce a sound that nor- in Skipproof. The modeling was based on this mally would require precisely synchronized right- analysis, with additional input gained by having an and left-hand gestures. The performer controls Skip- active dialog with the scratch DJ community. These proof either with the software interface or through models can in turn be used and manipulated in hardware devices connected to the computer. The real-time performance with simple control actions; hardware devices become alternative performance this makes it possible even for non-experts to play interfaces to the standard turntable, controlling the expressively within the stylistic boundaries of DJ Skipproof application with both low-level gestures playing practices. and high-level control actions in real time. The The motivations for writing Skipproof were mapping between hardware input and Skipproof to have a platform on which to model and sim- output is freely adaptable. Skipproof is in the proto- ulate scratch techniques, as well as a tool for type phase, but it has already been used in several studying how scratch techniques are used in ex- projects in recent years. pressive performances. Also, we wanted to explore Traditionally, scratching is performed through instrument-mapping strategies for scratching and to synchronized gestures: One hand controls the record experiment with alternative performance interfaces speed on the turntable (thus also the pitch), and for DJs. Skipproof can be combined with hardware the other hand uses the audio mixer’s crossfader to and other software, and has been featured in per- turn the sound on or off. The crossfader is a slider formances with DJs using quite different control that was originally designed for fading gradually interfaces (including the Radio Baton, the Reactable, between two turntables (or other sound sources) in and various gesture sensors). order to go seamlessly from one song to the next, To the authors’ knowledge, this is the first soft- but in scratching it is instead used for turning the ware that implements modeled scratch techniques sound of a single turntable rapidly on or off (often and provides high-level performance control of such while the other turntable is playing for instance a techniques. The paper is structured as follows: The rhythm track). The playing gestures are commonly following section gives a background on scratching known as scratch techniques and constitute a and DJ interfaces, and then the Skipproof application common language among DJs (Hansen 2002; Smith is described. Three performance situations where 2006). Similarly to other instruments, traditional DJ Skipproof has been used are presented, including playing skills must be acquired through dedicated results from informal user evaluation. Finally, the practice, which is reflected in a growing market for current implementations and possible future uses of Skipproof are discussed. Skipproof is available under the terms of the Computer Music Journal, 34:2, pp. 39–50, Summer 2010 GNU Public License (GPL) and can be downloaded c 2010 Massachusetts Institute of Technology. from http://www.speech.kth.se/∼kjetil/software/.

Hansen and Bresin 39

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2010.34.2.39 by guest on 25 September 2021 Figure 1. Three (Carluccio et al. 2000) of a bold part of the movement representations of the chirp’s record and is muted by the crossfader. chirp scratch. crossfader movements: (c) Sensor output from the (a) Spectrogram of two The plot shows the record and crossfader of a chirps in succession phonograph needle recording used in performed on the fresh position relative to the Skipproof. sample. (b) Common fresh sample (vertical axis) graphical notation as a function of time; the

(Hansen, Fabiani, and Bresin submitted). The study identifies how performance parameters are used ex- pressively in scratching, and also some of the instru- ment’s functional ranges. These and previous find- ings have been used to set parameters in Skipproof. DJs move the record to manipulate the playback speed or pitch of the sound sample, but also to vary the playing position in the sample and change between samples. The crossfader is used to mute the sound temporarily, either to control tone durations, or to create short sound bursts or silent gaps down to around 10 msec. Onset properties, or tone attacks, are shaped both with the record and the crossfader to make, for instance, smooth attacks of otherwise sharp sounds. Generally for scratching, the crossfader is adjusted so that moving the fader only 1 mm will change the output from muted to maximum sound level. The fader knob can then easily be wiggled over the small area to give rapid onsets and offsets, generating a high tone density. More than 100 scratch techniques have been described (DJ Q-bert 2007), and new ones appear regularly. The scratches are often given descriptive names, such as the chirp scratch: Chirps are per- formed in a series of rapid repetitions, where the record is moved in fast forward–backward motions and the crossfader is switched off and on to mute the sound at the start, at the turn, and at the end; see Figure 1b. The onset and offset of each sound are sharp, and the pitch quite stable and high because of the fast vinyl movements (Figure 1a), producing sounds that resemble chirping birds.

Traditional DJ Equipment

Record players were not originally intended to be used as musical instruments. Therefore, they have some less advantageous attributes, such as the vulnerability of the needle and pick-up system, the vinyl as sound storage, and, not least, the size and weight. A very basic setup of two turntables and a mixer weighs around 35 kg, and for a 5-hour set the Background vinyl albums needed could be approximately 50 kg. Turntables are fragile in the sense that the needle A recent study of DJ performances describes the can easily skip and even break. To avoid needle playing strategies and acoustics of the instrument jumps and noise, the record should be undamaged,

40 Computer Music Journal

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2010.34.2.39 by guest on 25 September 2021 Figure 2. Vinyl 15 minutes of scratching. deterioration is visible in For this particular vinyl, the five spectrograms the level of wear was quite when comparing a record constant between in new condition (leftmost approximately 2 and spectrogram) to the state 12 minutes. From Hansen after 1, 2, 12, and and Bresin (2003).

the pick-up arm correctly adjusted, and the table TTM notation system (Turntablist Transcription and floor free from vibrations. Methodology; Carluccio, Imboden, and Pirtle 2000); A vinyl record has both disadvantages and and Auto-DJ (Wun, Yong, and Chan 2007) is an advantages as a storage medium: For example, each application for generating mobile-phone ringtones, side holds only around 10–20 minutes of music, but using voice recordings manipulated by scratch offers a good visual representation of the content techniques (chirps and stabs). and track positions by the center label, stickers, Skipproof combines features found both in real- and groove density. The physical damage the needle time control interfaces and in sequencer-based inflicts on the vinyl is considerable (see Figure 2), interfaces. The input devices used to control the yet DJs endorse the worn sound. software can belong to any of the three aforemen- tioned interface groups. An intuitive, easy-to-learn interface provides real-time control and manipu- Alternative DJ Controllers lation of the most common scratch techniques as performed by DJs, so that the low-level, traditional The product catalog of commercial physical con- techniques need not be practiced and mastered. The trollers for DJing with other sound formats than next section describes the Skipproof application, vinyl is growing, and academic conferences such including its interface and mapping. as New Interfaces for Musical Expression (NIME) have reported on innovative interfaces (for instance, Andersen 2003; Beamish, MacLean, and Fels 2004; The Skipproof Application Hansen and Bresin 2006; Lippit 2006; Fukuchi 2007; Pabst and Walk 2007; Villar et al. 2007; Hayafuchi The name and concept came from the “skip-proof” and Suzuki 2008). New physical interfaces broadly feature in specialized vinyl records, which was fall into three categories: (1) interfaces converting the introduced by DJ Swamp (1998). In a skip-proof rotation speeds of traditional turntables to control section of such a record, a sound with a duration signals, (2) interfaces using the turntable metaphor, corresponding to one rotation is repeated for a such as with a rotating pad or jog wheel, and (3) in- couple of minutes; thus, if the needle jumps out terfaces using other metaphors, for example by of its current groove in the record, it will probably interacting with graphical representations of sound stay in the same temporal location within the same files on touch-sensitive surfaces. Categories 1 and 2 sound sample. are currently dominant in the commercial market. Skipproof was developed in Pure Data (Pd, In addition to the DJ control interfaces, sequencer- Puckette 1996), which facilitates control interfaces based interfaces with non-real-time interaction have that use common data protocols such as Open also been developed, which enable users to compose Sound Control (OSC, Wright and Freed 1997), MIDI, scratch phrases, design new techniques, and play USB, or TCP/IP. Pd was chosen as it is good for fast with different samples. For example, Scratcher prototyping and easy collaborations. (Faulstich 2007) is a Max/MSP patch for creating A high-level scratch preset control mode and a scratch phrases taken from a list or drawn in the low-level scratch improvisation control mode can

Hansen and Bresin 41

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2010.34.2.39 by guest on 25 September 2021 be interchanged seamlessly during a performance. vinyl and the platter is regulated by placing slipmats The improvisation mode, simulating the crossfader in between. The hand position parameter emulates and turntable, is the default. The scratch preset the effect of adjusting the hand’s distance from mode is temporarily activated when the performer the record edge (where the same gesture generates plays one of the modeled techniques. shorter movements with a peripheral hand position than a central). Turntables allow one to lift and move the nee- Turntable Simulation dle between tracks while the record is spinning (so-called needle-dropping), and to physically mark Skipproof provides the main functionality of a the vinyl with (for example) adhesive stickers to turntable and a mixer: the user can play different quickly locate and cue the sound. Analogous fea- sound samples and alter the speed and amplitude tures are available in Skipproof: Needle-dropping manually. The typical controllers found on the is possible as the sample can be changed without traditional instrument—such as a start-stop button, affecting the playing position, and cueing is emu- a pitch slider for adjusting the motor speed con- lated by a function that returns the sample to its tinuously, revolutions-per-minute (RPM) selectors, start. and volume sliders—are emulated. For instance, a power-toggle and tap function can be used to simu- late how DJs generate a slow, stepwise deceleration Scratch Techniques by turning off the motor while lightly tapping the vinyl. Similarly, motor-speed toggles for switching The main feature of Skipproof is the implementation between 33 and 45 RPM can be used to simulate a of twelve example scratch techniques, or “scratch vibrato-like effect that is created on turntables by presets.” These are modeled based on the analysis of pressing down the RPM selectors alternately. recordings by professional DJs, and they have been The volume fader, which is used sparingly in described in previous work (Hansen 2002; Hansen scratching, was implemented as a linear amplitude and Bresin 2003). The techniques included in the control. Because the crossfader is traditionally used default list are baby, tear, rolltear, chop, forward, in a way resembling an on–off switch, and because silent back, scribble, uzi, chirp, flare, crab, and scratching usually involves only one turntable, twiddle; see Figure 3. the crossfader was implemented as a logarithmic, Skipproof has a default setting in which record triggered fading ramp going from sound to silence or gestures and the corresponding crossfader gestures vice versa. In Skipproof, it is thus not possible to fade are synchronized with each other. However, it is also between two sound sources or move the crossfader possible to decouple the two gestures for creating gradually. However, as on a scratch mixer, the new scratches and experimenting with them. This steepness of the fading ramp can be adjusted, and allows simulating a common performance practice the crossfader direction can be reversed by setting where new techniques are created by changing the the end position to be muted instead of sounding (on offset between the gestures (as can be observed in, for mixers, this is called a “hamster switch”). instance, DJ Q-bert 2007). In addition to changing Also, mechanical properties of the turntable were the offset between gestures, the user can create emulated and implemented as variable parameters; new presets by sketching the record and crossfader these include the pitch (speed change) amount, movements in tables. inertia, motor strength (torque), friction, and hand We will use the previously mentioned chirp position on the record. Adjusting the inertia and technique as an example of how a scratch preset torque will mainly have an effect on the time has been implemented. A professional DJ recorded needed for the turntable platter to start up, slow isolated chirps, and chirps in sequences, using down, and restore speed. The friction parameter can a turntable and mixer equipped with sensors to be adjusted to simulate how the friction between the get the rotation speed and crossfader movement.

42 Computer Music Journal

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2010.34.2.39 by guest on 25 September 2021 Figure 3. Scratch presets in movement. In the baby, Skipproof. The upper tear,andscribble diagrams in each row are scratches, the crossfader is the record movement; the constantly on. lower are the crossfader

Tables for recorded scratches

Baby Tear RollTear Chop (short) table2 table4 table6 table8

Record -->

table1 table1 table5 table7

Crossfader -->

------

Forward Silent backdrawScribble Uzi table10 table12 table14 table16

Record -->

table9 table11 table1 table15

Crossfader -->

------

Chirps Flare Crab Twiddle table18 table20 table22 table24

Record -->

table17 table19 table21 table23

Crossfader -->

The best chirps from the recordings were selected coordination between record and crossfader in order and extracted, and the sensor values stored in to sound right. Although the parameters of a preset lookup-tables in Pd; see Figure 1c. The tables are scratch can be changed, there is little possibility resampled to have quantized durations and typical for real-time control during the playback; a scratch movement ranges (based on Hansen, Fabiani, and technique is typically shorter than 200 msec. Bresin submitted). A reading of the table, triggered by Certain scratch techniques like scribbles and the user, then sets the amplitude and deviation from chirps are usually played in sequences of the same nominal playback speed, thus generating a chirp gestures, whereas others like crabs and tears can be scratch originating from the real DJ performance, played singly. Chirps are therefore triggered in series but with properties that can be changed. in Skipproof; however, this series can be interrupted. Although the speed and the extent of both As an alternative to the recorded scratch presets, the crossfader and the vinyl movements can be it is also possible to play idealized versions of the manipulated in the playback of the gesture, it was scratches, which are based on equations derived important to preserve the character of the technique: from interpolation of the gesture data from selected Most techniques, like the chirp, require precise recordings. These models include both record and

Hansen and Bresin 43

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2010.34.2.39 by guest on 25 September 2021 crossfader movements, and can be manipulated in the gray rectangle in the interface represents the the same way as the recorded presets. vinyl record, and moving the mouse in this area will affect the playback. In the default mapping, vertical movement changes the speed (and direction), while Audio horizontal movement and position control the hand position and friction parameters, as well as To simulate skip-proof sections of DJ records, all parameters of added audio effects like echo (see sound samples that are used in Skipproof have a bottom right in Figure 4). The vertical position does duration corresponding to one revolution on the not correspond directly to the sound file position, turntable (1.8 sec at 33 RPM). Technically, however, as this mapping was considered by DJs to be less sound files can have arbitrary length. As with a consistent with the idea of a rotating turntable. repeated groove on a record, the sound file is looped, Conceptually, the user moves the record with the so it is possible to scratch beyond the start and mouse (or other gesture controllers), and not the ending of the sound. needle’s position in the sample. The progress bar For a more realistic simulation of the vinyl record immediately to the left of the waveform display sound, the user can choose a level of wear, achieved indicates the current playing position. by filtering the sound and adding hiss and click Compared to vinyl, digital audio makes adding noises that are looped with the sample. Deteriorated new sounds straightforward, with all music avail- record quality is also an important characteristic of able in a searchable library. However, when using the scratch sound, where the high-frequency content computer interfaces, the immediate visual assess- in the vinyl signal will be replaced by broadband ment of playback position that is available when noise (see Hansen and Bresin 2003 and Figure 2). using physical vinyl records will often be replaced Even audio sources other than the Skipproof by a visual indication that is now decoupled from sound samples can be manipulated. For instance, the the sound source and the place of the action. DJ gestures can control sound-synthesis parameters Although new interfaces present new visualiza- or—as in the case of the Reactable, explained tion possibilities, many musicians would naturally shortly—another system with its own audio engine. be reluctant to have to relearn their instrument. As Sound quality in digital DJ tools has so far not Skipproof uses only short repeated sounds, visual- been fully satisfactory; DJs particularly object to ization was not considered to be a critical issue. sound quality at low speeds. Also, even the slightest latency between the gesture and sound is reported as annoying. However, the main challenges for Sensor and Parameter Mapping high-level control of the models are not necessarily latency and audio quality, but rather the controller Instruments intended for virtuoso playing, as most and parameter mapping. traditional acoustic instruments are, need to be predictable and endorse skillful handling; therefore they require low-level control. For example, a GUI and Visual Feedback violinist must carefully regulate the bow force to achieve a desired amplitude. With computer-based Skipproof’s graphical user interface (GUI) was instruments, however, it can be beneficial to give the designed mainly for the purposes of testing the player simultaneous control over several low-level application and studying scratches. It was not parameters, as for instance in the case of playing a intended to be used as a performance interface; synthesized violin on a keyboard, where key velocity however, it facilitates visual feedback when used in is mapped to amplitude—ranging from barely combination with hardware controllers. audible to loud—without allowing imperfect tones. Each function has been made available to the The low-level control-based instrument of con- user through the GUI (see Figure 4). For instance, cern here is, of course, the typical DJ equipment

44 Computer Music Journal

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2010.34.2.39 by guest on 25 September 2021 Figure 4. A screenshot of button. On the right side effects. The middle part track, and, between these the Skipproof GUI. Typical are scratch technique has a scratch area (large two, a vertical progress bar instrument controllers are presets, preset settings, the gray rectangle), in which indicating the current placed on the left side: mechanical parameters mouse motions change playback position relative motor power, sound such as friction, motor playback speed and to the waveform display. samples, volume controls, properties and record wear, direction, a vertical pitch controls, and start and additional audio waveform display of the

for scratching, and Table 1 shows the mappings Table 1. Control and Sound Parameters for between the most significant control parameters Scratching with a Physical Turntable and Mixer and sound parameters for this instrument. Instru- ment mapping for DJ scratching has been described Duration Pitch Onset Dynamics Timbre earlier in Hansen and Bresin (2006); and in Hansen, Record ••• ◦◦ Fabiani, and Bresin (submitted) it was found that speed for expressive performances, durations and onsets Sound ◦ • •◦ • were varied more than (for instance) pitch. Timbre sample and dynamics were found to vary mainly because of Sample •◦•◦ ◦ other parameters that were being manipulated. position Crossfader •• Control and sound parameters in Skipproof can be ◦ • ◦ mapped either like the traditional DJ equipment or Volume more unconventionally. For example, the pushing- fader Tone ◦ and-pulling gesture has become a stereotypical control trademark of DJs, but now we can explore other metaphors for changing the playback speed—for • = The most important features; • = Other common features; ◦ = instance, by using pressure or light intensity as the Features that are little used or implicitly controlled. control input. The mapping process in Skipproof was general- with fast changing of the different mapping layers ized in Mandoux and Wohlthat (2004). Here an easy between controllers and sound parameters, follow- connection and calibration of sensors was provided, ing the schemes suggested by Hunt, Wanderley,

Hansen and Bresin 45

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2010.34.2.39 by guest on 25 September 2021 Figure 5. Performance finger transmitter and situations with two antenna; (b) the Reactable different interfaces for scratch objects played by controlling Skipproof: two musicians, showing (a) Radio Baton with the virtual connection Skipproof GUI on a between objects. monitor, and close-up of

and Paradis (2003). However, it is not trivial to find sensible mappings between sensors and the high number of control parameters available. This is especially true for high-level control, where the con- ceptual coupling is vague between (1) the simplified control action that generates a simulated scratch technique and (2) the combinations of gestures that are normally needed to create the same sound using physical equipment.

Skipproof Use Cases

In the following section, we present some use cases in which Skipproof has been played in combination with different interfaces and sensors. The musician’s interaction and the resulting performances have been informally evaluated to various extents and for different purposes.

Scratching with the Radio Baton

The Radio Baton (Mathews 1991) is a gestural controller developed primarily for conducting of synthesized music sequences. A rectangular antenna registers the 3-D position of two transmitting batons held above its surface. The position data are processed to control musical events. of onsets and durations. Conceptually, breaking a In the first public performance featuring the light beam could be compared to muting a sound, Skipproof software (see Figure 5a), the Radio Baton but the gesture did not resemble that of moving the was used as the turntable controller. The player wore crossfader. It was possible to perform fast techniques a transmitter fitted onto a finger, and movement like the crab scratch (see Figure 3, bottom) easily above the antenna was mapped to the various by using the fingers to cross the light beam in a parameters of the turntable simulation. As on comb-like way. a turntable, speed was controlled by horizontal High-level control was used in combination with movements on the y-axis. Vertical (z-axis) distance the turntable simulation. The scratch techniques to the antenna was mapped to hand position on the were triggered either by foot pedals or by gestures vinyl, i.e., holding the hand close to the antenna that approached defined areas on the antenna. The corresponded to grabbing closer to the record center, finger’s speed and its height at the trigger point resulting in more-effective hand movements. In determined the speed and the extent, respectively, addition, the x-axis hand position controlled audio of the scratch. effects such as echo. In the first performance, the DJ used a standard Evaluation mixer, but in later performances, crossfading was done by obstructing light sensed by a narrow-field The interaction was evaluated by the performing DJ optical sensor, allowing a nimble and precise control during the development phase and also after a public

46 Computer Music Journal

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2010.34.2.39 by guest on 25 September 2021 performance. Mostly, the evaluation compared the metronome. The subdivisions and durations were hardware interface to the traditional instrument. It controlled by the distance between connected ob- was considered that having no mechanical contact jects and the distance from the center of the table, or feedback opened up new possibilities, especially in compliance with expected Reactable behavior. with the two added dimensions (sideways and To experiment with low-level control, which vertical hand position). Scratch preset playback was is normally not the intended operation of the reported to be a welcome feature; it was used to a Reactable, “manual” sample and crossfader objects large extent during the performance. were designed. The sample object approximated an Among the drawbacks of the Radio Baton expected Reactable behavior where object rotation interface, the performer mentioned latency, mapped directly to record movement, although here the fact that the finger-held transmitter had a it was implemented as a temporary deviation from wire, movement-detection imprecision, and the a nominal speed. When the crossfader object was decoupling of visual feedback from the instrument. moved, it would in one setting pass sound from Regarding the Skipproof functionality, the lack of the sample object and in another setting mute the beat synchronization of the scratch techniques and sound from the sample player. This simulated the the impossibility of setting a general tempo were crossfader, but deviated from the expected Reactable indicated as problematic. behavior. As expected, a video-based platform could not quite perform fast enough for low-level, real-time Skipproof on the Reactable tasks. For high-level control, however, the latency introduced by the image processing was not critical. The Reactable is a circular tabletop musical instru- ment played by controlling optically tracked objects Evaluation (Jorda` et al. 2007). The physical objects positioned on top of the semi-translucent surface are linked Skipproof on the Reactable was tested with an to virtual objects that connect to each other. This expert Reactable player and an expert scratch DJ, connection is visualized with a rear-projected GUI, who both did three practice sessions followed by which in turn makes virtual connections between questionnaire and interview evaluations (Hansen the real objects. In essence, the Reactable is a and Alonso 2008). Between sessions, the design was tangible performance interface for Pd. modified based on their comments. We found that Skipproof was integrated into the Reactable sys- their appreciations of the interface were diverging. tem as a set of new or modified Reactable objects, Both rated high how they liked the instrument, and it was tested with professional musicians. The but overall the Reactable player got more satisfied Reactable scratch objects have been described in across sessions, whereas the DJ got less satisfied. Hansen and Alonso (2008). In summary, the focus Already from the first meeting, the musicians was on high-level control with three objects repre- could perform rather complex and realistic-sounding senting the sample, the record movement, and the scratch patterns, and after a couple of hours of crossfader movement, respectively. These objects practice, they performed expressively. They judged could be manipulated, allowing the performer to that the instrument needed a lot of practice, and also control technique type, execution properties, and that their performance did improve with practice. sample position. Both were satisfied with the visual feedback. We Unlike the situation with the Radio Baton, found differences related to their field of expertise; the crossfader and record gestures were not syn- only the DJ accepted non-standard Reactable behav- chronized. This permitted new combinations and ior, whereas the Reactable expert was more forgiving techniques even with only a few basic movement of latency and movement inaccuracy. Both found patterns. The objects had durations corresponding that it got easier to perform scratching musically, to rhythmical beat subdivisions provided by a global predict the instrument behavior, and have more

Hansen and Bresin 47

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2010.34.2.39 by guest on 25 September 2021 control, but also that it got harder to perform with to generate violin-like sounds with scratching was the scratch technique objects over time. One reason quite problematic. Thus, even if the interaction was for this could be that they performed increasingly found engaging, the musical output was not. more complex phrases as they practiced, but that the instrument could not provide a proper balance between their control of the interface and their Other Examples of Using Skipproof musical ambition with the objects. In addition to these examples, parts of Skipproof have been featured in other prototypes and applications. Scratching with a Friction Sound Model An ongoing project, Ljudskrapan [the Sound scraper], is aimed at helping children with cochlear implants Friction between two surfaces and scratching and limited motor control to explore their hearing. a record have several analogies: The scratching Among other models, scratch techniques are used gesture is associated with the needle’s sound from to manipulate sound samples in a playful way, the friction against the vinyl, and the sound of producing a more complex output than what scratching is arguably less “musical” than musical traditional musical instruments could. instrument sounds, but more similar to friction Another interesting area of development is noise. These observations led to an experiment in Skipproof-based applications for mobile devices, controlling a friction sound model with Skipproof. such as for creating ringtones (Wun, Yong, and Chan Serafin, Avanzini, and Rocchesso (2003) described 2007) and for expressive interaction. Sancho (2009) a physics-based model of frictional interaction recently developed an application for superimposing implemented in Pd. This model has many control scratch techniques from Skipproof on mobile phone parameters, such as force, velocity, and resonator music players, which lets the user trigger scratch and exciter characteristics, which all need to be phrases but not perform improvised scratching. carefully adjusted in correspondence to each other to provide the intended outcome, for instance to generate bowed string sounds. Discussion In a recent project (Hansen, Alonso, and Dimitrov 2007), scratch techniques were used for controlling Through various experiments like those described the friction sound model of Serafin, Avanzini, and herein, Skipproof has shown to be a versatile tool Rocchesso (2003). The model was controlled both for working with scratch techniques. Skipproof with the typical mapping where playback speed and includes only some of the basic techniques crossfader control frequency and amplitude, respec- as presets, but because most techniques are tively, and with alternative mappings to control (for variations on distinct record and crossfader gesture instance) velocity and force parameters. A combined combinations, new models can easily be added. The interface between Skipproof and the Reactable was Reactable setting showed that even with a limited tested to control the friction sound model within a subset of techniques available, the user can produce specified functioning range to generate bowed string- a variety of different scratches. like sounds (Dimitrov, Alonso, and Serafin 2008). Experimenting with different interfaces and map- pings makes it possible to study how both DJs and other players use scratch techniques expressively in Evaluation performances. With the Radio Baton, simple gestures The interaction was informally evaluated by the would trigger “perfect” scratches, but the player did authors and colleagues, and also by the Reactable not have much control of the continuous composi- expert and DJ mentioned earlier. Controlling the tion of techniques, as they were triggered individu- frequency and amplitude of friction sounds worked ally. With the Reactable, it was easier to create richer well, but attempting to tune the physical model scratch phrases as the objects would keep generating

48 Computer Music Journal

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2010.34.2.39 by guest on 25 September 2021 patterns while connected, and the player’s effort control of the instrument. Although scratch DJs could be focused on changing and manipulating may be perfectly content with existing interfaces, the techniques. In this sense, the Reactable offered the higher-level control makes scratching sounds a more effective high-level control, which would accessible to musicians who are eager to experiment suggest that DJ user interfaces do not necessarily but less devoted to learning the instrument. Also, have to resemble the traditional instrument. high-level control of scratching can be useful in When performing with sensors, we tried to other musical applications such as sequencers, as is minimize the visual feedback, but when using the the case for most musical instruments. Reactable, the visual feedback became an integrated part of the instrument. Even though we did not anticipate that visual feedback was very necessary Acknowledgments for short looped sounds, it turned out that the DJs preferred such feedback as close to the source as The authors would like to thank the Vestax Corpora- possible. tion for equipment. Financial support was received Scratching is a topical case study for interface in the early stages from the EU projects Sounding Ob- 2 design, as DJs represent a large group of experts who jects, AGNULA and S2S (Sound to Sense, Sense to interact expressively with hardware intended for a Sound), and recently from the EU projects Braintun- different purpose. Many scratch DJs develop am- ing (FP6-2004-NEST-PATH-028570) and SID (Sonic bidextrous skills, changing which hand controls the Interaction Design, European Cost action IC0601). crossfader and which hand controls the turntable— this ability is imperative for certain DJ playing References styles related to scratching, such as beat-juggling. Two-handed input is specifically mentioned as a Andersen, T. H. 2003. “Mixxx: Towards Novel DJ future research direction in human/machine inter- Interfaces.” Proceedings of the Conference on New action (Buxton et al. 2002), and scratching can thus Interfaces for Musical Expression. Montreal: Canada be an example of an interface where both hands Faculty of Music, McGill University, pp. 30–35. independently perform highly trained actions, and Beamish, T., K. MacLean, and S. Fels. 2004. “Manipulating where the users can perform the tasks with either Music: Multimodal Interaction for DJs.” CHI ’04: the left or the right hand. Proceedings of the SIGCHI Conference on Human Because there are many new interfaces targeting Factors in Computing Systems. New York: ACM, pp. 327–334. DJs, it is worthwhile to reflect on which features Buxton, W., et al. 2002. “Human Input to Computer could improve the possibilities for expressing Systems: Theories, Techniques and Technology.” Work- musical ideas. A simulated turntable such as ing draft, Available on-line at www.billbuxton.com/ Skipproof can, for instance, be made to go beyond the inputManuscript.html. Accessed 22 August 2009. physical constraints of the instrument. For instance, Carluccio, J., E. Imboden, and R. Pirtle. 2000. “Turntablist the scratch presets can easily produce unrealistically Transcription Methodology.” Available on-line at fast or slow scratches—which the DJs participating www.ttmethod.com/. Accessed October 2009. in our evaluation sessions gladly explored. The Dimitrov, S., M. Alonso, and S. Serafin. 2008. “Developing user can also manipulate sound sources other Block-Movement, Physical-Model Based Objects for than Skipproof’s internal audio samples, such as the Reactable.” Proceedings of the Conference on New sound-synthesis parameters and external audio Interfaces for Musical Expression. Genova: Infomus, Casa Paganini, pp. 211–214. engines. However, the characteristic features of the DJ Q-bert. 2003. Do-It Yourself Vol. 1. ThudRumble DVD: instrument, such as physical restrictions and the DIY001-DVD. vinyl audio quality, should be regarded. DJ Q-bert. 2007. Scratchlopedia Breaktannica: 100 Secret In our close contact with the DJ community we Skratches. ThudRumble DVD: SECRT001-DVD. have experienced that DJs are open to testing new DJ Swamp. 1998. The Skip-Proof Scratch Tool Volume 1. interfaces, but they have high demands on low-level LP: Decadent Records DEC002.

Hansen and Bresin 49

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2010.34.2.39 by guest on 25 September 2021 Faulstich, A. 2007. Scratcher. Software available on-line Mandoux, F., and S. Wohlthat. 2004. “Design- at www.skrasoft.com/. Accessed October 2009. ing of a HCI for Skipproof.” Technical report. Fukuchi, K. 2007. “Multi-Track Scratch Player on a Multi- Stockholm: Speech, Music and Hearing, KTH. Touch Sensing Device.” Entertainment Computing – Available on-line at www.speech.kth.se/∼kjetil/files/ ICEC 2007, Lecture Notes in Computer Science. Berlin, Mandoux Wohlthat report.pdf. Heidelberg: Springer Verlag, pp. 211–218. Mathews, M. V. 1991. “The Radio Baton and Conductor Hansen, K. F. 2002. “The Basics of Scratching.” Journal of Program, or: Pitch, the Most Important and Least New Music Research 31(4):357–365. Expressive Part of Music.” Computer Music Journal Hansen, K. F., and M. Alonso. 2008. “More DJ Techniques 15(4):37–46. on the Reactable.” Proceedings of the Conference Pabst, A., and R. Walk. 2007. “Augmenting a Rugged on New Interfaces for Musical Expression. Genova: Standard DJ Turntable with a Tangible Interface for Infomus, Casa Paganini, pp. 207–210. Music Browsing and Playback Manipulation.” Third IET Hansen, K. F., M. Alonso, and S. Dimitrov. 2007. “Com- International Conference on Intelligent Environments. bining DJ Scratching, Tangible Interfaces and a Physics- Ulm: Ulm University, pp. 533–535. Based Model of Friction Sounds.” Proceedings of the Puckette, M. 1996. “Pure Data: Another Integrated Com- International Computer Music Conference,Vol.2.San puter Music Environment.” Proceedings of the Inter- Francisco, California: International Computer Music national Computer Music Conference. San Francisco, Association, pp. 45–48. California: International Computer Music Association, Hansen, K. F., and R. Bresin. 2003. “Complex Gestural Au- pp. 269–272. dio Control: The Case of Scratching.” In D. Rocchesso Sancho, O. B. 2009. “DJ Tool: A Mobile Phone Audio and F. Fontana, eds. The Sounding Object. Florence: Player Application.” Master’s thesis, KTH Computer Mondo Estremo, pp. 221–269. Science and Communication, Stockholm. Hansen, K. F., and R. Bresin. 2006. “Mapping Strategies Serafin, S., F. Avanzini, and D. Rocchesso. 2003. “Bowed in DJ Scratching.” Proceedings of the Conference on String Simulation Using an Elasto-Plastic Friction New Interfaces for Musical Expression. Paris: IRCAM, Model.” Proceedings of the Stockholm Music Acoustics Centre Pompidou, pp. 188–191. Conference. Stockholm: Speech, Music and Hearing, Hansen, K. F., M. Fabiani, and R. Bresin. Submitted. KTH, pp. 95–98. “Analysis of the Acoustics and Playing Strategies of Sloly, D., and T. Frederikse. 2004. Basic DJ Techniques. Turntable Scratching.” Acta Acustica United with London: Sanctuary Publishing. Acustica. Smith, S. 2006. The Compositional Processes of UK Hayafuchi, K., and K. Suzuki. 2008. “MusicGlove: Hip-Hop Turntable Teams. Ph D thesis, De Montfort A Wearable Musical Controller for Massive Media University. Library.” Proceedings of the Conference on New Villar, N., et al. 2007. “The Colordex DJ System: A New Interfaces for Musical Expression. Genova: Infomus, Interface for Live Music Mixing.” Proceedings of the Casa Paganini, pp. 259–262. Conference on New Interfaces for Musical Expression. Hunt, A., M. M. Wanderley, and M. Paradis. 2003. New York: ACM, pp. 264–269. “The Importance of Parameter Mapping in Electronic Webber, S. 2007. DJ Skills: The Essential Guide to Instrument Design.” Journal of New Music Research Mixing and Scratching. Oxford: Elsevier Science and 32(4):429–440. Technology/Focal Press. Jorda,` S., et al. 2007. “The ReacTable: Exploring the Wright, M., and A. Freed. 1997. “Open Sound Control: Synergy between Live Music Performance and Tabletop a New Protocol for Communicating with Sound Tangible Interfaces.” Proceedings of the International Synthesizers.” In Proceedings of the International Conference on “Tangible and Embedded Interaction” Computer Music Conference. San Francisco, California: (TEI07). Baton Rouge, Louisiana: ACM, pp. 139–146. International Computer Music Association, pp. 101– Lippit, T. M. 2006. “Turntable Music in the Digital 104. Era: Designing Alternative Tools for New Turntable Wun, S., C.-H. Yong, and T.-E. Chan. 2007. “Musical Expression.” Proceedings of the Conference on New Extrapolation of Speech with Auto-DJ.” In Multimedia Interfaces for Musical Expression. Paris: IRCAM, ’07: Proceedings of the 15th International Conference Centre Pompidou, pp. 71–74. on Multimedia. New York: ACM, pp. 795–798.

50 Computer Music Journal

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/comj.2010.34.2.39 by guest on 25 September 2021