Proceedings of the International Computer Music Conference 2011, University of Huddersfield, UK, 31 July - 5 August 2011 TRACKING VIRTUOSITY AND FLOW IN COMPUTER MUSIC Chris Nash Alan Blackwell Rainbow Research Group, Computer Laboratory, University of Cambridge, William Gates Building, 15 JJ Thomson Avenue, CB3 0FD {christopher.nash, alan.blackwell}@cl.cam.ac.uk ABSTRACT 2. FLOW IN MUSIC NOTATION USE This paper explores the concepts of virtuosity and flow We are particularly interested in what motivates end- in computer music, by looking at the technological, users to interact with systems, to keep using them for an interactive and social factors in soundtracking; a text- extended period of time, and to develop mastery with based, computer keyboard-manipulated notation for their tools. To this end, we apply Csikszentmihalyi’s real-time computer-aided music composition. The role theory of flow [4], which describes a trance-like mental of virtuosity in both the personal user experience and the state of total focus and concentration on an activity, wider demoscene hacker-artist subculture are discussed. enabled by a balance of challenge and ability. Comparisons are made to mainstream music interaction Too little challenge and the individual becomes bored, paradigms, such as performance capture and sequencing, too much and they become anxious. A novice can where support for virtuosity is present in MIDI devices achieve flow in simple tasks, but gradually develops the and episodes of live performance recording, but ability to tackle greater challenges, towards complete otherwise impeded by mouse-driven interfaces designed mastery of the domain. At this ultimate point of around visual metaphors for novice use, rather than the development, macro-flow, the individual feels in development of practised skill. Discussions and complete control of the tools, the task and themselves, observations are supported by initial findings from a enabling action-awareness merging, where the large-scale, 2-year user study of over 1,000 tracker and individual becomes totally absorbed in the activity and sequencer users. oblivious to the outside world. Table 1 summarises the nine characteristics commonly found in descriptions of 1. INTRODUCTION flow experiences. We define “virtuosity” as the enabling factor of fluency 1. Clear goals and intentions. in a domain (i.e. music composition) through mastery of 2. Direct and immediate feedback. a device or system [1]. This paper explores the concept 3. Balance of challenge and ability. in the context of the users, interfaces, and technology of 4. Focus and concentration. soundtracking, a computer-aided composition technique 5. Intrinsically rewarding activity. that stands apart from mainstream digital music – in 6. Distorted sense of time. terms of user and listener demographic, technology and 7. Sense of personal control / autonomy. interaction style. 8. Loss of ego / self-consciousness. This paper begins with a brief description of flow and 9. Action-awareness merging. tracking technology, before discussing the role of virtuosity in both the tracker user interface (highlighting Table 1. Characteristics of flow experience [4]. appropriate contrasts with popular sequencing programs) and the demoscene, a computer art subculture that Flow has been observed in many activities, including specializes in realtime, non-interactive audio-visual music [5], video games and programming [1]. Tracking, presentations designed to demonstrate coding and as an intersection of these three domains, thus seems a artistic skill (detailed in Section 3.2). To illustrate and likely candidate for support of such experiences, which explore both phenomena, we present initial findings represent an intrinsically-motivated path towards the from a large-scale, 2-year user study of over 1,000 development of virtuosity [4]. tracker and sequencer users. Using real-time logs of interaction and surveys, the experiment has allowed us 3. SOUNDTRACKING to empirically study the nature and development of virtuosity in computer music. Analysis also reveals This section begins with a technical description of the indications of specific components of flow, within the notation and interface, followed by an historical account interaction. Video footage of one expert tracker user was of technology and social context, notably within the user also recorded, to augment the interaction log and allow a community and demoscene subculture, where the closer examination of a composer “in flow”. virtuosic coding talents of practitioners are recognised and valued as much as the artistic content. 575 Proceedings of the International Computer Music Conference 2011, University of Huddersfield, UK, 31 July - 5 August 2011 3.1. Technical overview Primarily using text to represent notational elements, a soundtracker (or simply, “tracker”) allows the user to create patterns of note data (often 4 bars) comprising a short passage of music. The music is realised in real- time, traditionally through an integrated sample engine and user-supplied set of samples. These patterns – resembling a spreadsheet in appearance, and analogous to a step-sequencer or player piano in function – are then arranged in a specific order to produce a song. The Figure 2. Pitch entry in tracker software. saved file (or module) stores the song together with all the notes, samples and instrument settings. Tracker programs are primarily controlled using the computer’s QWERTY keyboard, used for nearly all 01 A-3 01 16 D2F A-4 01 16 D2F C#5 01 16 D2F 02 ··· ·· ·· D00 ··· ·· ·· D00 ··· ·· ·· D00 tasks, including the entry and editing of musical data, as 03 ··· ·· ·· D00 ··· ·· ·· D00 ··· ·· ·· D00 well as management and navigation within the program. 04 ··· ·· ·· D00 ··· ·· ·· D00 ··· ·· ·· D00 05 E-4 01 24 D00 ··· ·· ·· D00 ··· ·· ·· D00 The text representation allows many parameters to be 06 ··· ·· ·· D00 ··· ·· ·· D00 ··· ·· ·· D00 simply typed, but pitches are entered using a virtual 07 F#4 01 28 D00 ··· ·· ·· D00 ··· ·· ·· D00 08 ··· ·· ·· D00 ··· ·· ·· D00 ··· ·· ·· D00 piano (Figure 2). Keyboard shortcuts and macros 09 E-4 01 32 D00 ··· ·· ·· D00 ··· ·· ·· D00 accelerate almost every aspect of the program, replacing 10 ··· ·· ·· D00 ··· ·· ·· D00 ··· ·· ·· D00 11 ··· ·· ·· D00 === ·· ·· D00 C#5 01 36 D00 the mouse’s typical role in block selection and cursor 12 ··· ·· ·· D00 ··· ·· ·· D00 ··· ·· ·· D00 navigation, with the provision of rich-cursor movement 13 ··· ·· ·· D00 ··· ·· ·· D00 E-5 01 40 D00 14 ··· ·· ·· D00 ··· ·· ·· D00 ··· ·· ·· D00 control. This allows the user to stay at the keyboard 15 ··· ·· ·· D00 ··· ·· ·· D00 F#5 01 44 D00 16 ··· ·· ·· D00 ··· ·· ·· D00 ··· ·· ·· D00 without incurring the time penalty or interruption of homing between input devices. (a) Example of tracker notation. (b) Equivalent score notation. Figure 1. A comparison of tracker and score notations. In the grid of the pattern, columns represent separate Figure 3. The Pattern Editor, in Impulse Tracker 2. tracks (or channels) and the rows represent fixed time slices, like a step sequencer. Each cell has a fixed The majority of interaction takes place in the pattern number of spaces to specify pitch, instrument, volume editor (Figure 3), which also provides a central visual (or panning) and a variety of musical ornaments (or focus for the user. Other screens offer control of song, effects), for example: C#5 01 64 D01 starts playing a note sample and instrument settings using more traditional [C#] in octave [5]; instrument [01]; maximum volume methods, such as buttons, sliders and text boxes, but [64]; with a slow [01] diminuendo [D]1. Figure 1 shows typically play a peripheral role after the initial set-up is an excerpt from a tracker pattern representing a single complete. However, even these other screens maintain bar of music, together with the equivalent phrase written fixed layouts and keyboard focus, permitting the in conventional score notation. learning of screen configuration (as spatial schemata [13]), for fast visual inspection, navigation and editing. The final three effect digits in the cell, can address a variety other musical (and sound) effects, including Playback is also controlled using the keyboard, other slides (e.g. portamenti, glissandi, filter, panning), providing dedicated keys for playing the current pattern oscillations (e.g. vibratos, tremolos), global variations in or the whole song, from either the beginning, or from the time or volume, or even branching in playback. A editing cursor. This last feature enables a tight coupling number of effects also allow changes that are not easily between editing and auditioning music, which we have expressible in score notation or MIDI – such as low- shown, in other work, to enable a fast feedback cycle, level control of sample playback or synthesis. supporting the embodied learning of music composition through the notation [8], where the notation is quickly 1 resolved into the musical sounds it represents, aiding Unlike MIDI, the notation represents relative changes in musical parameters over time, in a mode similar to score notation. experimentation and learning in the program. [1] 576 Proceedings of the International Computer Music Conference 2011, University of Huddersfield, UK, 31 July - 5 August 2011 Summarising the tracker user experience, Computer soundscapes, and highly complex pieces in almost any Music magazine highlighted the virtuosity supported by musical style – not just dance, but electronica, rock, use of the keyboard, observing,