Algorithms as Scores: Coding Live Music
a b s t r a c t Thor Magnusson The author discusses live coding as a new path in the evolution of the musical score. Live-coding practice accentu- ates the score, and whilst it is the perfect vehicle for the performance of algorithmic music it also transforms the compositional process itself into a live event. As a continuation of 20th-century artistic develop- raditionally the score is thought of as a mes- scholar we find the roots of many ments of the musical score, T live-coding systems often sage from a composer to an instrumentalist who interprets important ideas engaged here. embrace graphical elements and the given information. Although the emphasis varies, the mu- In the centuries after d’Arezzo, language syntaxes foreign to sical score can be conceived as both a description of music in Western culture has exhibited a standard programming lan- the form of written marks and a prescription of gestures for an strong desire to capture music: to guages. The author presents live instrumentalist. It has served as what we now call a “file for- represent it in the silence of the coding as a highly technologized mat,” where the paper (or parchment) stores the music for written marks and invoke it again artistic practice, shedding light on how non-linearity, play and later realization or consumption. However, the score is more through the interpretation of signs. generativity will become promi- than encoded music. It is also a compositional tool through It is a tradition interlocked in a nent in future creative media which composers are able to externalize their thoughts onto strongly formalistic attitude toward productions. a medium that visually represents the sonic data. The score encoding and decoding musical in its various forms is a mnemonic device that enables more data, resulting in the establishment complex compositional thinking patterns than those we find of an industry of composing and in purely oral traditions. performing music and, importantly, the technology and in- A careful investigation into its history will illustrate that the frastructure that supports recording, distributing and selling score is not a simple object whose nature can be easily defined. music aimed for machine playback. Since its origins in the It has had multiple functions in various traditions at differ- 11th century, the traditional score has evolved into a highly ent time periods. This article considers live coding as a new sophisticated language for encoding musical expression with evolutionary branch of the musical score. It will investigate various idiosyncrasies or personal styles. Currently, the bound- the background of diverse scoring practices as applied in live aries of what can be defined as a musical score are fuzzy, as coding, where the score is written in the form of an algorithm, the diverse musical compositions, performances and digital either graphically or textually, yet always encoded in the func- systems require different representations of the musical data. tionality of a programming language. I present my live-coding Nevertheless, the great success of the traditional score as a language ixi lang as a case study. musical technology has established it as the fundament on which our musical education is built. A Note on the Score Unless sounds are remembered by man, they perish, for they cannot be written down. Fig. 1. Guidonian —Isidore of Seville, 7th century hand with somization syllables. An early Although musical marks have been written for millennia, the technique for music invention of the musical score as we know it today is normally instruction, as well attributed to Guido d’Arezzo (b. 991). A magnificent scholar of as a mnemotechnic device for musical music, known for the creation of the solfège, he also invented thinking. (Photo © systems for algorithmic composition, for example, where pitch Jean Gray Hargrove values would be systematically derived from syllables in the Music Library, Uni- text. Furthermore, d’Arezzo is known for the Guidonian hand versity of California, Berkeley) [1] (Fig. 1), a system of prescriptive instructions for conduct- ing music, where each part of the hand’s digits represents a musical note for the performers. In the work of this medieval
Thor Magnusson (educator), School of Arts and Media, University of Brighton, Brighton, BN2 0JY, U.K. E-mail:
©2011 ISAST LEONARDO MUSIC JOURNAL, Vol. 21, pp. 19–23, 2011 19
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LMJ_a_00056 by guest on 30 September 2021 Player Pianos and Punch Cards The musical parameters that have been important in the traditional score are primarily the rough denotations of pitch, velocity and duration. These are features that could easily be encoded onto rolls or cylinders for musical automata, and there are records of early such machines dating back to the 9th century. In the 19th century, punch cards were a com- mon engineering solution, for example, in textile looms, and they were used to automate the popular player pianos of the early 20th century [2]. As instanti- ated through these technologies, musical notation became notably static and lin- ear. An ideal had been fulfilled: perfect performances to be written for perfect instruments. Not surprisingly, computers were ini- Fig. 2. A snapshot of Claudia Molitor’s 3D Score Series. (© Claudia Molitor) tially given the same notation system as player pianos, namely punch cards, and the piano roll has become the key visual metaphor in musical software designed for composition and arrangement. It is interesting to observe in this context that this capacity of machines—such as pianolas or early computers—to per- fectly render musical scores coincided with a development in which composers increasingly began experimenting with the scores themselves, incorporating elements that emphasized performer engagement and interpretation. This is analogous to developments in painting when, with the advent of the photograph, the machine liberated painters from re- alism, resulting in impressionism and various other isms such as Cubism and Surrealism [3]. If algorithmic music can be traced to the same 11th-century origins as the modern score in d’Arezzo, we may ques- tion its apparent absence through the ages. On closer scrutiny it is clear that Fig. 3. David Griffiths, Scheme Bricks, 2008. Here the functional programming language there has never been any lack of algorith- Scheme is represented graphically as bricks that can be plugged into each other, thus build- mic music: It merely presents an interest- ing complex functional graphs out of simple ingredients. (© Dave Griffiths) ing problem of encoding. As an example, every group of people improvising or playing together succumbs to implicit Currently the main obstacles for such The mid-20th century saw many interme- rule sets that could easily be reduced to musical productions are conceptual, not dia experiments involving visual media explicit algorithmic steps. However, since technical, but new media technologies and sound. Artists such as Kandinsky algorithms can be generative, resulting are transforming this situation very rap- and Klee experimented with how a syn- in diverse outcomes, it has proved dif- idly, with composers and software devel- chronic medium such as painting could ficult to find the ideal format to write opers increasingly grasping the potential be represented as a diachronic process them as musical scores. Although various for generative music. (such as music). These experiments mechanical automata, such as Winkel’s continued with the Fluxus movement of Componium of 1821, were capable of al- Algorithms as the 1960s introducing a novel approach, gorithmic music [4], and people such as Graphic Scores namely the celebration of the algorithm Ada Lovelace in 1842 speculated about as an art form. Good examples are the computational creation of music [5], What is an algorithm if not the con- LaMonte Young with works such as “Draw it is the advent of the digital computer ceptual embodiment of instrumental a straight line and follow it” (Composition that establishes the ideal medium for rationality within real machines? 1960 No. 10) or Yoko Ono with her in- writing music in the form of algorithms. —Andrew Goffey struction pieces, e.g. “Drill a hole in the
20 Magnusson, Algorithms as Scores
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LMJ_a_00056 by guest on 30 September 2021 sky. Cut out a paper the same size as the techniques such as the casting of dice or score. Indeed, graphic scores are often hole. Burn the paper. The sky should be formalizing rule sets that the performers not written for human performers but pure blue” [6]. have to follow. Since the divide between rather as instructions for the computer. Graphic scores can represent a special composition and performance becomes The UPIC software system designed by form of algorithm. They are instructions vague, it naturally follows that composers Xenakis is a good example of an early with which the composer has found a using these techniques participate regu- such system [9]. Furthermore, graphic reason to go beyond traditional nota- larly in the performance of their work. scores can be descriptive, i.e. used as visu- tion. Many composers exploring the The above examples manifest how con- alizations of musical works, enabling lis- open work, such as Christian Wolff, temporary music has established a strong teners to engage with the piece through Karlheinz Stockhausen, John Cage, basis for live-coding practice. another modality. Rainer Wehinger’s Cornelius Cardew and Iannis Xenakis, These 20th-century experiments with 1970 visual rendition of Ligeti’s Artikula- have resolved to use the graphic score to the score transformed it into an object of tion is a fine specimen of such visualiza- extend the musical language [7]. This art in itself. In musical performance, the tion [10]. is often done with the aim of enabling function of graphic scores is not limited non-linearity, increasing performer in- to that of instructions for performers. terpretation and presenting elements Often the audience is given a chance to Composing Algorithms of surprise [8]. The graphic score opens view and engage with the score as well. Live coding is the offspring of the two new dimensions in the ontology of mu- Claudia Molitor’s 3D Score Series (Fig. 2) strong traditions described above: the sic by rejecting linearity in musical no- is a good example of this approach. The formalization and encoding of music, tation, thus paving the way for encoded score as an object of art was a path of often for machine realization, on the generative or algorithmic music. The investigation pursued by John Cage in one hand, and the open work resisting emphasis is put on improvisation or the editing the book Notations, where, in- traditional forms of encoding on the performers’ role in the realization of the terestingly, Xenakis’s Fortran code for other. Live coding is a form of musical piece, often using aleatoric or rule-based Stochastic Music is presented as a graphic performance that involves the real-time composition of music by means of writ- ing code. This is done in front of an audience, which follows the proceed- ings on a projected screen. Typically Fig. 4. David Griffiths, Al-Jazari, performers start with a clean sheet, a 2007. A live-coding environment that tabula rasa, and build their composi- engages with the world of gaming. tions from scratch. The compositions The robots are programmable, and evolve through the writing of new code, the interface used to program them is the typical gamepad used in video changing code, pausing code or copying games. (© Dave Griffiths) a large block in order to transform it into something entirely different. The code is in constant change, often modifying itself. For this reason McLean et al. [11] talk about “codeomorphology,” since the code and the music evolve together in an interweaved process observed by the audience. In addition to tracing the compo- Fig. 5. A screenshot of Alex McLean’s Text programming language (2011), where spatial sitional process, the code is also itself locations of the elements have syntactic relevance, as opposed to many graphical languages a representation of what occurs in the such as Pure Data. (© Alex McLean) sonic domain. Without it, diverse musi- cal patterns could appear to be one, or a single process could be perceived as many. By inspecting the code one can learn about the instruments, the voices and the form of the musical piece. This representational aspect of code enables performers to externalize their thoughts, thereby freeing some cognitive load and enabling other explorations. A shift has taken place in that the composer is not thinking with a pencil and staff-lined paper but writing instructions in a text document by manipulating graphical forms. Such externalization is an impor- tant feature of composition [12], and the different coding environments enable the composer to think differently about music and musical problems. Even if computer science is the foun- dation of all live-coding environments,
Magnusson, Algorithms as Scores 21
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LMJ_a_00056 by guest on 30 September 2021 and for the audience to follow the musi- cal events), updated in real time, not only Fig. 6. ixi lang exists at the border of the graphical and by the performer but also recursively and the textual, as spaces are algorithmically by the very score itself. important functions in Judging from user feedback and personal the scores and iconicity is experience, the language serves as a con- applied in the way methods look. It also introduces the strained musical tool that affords certain use of a code document that compositional practices but prevents is updated when the code others. It is within the limits of the lan- modifies itself (2010). guage that the musical exploration takes (© Thor Magnusson) place, and users report that these limita- tions encourage creativity [13]. As such it is not clear whether the language is a musical work in itself or a musical tool. This is an example of how many of the old distinctions, such as work and tool or composer and performer, blur in current media practices.
Real-Time Composition and Performance The live coder is primarily a composer, writing a score for the computer to per- form. It is therefore appropriate that the computer science term for the system in which the live coder evaluates code is normally “interpreter.” The novelty of these environments are typically inno- where the agent is called jimi, playing a live coding is not simply that composi- vative systems that can be defined as art string instrument with a score containing tion has become a real-time activity [14] themselves. The goal is to explore musical indexes to the notes in a chosen scale. but also that the compositional tool is representation, interaction and thinking Spaces between the notes represent si- brought forth to the degree that it is seen through systems that have other criteria lence, spatial organization therefore be- as a musical composition in and of itself. than those of traditional programming coming a primary syntax of the language. One could argue that this entails taking languages. The works of Dave Griffiths, When other agents are written below jimi live performance of computer music to such as Scheme Bricks (Fig. 3) and Al-Jazari on the document, this spatiality becomes its logical conclusion, or in the words (Fig. 4), are good examples of graphi- helpful in arranging events in time. We of a journalist, “what music made cal representation of algorithmic music also see (Fig. 6) that there are three Wired with computers could—and should—be” scores. As another example, the veteran different modes available for scoring: [15]. live coder Alex McLean has recently melodic, percussive and concrète. The Current content production in new performed with his new Text live-coding melodic mode takes numbers from a media, in particular for mobile media language (Fig. 5), in which, even if it is chosen scale, the percussive works with devices, indicates that the mp3 is being text-based language, spatial relations de- samples that are mapped to the letters superseded by the app. Here musical fine the meaning of the code. It is yet of the alphabet and the concrète mode is scores, compositional systems or instru- to be seen if this language can be useful intended for the looping of longer sound ments for performance are often inher- for more production-based tasks, but it files. Each of the modes can take suffixes ent with so much music that they should works well as an artistic tool. in the form of note duration, note ac- cent, silence between loops, speed and be considered musical pieces themselves. In this context, live coding can play an ixi lang transposition. These can be seen in the case of agent noel, where ^ represents important role in the investigation of ixi lang is a live-coding system designed accents, ( is note lengths, + is transposi- algorithmic composition and real-time with the criteria that it be fast (maxi- tion and ! is silence before the melody is music performance for these devices. mum 5-second wait before some sound played again. This is especially important as music is heard), be understandable by the au- ixi lang includes actions (methods) technologies become increasingly com- dience and allow the user to be relaxed that can be applied to the agents: The plex and dominant, strongly influencing whilst performing. The engagement performer can instruct jimi to reverse, our musical creativity. As an art form that should be primarily at the level of mu- shake up, shift or swap his or her score. plays with the core of the tools used for sical composition rather than computer When an action is performed on an artistic creation, live-coding systems pro- science. The language is high-level and agent from another location in the vide a level of self-reflexivity that can be simple, focusing on the arrangement document, the agent’s code is updated. inspiring and educative for designers and and manipulation of musical events in Furthermore, these actions can be auto- composers of other new media art. time. Three main metaphors are utilized: mated and scheduled repeatedly in the agent, instrument and score. The agent is future, resulting in a code document in Conclusion given an instrument and a score through constant change. The coding environ- Live coding has become a widely prac- a simple instruction such as this: ment thus serves as a score with a double ticed and prominent art form, as man- jimi -> string[8 7 5 43 1 ] function (for the computer to interpret ifested by this year’s International Com-
22 Magnusson, Algorithms as Scores
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LMJ_a_00056 by guest on 30 September 2021 puter Music Conference call for papers interconnected and computational me- ference Proceedings (London, 2010),
Magnusson, Algorithms as Scores 23
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LMJ_a_00056 by guest on 30 September 2021 Call for Papers
Environment 2.0
Guest Editor: Drew Hemment
A special call for papers seeking cross-disciplinary thinking on environmental sustainability and participatory observa- tion on the environment, climate and biodiversity. Leonardo is soliciting texts that document the works of artists, researchers and scholars involved in the exploration of environmental sustainability and participatory observation. Themes and issues may include: • Participatory mass observation of the environment, climate and biodiversity • Citizen science and issues of participation • Everyday creativity and social innovation in environmental sustainability • Ubiquitous, pervasive, locative and mobile communication technology. In urban environments in particular we are separated from the consequences of our actions as surely as the tarmac of the road cuts us off from the earth beneath. This physical boundary encourages a phenom- enological separation. Innovative approaches to participatory observation and mapping can overcome this separation, when combined with the way the internet and digital media have enabled individuals to produce and share information globally and instantly. Cracks in the pavement can be widened by creative intervention or social innovation, including artworks, social entrepreneurship, scientific intervention or innovations that harness everyday creativity. Linked activities exploring the Environment 2.0 theme have been led by FutureEverything (Futuresonic) and Lancaster University (U.K.). Authors are encouraged to submit manuscripts or proposals to
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LMJ_a_00056 by guest on 30 September 2021