Integrated Algorithmic Composition

Proceedings of the 2008 Conference on New Interfaces for Musical Expression (NIME08), Genova, Italy

Fluid systems for including notation in music composition cycle

Andrea Valle CIRMA, Universita` di Torino via Sant’Ottavio, 20 - 10124 Torino, Italy [email protected]

ABSTRACT Audio FFT GUI In This paper describes a new algorithmic approach to instrumental musical composition that will allow composers to Audio explore in a flexible way algorithmic solutions for different Input CAC FFT In compositional tasks. Even though the use of computational tools is a well established practice in contemporary instru- GUI mental composition, the notation of such compositions is Analysis UI IAC still substantially a labour intensive process for the composer. Integrated Algorithmic Composition (IAC) uses a Compositional Output Stochastic procedures Notation fluid system architecture where algorithmic generation of generator notation is an integral part of the composition process.

Stochastic Notation generator Keywords Algorithmic composition, automatic notation Figure 1: Rackbox v.s fluid architecture. 1. INTRODUCTION Algorithmic composition can be defined as a composition completely algorithmic workflow –from the first idea to the practice that employs formalized procedures (algorithms) final score–, can be defined as Integrated Algorithmic Com- for the generation of the representation of a musical piece. position (IAC). An IAC approach pursues the integration Apart from the many ante litteram examples, algorithmic of notation generation with musical data manipulation, so musical composition has been proposed and practiced widely that any manual process could be removed from the com- starting from the ’50s. In particular, from the late ’50s a position pipeline. computational perspective started spreading across the two The paper is organized as follows: first IAC/CAC approaches Western continents (see [1] for a detailed discussion). An in- are discussed in relation to different software architectures; teresting shift in perspective has occurred roughly from the then, the need for a specific architecture is motivated in ’60s up to present day. The first approaches to algorithmic relation to automatic generation of music notation; finally, composition were driven by instrumental scoring. But, even two cases are presented. if computer tools are largely widespread in contemporary instrumental scoring through computer-assisted composition systems (hence on CAC, e.g. PatchWork, Open Music, [2], 2. RACKBOX VS. GLUE ACHITECTURES PWGL [9], but also Common Music, [11]), the idea of a CAC systems are intended to aid the composer in the purely algorithmic approach, in which a strict formalization computational manipulation of musical data: these data, rules the whole composition process, is no more pursued in at the end, can be exploited in traditional score writing. its integrity and has migrated from the instrumental domain Typically based on Lisp, CAC systems offer a large body of to the electroacoustic one. In fact, considering the final out- functionalities: pitch/rhythm operations remain the core of put of the composition process, while in the electroacoustic the system, with the inclusion of input modules for audio domain the synthesis of the audio signal is a trivial task analysis and sound synthesis modules in output. All this per se, in the instrumental domain the generation of mu- functionalities are typically accessible through a GUI envi- sical notation still remains a very difficult task ([4], [10]). ronment. While GUI is the main interface to the system, This notational issue has prevented the diffusion of real al- offering an easier access for the less programming-oriented gorithmic practice for instrumental composition. Such an composer, a high degree of flexibility is offered by enabling approach, in which the composition process is turned into a the user to extend the program via the Lisp language. Still, CAC architectures are based on the assumption that new functionalities must in some way be adapted to the host- ing environment. A CAC application architecture can be Permission to make digital or hard copies of all or part of this work for thought as a rackbox containing a certain number of mod- personal or classroom use is granted without fee provided that copies are ules (Figure 1, left): the box can leave large room for other not made or distributed for profit or commercial advantage and that copies modules to be inserted in it. Still the container is solid bear this notice and the full citation on the first page. To copy otherwise, to and consequently rigid, its capacity is finite, and the mod- republish, to post on servers or to redistribute to lists, requires prior specific ules, in order to be inserted, must meet the requirements of permission and/or a fee. NIME08, Genova, Italy the box geometry. By reversing the perspective, a different Copyright 2008 Copyright remains with the author(s). approach to computer-based composition environments can

253 Proceedings of the 2008 Conference on New Interfaces for Musical Expression (NIME08), Genova, Italy be conceived. It can be noted that each of the elements Start of the diagram in Figure 1 (left) may be replaced by an 1: input data Compositional undetermined plethora of components. As a consequence, parameters 2, 9: computational processes 1 instead of starting from a solid framework where to insert 3: data structure modules, it is possible to start from an indefinite variety of 4, 5, 7: manual operations 6, 8: documents 2 Compositional available modules to be plunged –when necessary– into an algorithms open environment (Figure 1, right). Such an environment is fluid because it is intended as a glue capable of attaching 4 5 3 Data structure different modules together. This reversed perspective im- Compositional Notational control control plies a different approach by the user. By definition, a fluid Ok? system such as the discussed one cannot be implemented in a closed application. On the contrary, a programming A B language is the most flexible way to glue together differ- 6 Data list ent software modules. Thus, in a fluid system, the communication among the modules can be operated by a glu- Transcription Transcription algorithms ing language: the language writes scripts addressing each 7 9 module’s specific interface and executes them by calling the related program from OS. The language is responsible for 8 Musical score symbolic manipulation representing the selected music fea- tures and for the communication with the modules in in- Ok? put/output. Not every programming language is suitable for such a gluing task. Requirements can be summarized Stop as follows: high-level, dynamic typing, richness in dynamic data types, interactivity, string processing, interfacing to many system calls and libraries. The first two requirements Figure 2: Algorithmic composition cycle. are needed to let the composer concentrate on composition algorithms and not to deal e.g. with compilation or complex project structure issues, and to allow for continuous feed- by looking at the resulting notation. The control processes back while experimenting in composition. The last two are are high level musical tasks, and potentially they can be necessary in order to ensure the gluing mechanism. The re- performed very quickly, even in terms of the few seconds quirements for softwares modules are in fact programmabil- necessary to have a glance at the resulting notes. On the ity and command line interfacing. Indeed, there is a strong other hand, the transcription step is a low level musical coupling between CAC systems and rackbox architectures: task, which is always very time-consuming –its timescale be- CAC applications are oriented toward composers interested ing typically measurable in hours. This slowness depends in a computational approach to symbol manipulations but both on the complexity of notation in se and on the dif- are not programmers and are not interested in automatic ficulty to clearly anticipate from a data list the peculiar- composition. On the other side, an IAC approach, i.e. a ity of the resulting notation. The crucial move towards fully-integrated computational approach to music composi- an IAC approach consequently requires to automatize the tion including notation, can evidently benefit from the flex- transcription step (Figure 2, from A to B). In this way, ibility of a fluid system. More specifically, the case of auto- the composer can focus on the higher level aspects of con- matic notation generation demonstrates that a completely trol: this would speed up the composition cycle (1-8) which algorithmic approach to composition can be achieved only can be executed until a satisfying result is obtained, thus through a fluid architecture. leading to an interactive, trial-and-error methodology. The task of automatically generating musical notation is a priv- ileged example of the power and, at the same time, of the 3. AUTOMATIC MUSICAL NOTATION necessity of a glue-connected, fluid system. CAC systems Algorithmic composition requires to define a mapping proposes generic transcription algorithms, intended to pro- from data structures (the output of composition algorithms) vide a draft of a possible notational output. Consequently, to a subset of notation symbols (the final output of instru- even if the resulting scores can be quite sophisticated, they mental composition practice). In the “classic” approach to are still drafts to be reworked manually by the composer. algorithmic composition (Figure 2), this sensitive step is In an IAC approach, this handmade notational work is re- performed in first person by the composer, whose work de- versed into the definition of an algorithmic procedure for fines the two extremes of the workflow: s/he provides com- the control of a notation module. It must be stated clearly position parameter in input and defines algorithms; from a that music notation cannot be derived exclusively from mu- certain data structure the computer generates a data list sical data structures because notation information involves in textual form; the composer controls the adequateness of graphic data which are autonomous from musical data, but the output, eventually modifying his/her composition strat- are important at the same degree for the final composition egy; then, s/he proceeds to transcribe the data list in musi- output: in short, music notation is not only music represen- cal notation. Finally, s/he evaluates the notational result, tation ([7]), and the composer must take into account both. eventually modifying some steps of the process. To sum- More, in real practice composition and notation are related marize, with respect to the computer output, the composer by a feedback loop, so that any decision on one side has al- works both as a compositional/notational controller and as ways to be verified on the other one. A fluid architecture is a transcriber. indeed needed to automate such a task, so that transcrip- The composition cycle thus requires two controls which are tion modules can be opportunely defined and fine-tuned. iterated in two different moments. Before the transcription, More, specific modules can be “plunged into the fluid” to the composer can evaluate the generated data and foresee meet the requirements of different notations: as an exam- specific issues that would raise up during transcription. Af- ple, in the case of graphic notation, a drawing module may ter the transcription, the same control can be carried out fit better that a musical notation one, even if the latter is

254

ppp 3

3 ppp

3 mf

p 3

f p

3 3

p mf ppp 3 3 3 ppp 3 pp ppp mf

Music engraving by LilyPond2.5.29— www.lilypond.org

Music engraving by LilyPond 2.5.29 — www.lilypond.org

pp Music engraving by LilyPond 2.5.29 — www.lilypond.org mf 3 3

3 mf f Music engraving by LilyPond 2.5.29 — www.lilypond.org

Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engravingMusic by engraving LilyPond2.5.29 by—www.lilypond.org LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29—www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving2.5.29 by—www.lilypond.org LilyPond p Music engraving by LilyPond2.5.29— www.lilypond.org 3 3

Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29—www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org p p Music engraving by LilyPond2.5.29—www.lilypond.org 3 Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org ppp 3 pp Music engraving by LilyPond 2.5.29 — www.lilypond.org

Music engraving by LilyPond2.5.29— www.lilypond.org 3 Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving2.5.29 by—www.lilypond.org LilyPond Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving2.5.29 by—www.lilypond.org LilyPond

Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29—www.lilypond.org f Music engraving by LilyPond 2.5.29 — www.lilypond.org

Music engraving by LilyPond 2.5.29 — www.lilypond.org

Music engraving2.5.29 by—www.lilypond.org LilyPond Music engraving by LilyPond 2.5.29 — www.lilypond.org 3 Music engraving by LilyPond 2.5.29 — www.lilypond.org

3 3 3 Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29—www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving2.5.29 by—www.lilypond.org LilyPond p

ppp Music engraving byMusic LilyPond engraving2.5.29 by LilyPond— www.lilypond.org2.5.29 — www.lilypond.org 3 Music engravingpp by LilyPond2.5.29— www.lilypond.org pp Music engravingp by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org

Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org

Music engraving by LilyPond 2.5.29 — www.lilypond.org

3 3 3 Music engraving by LilyPond2.5.29— www.lilypond.org 3 3 3 3

Music engraving by LilyPond 2.5.29 — www.lilypond.org ppp pp Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org p p Music engraving2.5.29 by—www.lilypond.org LilyPond ppp mf ppp

Music engraving by LilyPond 2.5.29 — www.lilypond.org

Music engraving by LilyPond2.5.29— www.lilypond.org

Music engraving by LilyPond2.5.29— www.lilypond.org pp Music engraving by LilyPond 2.5.29 — www.lilypond.org

Music engraving by LilyPond 2.5.29 — www.lilypond.org ppp pp p Music engraving by LilyPond2.5.29—www.lilypond.org Music engraving by LilyPond2.5.29—www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving2.5.29 by—www.lilypond.org LilyPond

Music engraving by LilyPond2.5.29— www.lilypond.orgMusic engraving by LilyPond2.5.29— www.lilypond.org

Music engraving by LilyPond 2.5.29 — www.lilypond.org

Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org

ppp p Music engraving by LilyPond 2.5.29 — www.lilypond.org

Music engraving by LilyPond2.5.29—www.lilypond.org Music engraving by LilyPond2.5.29—www.lilypond.org

Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org ppp Music engraving byMusic LilyPond engraving2.5.29 by —LilyPondwww.lilypond.org2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org ppp ppp Music engraving by LilyPond2.5.29— www.lilypond.orgppp Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org p p ppp Music engraving2.5.29 by—www.lilypond.org LilyPond

Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org

Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org

Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org

Music engraving by LilyPond2.5.29— www.lilypond.orgppp Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org ppp Music engraving by LilyPond2.5.29— www.lilypond.org pp Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engravingMusic engravingby LilyPond by 2.5.29LilyPond2.5.29— www.lilypond.org— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org ProceedingsMusic engraving by LilyPond of2.5.29 the— www.lilypond.org 2008 Conference on New Interfaces for Musical Expression (NIME08), Genova, Italy Music engraving by LilyPond2.5.29— www.lilypond.orgMusic engraving by LilyPond2.5.29— www.lilypond.org

Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29ppp— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org

Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org

Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org

Music engraving by LilyPond 2.5.29 — www.lilypond.org Python Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.orgMusic engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org ppp ( Scalptor ) ppp Music engraving by LilyPond 2.5.29 — www.lilypond.org ppp pp pp pp pp .070Music engravingppp by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29—www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org

.063 Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org

Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org .070 Music engraving by LilyPond2.5.29—www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org vertices .061 Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org

ppp Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org .051 .058 Music engraving by LilyPond2.5.29—www.lilypond.org Music engraving by LilyPond2.5.29—www.lilypond.org

Music engraving by LilyPond2.5.29— www.lilypond.org ppp ppp ppp pp 1 Music engraving by LilyPond2.5.29— www.lilypond.org.065 Music engraving.058 by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29—www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org

2 notation .044 Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org

LilyPond Music engraving by LilyPond 2.5.29 — www.lilypond.org Labels (vertices)

Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org .025 .083 Music engraving by LilyPond2.5.29—www.lilypond.org Music engraving by LilyPond2.5.29—www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org

ppp ppp ppp ppp ppp pp Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29—www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29—www.lilypond.org .059 Music engraving by LilyPond 2.5.29 — www.lilypond.org ppp Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org 3 Labels (edges) .141 Music engraving by LilyPond2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29—www.lilypond.org Music engraving by LilyPond2.5.29—www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org ppp Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org

ppp Music engraving by LilyPond 2.5.29 — www.lilypond.org

.124

Music engraving by LilyPond 2.5.29Music —engravingwww.lilypond.org by LilyPond 2.5.29 — www.lilypond.org ppp Music engraving by LilyPond2.5.29— www.lilypond.org Edges Music engraving by LilyPond2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org

Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org ppp Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org

Music engraving by LilyPond2.5.29—www.lilypond.org

graphics Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org ppp Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29—www.lilypond.org

MetaPost

Graph Music engraving by LilyPond2.5.29— www.lilypond.org .043 Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org ppp Music engraving by LilyPond2.5.29— www.lilypond.org

ppp ppp ppp Music engraving byMusic LilyPond engraving2.5.29 by —LilyPondwww.lilypond.org2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.orgIppp Music engraving byMusic LilyPond engraving2.5.29 by LilyPond— www.lilypond.org2.5.29 —pppwww.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org ppp Music engraving by LilyPondMusic2.5.29 engraving— www.lilypond.org by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org 4 Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org text Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Annotations TeX Prestissimo possibile, ma preciso "Graph I" Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving2.5.29 by—www.lilypond.org LilyPond Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving ConTby LilyPond 2.5.29 —eXtwww.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org

In ogni arco, l’etichetta indica il valore a cui deve essere legata l’ultima nota del vertice da cui l’arco inizia. Music engraving by LilyPond2.5.29— www.lilypond.org Tutto deve essere suonato alla 15ma superiore. Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving2.5.29 by—www.lilypond.org LilyPond Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving byMusic LilyPond engraving2.5.29 by —LilyPondwww.lilypond.org2.5.29 — www.lilypond.org Figure 3: A graph model is fed into Scalptor, gluing LilyPond and ConT Xt to generate a graphic notation. Music engraving by LilyPond 2.5.29 — www.lilypond.org E Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond 2.5.29 — www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org

Music engraving by LilyPond2.5.29— www.lilypond.org provided with some drawing capabilities. Examples of fluid fitting in this case. LATEX and ConTEXt are two typeset- Music engraving by LilyPond2.5.29— www.lilypond.org architectures implementingMusic engraving by LilyPond2.5.29— www.lilypond.org IAC systems are described in ting systems for documentMusic engraving by LilyPond2.5.29 preparation— www.lilypond.org Music engraving implemented by LilyPond2.5.29— www.lilypond.org as a Music engraving by LilyPond2.5.29— www.lilypond.org Music engraving by LilyPond2.5.29— www.lilypond.org [13] (where they are referred as Automatic Notation Gen- set of TEX macros. Both allow to work together with ad- Music engraving by LilyPond2.5.29— www.lilypond.org erators), [5], and [3]. In the rest of the paper, we discuss vanced graphic packages. ConTEXt ([6]) has been chosen two cases of IAC where different fluid systems are designed as it provides direct support for the MetaPost graphic lan- to fit different needs, allowing for a complete algorithmic guage and extends it by adding a superset of macros (named control over the final score. “Metafun”) explicitly oriented towards design drawing (e.g. allowing pdf inclusion). For this particular project, Python 4. GRAPHIC NOTATION has been chosen as the gluing language: it has a remark- ably clear syntax and meets all the previously discussed In the first project, the final scores (for piano solo) is com- requirements for an IAC language. Python takes into ac- posed of a page in very large format (A0) containing graph- count all composition data processing, i.e. graph generation ical notation. The formal composition model is a graph and manipulation algorithms, and also the gluing, scripting and the notation mirrors visually the graph structure (Fig- process. The Python module, named Scalptor (“engraver”), ure 3). All information associated to the graph data struc- generates the score by writing text files containing code for ture in the model has to be mapped into music/notation each of the involved modules and calling each module in information, so that notation can be generated automati- order to render it. cally. The score is made up of musical notation (vertices and edge labels), graphics (graph drawing), text (perform- ing annotations) (Figure 3, right): all these components 5. SPECTRAL COMPOSITION must be provided by programmable modules and their out- As previously noted, an IAC system should provide room put integrated in an unique document. A strong constraint for inserting modules specialized in audio analysis. Analy- is that musical tradition requires high typographic quality sis parameters can then be processed and used as starting both for the overall document and for the specific musi- material for musical composition. Figure 4 represents an cal notation elements. As all the involved components are implementation of an IAC fluid system for a composition alphabetic or geometric, vector graphic solutions are conse- project involving parameter extraction from audio signals. quently needed. In generale, as standard GUI applications In particular, the commission was to use as starting material are here not relevant, the possible candidates shares a TEX- an excerpt from Sophocles’ Antigon, which was read by a based approach ([8]), i.e. they are command languages, to philologist so to respect as possible the reconstructed Greek be input via textual interface and to be compiled in order classic pronunciation. Three voices sing melodies generated to generate a vectorial output. Concerning musical nota- from data resulting from the analysis of the original au- tion, among the possible candidates (for a review see [10]), dio file, in particular from the fundamental frequency and LilyPond, while still sharing a TEX-oriented approach, en- the first two formants. The Praat software has been cho- sures very high typesetting quality but on the same time sen for the analysis task, being it specialized in phonetic can be tailored for advanced uses, has a simple, human- processing. The SuperCollider application ([12], hence on readable syntax, it has undergone a fast development and SC) has been chosen both as system glue and as an audio it is now the most common text-based music notation ap- module: as a language, SuperCollider is rich in data struc- plication. LilyPond scripting solves the problems of gener- ture, highly expressive, provides an interfaces to the OS ating standard notation for the vertices of the graph, but environment, allows for string manipulation; as an audio the resulting files (one for each vertex, in pdf/ps format) server, it provides state-of-the-art sound processing. Most must then be included into the drawing of the graphic no- importantly, from a UI perspective, SC allows for interac- tation. It is interesting to see that many candidates are tive sessions and provides also programmable GUIs. Com-

255 Proceedings of the 2008 Conference on New Interfaces for Musical Expression (NIME08), Genova, Italy

- input Audio data

Compositional Audio data processing analysis Transcription SuperCollider

- processing 0.35861678Lei Audio Praat 5000 synthesis 4000

3000 Audio Python data 2000

UI Unicode Formant frequency (Hz)

displays conversion 1000 Graphic data III 0 - output 0 0.71723 Music PyX Time (s) 4 LilyPond notation $# # Soprano . $#& # # & # $#& $# # 2 ( ' # $# + + $# $# # # ( " " " $# + ( $# # " " " 1 pp mp # pp pp #& mp mf fff p'p * p*p e - y - o - - u # # o æ u a e # - æ e - - - - - æ - - o # Figure 4: From audio to notation: modules. Tenore $# $# # $# $# # % $# # 1 2 ( $p#p mp $# p$# + + $p#p # $# $# # # ( " " " $# # # # + p#p # # # ( $# $# #$# " " " 8 # #& p# # #& p#p $# # mp mf fff # p*p e i y # a % o - % o u # # o æ u - - a e # e - æ e æ e - æ - e æ - % o - - # munications between SC and Praat has been carried out Basso 3 - - % % % & via text files: Praat can be easily scripted by passing text 2 ( # ( " " " & + & ( ( " " / f # m#p f#f m#f m#f $# m#f p# files and it can, in turn, export text files, which can be read e ------a e # - æ e - - - - æ - - - back from inside SuperCollider. The whole composition cy- ) 0 cle can then be executed interactively from inside SC. As Voce 1 . ) ! ) ) 4 2 " " " " " " " " ( ! !& ! ! ! ! ( ' ! ! " ! ! (& " " + ! ! ! ! ! + " " " " before, a transcription module is responsible for the gener- Figure 5: Different outputs. SC GUI, Praatsf graph- sf ! sf * sf ation of LilyPond files which can then be rendered to final ics, LilyPond notation. s r h st' pi ç x k% r r h st' t' " pdf score file. The transcription algorithm also performs a 0 0 0 0 0 0 0 !,!& ! Voce 2 4 2 " " " " " " " " ( !) ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !& ! ! ! ! ! (& + !)& ! !& ! ! ! + " " melodic contour evaluation on the input data, so that con- sf * s'f tinuous pitch increases/decreases are converted into ascend- 7. REFERENCES s r h st' t' ------" $ # m ! f s r h r h st' ti [1] C. Ames. Automated composition in retrospect: ing/descending glissandos (see Figure 5, bottom). For each !, !, ! 0 0 0 Voce 3 1956-1986.4 2 " " Leonardo" " ,20(2):169–185,1987." " " " ( !) ! ! ! ! ! ! !& ! ! ! ! ! ! ! ! ! ! !& ! ! ! ! ! ! ! ! ! ! ! ! ! + note of a voice, a vowel symbol is assigned, as a result of a / s'f ! ! ! global evaluation of the two formants. As SuperCollider ac- [2] G. Assayag, C. Rueda, M. Laurson, C. Agon,s and" $ # m ! f s " $ # m ! f s " r h st' ti ç x ka h st' t' - - tually does not support Unicode, the LilyPond file has been O. Delerue. Computer-assisted composition at post-processed by a Python module replacing special ASCII IRCAM: From PatchWork to OpenMusic. Computer strings sequences with necessary Unicode glyphs. SC pro- Music Journal,23(3):59–72,1999. vides facilities to sonify in real time all the data, i.e. before [3] T. Ba¸ca. Re:8 Lilypond for serial music? LilyPond 12 $# $# # # S . # 7 $# and after processing and, through GUI packages, the same1 ( # & mailing$# # (& list( ([email protected]),# # # ( " " " Nov." 28" 2007." " " (& * $# # " (& * $# ( pp * p'p p mf pp mf pp data can be displayed on screen. For purposes of documen- # [4]- D. Byrd.- ø Music# % notation- - ø # software and intelligence. e - - - % - ø - y i e ------% a tation, high quality, vector graphics has been generated by Computer Music Journal,18(1):17–20,1994. % T - % - 7 & & $# # # $# # writing in SC the opportune modules. Such modules allow ( [5]# N. Didkovsky.(& ( $ Java# # # Music# ( Specification" " " " Language," " " " (& # # $# # # # " (& $# # # $# ( 18 $p#p # p#p # mp # $# p# pp $# $# p# mf # # # $# # # m*f pp # $# ' to interface Praat, which is able to create graphics from # o v103ø update.# In% Proceedingso - - - - ø # of the International e æ - a # o % o - % # ø y i - e - - æ - e y - # a % - all its data, and the PyX Python graphics package, which Computer Music Conference 2004, Miami, 2004. 7 % has been used to plot compositonal data structure. FigureB 53 - % % ( [6] H. Hagen. ConTeXt the manual# ( " ." PRAGMA" " Advanced" " " " (& " (& # + shows (from top to bottom) a GUI from SC plotting formant/ $m#f #& p #f & m#f f m#f p# # - Document- - Engineering,- o - - - Hasselt# NL, 2001. e ------i e ------a - data, the same data exported by Praat into an eps file, and [7] K. H. Hamel. A design for music editing and printing . !5 0 0 0 7 ! an excerpt from the final score by LilyPond. This rich sys-V1 software based on notational& ) syntax.& & Perspectives of 4 + !) ! ! ( " " " " " ( ! ! ! ! ! ! ! ! ! + " " " " ( !) ! + " " " " " " " " " " " " " ( ! tem output provides a constant feedback to the composer, sf !6 New Music,27(1):70–83,1989.s*f sf !6 s!f) x(% ( tum t&k t' s r h st' t' - ( tum ' gæ k% h allowing her/him to control interactively the composition [8] D. Knuth. The TeXbook. Addison Wesley, Reading, process. !& !& ! 0 0 0 7 !, V2 Mass., 1984. & ) & & & & - 4 + !) ! ! ! ! ( " " " ( ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! + " " ( !) ! ! ! ! ! ! + " " " " " " " " " " " ( sf [9]!6 M. Laurson, V. Norilo, ands*f M. Kuuskankare. sf !6 s!f) x(% ( tum ' gæ h st' ti s r h st' t' - " ps ks r h st' t' ( tum ' m ! s r k% 6. CONCLUSIONS PWGLSynth: A visual synthesis language for virtual !5 instrument!5 0 0 0 design and control. Computer0 Music0 0 7 !, ! ! !5 The case of musical notation is particularly relevantV3in 4 + ! ! ! ! ! ! ! ! ! ! ( " (& !) !& ! ! !& ! ! ! ! ! ! ! ! ! ! !& + ( ! ! ! ! ! ! ! ! ! + " " " " " " " " " ( ! sf) ! Journal!6 ,29(3):29–41,2005. sf) ! ! demonstrating the need (and the strength) of a fluid archi-/ 6 ' s*f ' 6 sf) x(% ([10]tum t&k t H.-W.um t&k t' - - Nienhuys" ps r and J. Nieuwenhuizen.s " r h st LilyPond,' t' - - " ps r a ( tum ' m ! f s r h st' ti t&k k% h tecture for an integrated algorithmic composition system. system for music engraving. In Proceeding of the XIV In itself, notation is not a simple mapping from musical data CIM 2003, pages 167–172, Firenze, 2003. to notation symbols, as it requires the composer to provide [11] H. Taube. An introduction to Common Music. specific typographic information. An IAC approach allows 16 20 Computer Music Journal,21(1):29–34,1997.# 7 $# $# #& # S . & # & # # # # the composer to develop case-specific solutions to such a1 " ( # ( " " " " " " " " " ( ' + " " " " " " pp ( " " " +m'p pp( " " $p#p ( pp [12]p*p S.* Wilson, D. Cottle, and N. Collins, editors.p Thepp * * problem, by plunging the selected modules into a fluid sys- # o y - - ø i ø e i - - o ø y tem: indeed, these can include not only notation but e.g. SuperCollider Book. The MIT Press, Cambridge, Mass., 2008. 7 many different UIs. The maximum flexibility is evidentlyT # % 1 " (& # # (& " " " " " " " " " ( # #$# + " " " " " " ( " " " + ( " " $#& ( % 8 pp # $# # pp # gained by using an interactive language as a system glue. [13]p*p H.* Wulfson, G. D. Barrett, and M. Winter.p# # Automatic# p#p # m#p pp # # p#p Some examples of automatic generated notation can be #notation generators. In Proceedings of they e æ 7tha # - ø i ø e i e i o # ø y found at international conference on New interfaces- for musical - 7 B 3 ( (& # # + + " ( + " " " " ( " " " + + " " expression$p#p # , pages 346–351, Newp# York,m 2007.#p # ACM.# # #& $# m#p http://www.cirma.unito.it/andrea/compositionNotation// #.$m#p p* * mp # mp #f #f m#p m#p # - # ------y i - e i e - - o

. ! 7 !) ! !, !) !& V1 ) ) & 4 ! ! + " + !) ! ! ( " " " " " " " " + ! ( " " " " " " ( ! ! ! ( ' ! + ! ! ! ! ! ! ! ! ( * ! ! ( " " 256 sf !6 ! s!f)& sf sf sf * sf * st' ti d# to tum ' k% h ç s r pi ç s " $ # m ! f pi ç s

0 0 !, 0 0 0 0 0 0 0 0 0 7 ! ! 0 0 0 V2 4 ! !& ! ! ! ! ! ! ! ! ! ! ! ( " " " " " " + ! ! ! ! ! ! ! ! ! ! ! ( " " " " ( !) ! ! ! ! ! ! ! ! ! ! + !) !& ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! (& ! s!f) sf sf ! * h st' t' - " $ # m ! s " ps k% h st' t' ------ç x pi ç s " ps ks ) ! # s r h st' ti ç x k% h st' t' - - " ps ks ) ! f s " r h st'

0 0 0 !,& 0 0 !, 7 ! 0 0 !, ! V3 4 ! ! ! ! ! !& ! ! ! ! ! ! ! ! ! ! ! ! ( " " " " + ! ! ! ! ! ! ! ! ! ! ! ! ! ! ( " " ( !) ! ! ! !& ! ! ! ! + !) ! ! ! ! ! ! ! ! ! !& ! !& !& ! !& ! / s!f) ' sf ! sf st' t' " ps r h st' t' - " $ # m ! f k% h st' t' - " $ # m ! s " r h ç x pi ç x ka h st' t' " ps s r h st' t' " $ # m ! f s " r h