Current Research in Computer-Generated Music

1JECT OVERVIEWS Current Research in Computer-Generated Music

The six overviews that follow reflect varied ongoing research. Reporting from such diverse locales as Singapore, Europe, and the US, the authors explore the spheres of computer-aided composition, synthesis of musical scores, computer simulation, and composing by musical analog.

Algorithms for Musical Composition: A Question of Granularity

Stephen W. Smoliar Institute of Systems Science, National University of Singapore Neng Mui Keng Terrace, Kent Ridge, Singapore 051 I

omputer programs that “com- the most frustrating applications of com- mathematical models as alternatives to pose” music have been around puter composition in the “Illiac Suite.” Markov processes. The spectral densi- almost as long as the computers The appeal lay in the belief that knowl- ties of a wide variety of physical quanti- thatc run them. Initial approaches se- edge of past events could serve as a ties tend to vary as the inverse of the lected notes for a composition using a valuable predictor for subsequent frequency: l(f. Voss and Clarke demon- random process with constraints to elim- events. The frustration lay in the dis- strated such a correlation using the au- inate or modify undesirable choices. The covery that there was either too much dio signal of Bach’s first Brandenburg computer composition “Illiac Suite for or too little of this knowledge. If the Concerto and then detected the same String Quartet,” by Lejaren HiIler and number of past events considered was correlation in signals from classical music Leonard Isaacson, demonstrates the too small, the result sounded as random radio stations.’ Voss and Clarke con- range of these approaches. as if the notes had simply been selected ducted similar studies of the music of Experiments with Markov processes from an arbitrary set of weighted prob- other composers, including Scott Joplin were probably the most appealing and abilities without any prior knowledge. and Milton Babbitt. Observing this dis- If the number was too large, what was tribution in many different sources of “predicted” was nothing more than a music. they reasoned that they could replication of the original data from synthesize new music by a random pro- Audio examples which the probabilities were computed. cess based on the same distribution. There seemed to be no middle ground Figure 1 shows an example. Examples of the music dis- between the absolute predictability of Eric Iverson has explored another cussed in these project overviews are available on CD and audiocas- duplication and the absolute unpredict- approach.* He applies principles of sette. Please see the order form ability of random noise. autocatalysis, a chemical process that on page 9. The quest for an appropriate middle enables the reproduction of molecular ground led to an investigation of new strings, to the synthesis of “strings” of

54 001%Ylh2/91/0700-005J$01.00@ 1991 IEEE COMPUTER music-either pitches or intervals. Fig- ure 2 shows an example of his results. The variety in approaches to algorithmic composition is impressive, but the results are disappointing. Do these results arise from a critical commonality in these many experiments? I believe the commonality is the assumption that musical composition requires decisions about the placement of notes. This assumption is natural enough. It has its roots in a pedagogic tradition in which the composition student is concerned with manipulating notes on music paper. However, music need not be notated to be composed. A composer can compose while playing an instrument. This is generally known as improvisation. Anyone who has heard good jazz improvisation can appreciate how unlikely it is that musicians who impro- vise read from some notation they write in their head. Thus, if we want a founda- Figure 1. Music synthesized by a random process based on a L’fdistribution. tion for algorithmic composition, the (Reproduced with permission.*) materials provided by the pedagogy of musical composition may be far less valuable than those of the actual prac- with the Dice Composer was to make resolved: These examples are models. tice of making music. In this article I sure that the terminals chosen for any In musical composition, such models argue that making music is concerned given measure were interchangeable. include specific compositions or excerpts with a higher level of granularity than Thus, by raising the level of granularity from compositions. David Cope is a good that of the notes on music paper. to entire measures, Mozart could use example of a working composer who chance techniques to produce accept- has managed to put a model-based ap- Raising the level of granularity. Mu- able music in the classical genre in a proach into practice. (See Cope’s arti- sical composition is not necessarily a manner never duplicated by any algo- cle beginning on page 22 of this issue.) matter of putting notes together in the rithmic technique concerned with se- right way. Long before there was jazz, lecting individual notes. An alternative agenda. When we talk people were making music by working If raising the level of granularity in- informally about musical composition, with a higher level of granularity. creases the power of random processes we rarely talk about individual notes. Mozart’s Dice Composer, for example, applied to musical composition, per- Composition is a matter ofworking with generated (to use the terminology of haps we are thinking about the wrong “musical ideas.” We use this phrase in- computational linguistics) random sen- things when we focus our attention on tuitively, without pinning down just what tences by selecting productions from a the selection of individual notes. Work it denotes. Nevertheless, following the simple context-free grammar. Each of in artificial intelligence shows that such lead of model-based reasoning, we can the terminal symbols in this casewas an low-level decisions may actually be sub- assume that musical ideas include our entire measure of keyboard music. Each ordinate to a model-based control struc- memories of past listening experiences. measure of the score, in turn, corre- ture. The underlying premise of such To some extent all composers pick up spondedto 11productions, eachof which control is that the knowledge sources materials from past experiencesand find filled in that measure with one of those used to make decisions include actual new ways to assemble them. terminals. The bulk of Mozart’s work examples of how problems have been What do such intuitions tell us about

Figure 2. Music synthesized by applying principles of autocatalysis to the selection of pitches and intervals. (Reproduced with permission. *)

July 1991 55 PROJECT OVERVIEWS

algorithmic composition? The primary music powerful enough to compel us to look where the ordinary is practiced to lesson is that our agenda for algorith- sit still and listen - even one of Bach’s find a foundation for studies of the ex- mic composition is off the mark. If we two-part inventions or Domenico Scar- traordinary. n are determined to seek out algorithmic latti’s many sonatas - is probably too rules, then our search should be direct- sophisticated for the analysis required ed by two questions: in algorithmic composition. Finding much past evidence of the ordinary in References l How do we identify units of mate- these extraordinary pieces may be diffi- rial of the appropriate granularity? cult simply because the ordinary rarely 1. R.F. Voss and J. Clarke, “‘l/f noise’ in l Given a collection of those units, survives the eroding forces of history. Music: Music From l/fNoise,“J. Acous- how do we properly assemble them? However, the ordinary is very much tical Sot. America, Vol. 63, No. 1, Jan. with us in the present. We tend to ig- 1978, pp. 258-263. What made Mozart’s Dice Composer a nore the routine work of composers 2. E. Iverson and R. Hartley, “Metaboliz- clever piece of work was his recognition obliged to provide themes and jingles ing Music,” Proc. Int’l Computer Music that he could think about simple dances for commercial applications. These com- ConF, Computer Music Assoc., San Fran- (following the style of the dance move- posers must create on demand, proba- cisco, 1990, pp. 298-301. ments of Bach’s keyboard suites) in bly to the extent that none of their tasks terms of individual measures. He could receive extensive cognitive effort. Here Stephen W. Smoliar is investigating several then ask how any one measure could be is where we may find artisans with a aspects of music cognition and perception in varied so that it would always “fit” in clear understanding of the units of ma- addition to his artificial intelligence research the context of preceding and succeed- terial with which they work. We should at the National University of Singapore. ing measures. He gave these variations enough flexibility that any alternative for the first measure could be followed by any alternative for the second measure, and so on for the duration of the entire composition. Scoresynth: A System for the Perhaps Mozart’s exercise was a par- ody of the rather routine approach to Synthesis of Music Scores Based on composition he observed in others around him. For Mozart, the model be- Petri Nets and a Music Algebra came music only after he suitably per- turbed it, but he could not address questions of perturbation until he had a model Goffredo Haus and Albert0 Sametti in place. The minuet movements that Laboratorio di Informatica Musicale, Dipartimento di Scienze dell’Informazione, Mozart wrote for his more memorable Universitri degli Studi di Milano, via Moretto da Brescia, 9, I-20133 Milano, Italy orchestral and chamber music could not readily conform to this dice model, since the model had no good way of imple- menting material that returns in a simi- usic structures can be de- traditional and nontraditional music lar, but not identical, form. *: P scribed, processed, and syn- objects). M,: I, ’ thesized using a more flexi- A music object means anything that The ordinary versus the extraordi- ble kind of representation than the staff. could have a musical meaning and that nary. The moral of the Dice Composer In this article, we show how. In fact, we we think as an entity -either simple or story is that identifying units of appro- have chosen and organized symbols de- complex, abstract or detailed -with a priate granularity and assembling them pending on instrumental needs within name and some relationship with other properly is no easy matter. The basis for common music notations. The level of music objects. We can describe music “automating” dice-composed binary representation within scores is more de- objects at various abstraction levels with- forms was Mozart’s recognition that it tailed than that of music composition in a hierarchical context -for example, was a highly routine activity. However, and more abstract than that of timbre the structural level, the score level, and many who pursue algorithmic composi- modeling. the timbre level. tion do so out of “genius envy.” They The new kind of representation we Common music notation is charac- are more interested in the extraordi- propose makes up a conceptual music terized by many different languages (one nary than the ordinary, and fail to rec- framework with as many different lev- or more languages for each level of rep- ognize that they cannot have the ex- els of abstraction as the musician needs. resentation). Our research is devoted to traordinary without first having the It allows us to explicitly describe and the definition of just one language, suit- ordinary as a point of departure. Any process what we call music objects (both able for every level of representation.

56 001%9162/91/0700-0056$Oi 00 0 1991 IEEE COMPUTER To identify the most suitable description tool, we sought a formal tool that

l requires few symbols, All Transitions l has a graphical form of notation, q l allows hierarchical description,

l allows the description of music objects processing,

l allows time description,

l allows deterministic/nondeterministic models, l allows macro definitions for common structures, and *shows the musician the score syn- Figure 1. Music object map dialogue with the two windows of Petri net node at- thesized as the model executes. tributes.

Score modeling. We have been using Petri nets (PNs)’ as the basic tool for music description and processing since 1980. In our research, we have developed certain programs to edit and execute PNs for both music and general-purpose applications.2 The most re- cent program we developed is called Scoresynth. In it, we use PNs to describe, process, and synthesize music scores. While we assignmusic objects (MOs) I to place nodes for describing music in- Figure 2. The score of the “Are You Sleeping” theme stored in the Theme file. formation, we assign causal relation- ships and transformation rules to PN structures and PN parameters (marking, numeric labels, algorithms, etc.). the editor and the executer of the mod- plane - the map of MOs as they are MOs can be processed by modifying els. appended in the score and any informa- the superimposition and juxtaposition The editor allows the description of tion requested by the musician about laws within PN structure. On the other hierarchical, timed, concurrent, deter- the genesis of MOs. hand, PN parameters allow us to create ministic/nondeterministic, and weight- instances of MOs that modify them- ed PN models. Both places and transi- Music objects. As stated previously, selves according to the behavior of PN tions are used as the “morphism” nodes the meaning of MOs is strictly related to models during execution. Furthermore, to implement hierarchy. Common PN place nodes. To us, an MO encompasses we can represent the same musical in- structures can be described as macros. not only any sequence of notes but also formation by PNs at a lower or higher An interactive graphic user interface is something more general that is not ex- level of abstraction by using suitable provided to build PNs by an iconic rep- clusively linked to listening and not con- alternative modeling approaches. resentation. nected to the idea of process because it Indeed, a PN model with a particular The executer performs the models, could last zero seconds. Obviously, the initial state (that is, initial marking) can synthesizes MIDI (Musical Instrument definition of an MO is partly affected by represent a family of scores;a particular Digital Interface) scores, and shows a the music code standard we use (MIDI) execution of the model synthesizes a graphic map of the MOs as they are and must be kept within the bounds of specific score. By modifying the initial generated by the model execution (see this standard (see Figure 2 as an MO marking of the model, we can change Figure 1). Looking at the map, the mu- example). All MIDI messages,even if the family of scores that can be pro- sician can interact with the model, stop not directly concerned with a note, can duced by model executions. A special the execution assoon as a bug is found in be put in an MO. situation exists when we have a fully the model, and proceed to the editor MOs can be coded according to three deterministic model. In such cases, we environment to make modifications. This syntaxes: can produce one only score (given a can be done simply by executing the particular initial marking). model and looking at a graphic window l a coding language that is specific in The Scoresynth system is made up of that shows - on a time-MIDI-channel Scoresynth,

July 1991 57 PROJECT OVERVIEWS

l an alphanumeric MIDI coding language, and l a coding format that implements the StandardMIDIFiles l.Oformat (both type 0 and type 1).

The program uses the first format in- ternally. The second format is based on Voices Input the availability of editing tools (such as common word processors). The third Are You Theme format has been implemented for file Sleeping import/export. MOs can be defined either by a common text editor or by a MIDI controller (a keyboard, guitar, (a) (b) microphone, etc.). The MOs can be stored in separate files (one file for each ob- Figure 4. The IEEE net (a); the “Are You Sleeping” net (b). ject) or within the PN file to which the associated place belongs. Figure 3a shows the simplest net to perform an MO. We joined the Theme place, which has been previously stored in a file according to any of the three allowed formats, to the Theme MO. The upper attribute of the place defines the number of tokens available, while the lower attribute specifies the maximum capacity of tokens allowed for that place.

The music algebra. The main idea of Scoresynth is to make it possible to transform both the parameters of sound (pitch, timbre, intensity, and duration) and the order in which notes appear, using algorithms. Algorithms join with transitions. (4 (b) If a transition has an associated algorithm, Scoresynth merges all the MOs in Figure 5. The Voices net: (a) deterministic version; (b) nondeterministic version. the entrance before making the transition fire. When the transition fires, it takes a token from each input place, applies the transformation to the single MO obtained, puts the token into all the Algl: (C: 1, $, [Theme, 1],1 (...a11 on MIDI channel 1; use “Theme” MO} output places, and places the outcoming M:l, S, 2) (...repeat 2 times] MO into every output place allowed for Alg2: (C: 1, $, [Theme, 1],2 (...a11 on MIDI channel 2; use “Theme” MO} MOs. The transformation can only be M:l, $, 2 (...repeat 2 times] applied to note events. All the other P[C]: 1, $, ? + 2) {...2 degrees transposition, C major tonality} MIDI commands are filtered and lost. If Theme stands for the MO on which Alg3: (C: 1, $, [Theme, 1],3 (...a11 on MIDI channel 3; use “Theme” MO) the algorithm is processed at the input, D: 1, $, ? I2 (...halve durations] the algorithm can be applied to the whole M:l, $, 4) {...repeat 4 times) Theme or can be limited to one of its Alg4: {C: 1, $, [Theme, 1],4 (...a11 on MIDI channel 4; use “Theme” MO) subsequences, as designated by the mu- I: 1, $ (...retrograde) sician. It is also possible to use informa- M:l, $, 2) (...repeat 2 times} tion from every MO available at that moment within the whole model. An algorithm can be formed by one or Figure 6. The parameter list for invoking the Voices macro Petri net. more single operators. Each operator

58 COMPUTER can affect either the parameters of the R (rotate) = operator for rotat- output arcs of P become output arcs of notes or their order. Each operator is ing notes. Out. Transitions can be refined in the applied to the outcoming MO from the same way. We can also model recursive operator before. Set in its header, the Figure 3b shows a very simple net nets (that is, nets that contain them- algorithm is applied to the range of that can transform MOs. Here, we can selves as refinement nodes). To avoid notes as each note occurs. The header join any algorithm we wish to the tran- endless recursive expansions, we must also defines the kind of parameter to sition Alg, decide if we want to enable assigna recursion level number to every which the operator expression must be the execution of the Theme and/or recursive net. applied-pitch, duration, intensity (key Theme.mod MOs (setting the specific Furthermore, when we design a nei velocity), MIDI channel - and the or- place attributes), and decide if the MO model, we often have to create many der of the affected notes. Let us exam- obtained from the transformation similar nets that differ in their place ine two examples of such algorithms. (Theme.mod) should be volatile or attributes and algorithms but are iden- stored in a file. In this case, a specular tical in structure. With a macro net, we (1) Specular inversion. This can be inversion algorithm may be P: 1, $, 2 * can use just one net as the base model done by meansof a reference note, called C-sharp 3 - ? and then modify its attributes and/or the mirror note; that is, every processed algorithms as needed. This enables the value of pitch is symmetrical to the mir- Firing and timing. PNs are particu- musician to create personal libraries ror note. The following is an operator to larly suitable for describing concurrent containing commonly used nets. We can invert the pitches with respect to C- processesand controlling their synchro- do this by writing a modifier list for sharp 3: nization (or lack thereof). The behavior every subnet place or subnet transition P: 1, $, [Theme, 1],2 * C#3 - ? of the net implicitly determines timings. we want for the net. This modifier list is We can synchronize MOs simply bysuit- loaded into memory and actualized be- where P stands for pitch; 1, $ stands for ably structuring the node connections. fore execution of the model. “from the first to the last note of the The structure of the net determines the Figures 4a, 4b, and 5a represent a input MO”; [Theme, 11determines the firing sequence. deterministic model based on the “Are use of information from the Theme MO The firing sequence changeswith dif- You Sleeping” theme, while Figures 4a, (not from the MOs joined to the input ferent executions of the net, so there is 4b, and Sb represent a nondeterministic places, which are the defaults and need no correspondence between a net and version of the same melody. (The alter- no explicit specification) starting from its firing sequence. We can have many native depends on Figures 5a and 5b. the first note (I); and 2 * C#3 - ? is the transitions qualified for firing at the Figures 4a and 4b are deterministic if expression that calculates the new pitch same time and not know which one fires they are considered stand-alone nets.) values. The ? is a metacharacter that first. The net does not describe this kind The IEEE net is the most abstract net. represents the value of the parameter of information. Every execution of a PN The Are You Sleeping place is then (in this case, the pitch), assigned to the model may give a different firing se- refined by the homonymous net with current processed note. quence. Input as input place and Voices as out- Other algorithm headers affecting Our approach represents timed MOs put place. The input arc to the Counter note parameters are: by places and their transformations by place has weight 3; this means that the C (channel) = operator on MIDI transitions. The firing of a transition firing of the transition will put three channels, has a null duration. In this way, when a tokens in the Counter place. The Rest D (duration) = operator on dura- token is put into a place with an associ- 414place has a 414rest as an associated tion, and ated MO, the token cannot be consid- MO. L (loudness) = operator on inten- ered for the firing of transitions con- To expand the Voices place with a sity (key velocity). nected to the place until the associated macro net, we can use one of the two MO has ended. nets in Figure 5. In the first case, we (2) Inversion of the order (retrogra- obtain a deterministic execution of the dation). The inversion operator rewrites Hierarchy and macros. In this sec- model; in the second case, we obtain a the input MOs backwards: I: 1, $, where tion, we introduce two new concepts: nondeterministic one. Stated another I stands for invert, and 1, $ stands for hierarchy (refinement) and macro. way, we cannot fix the firing sequence, “from the first to the last note.” A refinement, called a subnet, is a PN but the firing sequenceis determined by Other algorithm headersaffecting the that gives a more detailed description Scoresynth with a pseudorandom uni- order of notes are: of a node (either a place or a transition) form distribution of the order. K (kill) = operator for delet- of the upper level of abstraction. If we The Input place has no token as ini- ing notes, choose to define a subnet associated tial marking because it will receive to- S (save) = operator for pre- with a place P, then the daughter net kens from the upper level of the model serving notes, must have two special places: an input (in both the Are You Sleeping and Voic- M (multiply) = operator for multi- place In and an output place Out. Input es nets). Figure 6 shows the parameter plying notes, and arcs of P become input arcs of In, and list for invoking the Voices macro PN

July 1991 59 PROJECT OVERVIEWS

(note that we invoke only the actualiza- l executingnondeterministicPN mod- 89.00031.69 and 90.00678.PF69: “Intelligent tion of algorithms associated with the els many times. Music Workstation,” Goal C: Sistemi Avan- zati di Produttivita Individuale, Subproject four transitions of the macro net here, 7: Sistemi di Support0 al Lavoro Intellet- but we can do the same with respect to While editing PN structures and pa- tuale, Finalized Project Sistemi Informatici e all the possible attributes of the PN). rameters affects causal/structural rela- Calcolo Parallelo.” Figure 1 shows the map of the MOs tionships within the architecture of obtained from the IEEE model execu- scores, editing both MOs and algorithms tion. It contains four different instances affects the basic information units and of the Theme.mod MO belonging to the their transformations. We believe that References Voices net put on different MIDI chan- the Scoresynth system provides a pow- nels. These MOs differ from one anoth- erful means for describing and process- 1. J.L. Peterson, Petri Net Theory and The Morleling ofSystems, Prentice Hall. En- er because they are generated by differ- ing music in a way that is closer to music glewood Cliffs, N.J.. 1981. ent algorithms. The other MOs of the thinking and perception than common model are not present in the map (or the music notation is. w 2. G. Haus and A. Rodriguez, “Music De- output score) because Theme was set as scription and Processing by Petri Nets,” 1988 Advances Petri Nets, Lecture Notes an MO that is not meant to be played in Computer Science N.340, Springer- and Rest 414produces a delay, since it is Acknowledgments Verlag, Berlin. 1989, pp. 175-199. a rest MO and not a MIDI event. We thankEnricoBianchi,AntonioCamur- Transforming music structures. Our ri, Giovanni Degli Antoni, Giampiero Jaco- Goffredo Haus. scientific director of the research has shown that music struc- mini, Antonio Rodriguez, and Renato Zac- Laboratorio di Informatica Musicale at the tures can be transformed simply by caria for their contributions to previous University of Milan, does research in the research projects and experience related to university’s Department of Computer Sci- Petri nets and music. We particularly thank ence and composes music by computer meth- l modifying markings, MIDI channel Sergio Scolaro for his relevant contribution ods at both the score and sound level. He specifications, and capacities of to the design and development of some fea- helped found the Italian Association of Mu- places; tures of Scoresynth Rel. 2.0 (especially the sical Informatics. graphic interface and PN macros). More- l modifying labels of arcs; over, we especially thank members of the staff of the Laboratorio di Informatica Musi- Albert0 Sametti is the software manager of l modifying MOs associated with cale at the University of Milan for the scien- the Intelligent Music Workstation project at places; tific and technical support they provided Laboratorio di Informatica Musicale at the

l modifying algorithms associated during our research. University of Milan. He works on Petri net with transitions; This research is partially supported by the applications to music and is a scientific con- Italian National Research Council through sultant to Apple Computer on software de- l modifying the structure of PNs; and the Music Topic LRC C4, Grants No. velopment and validation.

Neurswing: An Intelligent Workbench for the Investigation of Swing in Jazz*

Denis L. Baggi Via Casagrande 12 CH-6932 Breganzona, Switzerland

eurswing is an intelligent sys- representing musical data. At runtime “knobs,” that is, by setting input ‘, tern to investigate swing in the system generates and plays the mu- parameters of a second, separate and N jazz by simulating the opera- sic of piano, bass, and drums in real asynchronous, net, which manip- tion of a rhythm section. From an input time. ulates the probability of some choices. consisting of the harmonic grid of a The user can control the performing The system can simulate most aspects standard tune, it constructs a network style of the rhythm section by turning of a performing jazz rhythm section. It

*This article is an abridged version of previous publications’ 2 that describe the work performed at the International Computer Science Institution in Berkeley. California. whosesponsorship is gratefully acknowledged. Figure 1. (a) Structural har- 6b Bb Bb /Bb7 Eb /Ed Bb IF7 Bb monic grid of Rhythm Changes, the basis of many pieces fa- vored by jazzmen: “Antbropol- Bb Bb Bb Bb/Bb7 EblEd BbIF7 Bb ogy,” “Dexterity,” and “I Got Rhythm.” (b) The harmonic net constructed from the grid. From top to bottom: measure and Am7 D7 G G7 c7 c7 F7 F7 beat number, piano note units, bass note units, drum note units, units of the given barmo- , Bb , Bb , Bb , Bb , BblBb7 , Eb/Ed , Bb/F7 , Bb , ny (squares), and units of harmonic substitutions. I I I I I I I I I (a)

: 2 3 4 2 6 7 8 8 18 11 12 :3 14 1s 16 f 7 18 19 28 41 22 23 24 15 26 27 28 33 38 31 32 3 3 34 35 36 I$38 33 48 %42...

bbld2 f2 d2 bbld2 f2 62 bb%iZ f2 62 bbldl f2 62 bblfl bblfl ebmblel bbmbifl fi cl bb3d2 f2 d2 bbld2 f2 d2 bbld2 f2 42 bbld2. +. fI.~ff~rffffffffffffftff~fffffTfff~ff~fffff 8b7 Eb lib Bb

.b)

substitutes chords, bass lines, and drum This is why jazz is improvised by defini- section and allows the user to modify its licks and has been used by practicing tion. Swing cannot be annotated in a parameters at three levels. At the top improvisers as a didactic tool. Further- music score; rather, it is captured in a are three stylistic knobs: more, the rules for substitution as well as record, which has been the document in the stylistic net are external to the basic jazz since 1917,the accepted date of the l hot-cool controls the overall impres- system and can be configured and altered first jazz record. sion of drive and push, as defined at will, allowing the user to treat the Swing is produced by - though it from the thirties to the fifties, with system as a workbench for experiments should not be confused with - certain the “hot” elements of swing in op- in the synthesis and analysis of swing. technical means such as rhythmic pat- position to the “cool” school of the terns, harmonic progressions, melodic early fifties; The problem. In jazz the word swing accents, and instrument timbres. One l dissonant-consonant controls the has many meanings. It really means a of its aspects is possibly the opposition overall level of dissonance in the holistic. “gestaltic” quality of almost between an implied regularity of pat- underlying harmony; extra-musical nature, by virtue of which terns, rhythmic or otherwise, and a con- the jazz message is passed to the listen- sistent yet controlled violation of that *as-is-ness amplifies or reduces the er. Swing is the medium for jazz, as regularity. This project addressessome level of free substitution versus ad- space is the medium for sculpting. This of these quantifiable, technical aspects herence to the given input - the sets jazz apart from classical music, for of swing. piece - and is thus a control for example, where the piece has a quality improvisation. independent of the moment in which it The system and its possible uses. Neur- is played. In jazz the opposite is true. swing attempts to simulate a jazz rhythm At the second level, all rules for harmo-

July 1991 61 PROJECT OVERVIEWS F’

Hot/ Dissonance/ As-is-ness/ cool consonance free

------m-m--

------

1 22 23 24 IS 26 27 28 93 38 31 32 38 39 40 :: 42,

DOI :: DO1

bbld2 12 d2 bbld2 12 d2 bblfl bblfl eb~blel bbPlbl,l f; ~1 bbld2 f2 42 bbld2 12 d2 bbld2 f2 d2 bbld2

tftttttttttttttttttfYtTttTtttttttt

Tempo wiations, delays

Harmonic net

Figure 2. The architecture of the system: stylistic net above, harmonic net below. The inputs of the stylistic net produce outputs - the books - indicated by asterisks and corresponding to those of Figure 3. These outputs affect the probability of possible choices in the harmonic net at runtime as well as tempo increase or decrease and delays of piano and bass in respect to drums.

62 COMPUTER ny substitution, piano, and drum patterns are in external tables and can be modified at will. At the third level, the presented stylistic network is just an example - an untrained neural net - to demonstrate how style can be modeled. It is external to the system, and some other network could substitute. In this sense, Neurswing is a workbench becauseit allows us to investigate which parameters have what influence on swing. Aside from its scientific interest, Neur- swing has been used by practicing musicians as a didactic tool, since for most purposes it can substitute for existing records with jazz rhythm sections (for example, Music Minus One and Japa- nese Karaoke), such records being in- flexible and of fixed duration, tempo, and key. Neurswing, on the other hand, is a flexible, programmable system that can play any tune in any key and at any tempo for as long as desired. It can even repeat difficult passages.Neurswingcan help aspiring musicians control rhyth- 1 output 1 mic accents and correct typical begin- ner’s errors, such as slowing in difficult passagesand accelerating in easy ones. Figure 3. The hierarchy of runtime choices expresses the dependency of the Control of tempo is an essential tech- types of hooks (indicated by an asterisk). For instance, selection of the drum nique in swing; a metronome may offer pattern constrains the choice of notes played by the drums. some help, but Neurswing is close to the ultimate practicing tool, a real rhythm section embodying harmony, rhythm, and style. notes, initialized from the given har- or melodic) predominates while many The input and the harmonic net. The mony. The four to seven units just be- others are implicitly activated in the harmonic grid (see Figure la on page low the measure and beat numbers at background. 61) is the input to the system. Each the top contain the initial notes of the square of the grid corresponds to a mea- piano units. The paths in the section The system architecture. Figure 2 sure, or four beats, and contains sym- representing harmony and the contents shows the system architecture. During bols representing chords that determine of the units of piano, bass, and drum operation of the underlying harmonic the harmonic structure, though not the notes change at runtime under control net, there are several points where it is actual notes. The meaning of these sym- of the stylistic net. The harmonic net possible to choose among different pos- bols, which are also used in guitar charts, operates in synchrony with the beat; sibilities - for instance: is described in the jazz literature. activation passes from all units on a l kind of harmony substitution; This section of the system uses a beat to those on the next beat and then l piano rhythmic pattern; knowledge base to construct a connec- wraps around. l type of walking bass- up, down, or tionist model of the jazz performance, The net, though only vaguely resem- note from the chord; drawn by the Rochester Connectionist bling a neural net or a connectionist l position of piano chord; Simulator and shown in Figure lb. The system (it never converges, and neither l drum pattern; and net units represented by a square con- relaxation nor learning takes place), ex- l piano or bass appoggiatura. tain data of the grid harmony, while the ploits the inherent parallelism of such units below show the constructed alter- models. The paradigm was chosen be- All these choices, called hooks, are un- native harmonies. The symbols above causeit seemsthe most convenient way der control of the stylistic net, shown at the squares represent drum hits, and to represent the process of jazz impro- the top; Figure 3 shows what they are above these are the units of the bass visation, in which one path (harmonic and how they relate.

July 1991 63 PROJECT OVERVIEWS

&”

The system could operate without the tested without changing anything in the Computer music projects have rarely stylistic net. This would be equivalent controlled program. Second, hierarchies been applied to jazz. Perhaps this work to a setting of 0.5 for all three knobs, of nets can be conceived to further con- will stimulate further investigation into and the system would not exhibit any strain some aspects of style without af- the subtle problems of jazz improvisa- stylistic consistency. Turning a knob fecting the operation of underlying nets. tion and swing. n activates the stylistic net, which in this example consists of three inputs, 360 Performance. Neurswing is a hybrid units in the middle layer, and 360 out- system using a knowledge base to con- puts. The outputs, or hooks (indicated struct a network that contains data for in Figures 2 and 3 by asterisks), are jazz improvisation and performance. It References numbers that change the probability of realizes controllable real-time jazz im- a given choice. For instance, an increase provisation using neural nets. 1. D.L. Baggi. “Neurswing: A Connection- in the hot/cool knob favors a dense rhyth- Neurswing performs acceptably for ist Workbench for the Investigation of mic piano pattern, more piano appog- tempos between 100 and 2.50beats per Swingin Afro-American Jazz,”Proc. 23rd Asilomar Conf. Signals, Systems, and giaturas. louder playing of the piano, minute (medium to fast) and optimally Computers, Vol. 1, CS Press,Los Alam- walking bass up, more bass appoggiatu- for 180 to 200 beats per minute. The itos, Calif., Order No. 2037, 1990, pp. ras, more nervous drum patterns, more time signature is 4/4. Extension to other 336-341. drum fill-ins, and more crash cymbal signatures such as 3/4,6/8,5/4, and 614is 2. D.L. Baggi, “Neurswing: A Connection- hits. An increase in the dissonance knob possible, but has not been realized. ist Workbench for the Investigation of favorsdissonantharmonicpatternsflat- While there are commercial systems Swing in Afro-American Jazz,” Proc. ted fifth in dominant seventh chord, that simulate a jazz rhythm section, none Workshop AI and Music, European Conf. inversion for piano chords, and half steps are capable of improvising, that is, of Artificial Intelligence, Stockholm, 1990. for piano and bass appoggiaturas. elaborating the input with substitutions, To be reprinted in Interface, J. New Mu- The advantages of modeling stylistic changes,anddrum accompaniment; they sic Research. constraints with a neural net are two- merely replay what the user typed in. fold. First, complicated interactions Hence, they are not interactive and do Denis L. Baggi, a computer scientist and a between related aspects, or parameters, not allow control during the perfor- musician, is guest editor of this special issue. can be easily modeled, modified, and mance, as the knobs in Neurswing do. His biography and photo appear on page 9.

HARP: A System for Intelligent Composer’s Assistance

Antonio Camurri, Corrado Canepa, Marcello Frixione, and Renato Zaccaria DIST - Dept. of Communication, Computer, and System Sciences, University of Genoa, Via Opera Pia 11/A, I-16145 Genoa, Italy.

ystems for computer-aided goals. The system can generate new rithms) and particular analysis process- t pieces of music as well as manipulate es. The symbolic subsystem, a declara- composition are generally based on tools and languages designed existent ones on the basis of composer- tive symbolicenvironment, stores high- s er level scores, composition rules, for low-level manipulation of music supplied material. The system also pro- scores and composition algorithms.’ In vides capabilities for formal analysis of definitions in general, and descriptions this article, we introduce HARP (Hy- both music and sounds, so it can extract of pieces of music. The symbolic sub- brid Action Representation and Plan- information useful in subsequent syn- system is based on a multiple-inherit- ning), a prototype high-level system for thesis processes. ance semantic network formalism de- computer-aided composition. HARP represents and manipulates rived from KL-One.2 There is a formal We think of the system as providing music knowledge using a twofold for- link between the two subsystems. For intelligent composer assistance. HARP malism. An object-oriented concurrent example, if a composer asks the seman- can store and process music and sounds, environment, the analogicalsubsystem, tic network to generate a particular in- and carry out plans for manipulating provides procedures to manage the stance of a music object, the system this material according to composers’ sound itself (samples, codes, and algo- automatically “fires” the appropriate

64 0018.9162,91/0700-006450, .OO 0 1991 IEEE COMPUTER procedures at the analogical level. Figure 1 shows part of the HARP symbolic knowledge base: Ellipses correspond to concepts in the taxonomy, double arrows are “is-a” links, and arcs with small boxes correspond to roles (slots, relations between pairs of concepts). For example, a music action is-a music fact, and one of its roles is intensity, whose value is amplitude. In other words, the intensity of a music action is an amplitude, as shown in Figure la.

The HARP knowledge base. We defined two basic music entities in the system: the music action and the compositional action (see Figure 1). Music actions can represent musical material at different levels of abstraction, from the sonological level (low-level descriptions of sounds) to the whole musical form of a piece (for example, fugue or uamy sonata). r music_intetval / // Compositional actions are meta- actions; that is, they are the “manipula- I ,’ rCompo$ional) tors” that operate on music actions (both classes and instances) to produce new music actions or to perform analysis tasks useful for subsequent manipulation. Compositional actions let composers manipulate music actions according to their objectives. For example, a composer can operate on a music action at its sonological level for the “tuning” of particular sound features. While plan- ning the overall structure of a part, the -“T Counterpoint ) 1 composer can seethe same music action as a more abstract, high-level symbolic entity and work on its possible relations with other music actions. The composer can introduce into the // \ counterpoint ) I system subclassesof music actions and instances. Subclasses are reusable objects, skeletons of music scores,or sound definitions. Instances correspond to complete, individual objects that can be heard reproduced by the HARP sound Figure 1. Part of the HARP symbolic knowledge base. output channels. HARP’s framework does not refer to a particular musical style or context. Each context requires the creation of suitable definitions in the symbolic part and the related analogical procedures. consequent, as shown in Figure lb. Suc- Basic features of music actions in- For example, in the context of Western cessively, the composer can introduce elude tonal music, a composer can define the - or ask the system to generate - a subclass canon with its structure: its particular canon characterized by a giv- l a temporal connotation (the begin component music actions antecedentand en antecedent and a suitable consequent. and end roles), allowing the com-

July 1991 65 PROJECT OVERVIEWS

poser to relate music actions to each ing to the system queries whose results tion. Here is another example: A canon other in the time domain, and can be graphically presented. Compos- is generated-by an imitation process

l a set of relations, such as dynamic ers can make queries on classes and (Figure lb). A composer can define a evolution, timbral and density con- subclasses, the definitional part of the particular canon subclass, say per dimi- tent, pitch, metrics, and rhythmic network, and on instances, that is, indi- nutionem canon, as generated-by a sub- properties. vidual music entities. class of imitation, sayper diminutionem The system shows subsets of the mu- imitation. The composer can hook more Assume that a composer wants to sical material, allowing different views detailed procedures to the second con- specify in a compositional action that a of the same material. In Figure 1, for cept, and one of them can implement a music action (a part of a piece) N be example, the query “show all objects proper diminution algorithm. By que- generated in relation to, say, the evolu- from which canon inherits properties” rying a proper consequent for this tion of another music action M, ex- results in a display that colors some canon, the composer can generate an pressed simply as a score fragment. An network elements gray to give promi- instance ofper diminutionem canon that analogical representation of an instance nence to a path in the taxonomy. An- starts from the above concepts and an of M is formed by a score fragment other simple query, “show all about the instance of antecedent. The system an- (hooked to the concept M) and by a set object music action except temporal swers by activating the proper proce- of functions, or methods, hooked to its relations,” results in a display of the dure hooked to per diminutionem imi- relations, which extract the related fea- concept music action with all its rela- tation, passing as parameters the ture from the score. The composer can tions except the begin and end inherited antecedent instance. (We produced ex- query the intensity of M, which corre- from music fact (see Figure la). amples 4 and 5 in the available audio sponds to a call to the intensity method recording by introducing particular can- of M, whose result is a presentation of Reasoning mechanisms. The sym- on definitions using suitable subcon- the behavior of the dynamics of M, ex- bolic subsystem is equivalent to a sub- cepts of canon and imitation. See the tracted from its score fragment. In this set of first-order logic in an artificial “Audio Examples” box on page 54 for way, the composer carries out a reason- intelligence system. The system answers information on how to obtain the re- ing process, partially in procedural form. queries using the symbolic subsystem cording.) No a priori assumption has been super- formal deductive apparatus. We have imposed on the leading features of mu- implemented a classifier2 to organize Force fields - A recorded example. sic actions. In classical Western music, concepts in the taxonomy. An important set of analogical descrip- the melodic aspect could predominate Music actions have “hooks” to score tions in HARP is based on the meta- in the manipulation of music actions, fragments, and compositional actions phor of force fields. These descriptions but the system lets the composer mod- have hooks to code “chunks.” This un- let the composer think of and perform a ify or redefine their relations and “ma- derlying level of procedural definitions set of compositional actions in terms of nipulators” according to his or her needs. constitutes the analogical level of HARP, the intuitive natural dynamics of navi- A compositional action is character- which is structured into a sonological gation in attractor fields, instead of us- ized by a procedural description of its component and a simulative component. ing a rule base. behavior and a temporal connotation The simulative component is formed by Force fields play a significant role in (with a time axis defined by the mbegin a set of procedures that acts as a coun- the generation of the examples on the and mend temporal roles, different from terpart to the symbolic deductive appa- available recording. The sixth music the time axis of music actions). As with ratus and integrates the inference capa- example is generated starting from a any other object defined in the system, bilities of the symbolic system. Both the phrase (the incipit), a definition of the composers can define both subclasses simulative and sonological components sound parameters, and two concurrent and instances of compositional actions. are driven by the symbolic level sub- force fields operating as follows: The Composers can define compound system, because proper activations of a initial incipit (easily recognizable as the compositional actions in the system as node in the net correspond to calls to its beginning phrase in the example) is sets of given compositional actions whose hooked code. imitated with continuous timbral varia- execution is ruled by proper constraints The system also provides mechanisms tions. At the same time, the aspect of on their temporal relations. for visual interaction with the analogi- pitch intervals progressively degrades, cal level of music information represen- giving place to the timbral aspect. In System interface. HARP lets com- tation. The mechanism is based on mu- HARP terms, there is a force field oper- posers use different visual interfaces to sic notation, sound graphic descriptions, ating on the timbral space, which per- communicate with the system. It pro- and visual metaphors. Here, queries forms a continuous timbral disassem- vides a visual presentation of the se- correspond to the activation of code bling according to its field function. mantic network formalism. Composers modules, with the passing of proper Concurrently, another force field, op- interact visually or textually with this parameter values. An example is the erating on the pitch axis, gradually su- part of the system by introducing or query on the intensity of the music ac- perimposes a variation of the initial in- editing elements in the net, and specify- tion M presented in the previous sec- cipit. The result is that the application

66 COMPUTER of this force field (which has a function eluding the ability to merge different 2. R.J. Brachman and J.G. Schmolze. “An knowledge baseswhile maintaining glo- Overview of the KL-One Knowledge with local minima on particular pitches) Representation System,” Cognitive Sci- to the incipit causes a gradual pitch bal consistency and detecting conflicts. ence. Vol. 9. No. 2, 1985, pp. 171-216. stretching, which conduces to the final We will implement other knowledge melodic contour. basesin the near future. Problems in the The metaphor of force fields is useful formalism, such as how to store consis- Antonio Camurri is a research computer scientist in the Department of Communication, for describing continuous changes in tent multiple views of music entities, Computer. and System Sciencesat the Uni- music actions. A composer can use sev- are areas for future research. H versity of Genoa. He studied music at the eral force fields to easily model com- Conservatory of Genoa and Parma, and re- plex behaviors (such as continuous tim- ceived his LZrurea degree and PhD in electronic and computer engineering from the bra1evolutions), which are very difficult University of Genoa. to model as rules. Force fields also give Acknowledgments a different viewpoint on music entities and provide powerful manipulation We thank Salvatore Gaglio, Steve Smo- Corrado Canepa creates electro-acoustic liar, and the referees for their useful sugges- primitives. compositions in his electronic music studio tionsandcommentsonthisarticle,andArthur and collaborates with the Departmenl of With HARP, composers can formu- Lomas for the improvements he made to our Communication. Computer, and SystemSci- late such complex analogical queries, English. Others who helped design and im- ences at the University of Genoa on the involving complex activations of inter- plement HARP include Claudio Massucco, description and processingof sound via soft- Carlo Innocenti, Armando Allaria, and Mau- ware. acting procedures, and build high-level rizio Ciccione. music actions. This work was partially supported by the Progetto Finalizzato Sistemi Informatici e Marcello Frixione is a PhD student in the The other examples. The recording Calcolo Parallel0 of the Italian Research Department of Philosophy at the University includes two groups of musical exam- Council, under Grant No. 90.00716.PF69, of Genoa and collaborates with the Depart- Pos. 115.14231, and by MURST Special ples: a simple improvisation environment of Communication, Computer, and Projects (Italian Ministry of University and System Sciences. He received his Ldurea ment in an Afro-American jazz style, Research). degree in philosophy from the University of and three excerpts from “Anceps Ima- Genoa. go,“a contemporary piece for two harpsichords and computer-generated mu- Renato Zaccaria is an associateprofessor sic (composed by Corrado Canepa in References of computer science in the Department of 1989). Communication, Computer, and SystemSci- The first three musical examples are 1. G. Loy and C. Abbott. “Programming ences at the Unviersity of Genoa and a re- simple improvisation fragments. The Languages for Computer Music Synthe- search fellow at the Institute of Electrical system’s goals were to generate and sis.PerformanceandComposition, “ACM Engineering. He received his Ldurea degree Computing Surveys.Vol. 17, No. 2, June in electronic engineering at the University of transform suitable musical phrases (the 1985, pp. 235-265. Genoa. soloist improvisation) to follow the composer’s defined harmonic schemasin an Afro-American jazz style (blues, bal- lad, and so on). We stored a set of pre- defined music phrases in the system in terms of assertional constants, or sub- Composition Based on concepts of music action. Composi- Pentatonic Scales: tional actions take as inputs subsets of these phrases for developing improvi- A Computer-Aided Approach sations. Examples 4 and 5 are two excerpts from the first part of “Anceps Image,” Yup Siong Chua in a canon form. Example 6 is described College of Computer und Information Sciences above. For the recording, we replaced University of North Florida the two harpsichords with computer Jacksonville, FL 32216 music of a similar timbre. We used cmusic as the low-level language to describe the computer-generated scores. The HARP software prototype is entatonicscales-musicalscales er-aided composition. This approach implemented in Prolog and C, running > ’ of five tones - are used in the can be likened to the study of painting on Apple Macintosh and 386-based P music of many cultures. My using a limited “palette” and may be of machines under Microsoft Windows. project uses pentatonic scales to ex- interest to readers with a background HARP has several other features, in- plore some basic techniques of comput- in computing and music who want to

July 1991 0018.9,62/91/070~10067$01 00 @ 199 I IEEE 67 PROJECT OVERVIEWS

experiment with computer-aided com- Examples of pentatonic music and mas are optional) for the first few bars position. scales. The examples in Figure 1 are of “Old MacDonald Had A Farm” is 11 excerpts of pentatonic music from many 11,112,1111,2-11,1111,112,1 Notation (pitch and duration). The cultures of the world. Two pentatonic 1 1 1, 2 -1 .5 .5, 1 1 1 .5 .5, . . . Other 88pitches on the piano are usually iden- scales are represented in these exam- considerations such as grace notes, n- tified (from lowest to highest) as AO, ples, namely (C4, D4, E4, G4, A4, C5) tuplets, staccato, and fermata can be A#0 (or BbO), BO, Cl,. . . , C4 (middle (variously transposed) and (C4, E4, F4, handled similarly using the numeric C), . . . , C8. The MIDI (Musical Instru- A4, B4, C5). These are not the only approach. ment Digital Interface) standard uses pentatonic scales, however. Other ex- A rhythm vector can be specified as integers (O-127) to identify a set of 128 amples of pentatonic scales include (C4, shown above or with the help of a com- pitches. The pitches AO-C8 are mapped D4, E4, G4, Bb4, C5), (C4, D4, E4, F4, puter program equipped with a pseudo- to the MIDI numbers 21-108. While A4, C5], and (60, 62.40, 64.80, 67.20, random number generator. The com- many modern MIDI synthesizers have 69.6,72]. (At the end of a scale the first poser specifies a biased distribution (or, alternate tunings, we will assume that note is usually repeated an octave high- simply, distribution) of duration values the default tuning is equal tempered er.) The last scale, given in numeric to reflect his or her preferences for the based on 440 Hz for A4 (MIDI No. 69) MIDI notation, is called the equidistant part of the composition being created. so that we will have a fixed frame of pentatonic, an idealization of a popular This distribution is expressed in the form reference. Indonesian scale known as the slendro. of a sequence of ordered pairs (k,, D,), i Numeric notation for pitches has def- = 1, 2, . , n, where k, is a positive inite advantages over the conventional Basic composition. The project illus- integer giving the count of the corre- names in dealing with computer manip- trates some basic techniques for mak- sponding duration value D,. The proba- ulation of pitch data. The relationship ing music under the guidance of a com- bility of duration D, being selected is between frequency F (in hertz) and poser and with assistance from a equal to its count k, divided by the sum MIDI note number M is given by the computer, using only a small set of of all k,, i = 1, 2, . . , ~1.For example, formula pitches for simplicity. A musical com- Rdistl = ( (2.25) (7 5) (4 1) (2 2) (14)) position and an English composition specifies a rhythm distribution that can F = 27.5 x .Y2’ share some structural similarities. The be regarded as an “urn” containing two hierarchy of an essay or a book (alpha- 16th notes, seven eighth notes, four where S is the 12th root of 2 (frequency bet, words, sentences,paragraphs, chap- quarter notes, two half notes, and one ratio of a semitone or half step) and 27.5 ters, and volume) is analogous to that of whole note. Items placed in a rhythm Hz is the frequency associated with AO. a musical composition, which starts with urn are not limited to duration values of In other tuning systems, musicologists an alphabet of pitches and duration val- single notes. Rests and rhythm phrases divide the semitone into 100 equal (in ues. These are combined to form (short rhythm vectors) are also permit- terms of frequency ratio) intervals called phrases (motives and themes). Devel- ted. For example, Rdist2 = ( (4 1) (2 (.33 cents, which can be easily incorporated opment and variation of these basic .33 .34)) (2 -2) (5 (2 1 1)) (5 (1.5 .5 1.5 into the numeric MIDI notation by add- phrases give rise to other phrases. A .5)) ) is a rhythm distribution in which ing two decimal places. For example, a sequence of related phrases forms a the probability of choosing a triplet (.33 pitch halfway between C4 (60) and C#4 movement, and one or more movements .33.34) as a unit is 2 out of 18, or l/9. A (61) is said to be 50 cents above C4 and constitute a composition. Additionally, function RV assists in generating a is given a MIDI number of 60.50. The an alphabet of chords permits the for- rhythm vector, given a rhythm distribu- equation above still applies for noninte- mation of chord progressions, the har- tion (Rdist) and the total desired dura- gral M. In addition to the computation- monic foundation of a composition. tion (TD) of the phrase. al flexibility it offers, the MIDI note- Elementary computer-aided compo- numbering system facilitates the sition techniques’ include random sieves, Pitch. A sequence of pitches repre- translation of musical data into stan- biased choices, and Markov processes. sented in numeric MIDI notation is dard MIDI song files for playing with For simplicity, we concentrate on the called a pitch vector. Each element of a MIDI synthesizers. generation of musical phrases with em- pitch vector can be specified explicitly, We use the number 1 to represent the phasison the two fundamental attributes or a distribution approach can be used relative duration of a quarter note. The - rhythm and pitch - using an ap- to generate pitch vectors containing exact duration is a function of the tem- proach that can be characterized as a composer-imposed biases. Consider a po - for example, the number of quar- guided random process based on distri- given pitch set (a sequence of MIDI ter notes per minute. Other durations butions imposed by the composer. numbers in increasing order) and a dis- follow naturally: 0.5 for an eighth note, tribution of skips (defined as relative 2 for a half note, and so forth. Rests are Rhythm. A rhythm vector is defined movements within the pitch set). For notated as negative durations; for ex- as a sequence of real numbers repre- example, Psetl = ( 60 62 64 67 69 72 ) ample, -1 for a quarter rest, -1.5 for a senting relative note and rest durations. specifies a pitch set corresponding to dotted quarter rest. For example, the rhythm vector (com- one octave of a pentatonic scale. Pset2 =

68 COMPUTER “Auld Lang Sync” Scottish air

“Hard Times Come Again No More” Stephen C. Foster

“Alberta, Let Your Hair Hang Low” American folk song

“Sakura, Sakura (Cherry Blossom)” Japanesefolk song

“Warriors’ Song” Ba-Ronga. South Africa

“Evening in the Country” B. Bartok

“The Power of the Snake” Amerind Suite -Henry Cowell

“Deep River” African-American spiritual

“Xiao Lu (The Trails)” Chinese folk song

“Seedling” (Hungarian folk song) Collection: Z. Kodaly

“Loch Lomond” Scottish air

“Teacher’s Blues” American folk song

Figure 1. Excerpts of pentatonic music from many cultures.

(55576062646769727476)specifies discard that skip and pick another one. count and S, is a small integer indicating the same scale with a wider range. A Another approach is to transpose the a skip. For example, the skip distribu- skip of +l means move up one note on resulting out-of-bounds pitch (octave tion SKPdist = ( (1 -5) (2 -4) (4 -3) (4 - the scale from the current note; 0 means up or down) so that it is within the pitch 2)(6-1)(40)(61)(52)(83)(44)(25)1 repeat the current note; -2 means move range and still satisfies the maximum indicates that the probability of making down two notes on the scale. skip limitation that the composer may a -2 skip is 4 out of 46, or 2123;a +5 skip, Depending on the current note and have set. 2 out of 46. or 1123:etc. Function PV the skip size, there is a possibility of A skip distribution is specified as a generates a pitch vector based on a giv- going “out-of-bounds.” One easy way sequence of ordered pairs (k,, S,), i = 1. en pitch set, a skip distribution, starting to handle an out-of-bounds skip is to 2, . , II, where k, is a positive integer and ending notes for the pitch vector,

July 1991 69 I’FOJECT OVERVIEWS

MORNEVG STROLL p. 2

MORNING STROLL

Y. s.cllua J =loo

and the rhythm vector it is to match. posers constantly create new (some- pitch vector (inversion). Although skip sizes are controlled by times inexplicable) ways to develop l Transpose the pitch vector. the skip distribution, interval sizes (gen- and manipulate themes. A few possi- l Retain the pitch vector and change erally not the same as skip sizes) can be bilities are the rhythm vector, or vice versa. checked and modified as desired. *Take portions of the theme and l Reverse the order of elements in a develop each portion individually. Development. A phrase used as a pitch vector, a rhythm vector, or both theme can be stated, restated, manipu- (retrograde motion). A composition is formed through lated, and otherwise modified. Com- *Invert the intervals or skips of a the creative sequencing of phrases ac-

70 COMPUTER which was imported into music publish- ing software (Escort and Score) to produce the score. The score accurately reflects the pitch and rhythm produced by the program without additional man- MORNING STROLI. P. 3 ual intervention. At this point, the composer can revise parts (or all) of the composition. Other elements of the composition, such as dynamics and timbres, can be added as well.

Opportunities. The declining cost of computers and synthesizers has made computer-aided composition accessible to anyone with programming skills and musical training. While off-the-shelf products help the computer musician play and print music, to effectively experiment with innovative computer- aided compositional techniques, the computer musician must be very closely involved in translating these techniques into algorithms. The possibilities are endless, even if we restrict ourselves to generating and manipulating musical phrases. For readers who find this introduc- tory article and its illustrative techniques of interest, I highly recommend the survey by LoY.~ It is a good starting point for those wishing to learn more about computer composition. n

Figure 2. Score of a short computer- References aided composition, 1. R.F. Moore, Elements of Computer “Morning Stroll.” Music, Prentice Hall, Englewood Cliffs, N.J., 1990.

2. G. Loy, “Composing with Computers - A Survey of Some Compositional For- malisms and Music Programming Lan- guages,” in Current Directions in Com- puter Music Research, M.V. Mathews and J.R. Pierce, eds., MIT Press, Cambridge, Mass., 1989, pp. 291-396. cording to some compositional form. A Biases in both the rhythm and skip pat- computer can be used to experiment terns are evident upon examining the with ideas that may be too cumber- score (see Figure 2). It may also be some to handle manually. evident that the parts borrowed rhythm and pitch skip patterns from one anoth- Yap Siong Chua is an associate professor of computer and information sciences at the A short composition. A short com- er. I wrote a program to generate and University of North Florida. His research position based on the scale (F3, G3, manipulate the phrases. The pitch and interests include computer graphics, image A3, C4, D4, F4) was generated using rhythm data thus produced were trans- processing, computer-aided geometric de- the techniques described in this article. lated into a standard MIDI song file sign, and computer music.

July 1991 71 PROJECT OVERVIEWS

Composing by Musical Analog: A Look at Planetary Orbits

Robert Keefe School of Music, Ithaca College Ithaca, NY 14850

rom the time of Pythagoras, the of complex calculations. The formulas tion (see Voss and ClarkeZfor an expla- term number made audible has themselves may be traced back to the nation of l/f distribution). F come to mean the relationship works of the astronomer Johannes (2) Since each planet has its own or- between the pure world of number and Kepler (1571-1630). bital speed, any difference between ad- the sounding world of music. Today, a The orderliness and motion of the jacent latitudinal readings will be new numerology is generating musical universe has always intrigued me, and I greater with faster moving planets and compositions using models from proba- felt that this order could be transferred lesser with slower moving planets. This bility statistics, fractal geometry, and to a musical composition. I had been characteristic is used to generate the other nonlinear and chaotic systems. writing computer-assisted composition durational and rhythmical activity of a This new way of making numbers audi- programs since 1982 with some very composition. Conversely, the faster the ble is now referred to by some compos- satisfying musical results. Hence, I de- orbit of a planet, the slower the rhyth- ers as a mapping technique or mapping. cided to combine my programming skills mical activity, and the longer the dura- Mapping pairs the parameters of an with my interest in the planets and cre- tion of a pitch. object or event to the parameters of a ate a modern-day music of the spheres. (3) There is a discernible difference musical composition, thus creating a My celestial coordinates are made in latitude among the planets. Mercu- musical analog. The object or event can audible through an interactive Pascal ry’s swing in latitude for any given year occur in nature or be man-made and computer program that generates ce- might be between -4.91 degrees and will be represented by a series of num- lestial coordinates for all the planets +3.36 degrees. Saturn’s swing in lati- bers, letters, or symbols. Compositions and the earth’s moon as seen from any tude for the same year might be be- based on such musical analogs are found point on earth for any given time frame. tween -1.08 degrees and -0.48 degrees. in Heitor Villa-Lobos’ New York Sky- The program allows me to map these Due to these latitudinal variations, each line Melody (1940) and Montanhas do coordinates onto the musical parame- planet develops its own melodic profile Brasil(1944), which used a scaled-down ters of pitch, dynamics, rhythm, and when mapped onto a synthesizer key- version of the 1940’s New York skyline duration of a composition in a MIDI board. Mercury and Venus generate and a mountain range in Brazil, respec- (Musical Instrument Digital Interface) disjunct melodies while Mars, Jupiter, tively; John Cage’s Atlas eclipticalis environment, producing a composition and Saturn generate conjunct melodies. (1961-62) and Luigi Dallapiccola’ssicut that is a pure musical analog or choos- The melodies generated from Uranus, umbra... (1970), both of which employed ing only a few musical parameters cre- Neptune, and Pluto center on only a few star maps and star constellations; Charles ating a composition that is a partial pitches within a limited range (do, re, Dodge’s Earth’s Magnetic Field (1970), musical analog. mi, fa, sol, for example). where numbers representing the earth’s magnetic field created the musical ana- Compositional environment. Three Once the celestial data is generated, I log: and Larry Austin’s Canadian Coast- factors influenced my use of The Ephe- map the latitudinal coordinates onto a lines (1981), where the coastlines of merides Tables in a compositional envi- pitch letter name, an accompanying ac- Canada were the object. Some compos- ronment: cidental (a note foreign to a key indi- ers have even used the Dow Jones In- cated by a signature), and an octave dustrial Average and DNA sequences (1) Each planetary orbit is unique displacement (range on the keyboard). to generate musical analogs. unto itself, similar to itself,’ and similar The system I devised performs three My interest in numerical, nonmusical to other planetary orbits because they operations on each latitudinal reading. structures began in the fall of 1985 after all adhere to Kepler’s three laws of plan- For example, if the latitude coordinate I found a set of books called The Ephe- etary motion. When a planetary orbit is is 0.25, the following sequence of events merides Tables. These volumes contain mapped onto selected parameters of a occurs: celestial latitudinal and longitudinal composition, common musical elements coordinates for the sun, the moon, and among single orbits, adjacent orbits, and (1) The rightmost digit “5” from 0.25 naked-eye planets as seen from the bib- nonadjacent orbits might be readily maps onto a pitch letter name from A to lical city of Babylon, the site of one of heard. These characteristics lend them- G. Since our number system is base 10 the first observatories. The coordinates selves quite readily to hierarchical rela- and there are seven pitch letter names in the tables are generated from a series tionships and also exhibit a l/f distribu- (not including accidentals), some pitch

72 001%9162/9110700-0072$01 000 1991 IEEE COMPUTER r

-8

19 4 20 5 .?I 7 23 8 24 10 26 I1 27 12 28 I5 3l 16 2 18 3 19 5 21 6 2.2 7 23 JU(Y Aug. Sept. Ocf NOL! Dec. Jan. Feb. Mar Apr May June July Aug. Sept 1595 1596

Figure 1. A partial musical analog of “Venus.”

names must map onto more than one immediately linked to the pitch letter might prove beneficial to someone us- digit. This scheme is user-variable and a name. I use only three types of acciden- ing a microtonally based tuning scheme. typical mapping might be tals (sharps, flats, and naturals, leaving out double-sharps and double-flats), so Durational schemes. As the pitch al- A=Oorl accidentals and digits must also be dou- gorithm is running, a durational algo- B=2or3 bled up. One mapping scheme may be rithm maps latitudinal or longitudinal C=4or5 coordinates onto composer-defined du- D=6 Sharp=0,1,2,or3 rations. The composer has three types Flat = 4,5, or 6 of durational schemes from which to E=7 choose: F=8 Natural = 7, 8, or 9 G=9 (1) Taking the absolute value of ad- (3) Finally, the entire latitude num- jacent latitudes or longitudes as deter- The 100th’s place of the latitude co- ber maps onto an octave displacement mined by the formula abs(lat2-latl) or ordinate also acts as an on/off switch or (vis-a-vis a place on the synthesizer key- abs(lat2ilatl) and using that as a dura- volume control, and a digit may gener- board). The wider the swing in latitude, tional value in seconds or fractions of a ate a rest instead of a note. The com- the greater the range covered on the second. poser can also introduce more rests if musical keyboard. If the latitude read- (2) Using a coordinate that falls be- desired. Introducing rests allows the gen- ings between 0.00 and +l .OOare assigned tween a specified range and mapping erated musical phrases to be phrases to the range between middle C and C an the value onto a designated rhythmic instead of streams of notes. Additional- octave higher, the reading 0.25 would value. For example, latitudes that fall ly, the longitude of a planet might also fall within the third octave. The entire between +l.OO degrees and +1.99 de- generate loudness or dynamics. If a plan- mapping process would yield the pitch grees may be assigned to a quarter-note et reaches 360 degrees in longitude, the C#3. rhythmicvalue. Latitudes between +2.00 loudness is 100percent (a dynamic level and +2.99 may be given a half-note val- of fff, or fortissimo); if the planet is at In an earlier algorithm, I reversed the ue, and so forth. 180 degrees longitude, the loudness roles of the numbers. The tenth’s place (3) Superimposing a constant rhyth- would be 50 percent (a dynamic level of acted as the pitch generator with very mic value over all pitches such as all mf, or mezzo-forte), and so forth. unsatisfying musical results. The musi- 16th.notes, all eighth-notes, or other (2) The tenth’s place of the latitude cal output centered on only a few pitch- rhythmical values. number (“2” of 0.25) maps onto an acci- es within a limited range and with many dental (also composer variable) that is chromatic inflections. However, this Figure 1 shows “Venus” (from my

July 1991 73 P?RO JECT OVERVIEWS p A

Mercury

Venus

Moon

Mars

Jupiter

Saturn

Uranus

Neptune

Pluto

Figure 2. An excerpt from the Three Movements for Imaginary Dancer, movement 1: “De Stella Nova,” a polyphonic version of the planetary analog.

composition The Earth is the Circle Which pact disk available in conjunction with planetary data. (The tape and CD con- is rhe Measure of All (1988)) as a partial this issue (see the order blank on p. 9). tain an excerpt from Three Movements musical analog because it only uses the Figure 2, an excerpt from my compo- for Imaginary Dancer.) pitch and volume portion of the algo- sition Three Movements ,for Imaginary rithm. (Figure 1 shows only the first 10 Dancer. movement I: “De Stella Nova” Decisions on options. Creating a mu- seconds of the work.) A constant dura- (1991). is a polyphonic version of the sical analog from planetary orbits re- tional value of Ihth-notes is superim- planetary analog. In this excerpt, all quires a sensible understanding of the posed over all computer-generated pitch- eight planets (as well as the earth’s moon) simple physical properties of our solar es. The upper part of Figure 1 shows the are made audible as they are panned system and incorporating them into the latitude of Venus’ orbit from July 19. across the stereo listening field from compositional process. The more I know 1595. to October I. 1596, as seen from left to right. Mercury to Pluto, resulting about the celestial model, the better Graz. Austria. The lower part of Figure in a modern-day music o,f the spheres equipped I am in making a musical an- 1 displays the musical analog of the plan- composition. Figure 2 is an example of alog that reflects the structure of the etarv orbit. “Venus” can be heard in its a pure musical analog, where all musi- model. I then have the option of decid- entirety on the cassette tape and com- cal parameters are generated by the ing whether to make a pure musical

74 COMPUTER analog where all of the musical param- NANYANG TECHNOXGICAL UNIVERSITY eters are generated by the computer Singapore program (as in “De Stella Nova”) or a (To be established on 1 July 1991) partial musical analog where I take ar- tistic control over the computer music SCHOOL OF APPLIED SCIENCE In July 1991, the Nanyang Technological Institute (NTI) will be reconstituted and renamed the Nanyang output (as in “Venus”). TechnologicalUniversity(NTU). It will conductawiderangeofcoursesatdegreeand postgraduatelevels. This is by far the most difficult deci- The School of Applied Science currentl offers undergraduate degree courses in COMPUTER sion to make when dealing with com- TECHNOLOGY and MATERIALS ENGI f.YEERING. Applications are Invited from suitably qualified persons to fill teaching positions in the following areas. puter-generated or computer-assisted music and one that I face each time I COMPUTER TECHNOLOGY compose. I cannot codify my intuitive 1. Microprocessor-Based System Design 10. Image Processing/Computer Vision 2. Computer Architectures 1 I. Artificial Intelligence process in a few paragraphs, but I can 3. Advanced Microorocessors 12. VLSI Svstem Design say that my compositional creativity is 4. Computer Comniunications 13. Robotics 5. CAD/Computer Graphics 14. Compiler Design influenced by all the different types of 6. Real-Time Systems 15. Distributed Computing music I have listened to and/or per- 7. Computer Peripherals 16. Fault-Tolerant Computing 8. Control 8 Instrumentation 17. Performance Prediction 8 Analysis formed over my lifetime. Preference will be given to applicants with expertise in both hardware and software The fine line between adjustment and acceptance of computer output is con- MATERIALS ENGINEERING stantly being redrawn with each new I. Metal Casting 9. Modern Analytical Examination of composition.However,myintuitivepro- 2. Powder Metallurgy Materials 3. Welding IO. Metal Working and Forming cess seems to maintain a series of sub- 4. Electronic and Magnetic Materials 11. Heat Treatment conscious checks and balances over each 5. Surface Treatment 12. Polymer Science and Processing 6. Corrosion 13. Composite Materials composition, whether it be a computer- 7. Failure of Materials 14. Ceramics generated tape piece or a work for acous- 8. Materials Selection and Design 15. NDT Techniques 16. All aspects of Physical Metallurgy tical instruments3 Nonetheless, even my Considerations Qualifications intuitive process may be altered with a AppEcantsshouldpossessagoodHonoursdegreeandarelevanthigherdegree.Preferencewill be iven new stimulus or input. One of the more to those with teaching/research or industrial experience. Appointees will be expected to initiate an 8 take fascinating aspects of computer- part in research programmes and to participate in academlo’profes9ionaI activities that complement the industrial development of Singapore. assisted composition is that sometimes Gross annual omduments range as follows: the computer gives you unexpectedly Lecturer : S.$ 53,160 - S$ 64,200 pleasant musical results that might not Senior Lecturer : SS 58,660 - St1 00,310 have been otherwise realized. W Associate Professor : S$ 88,650 - S$I 22,870 Profassor : St1 06,670 - S$146,970 (UStl.09 = SS1.73 approximately) Inaddition totheabov?, the InstituteadoptstheGovernmenrspracticein thepaymentofavariable bonus, the quantum of which IS tied to national economic performance and has, in past years, ranged from 1 to 2’18 months of December salary. References The commencing salary will depend on the candidates’ qualifications, experience, and the level of appointment offered. 1. C. Dodge and C.R. Bahn, “Musical Frac- Leave and medical benefits will be provided. Depending on the type of contract offered, other benefits tals,” Byte, June 1986, pp. 185-196. include: provident fund benefits or an end-of-contract gratuity of 25% of the staff member’s last drawn monthly salary for each completed month of service, a settling-in allowance, subsidised housing, education allowance up to a maximum of S$30.000 per annum, passage assistance, baggage allowance 2. R.F. Voss and J. Clarke, “‘l/f Noise’ in and car loan. Music: Music From l/fNoise,” J. Acous- The Institute encoura es its staff to undertake outside consulting work of a specialist nature. They are tical Society of America, Jan. 1978, Vol. permitted to earn an 1 retarn such consultation fees up to a maximum of 60% of their gross annual 63. No. 1, pp. 258-263. emoluments in any one calendar year.

3. R. Keefe, composer, “The Ephemerides TheCampug nstrtute has a modern campus with upto-date facilities for teachin and research as well as for Harp and Percussion: Moon, 16.50. residential and recreational facilities for staff and students. It has one of the ar est high-speed campus wide network of computing facilities for supporting teaching, research andB o?r ice automation in this 1657,” Centaur Records, CDCM (Con- region. Over a hundred en ineering workstations and a thousand PCs are sup orted on NTlnet which sortium to Distribute Computer Music) runs into all academic sta w offices and laboratories. Large computation and h7 e servers available on Computer Music Series. Vol. 9, recorded NTlnet indude three large VAX882Os. one VAX6530. one VAX6200. five VAX3500 servers and ten 1991. MicroVAX II servers. In addition dedicated CAD/CAM facilities are available in the various Schools. The Institute also provides its users with a communication link via NTlnet to a supermmputer installed in the Singapore Science Park. Further information on the above may be communicated to the Institute through BITNET to:TFWANG@NTIVAX Robert Keefe, who teaches music theory at Ithaca College, has been using the computer Candidates wishing to be considered should write to: as an additional compositional resource since THE DIRECTOR 1982,and his compositional output has been PERSONNEL DEPARTMENT solely directed ai creating muiical analogs NANYANG TECHNOLOGICAL INSTITUTE since 1986.He earned MM and PhD degrees NANYANG AVENUE, SINGAPORE 2263 at the University of North Texas. giving their curriculum vitae and the names and addresses of three referees.

July 1991