From: AAAI Technical Report SS-93-01. Compilation copyright © 1993, AAAI (www.aaai.org). All rights reserved. EXPERIENCE-BASED MUSIC COMPOSITION CARL WESCOTT and ROBERT LEVINSON Department of Computerand Information Sciences University of California, Santa Cruz Santa Cruz CA 95064 USA 1 Overview 2 Related Research "If a complex structure is completely "[Musical fragments] that please me unredundant - if no aspect of its struc- I retain in memory, and am accustomed, ture can be inferred from any other - then as I have been told, to hum them to my- it is its ownsimplest description. Wecan self. If I continue in this way, it soon exhibit it but we cannot describe it by a occurs to me how I mayturn this or that simpler mechanism." - Herbert Simon, morsel to account, so as to make a good The Architecture of Complexity, p. 478. dish of it, that is to say, agreeably to the rules of counterpoint, to the peculiar- ities of the various instruments, etc. All Humancreativity produces new forms, or this fires my soul, and, provided I amnot patterns, from previous experience, in such di- disturbed, my subject enlarges itself, be- verse activities as composingmusic, writing prose, comes methodized and defined, and the and playing chess. We believe that humans have whole, though it be long, stands almost domain-independent pattern-based neurobiologi- complete and finished in my mind, so cal mechanisms that provide the foundations for that I can survey it, like a fine picture creativity, and propose that systems which com- or a beautiful statue, at a glance. Nor bine and alter simple patterns to model complex do I hear in nay imagination the parts systems can effectively simulate creative intelli- successively, but I hear them, as it were, gence. all at once." Wolfgang AmadeusMozart, To study these mech- in Vernon, Creativity, p. 55. anisms and experience-based creativity, Smurph, System for Musicomposition Using Repeated Pat- The original computer music composi- tern Hierarchies, has been developed. Smurph is tions were stochastic music composed using ran- intelligent software that learns to composemusic dom numbers or processes [Hiller & Isaacson, based on discovery of repeated musical patterns 1959] [Xenakis, 1971]. Often, these aleatoric de- and its creative combinations and variations of vices were interactive compositional tools to assist those patterns. The goal is for Smurphto learn to the humancomposer, such a.s Koenig’s mentor sys- compose original works of music entirely from its tems Project One and Project Two [Laske, 1981]. experience of ’listening’ to music and its analyses Another approach has been to use formal gram- thereof, with virtually no music-theoretic knowl- mars to model and generate music [Laske, 1973] edge or heuristics. [Bernstein, 1976] [Smoliar, 1976] [Winograd, 1968] This paper summarizes related research, [Holtzman, 1981] [Lerdahl & Jackendoff, 1983] describes the design of our experience-based chess [Roads, 1978]. and music programs, expounds on the patterns, More sophisticated music composition hierarchical structures, and linguistic properties programs have been written that make use of rules of music, and concludes by outlining ongoing re- based on the music-theoretic heuristics of tradi- search. tional counterpoint species [Schottstaedt, 1984] 119 [Ebcioglu, 1986]. These systems attempt to min- pattern ranging from 0 to 1, with 0 indicating a imize penalties for broken rules using backtrack- sure loss and 1 indicating a win. Unlike most in- ing to compose an original piece of music or har- telligent chess programs, Morph does not search monize an existing one phrase-by-phrase. More the tree of possible movesfor future consequences recently, there have been several neural net im- of its actions. Instead, Morphevaluates the pat- plementations of compositional algorithms [Todd, terns that it finds in its possible legal movesand 1991] [Mozer, 1991] [Lewis, 1991] [Kohonen, Laine, chooses the movewith the best possible combina- Tiits, & Torkkola, 1991] and algorithms to learn tion of patterns. After the game is over, Morph and produce well-formed melodies [Underwood, adds and deletes patterns and adjusts its pattern- 1992] [Gilbert, 1993]. Someresearchers are study- weights using temporal difference learning [Sut- ing the related field of real-time musical accompa- ton, 1988] and the simple feedback of whether it niment [Dannenberg, 1984] [Vercoe, 1984] [Baird, won, drew, or lost the game. Blevins, & Zahler, 1989]. To study the application of similar princi- Our computational music research builds ples to another domain besides chess, music com- on the work of David Cope, whose Experiments in position was chosen, since: Musical Intelligence (EMI) software has created ¯ pieces in the style of Bach, Mozart, Joplin, Gersh- Music has relatively simple, discrete patterns win, and Bartok, amongothers. In many respects, that can be sequentially analyzed and unam- Cope combines some of the above techniques: EMI biguously represented. uses an Augmented Transition Network (ATN) * Due to the hierarchical structure of music, grammaras well as a rule-base to describe the syn- each note can be interpreted on manydiffer- tax of music. EMIbuilds a dictionary of signature ent levels, as part of a bar, motive, phrase, phrases by listening to a composer’s works, and section, or theme. then composesmusic using these stylistic patterns and its rule-base [Cope, 1991a]. ¯ Music composition, like winning chess, is of- Smurph does not use transformational ten cited as an exampleof intelligent and cre- grammarsnor musical heuristics. Instead, it uti- ative behavior. lizes only its probabilistic analyses of input music ¯ There are digitally stored musical databases to determine the most suitable patterns to follow that can easily be accessed on-line. its current output. It stores repeated patterns in a hierarchical database, and combines and alters ¯ Psychological aspects of human music com- them to form new musical phrases. position have been widely studied and pub- lished. 3 Morph & Smurph Like Morph, Smurph stores musical pat- terns that it recognizes in a hierarchical database "While Babbage dreamt of creating using a more-general-than operator. Rather a chess or tic-tac-toe automaton, [Lady than storing weights with these patterns, though, Ada Lovelace] suggested that his Engine, Smurph keeps track of the probabilities of oc- with pitches and harmonies coded into its currence of these patterns, modeling music as spinning cylinders, ’might compose elab- the result of a multiple-order Markovinformation orate and scientific pieces of music of any source. Thus, when composing, Smurph combines degree of complexity or extent.’ " - Dou- these smaller patterns to produce music that has glas Hofstadter, Godel, Escher, Bach, p. some of the style or signatures of the input music 25. pieces that it has listened to. Smurph’s patterns currently use only the Morph, an experience-based, adaptive, melody of a musical piece, but Smurphwill eventu- pattern-oriented program that learns to play chess ally analyze all the voices in the input data stream. by watching or playing games, has already been The input music is all transposed to the key of C, built [Levinson, 1991]. Morph was developed and rhythms are represented only in a temporally based on our APS (Adaptive-Predictive Search) relative manner. Thus, two eighth notes followed paradigm for machine learning. Starting with lit- by a quarter note are perceived to be identical to tle a priori knowledge, Morphstores the patterns two quarter notes and a half note, assuming of that it finds on the chess board in a hierarchical course that the pitches mapone-to-one after trans- database, along with its estimated weight for each position. Smurph’s database routines are written 120 in C++ and interface with Macintosh MIDILISP unary operators like Generalize, Invert, and Re- code to parse and output MIDIevents to and from ?)erse. the computer. 5 The Hierarchical 4 Patterns Structure of Music "One of the most interesting aspects "As a piece of music unfolds, its of the world is that it can be considered rhythmic structure is perceived not as to be made up of patterns. A pattern is a series of discrete independent units essentially an arrangement. It is charac- strung together in a mechanical, additive terized by the order of the elements of way like beads, but as an organic pro- which it is made, rather than by the in- cess in which smaller rhythmic motives, trinsic nature of these elements." - Nor- while possessing a shape and structure bert Wiener, in Gonzalez & Thomason, of their own, also function as integral Syntactic Pattern Recognition: An Intro- parts of a larger rhythmic organization." duction, p. 1. - Grosvenor Cooper & Leonard Meyer, The Rhythmic Structure of Music, p. 2. The easiest way to form complex patterns from simple ones is to group the smaller patterns According to Heinrich Schenker, the orig- together. Experimental evidence shows that the inator of modern harmonic analysis, music can be Gestalt principles of temporal proximity and sim- understood on three principal levels: foreground, ilarity are important determinants of grouping in middleground, and background. Schenker advo- music. Grouping objects together improves per- cates a universal deep structure for music based on ception and recognition visually as well as aurally: his formalisms not unlike Chomsky’s. This struc- "Regular, symmetrical, simple shapes will be more ture has two components, a fundamental melodic readily perceived,