Towards a Taxonomy of Musical Metacreation: Reflections on the First Musical Metacreation Weekend

Musical Metacreation: Papers from the 2013 AIIDE Workshop (WS-13-22)

Arne Eigenfeldt Oliver Bown Philippe Pasquier Aengus Martin School for the Contemporary Arts, Design Lab, Interactive Arts & Technology, Faculty of Engineering, Simon Fraser University, University of Sydney, Simon Fraser University, University of Sydney, Vancouver, Canada Sydney, Australia Surrey, Canada Sydney, Australia [email protected] [email protected] [email protected] [email protected]

Abstract ing the computer as an active participant in the creative The Musical Metacreation Weekend (MuMeWe), a series of process. Several of the systems presented at MuMeWe four concerts presenting works of metacreation, was held in were within the paradigm of improvisation; interestingly, June 2013. These concerts offered an opportunity to review improvisation provides a useful parallel to that of musical how different composers and system designers are ap- metacreation, in that there is a varying relationship be- proaching generative practices. We propose a taxonomy of musical metacreation, with specific reference to works and tween intent (composition) and practice (performance). In systems presented at MuMeWe, in an effort to begin a dis- the case of improvisation, one needs to consider what the cussion of methods of measuring metacreative musical relationship is between the underlying composition, and its works. The seven stages of this taxonomy are loosely based realisation – the relationship between different perform- upon the autonomy of the system, specifically in its ability ances of the same work – and thus the influence of the per- to make high-level musical decisions during the course of the composition and/or performance. We conclude with a former/improviser. Similarly, in metacreation, we need to discussion of how these levels could be interpreted, and the consider the relationship between the composer/system potential difficulties involved in their specification and ter- designer’s direct influence on the system, and the potential minology. differences between performances of the same work, in order to determine the system’s autonomy. Introduction The Musical Metacreation Weekend (MuMeWe)1 was held Previous Work in Sydney, Australia, on June 15th and 16th 2013. A series of four concerts were presented that featured peer-reviewed Benson provides his taxonomy of improvisation (2003), in works: these concerts presented a range of music generated which he specifies sixteen different “shades” of improvisa- by systems with varying degree of autonomy. We feel the tion, each of which present a relationship between a given weekend festival provides a useful measure on the current original composition and its realisation. These begin with a 2 state of musical metacreation systems ; furthermore, these “filling-in” of details that are not notated in the score, such events will allow us to provide a brief overview and dis- as tempi, timbre, and dynamics, to which he notes “no per- cussion of several of the systems, and conclude with a pro- formance is possible without some form and degree of im- posed taxonomy of metacreative systems. Such a taxon- provisation”. The next eleven progress through various omy will allow the comparison of systems without regard degrees of freedom in relationship between performer and to perceived musicality, complexity, or traditional auton- original composition, with the thirteenth being one in omy, concepts we feel are related to, but separate from, which the relationship between the performance and the describing the level of metacreativity of a system. original composition is evident only to the performer and Musical metacreation is concerned with autonomy and composer of the piece, a model that one tends to find most agency in composition and performance: simply put, hav- often in contemporary non-jazz improvisation. Three additional forms of improvisation are provided that Benson Copyright © 2013, Association for the Advancement of Artificial Intelli- considers “compositional”, such as Mozart “improvising” gence (www.aaai.org). All rights reserved. upon opera buffa in Così Fan Tutti. 1 http://www.metacreation.net/mumewe2013/index.php Boden and Edmonds (2009) provide a taxonomy of gen- 2 See Bown et al. (2013) for a detailed discussion of the event. erative art, separating various forms into categories in an

40 effort to “help toward a critical framework” of new forms tem in order to get “a better sense of the system’s auton- in art technology. The authors provide an interesting dis- omy in a range of contexts”. cussion in regards to their definition of computer-generated Lastly, the amount of control an artist has over the sys- art (CG-art), in which they first define as artwork resulting tem may be difficult to ascertain in non-performative situa- from autonomous systems that allow for “zero interference tions. As Collins (2008) points out, the established notion from the human artist”; however, they immediately ac- of algorithmic composition, as discussed in Pearce, Mere- knowledge that such a definition is extremely strict, which dith and Wiggins (2002), place the computer in the role of ignores a prominent subclass of computer-generative art composer’s assistant in an offline system. Collins, in sug- that includes interaction. Their more general classification gesting methods of analysis for generative music, clearly of generative art (G-art) – artwork produced that “is not outlines what he considers to be “generative music” – in under the artist's direct control” - raises the question of Boden and Edmonds terms, CG-music – and excludes from what exactly “direct control” implies. It is this level of con- his discussion art that is algorithmic, such as live-coding, trol that we will attempt to categorise in this paper: while that allows for human intervention during execution. Our some metacreative systems, as demonstrated by those pre- notion of musical metacreative systems, and the taxonomy sented at MuMeWe, may strive towards LeWitt's concept suggested here, is less rigid, and acknowledges the contin- of a “machine that makes the art”, where “all of the plan- uum between strictly reactive or inert music software and ning and decisions are made beforehand and the execution fully generative ones. This continuum goes through the is a perfunctory affair” (LeWitt in Boden 2007), it turns various stages of computer-assisted creativity, as detailed out that having some direct control over the system is still below. an important element of metacreation. This reflects Boden and Edmonds statement that such interactivity is the result of artists who “rely on their intuitive judgment to make Descriptions of selected pieces selections during an artwork's evolution.” In curating the concerts, it was extremely difficult to com- Bown and Martin (2012) provide a discussion on the na- pare various systems and representative performances, ture of autonomy in software systems, and suggest that without regard to the artistic output of the systems. While building systems that attain a high level of autonomy may some submissions were artistically strong, they were only be trivial in comparison with building systems that produce vaguely metacreative; others were, perhaps, less successful output that is contextually relevant. Referencing Newell et in their artistic results, but much more autonomous. As al. (1963), Bown and Martin suggest that the requirement such, we are presenting and describing a few of the sys- for usefulness should never be abandoned in favour of tems/compositions performed at MuMeWe as examples. autonomy. Furthermore, they suggest that a measure of a system’s agency, complexity, and intelligence are an Improvising Algorithms equally important measure of the system’s autonomy. Metacreative systems, as proposed by Eigenfeldt et al. The first concert presented systems in reaction to a live (2012), and as demonstrated at the MuMeWe, tend to fall improvising performer. Five of the systems interacted with into two distinct genres: improvisational systems (i.e. on- an acoustic instrument (Ollie Bown: piano, Oliver Han- line), and composition systems (i.e. offline). In the case of cock: piano, Bill Hsu: clarinet, Ben Carey: saxophone, the former, it makes sense to judge such systems in relation Alesandro di Scipio: trumpet), while Andrew Brown/Toby to how a system interacts with a live performer: in Bown Gifford’s system interacted with a MIDI synthesizer. As a and Martin’s terms, the complexity, intelligence, agency, result, the latter piece was the most pitch-based in its inter- and autonomy of the system in response to the live input. actions, most likely due to the availability of exact pitch In the case of offline systems, it makes sense to judge such information from its MIDI input. systems in relation to an underlying compositional idea: to Bown and Hsu were present, and operated their own what extent does the system produce its own musical struc- software; in both cases, they interacted with their software ture and details, and how much does it depend upon the during performance. Carey and Brown performed as musi- user to move it forward? cians with their own systems. Hancock and di Scipio sent Determining the autonomy of either system is problem- their software, and required someone else to rehearse with atic, as it would require multiple exposures to discover the the performers, and operate the software during perform- true range of a system’s output. In fact, Bown and Martin ance: in both cases, it was not clear to whether any interac- conclude that “a singular measure of autonomy itself may tion between operator and system occurred during per- not actually be as interesting as the various measures that formance. can be considered to constitute it”, and suggest the poten- While each system was unique, and each successfully tial for a second software system to probe the creative sys- exhibited “musical” interactions – which, in itself, were an indication of the current state of maturity in improvisa-

41 tional metacreative systems – we will choose Bown’s sys- predictability in its short-term decisions but high- tem as an example to describe in more detail. One charac- predictability in its long-term style. teristic that separated Bown’s work from the other im- Other performances included Oliver Hancock, echo- provisational systems was his avoidance of the record– system; Bill Hsu, Figment; Benjamin Carey, _derivations; playback paradigm demonstrated by the other systems. The Andrew Brown and Toby Gifford, Unity in Diversity; fact that the other systems were not playing a distinct tim- Agostino di Scipio, Modes of Interference / 1. bre or instrument, and instead played back recordings made of the live instrument, made the perception of their Algorave identity and autonomy more difficult to determine: playing A second event was held in a club venue, focusing upon a processed version of the input blends it in a way that is generative dance music. The term Algorave specifically reminiscent of effect processors, and of the turn-taking relates to the notion of algorithmically generated dance mechanisms of reflective interaction, as established and music, an event begun by Alex MacLean and Nick Collins studied in the context of the Continuator (Pachet 2003) and circa 2012. The event began with a 30 minute set of gener- MIROR (Addessi 2013) platforms. ated EDM tracks in the Breakbeat style by Chris Ander- It is informative to compare two pieces. Bown's system, son’s GEDMAS system (Anderson et al. 2013). As the Zamyatin, is “comprised of an audio analysis layer, an in- tracks were pre-generated, there was no performance as- ner control system exhibiting a form of complex dynamical pect, and thus no interaction. Anderson’s set was generated behaviour, and a set of output modules that respond to the offline, using a corpus of 24 transcribed Breakbeat tracks. patterned output from the dynamical system. The inner The system was autonomous in that it allowed for no user systems consists of a Decision Tree that is built to feed interaction during generation; however, timbres were hand- back on itself, maintaining both a responsive behaviour to chosen once the tracks were generated. the outside world and a generative behaviour, driven by its Following the presentation of pre-generated audio, two own internal activity” [Bown, program notes]. live-coding performances were presented. Bell’s perform- Hancock's system is “a group of computer agents, each ance of Live with Conductive demonstrated “post-dubstep one acting as a delay, and basing its playback on the bass music”, which was followed by the live-coding duet rhythms of the live improviser, and all the other agents in of Sorensen-Swift, who provided a mellower and com- the system. Together they behave like chorusing insects or pletely tonal set of electro and minimal house. Although frogs, with some degree of irregularity and unpredictabil- the live-coders used different environments – Bell used ity. Broadly the system matches the activity of the impro- Haskell3, while Sorenson/Swift used Extempore4 – both viser, but it can blossom and race unexpectedly, carry on triggered gestures and events that were typed by hand. In alone in a lull, or leave the improviser to play in relative the case of the latter, the laptops shared a common sync, isolation.” [Hancock, program notes]. which allowed them to maintain a common pulse. Bown's approach consciously avoids designing behav- Other works presented included Sick Lincoln, Al- iour into the core of the system in a preconceived manner. goravethm for 31770; Arne Eigenfeldt, Breakbeats by A general purpose system is instead created and given ba- GESMI. sic functionality through an evolutionary goal. However, the musical work is then additionally developed around Composed Process / Studies in Autonomy this core system. In contrast, Hancock follows the paradigm of resampling the performer's input and applying Two additional concerts were held in more traditional con- different composed responses to the playback of these cert venues, and provided a somewhat more formal atmos- samples. In both cases, the system in no way directly mim- phere. Roger Dean’s Serial Identities began with Dean ics or tracks the performer, and is understood as an inde- improvising atonally at the piano; he paused to initiate his pendent agent that stimulates the performer through inter- software that began generating similar atonal/serial music action. In Bown's case this is extreme, and the performer that triggered a sampled piano module. The system was can be understood as perturbing the output of an otherwise reactive, rather than interactive, in that Dean reacted to the reasonably stoic system, whereas in Hancock's case, musi- generated material, but the system was not “listening” to cal materials (timbral, melodic, rhythmic) from the per- his performance. former become part of the system's response and the inter- Kenneth Newby’s The Flicker Generative Orchestra action has the form of a rich loop or echo system. was a fixed media example of “the application of a variety Formally, in both cases, the performer does not directly of generative processes that result in music that resembles control the system – thus, a lack of heteronomy – the sys- that of large symphony orchestras playing in a contempo- tem is neither fixed nor highly random, and is in part influ- 3 enced by the performer. In both cases the system has low http://www.haskell.org/haskellwiki/Haskell 4 http://benswift.me/extempore-docs/

42 rary idiom” (Newby, program notes). The system gener- On Reflection ated individual musical lines that were convincing and be- lievable in their specific representations. For this particular The weekend provided the opportunity to witness state-of- rendition, Newby selected a specific pitch and rhythmic set the-art metacreative systems, and demonstrated a wide for the system to explore. variety of approaches to the notion of autonomous creative David Kim-Boyle presented Line Studies no. 1, a real- systems. Without commenting on the artistic merit of any time generated graphic score which was immediately in- of the compositions and performances, the experience did terpreted by a software improviser, performed on a sam- suggest a basic distinction between the approaches. pled piano. The score itself was visually beautiful – a se- Musical metacreation, borrowing from the more general ries of “concentric rings segmented into arcs of varying definition of computational creativity, is the idea of en- lengths and thicknesses”. During the performance, the arcs dowing machines with creative behavior. Musical are continually “drawn, extended, overwritten, rotated, and metacreation can be divided into a number of canonical erased. Each arc denotes a pitch that is performed by the problems: pianist, with arc lengths and arc thicknesses corresponding to pitch durations and dynamics respectively” (Kim-Boyle, 1. Composition – being the process of creating a series of program notes). Although the piano was interpreting the performance instructions for musical performers, (i.e. a score, the unusual presentation suggested the reverse (to score); the first author): a visualisation of an improvised musical 2. Interpretation – being the process of performing a mu- performance. sical composition and producing an audio rendering; Nomos ex Machina was an example of Shlomo Dub- 3. Improvisation – which combines (1) and (2) in real- nov’s Audio Oracle in action, as it reacted to a live per- time performance; former and responded with audio recordings made during 4. Accompaniment – being the process of following a live the performance. Unfortunately, the system’s ability to performer in an accompanying role, possibly performing interact with the performer through its complex processes pre-composed music; of time mapping wasn’t clear, as the performer choose to 5. Continuation – being the process of continuing a given improvise in a consistent “full-on” manner that ignored the musical input in the same style. system’s responses. Other works presented included Brian Kelly, Phi; Mic- The MuMeWe provided two examples of #1 in the Al- quel Parera Jaques, neix_2013a; Ivan Zavada, Algosonic; gorave (albeit, the directions for performance were Ableton Anisha Thomas, Kundaliktra; Kerry Hagan, Morphons and Live scores). While #2 was not presented on its own, it was Bions; Benjamin O'Brien, Densité. encompassed within several instances of #3 through system performance. #4 was tangentially demonstrated within Installations the improvisation systems; in fact, some of the performances were more suggestive of accompaniment, rather than Eigenfeldt’s Roboterstück was one of three installations equal interaction, due mainly to the performer’s choice to that ran continuously during the weekend. Eigenfeldt’s remain as the foreground instrument. #5 was mostly dem- work played every 15 minutes, when agents would “wake- onstrated by the Brown/Gifford piece, itself inspired by the up” and begin interacting. The agents sent MIDI informa- reflective interaction paradigm of Pachet (2003). tion to the NotomotoN mechanical musical instrument, which struck a variety of objects, including clay flower- pots, metal bars, plastic blocks, and a drum. The agents A Taxonomy of Musical Metacreation negotiated a texture – based upon opposing metrics of slow-fast, loud-soft, dense-sparse, rhythmic-arrhythmic – The taxonomy presented is meant as a classification system then improvised within that texture until a majority of the for metacreative systems in terms of their relationship to agents “got bored”, at which point a new texture would be the human designer/composer’s control over the final mu- negotiated. Once the same texture was performed three sical result, which can also be interpreted as the level of times, the performance ended. Each performance lasted musical autonomy. Just as there are different levels of im- between three to six minutes. provisation (and different relationships between a per- Other installations included Miles Thorogood and Phil- former’s effect upon the final result in comparison to the ippe Pasquier, Audio Metaphor; Fredrik Olofsson, low-life. original “composition”), there are different degrees that a system has control over the final musical artifact, whether it is generated live (online) or fixed (offline). This can be reduced to how much creative decision-making was left to the system. Another way of looking at it is how much in-

43 fluence is required from a human in order for a metacrea- -a random selection of playback speed (tempo); tive system to perform musically5. -using an algorithmic process to alter volume and/or on- Importantly, a human designer/composer's sense of con- set data in a MIDI sequencer. (Note that the “humanisa- trol over the musical result of an action depends strongly tion” of pre-generated sequences through such variation on his/her understanding of the mapping between the input can be considered within the interpretation task of a and output of the system. A mapping that is completely metacreative system.) deterministic but complex might deny a sense of control Such variation can be random (discrete distributions, or and therefore exhibit a significant level of metacreation. continuous), constrained random (within a preset range), Thus, the level of metacreation is at least in part something constrained random whose range is varied by a performer, that is attributed to a system by a user or observer, and not random variation of a preset value, or a process – such as something that might be directly measurable. This is con- an LFO based upon a complex mathematical function – sistent with Paine's view that interactive MuMe systems whose output may not be entirely predictable. It can also require “a level of cognition” (Paine, 2002), i.e. something be stochastic, or statistically learned from a corpus of hu- clearly attributed to it by a human observer. It is also con- man interpretations. sistent with Jòrda (2007), who links interactivity in MuMe systems to the potential of a system for producing unex- 2. Compositionality: the use of any process to deter- pected results, whether through `non-linearity, randomness mine the relationships between pre-defined musical or ungraspable complexity'. gestures. Our taxonomy consists of seven levels, and is ordered The previous level applied to a single gesture; this level is from the least autonomous to the most. Later levels do not concerned with the relationship between two gestures need to exhibit properties of earlier levels (although they and/or processes that are in themselves fixed. This may could): instead, it should be understood that earlier levels involve initiating multiple layers of pre-generated material, are incapable of exhibiting defining properties of later lev- triggering pre-recorded material, or even initiating com- els. plex signal processing in relation to a live performance in An eighth level (level 0) can be inferred as musical which the process is heard as a separate gesture to the compositions that contain no metacreativity; however, note original (i.e. complex delays). Initiating events and/or that such a level does not imply fixed-media works, as processing in relation to a fixed score (i.e score-following) autonomy can exist in the creation, separate from the per- would be another example: from the perspective of pre- formance/execution. dictability and causality, whether events are initiated by the press of a button, or waiting to be triggered by an event in 1. Independence: the use of any process on a musical a score, the results are the same. gesture that is beyond the control of the composer. Similarly, aspects of live-coding (Bell’s Live with Con- The most basic creative influence a system can have on a ductive, Sorenson/Swift’s Palle in Aria) in which direct musical work is the application of a process upon a ges- sequences are initiated, are likewise catagorised here. ture6, in which the process itself can vary with each appli- In all these cases, the exact relationship between indication; in other words, a process that involves some ran- vidual musical lines, while deterministic (due to their fixed domisation, for example. In a studio environment, such material), possibly remain unknown when initiated, and are processes can be auditioned and selected, while in per- thus considered metacreative. formance, a certain amount of trust will be placed upon the system to apply the process in a predictable way: in other 3. Generativity: the generation of musical gestures. words, delegating some creative responsibility to the sys- Similar to #2, albeit with the addition of an algorithm that tem. generates and/or substantially varies pre-generated se- Examples of processes that fall into this category would quences; any reactive system that requires input in order to include: function. -complex signal processing that involve random parame- An example would be triggering processes that contain ters, in which the final result is heard as a single gesture pitch/rhythm generation algorithms; another example (i.e. reverberation); would be to trigger gestures that are undefined (i.e. generative) in their pitch/rhythm/volume in response to a per- 5 former’s action. Similar to #2, live systems that are reac- By “musically”, we mean as done by a human, either improvi- tive, in that they use live input to generate material, are sational or compositional, in a way that would be considered “musical”. categorised here for the same reason score-followers are 6 By “gesture”, we may any musical idea that has a beginning and placed in #2. an end. This includes not only traditional “phrases”, but also “performer-less” notions, such as LFO drones.

44 Feedback systems are contained here, in that the gesture An example of 5a would be a generative system that is indeterminate in its details, yet require the proper envi- generates its own musical structure (i.e. Newby’s The ronment to be instantiated. Flicker Generative Orchestra). Offline examples would Examples from the MuMeWe described include Kim- include Anderson’s GEDMAS, which produces complete Boyle’s Line Studies no.1, and Dean’s Serial Identities. electronic dance music tracks based upon a restricted cor- Additionally, some aspects of live-coding (Bell’s Live with pus. Conductive, Sorenson/Swift’s Palle in Aria) are included Examples of 5b would be evolutionary systems not here, in which sequences were initiated that included ran- heard at MuMeWe, such as McCormack’s Eden, and dom, or stochastic, selection from a constrained set of Bown's Sonic Eco-System, in which the system produces pitches. fundamentally different output over time. Another simpler example would be a physically modeled system that un- 4. Proactivity: system/agents that are able to initiate dergoes a single-phase transition during its lifetime. their own musical gestures. As opposed to #3, there no is direct trigger that an agent 6. Versatility: agents determine their own content with- awaits before beginning. Thus, this is the first notion of out predefined stylistic limits. autonomous agent-based behaviour. Agents are not merely As opposed to #5, which essentially generates a variation reactive, requiring input in order to function, but can gen- of the same composition each time, with variation deter- erate material of their own accord; this includes Black- mined by user settings (i.e. a user-set pitch structure, a well’s autonomous musical agent, which “goes beyond fixed relation between agents, a given corpus, or any kind mere automation in determining responses to musical in- of formal template), here the agents would generate truly put, by deriving novel but relevant responses” (Blackwell, different compositions with each generation. The use of in Bown). In the agent literature (Wooldridge 2009), this is (and need for) templates for formal construction have been know as proactivity, and is a defining characteristic of eliminated. In terms of Boden’s exploration vs. transforma- autonomy. tion, this system would offer the potential for transforma- Examples of proactive systems would be interactive sys- tion of the creative space. No such system was presented at tems that have an independent response to a performer (i.e. the MuMeWe. Bown’s Zamyatin, as well as George Lewis’ Voyager); another example would be a multi-agent system (i.e. Ei- 7. Volition: agents exhibit volition, deciding when, genfeldt’s Roberterstück). what, and how to compose/perform. A truly free-standing creative system, independently capa- 5. Adaptability: a) Agents behave in different ways over ble of deciding whether it wants to create, and eventually time due to their own internal evolution; b) agents in- why it would want to do so (which is the problem of intrin- teract and influence one another. sic motivation); a system capable of deriving its own con- This level is a subtle shift in autonomy in comparison to #4 ceptual spaces (Gärdenfors 2004, in Bown); a system ca- – the primary difference is the amount of change the agents pable of autonomous critical evaluation (Galanter 2012). are capable of achieving proactively, in a restricted period Such a system, as of yet, does not exist. Human com- of time (i.e. the composition). In some agent-based works, posers, on the other hand, presumably exhibit such actions an agent’s behaviour is governed by an internal state that whenever they create: at least truly creative artists would, can be altered by the composer/performer so as to achieve one hopes. Such a system would require long-term learn- different reactions in different situations; as such, human ing, sophisticated feedback mechanisms that include peers interaction is still required. In systems within this fifth and community, the ability to form aesthetic judgment, and level, agents determine when to alter their behaviour, and the ability to derive its own motivations. how, proactively. This is separate from defined parametric change, in which gestures may alter over time based upon a predefined applied function. Discussion 5b includes agent interaction, thus completing the multi- The reader may notice that the above taxonomy does not agent model, in which agents are not only autonomous (#4 take into account any level of complexity of a system. An - proactivity), but they are also social, and their interac- extremely complex system that requires the designer to tions with other agents determine how they operate in the “nudge it along” during performance will remain at level 4 future. However, social agents do not, on their own, guar- (proactivity), while a system that generates simple random antee that they will evolve their actions over time in a sig- melodies, but changes how those melodies are produced nificant way. As such, agent interaction is separate from proactively during the composition/performance – even if agent autonomy, and does not warrant its own level. only by using a randomly generated form – will be placed

45 at level 5 (adaptability). While this may appear counterin- and thus do not require any ability to listen; they do, how- tuitive, the autonomy of the two systems is different: the ever, require an ability to “think”. As such, a system may latter system can produce a complete (albeit musically lim- display an ability to interpret live input exceedingly well, ited) work, while the former requires a response to be trig- but if it cannot rise above echoing such input, will have gered and/or altered. This division is a deliberate nod to difficulty rising above level 3 (generativity) or 4 (proactiv- LeWitt, and his suggestion of a “machine that makes art”: ity) on our scale. On the other hand, a system may com- without the ability to generate its own forms, the machine pletely ignore a live performer (and thus listen very poorly) will only be able to assist a human in making the artwork, yet function as a highly metacreative system if it can de- as the human is still required to make fundamental deci- termine, proactively, when to make important musical de- sions regarding the evolution of the artwork in time. While cisions. Of course, a system can also do both; as such, it Colton et al. (2012) have created a fully autonomous po- might be justified to use a cognitive model of listening, or etry generation system which requires the use of form tem- more specifically, musical thinking, as proposed by Max- plates, we contend that such use of templates are a substi- well et al. (2012). tute for direct interaction: the higher levels in our taxon- We acknowledge that it is difficult to ascribe a clear omy must generate their own forms autonomously. definition of “thinking”, which is as hard to define as autonomy and proactivity; however, we believe a collo- On Proactivity quial understanding of the term is useful here. A defining aspect of many of the proposed levels is related to the system’s ability to make higher-level musical deci- On Usefulness sions proactively: or “on its own”. Clearly, proactivity, like Roger Dean has suggested that non-interactive systems in autonomy, is extremely difficult to define, particularly improvisational contexts are musically useful: “A genera- from an external viewpoint. How can listeners determine tive algorithm might also be conceived as poorly interac- whether a musical event was the result of serendipity in tive, but providing a sound environment in which other performance, or the emergence of relationships within the sound is improvised. Unpredictability in the algorithmic system? Conversely, how can a profound change in the output is also a potential positive contributor to its impro- system be recognised by the audience if such changes do visatory utility.” (Dean 2003). The musical “usefulness” of not result in substantial changes within the music? We any system is not accounted for in our taxonomy, whether cannot assume that designers of metacreative systems can it is Dean’s application of the word, or in terms of creativ- have a clear operational grasp of what it true proactivity ity research (Newell et al. 1963). Dean’s example system means, and how it is applied in a range of human and ma- may provide generated material, and whether it is created chine scenarios, nor do we have clear enough methods to in real-time or not, places it at level #3 (generativity): the determine how to unambiguously make something that requirement for the next level (proactivity) is that it de- does something “on its own”. cides when to initiate. We consider the discussion of the While we wish to avoid including the notion of com- usefulness of a system to be separate. plexity within our level criteria, it may need to be included in differentiating level 3 – triggered generativity – from On Autonomous Fitness Functions level 4 – proactivity. As previously suggested, if a system Boden and Edmonds discuss autonomy in relation to rule- waits for a pre-determined input, it would not be consid- based systems versus evolutionary systems: since the latter ered proactive due to its predictability and causality; but by are making decisions proactively as to how to evolve (as- combining a series of such triggers (i.e. a series of when() suming a non-interactive fitness function), such systems methods) with some randomness, the predictability and will be placed at level 5 (adaptability) in our taxonomy. causality could be eliminated, and the system would seem While the creation of a fitness function is, in itself, a hu- to operate with greater complexity, and perceived proactiv- man decision, if the fitness function operates on its own ity. during the composition/performance, its level of autonomy In summary, an understanding of a system’s ability to is greater than a rule-based system that requires interaction. act “on its own” may be our deepest analytical and phi- However, the human-created fitness function will also limit losophical challenge. it to this location: in order for such a system to be placed at level 6 (versatility), it would require the ability to develop On Thinking and Listening its own non-human fitness function. Similarly, any corpus- We can wander into another conceptual minefield by sug- based system that involves a fixed corpus provided by a gesting that we are attempting to distinguish a system’s user will limit its level in the same way. ability to “think” (its autonomy) from its ability to “listen” to any input. Metacreative systems do not require input,

46 On Large-scale Structure Benson, B. E. 2003. The improvisation of musical dialogue: a phenomenology of music. Cambridge University Press. Lastly, it became apparent at MuMeWe that successful Boden, M., and Edmonds, E. 2009. What is generative art? Digi- large-scale musical structure is something that is the most tal Creativity 20.1-2: 21–46. difficult for an artist to delegate to a system (voluntarily or Bown, O., and Martin, A. 2012. Autonomy in Music-Generating otherwise). This makes complete musical sense, as this is Systems. In Eighth Artificial Intelligence and Interactive Digital one of the most difficult aspects of music for young com- Entertainment Conference. 7 posers to learn . Just as young (human) composers can Bown, O., Eigenfeldt, A., Pasquier, P., Martin, A., and Carey, B., generate short forms, so can many metacreative systems; 2013. The Musical Metacreation Weekend: Challenges Arising selecting “what to do next” is an incredibly difficult musi- from the Live Presentation of Musically Metacreative Systems. In cal decision to make, one that takes into account the cur- Proceedings of the Artificial Intelligence and Interactive Digital Entertainment (AIIDE’13) Conference. rent musical context, the past musical context, and the po- Collins, N. 2008. The analysis of generative music programs. tential future contexts. Asking – or expecting – a computer Organised Sound 13.3: 237-248. to successfully navigate this terrain is within the nascent Colton, S., Goodwin, J., and Veale, T. 2012. Full face poetry field of computational aesthetics: see Galanter (2012) for generation. In Proceedings of the Third International Conference an overview of this field and its potential for musical on Computational Creativity, 95–102. metacreation. Dean, R. T. 2003. Hyperimprovisation: Computer Interactive Sound Improvisations. AR Editions, Inc. Eigenfeldt, A., Burnett, A., and Pasquier, P. 2012. Evaluating Conclusion Musical Metacreation in a Live Performance Context. In Pro- ceedings of the International Conference on Computational Crea- Once a system is clearly placed within this taxonomy, a tivity. discussion can occur as to the complexity of its output, the Galanter, P. 2012. Computational aesthetic evaluation: past and level of variation in its output, and its “musicality”. future. Computers and Creativity, 255–293. Springer Berlin Hei- Clearly, some very complex and musical systems exist at delberg. the lower levels of this taxonomy, as evidenced by Jòrda, S. 2007. Interactivity and live computer music, N. Collins MuMeWe: what we propose is a method to compare the and J. d’Escrivan, (eds.), The Cambridge Companion to Elec- metacreative level of systems independent of their “musi- tronic Music, 89–106. Cambridge University Press, Cambridge, cal maturity”. UK. Perhaps the distinction in all of these examples is to Maxwell, J., Eigenfeldt, A., Pasquier, P., Gonzalez Thomas, N. think of each level as a principle, and consider each system 2012. Musicog: A Cognitive Architecture for Music Learning and Generation. In Proceedings of Sound and Music Computing in terms of whether (a) it aspires to that principle and (b) it (SMC 2012) Conference. masters that principle. In that way, if certain systems could Newell, A., Shaw, J. G., and Simon, H. A. 1963. The process of be said to aspire to principle #6 (versatility), then we can creative thinking, H. E. Gruber, G. Terrell and M. Wertheimer critically examine how these systems fall short. (eds.), Contemporary Approaches to Creative Thinking, 63–119. Lastly, it should be pointed out that there were many New York: Atherton. musical works at MuMeWe that were of a high musical Pachet, F. 2003. The continuator: Musical interaction with style. quality. Artists within this nascent field will always want to Journal of New Music Research, 32(3), 333–341. create works of which they feel are artistically strong, and Paine, G. 2002. Interactivity, where to from here? Organised may feel the need to retain interactive control of their sys- Sound, 7(3): 295–304. tems; scientists within the field will be drawn to creating Pearce, M., Meredith, D., and Wiggins, G. 2002. Motivations and systems that are more autonomous. The dilemma for all methodologies for automation of the compositional process. Mu- sicae Scientiae, 6(2): 119–147. practitioners in the field is to balance these two goals, Wooldridge, M. 2009. An Introduction to MultiAgent Systems – which should not be mutually exclusive. Second Edition. John Wiley & Sons

References Addessi, A. 2013. Child/machine interaction in reflexive environment: The MIROR platform. In Proceedings of the Sound and Music Computing (SMC 2013) Conference. Anderson, C., Eigenfeldt, A., Pasquier, P. The Generative Elec- tronic Dance Music Algorithmic Systems (GEDMAS). In Pro- ceedings of the Artificial Intelligence and Interactive Digital En- tertainment (AIIDE’13) Conference.

7 At least of Western art music.