Automatic Orchestration for Automatic Composition Eliot Handelman Centre for Interdisciplinary Research in Music Media and Technology Montreal, Canada
Total Page:16
File Type:pdf, Size:1020Kb
Musical Metacreation: Papers from the 2012 AIIDE Workshop AAAI Technical Report WS-12-16 Automatic Orchestration for Automatic Composition Eliot Handelman Centre for Interdisciplinary Research in Music Media and Technology Montreal, Canada Andie Sigler Centre for Interdisciplinary Research in Music Media and Technology and School of Computer Science McGill University Montreal, Canada David Donna School of Computer Science McGill University Montreal, Canada Abstract In this paper we define a narrow version of the orchestra- tion problem that deals with assigning notes in the score to The automatic orchestration problem is that of assign- ing instruments or sounds to the notes of an unorches- instrumental sound samples. A more complete version of the trated score. This is related to, but distinct from, prob- orchestration problem includes articulations and dynamics, lems of automatic expressive interpretation. A simple while an extended version could include varying and adding algorithm is described that successfully orchestrates to the score (“arranging”). A more constrained version of scores based on analysis of one musical structure – the the orchestration problem would take into consideration the “Z-chain.” limitations and affordances of physical instrumental perfor- mance. One way for an autonomous system to compose is to ini- tially write parts for different instruments or sounds. A sec- Strategies for Expressive Interpretation and ond strategy (one closer to some human practice) is to write a “flat” score first and then orchestrate it. The latter poses Automatic Orchestration a more general problem, since a system capable of orches- There has not been much previous research in the area of au- trating its own compositions could also be capable of or- tomatic orchestration in the sense of autonomously assign- chestrating other input, from Beethoven sonatas to Tin Pan ing sounds to a score.1 However, there has been research Alley tunes to Xenakis. The goal of being able to orches- on expressive interpretation. The goal there is generally ei- trate “known” music makes the system more suceptible to ther to imitate human performance, to impart a mood as analysis and evaluation, and therefore to development. described by an adjective, or both. Like automatic orches- Orchestration is a problem of music generation through tration, automatic interpretation starts with a flat score and analysis, occupying a place on the continuum from expres- modifies it through analysis, adding new layers of informa- sive interpretation (often framed as timing, articulation and tion. New layers could take the form of instructions for mod- dynamics) through to arrangement and variation-making. ifying dynamic levels or tempo, addition of accents, or as- This class of problems is different from (and somewhat more signment to instrumental sounds. constrained than) the goal of generating music from scratch. Approaches to expressive interpretation are generally Since a successful orchestration will be reflective of, or syn- based on determining sets of rules for assigning dynamic chronized with, things that are happening in the music, an and tempo curves, articulations, and accents to notes. These analytical level is essential. rules are developed from music theories (Bisesi, Parncutt, The flip side of the orchestration problem is that if we and Friberg 2011), inferred through listening consultations treat orchestration as a kind of sonified analysis, then not with professional musicians (Friberg, Bresin, and Sundberg only can analysis help us make orchestrations, but careful 2006), or derived through machine-learning analysis of hu- listening and re-analysis of orchestrations generated through man performances (Widmer 2003). Rules are often based on computational analysis can help us evaluate and refine our 1 analytic methods. Research on “automatic orchestration” at IRCAM focusses on analysis of timbre and instrumental capabilities for assisted orches- Copyright c 2012, Association for the Advancement of Artificial tration, not the automatic assignment of sounds to a score (Carpen- Intelligence (www.aaai.org). All rights reserved. tier et al. 2012). 43 analysis of grouping (phrase) structures at different hierar- Experiment chical levels, as well as local melodic and harmonic events An ultimate goal is to generate full orchestrations with bells (Widmer and Goebl 2004; Friberg, Bresin, and Sundberg and whistles in appropriate places. This will entail a large- 2006; Canazza et al. 1998; Todd 1992). As a result, the per- scale orchestration system with analytical access to many formances are focussed on local effects, with global consis- interacting musical structures as well as means of reason- tency stemming from consistency in rule parameters and in ing about combinations within a large database of articulate musical structure. samples. The two sides of the coin, then, are reasoning about One general way these approaches could be extended the score, and reasoning about the sounds.3 In this paper, we would be to analyze longer range structure, so that it would report on a preliminary experiment in which the focus is on be possible to control differentiated patterning of perfor- analysis of the score for orchestration. mance expression along a series of repetitions or variations. Harold Cohen asks: “What is the minimum condition un- This kind of analysis might bring the addition of expres- der which a set of marks functions as an image?” (Cohen sive layers a little further along a performance-arrangement- 1994). Likewise, it is of interest to find a minimal set of re- variation-composition continuum of music creation. quirements which will result in orchestrations that function The orchestration problem is closer to the compositional musically – that sound structured in relation to the perceiv- level of musical action than are the problems of expres- able musical structure. sive performance. Therefore a different set of challenges are A simple algorithm was developed in order to evaluate posed, in which the set of interesting solutions is perhaps one kind of structure (called a Z-chain) for its utility in or- more open. We need not imitate human-written orchestra- chestration. tions, nor model the expression of musical mood. We would Z-chains are recursive structures built based on orienta- be very happy to produce surprising, innovative orchestra- tion (in the present case, orientation of pitch). They exist on tions, as opposed to modelling a given style. In fact, we local and large scales, with large-scale Z-chains built out of would like to use computational music analytic methods as local-scale ones. Z-chains are built on the principle of chain- a means for discovering new kinds of orchestrations. ing like to like, so that a Z-chain contains substructures with In contrast with approaches to expressive imitation, there- 2 identical sets of recursive orientations. Z-chains therefore fore, we can proceed without a set of rules. Instead of mod- capture a certain kind of repetition structure which is on the elling a desired outcome, we can experiment with processes one hand deterministic and rigorous, but on the other hand that might generate musically interesting yet unpredictable low-dimensional enough so that musical passages that are orchestrations. quite different can be part of the same Z-chain. A null-hypothesis strategy for the orchestration problem is to assign sounds to notes randomly. Another pre-analytic Chains and Z-chains solution would be to assign each pitch to a different sound (so every C5 sounds like a trumpet, etc.). Beyond this, anal- Suppose we have sequence of pitches (or of durations, loud- ysis is required. nesses, or any other “oriented” feature in which for any two The proposed strategy is to synchronize aspects of struc- elements x and y, either x is greater, y is greater, or they ture in the score with structure in the orchestration layer. An are equal). Chains are a way of taking a sequence apart into analytic system is used to select structures within the score different “dimensions” so we can look at different aspects of (where a “structure,” generally speaking, is some abstractly the sequence separately. specified data type which is instantiated by a set of notes A chain is a consecutive subsequence of elements going in the score), and each structure is assigned to some sound. in just one orientation. Figure 1 shows the three orientations Given an analytic system that identifies overlapping struc- of (pitch) chains in the beginning of the Menuet in G from tures of several different types, this is a strategy that can the Anna Magdalena Bach Notebook. The top view shows generate a large number of different orchestrations. The se- chains going up (shaded pitches connected by lines). The lection of structures and assignment of sounds can be done middle view shows chains going down, and the bottom view randomly, with human assistance, or with a more specific shows “be” chains (repetition chains). algorithmic approach. We can proceed by taking each view as a separate di- The technical part of this paper describes an experiment mension, and recursively “chaining” chains together. Recur- in orchestration using just one of several musical structures sive chains are called Z-chains – named for their zig-zigging currently being developed. Unlike the phrasing and grouping shapes. Figure 2 shows chains up with their top pitches re- structures often used for expressive interpretation, the struc- spectively going up: a Z-chain up-up. Figure 3 continues the tures used here are non-hierarchical and non-contiguous, recursion, finding a Z-chain up-up-down-be. The Z-chains affording interleaving and overlapping of colors in the or- include entire chains, not just the top pitches. chestration. The experimental hypothesis is that innovative We need not only look at the top pitches of chains. Figure and yet musically sensible orchestrations can be produced 4 shows Z-chains with respect to bottom pitches: a Z-chain by synchronization with a (particular) structural level of the up-down-down-be-down, which spans most of the Menuet.