Supporting Musical Creativity With Unsupervised Syntactic Parsing Reid Swanson*†, Elaine Chew*‡ and Andrew S. Gordon*† * University of Southern California Viterbi School of Engineering, Los Angeles, California †University of Southern California Institute for Creative Technologies, Los Angeles, California ‡Radcliffe Institute for Advanced Study at Harvard University, Cambridge, Massachusetts Abstract sonic elements that do not have a culturally defined Music and language are two human activities that fit well meaning. However, even in this restricted view he believes with a traditional notion of creativity and are particularly that music contains many linguistic elements including suited to computational exploration. In this paper we will symbols and grammar that allow linguistic analysis to be argue for the necessity of syntactic processing in musical enlightening. applications. Unsupervised methods offer uniquely One of the more specific relationships between music interesting approaches to supporting creativity. We will and language is the natural ability to recursively group demonstrate using the Constituent Context Model that primitive elements together to form larger and larger units syntactic structure of musical melodies can be learned organized in a hierarchical structure. Syntax, or grammar, automatically without annotated training data. Using a is a long and actively researched topic in the field of corpus built from the Well Tempered Clavier by Bach we describe a simple classification experiment that shows the Linguistics that has been dedicated to these structures. relative quality of the induced parse trees for musical Music does not have the rich history that Linguistics does melodies. in the area of syntax, but some work has been done, most notably by Lerdahl and Jackendoff (1983) and by Steedman (1996). There is even evidence that some of the Introduction syntactic processing for both types of data are processed in Creativity is a difficult concept to define precisely, yet we the same region of the brain (see, for example, Patel 2003), all have an intuitive feeling for what is and is not creative. although not necessarily in the same way. Over the last Although creativity is used to describe innovative and decade syntactic processing has permeated nearly all unique methods of accomplishing just about any task, the aspects of natural language processing topics. Not only has arts are most prototypically associated with creativity and the use of syntactic information directly improved the the creative process. Music and language are two human results of many traditional tasks, it has also enabled more activities that tie into this traditional notion of creativity complex systems to be built that were not possible without well, and are particularly suited to computational syntactic information. exploration for several reasons. The ubiquity of these Although, the use of syntactic information in the musical expressions and the ability for most people to have at least composition process and in music analysis is frequently some limited experience or ability in these areas are key discussed and often considered fundamental, there has aspects that make these topics so appealing for research. been relatively little work on integrating this information Another reason is the relative ease in which basic units of into automated approaches to music analysis and “meaning” can be represented in various machine-readable generation. Two examples are the computer formats. Language and music also share many implementations of Lerdahl and Jackendoff’s General characteristics that could allow key insights from one Theory of Tonal Music (GTTM) by Hirata and Matsuda domain to shed light on the other. (2003), and by Hamanaka, Hirata, and Tojo (2006). There are many relationships that can be found between However, due to ambiguities in the GTTM rules, the music and language dating as far back as Socrates, Plato programs require human intervention to parse an input. and Aristotle. Dobrian (1992) elaborates three categories Hierarchical processing has also been integrated into that are particularly recurrent in this discussion. The most some computing interfaces for music composition. relevant to this work is the concept of music as a language Tuneblocks explicitly makes use of hierarchical units for itself. When viewed in this way it is natural to apply teaching composition techniques to beginning and novice linguistic theories of syntax and semantics to try to analyze composers (Bamberger, 1999). Concepts of hierarchical and derive meaning from music. Dobrian (1992) ultimately construction have also been implemented in the maquette 1 argues that music is not a language in the same way in OpenMusic . Few automatic methods for generating English is, for example, because there are simply too many music actually consider such hierarchical structures. Just as using syntactic representations has been useful in human Copyright © 2007, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. 1 http://recherche.ircam.fr/equipes/repmus/OpenMusic guided learning processes, syntactic information could be determine the creativity of the system based on the process useful for a host of computational music applications. it uses to derive its output. One starting point for developing automatic parsing Having humans judge the level of creativity of a techniques for music would be to adapt successful machine’s output poses two significant theoretical techniques from the language community to music. The problems. Although we would like to test how creative the most successful language parsing techniques require the machine is, by using humans we are in a sense testing how development of large annotated corpora, which is riddled creative those people are in ascribing meaning to the with difficulties. Early work in linguistic syntax analysis output. Depending on the background of the person, their also showed that hand authored grammars are extremely experiences and knowledge of the genre any computer- difficult to create, debug and maintain. Until recently the generated material may take on wildly different quality of automatically learned grammars was found to be interpretations for different people, or even the same insufficient for even the most rudimentary tasks. Recent person in different circumstances. While this may be useful progress in unsupervised language parsing techniques have for inspiring new ideas in humans this is not adequately produced results that are much more competitive with state addressing the question “is the computer creative?” The of the art systems, and may provide the ability to identify other concern is epistemic. Even if our human judges were reasonable syntactic structure in music. able to agree on an objective common knowledge for In this paper we argue for the necessity of syntactic grading creativity, it would limit the performance of the processing in musical applications, particularly for two key machine to that of the human intellect. Although the aspects of creativity: enabling human creativity and being human intellect is not likely to be the limiting constraint in innately creative. We propose that unsupervised methods the near future, it is not wise to build such a limitation into offer a uniquely interesting solution to both aspects. To the definition of creativity. Finally, introspection about the demonstrate that these techniques are not just theoretically creation should not just inspire us that it is good but why it possible, we will describe how an unsupervised parsing is good, and afford the possibility of learning something technique developed by Klein and Manning (2002) for use beyond our own biases and capabilities. with language can be used in the musical domain to parse Computational systems can embody these two types of musical melodies. In lieu of an annotated corpus of creative processes in a multitude of ways. At one extreme melodies, we describe a simple experiment that estimates are deterministic rule based systems, and completely the quality of the learned syntactic structures and shows unsupervised learning agents at the other end. Rule based their plausibility and promise for future research. systems are more naturally aligned with enabling humans to be more creative for several reasons. Often, as in the Computational Creativity case of a musical score editor, the arduous tasks are well defined, and a series of rules can be written to alleviate There are two main ways in which computers can play a much of the burden. Also, their behavior is typically significant role in the creative process of musical expected or predictable, which can be beneficial for many innovation. The first is by enabling the human to be more applications, although the system’s predictability usually creative through facilitating mundane or arduous tasks not subverts the possibility of its being creative. Rule based directly focused on the idea, yet are typically necessary for systems are not always predictable however. Wolfram completing the process. For example, a musical score tones4 are a good example of how a seemingly simple rule editor (e.g., Musicease2 or Sibelius3) can greatly ease the can lead to unexpected emergent properties. Dabby (1996) creation and management of writing formal music notation. also describes a music generation system that transforms This can allow more focus and energy to be dedicated to human composed pieces in unpredictable and interesting