Musical Path Planning for an Improvising Robot
Total Page:16
File Type:pdf, Size:1020Kb
Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17) Integrating the Cognitive with the Physical: Musical Path Planning for an Improvising Robot Mason Bretan Gil Weinberg [email protected] [email protected] Georgia Institute of Technology Georgia Institute of Technology Atlanta, GA Atlanta, GA Abstract These benefits, however, do not come without the addi- tional constraints that are coupled to the natural world in Embodied cognition is a theory stating that the processes and which we live. An artificial intelligence (AI) controlling a functions comprising the human mind are influenced by a per- mechanical body that is bound to the physical laws of na- son’s physical body. Embodied musical cognition is a theory ture must address its own presence in 3-dimensional space of the musical mind stating that the person’s body largely influences his or her musical experiences and actions (such in order to function properly. This includes not only know- as performing, learning, or listening to music). In this work, ing the location of each of its degrees-of-freedom (Dofs), a proof of concept demonstrating the utility of an embod- but also understanding how multiple DoFs need to behave ied musical cognition for robotic musicianship is described. and work together in order to complete a task. Perhaps most Though alternative theories attempting to explain human mu- importantly, the AI must also understand when a task is im- sical cognition exist (such as cognitivism and connectionism), possible (given its physicality) and anticipate failure before this work contends that the integration of physical constraints it causes damage to itself or corrupts certain aspects of the and musical knowledge is vital for a robot in order to opti- task making success completely unattainable even with ad- mize note generating decisions based on limitations of sound ditional help. generating motion and enable more engaging performance through increased coherence between the generated music In the musical domain these constraints are often (if not and sound accompanying motion. Moreover, such a system always) compounded by issues of timing. It is not enough to allows for efficient and autonomous exploration of the rela- simply arrange the DoFs in a desired order; the timing and tionship between music and physicality and the resulting mu- sequencing of the movements must also be considered. Of- sic that is contingent on such a connection. ten, depending on the time constraint and robot’s physical design, the path in which each DoF moves and relocates it- self must be optimized in order to reach specified locations Introduction in a timely manner. This constraint satisfaction process of developing coordinated movements is referred to as ‘path Several advantages and opportunities emerge as a result of planning’. the physical embodiment of machine musicians compared to that of their pure software counterparts. These “robotic mu- Here, we explore path planning and its relationship to the sicians” assume additional abilities to entertain, engage, so- notion of embodied musical cognition, which states that an cialize, and produce sound. Such desirable capacities are in- individuals’s body largely influences his or her understand- trinsic to most social robotic platforms in general, as a result ing, experience, and decision processes pertaining to music of physical presence and embodiment, and benefit both those (Godøy and Leman 2010; Leman 2008). In particular, we directly immersed in the interaction as well as those sim- examine how propioception and embodiment can (and we ply witnessing it (Kidd and Breazeal 2004). Though many argue should) influence the musical decision process of an of these interactive assets must be explicitly designed to improvising robot musician. The hypothesis is that a robot address human perceptual and social tendencies, in music that utilizes a music generation method that jointly opti- some advantageous characteristics arise entirely as a result mizes for its physical constraints as well as its general mu- of the inherent coupling between the robot’s spatial move- sical knowledge will increase performative expressivity by ments and sound generation. The byproduct of sound gen- providing increased coherence among higher level musical erating movements is increased levels of rhythmic coordi- ideas, sound generating motions, and sound accompanying nation and synchronization within ensembles because inter- motions. Additionally, such an integrated musical decision acting musicians are able to anticipate the robot’s musical process will result in music that is defined by the robot’s onsets or behaviors through visual cues (Hoffman and Wein- physical identity, hopefully leading to interesting music and berg 2010; Lim et al. 2010). even entirely new genres. Traditionally, machine musicianship has focused on soft- Copyright c 2017, Association for the Advancement of Artificial ware applications that respond to and generate music (Lewis Intelligence (www.aaai.org). All rights reserved. 2000; Drummond 2009; Whalley 2010). Robotic musician- 4371 ship has been defined to be comprised of two components system should generate paths with lengths that represent (Bretan and Weinberg 2016). One of which is ‘machine mu- complete musical ideas (though it is possible a complete sicianship’ or the development of cognitive models depict- idea is indeed only a single note). These musical chunks ing aspects of music perception, composition, performance, may be portions of phrases, complete phrases, or even en- and theory. The other component is ‘musical mechatronics’ tire structured improvisations from beginning to end. or the study and construction of mechanical systems capable of sound generation. 4. Is there a single integration approach that can be use- In musical mechatronics there are several things that a re- ful for many robotic platforms that have vastly dif- searcher considers. There is no single physical design of a ferent designs and functionalities? A joint optimiza- musical robot that is perfect and there have been no claims tion methodology and single algorithm may be suitable that one such design exists. Instead, designers make trade for many music generating robots, however, a single state offs that may account for a robot’s size, mass, possible an- space representing all platforms is probably not possible. thropomorphic design, the instrument it will play, the spe- The physical design of robotic musicians tend to vary sig- cific genre(s) it will play, method of sound actuation, abil- nificantly from platform to platform. Instead, adjustments ity to provide useful visual cues, the ability to provide so- will likely need to be made that address a specific robot’s cial cues, energy consumption, price, and aesthetics. In other physical characteristics and intended interactions and be- words, designers make decisions that are influenced by both haviors. music-specific ambitions and physical characteristics. In this remainder of this paper we describe specific mo- This paper focuses on the machine musicianship aspect tivations for such a joint optimization music generation of robotic musicianship and it presents the notion that the method. We discuss related work in the music and cogni- music making decisions of a generative algorithm must also tive science domains and address how embodied cognition be influenced by the robot’s own physicality. Similarly to concepts have influenced our design. Finally, we provide the processes that go into formulating the physical design an overview to our generative musical path planning system of a musical robot, a robotic musician should have an intel- that creates different musical motifs in which the source of ligence that integrates the cognitive and physical domains variance stems from the physical constraints. such that the musical and physical behaviors are a result of a decision process that jointly optimizes musical goals, path planning, and human perception. Motivation This work addresses several research questions: Though there are various theories of mind and several exam- 1. How can autonomous decision processes based on mu- ples of robots being used to study natural human behavior sic incorporate the physical domains? Including addi- and brain science (Atkeson et al. 2000; Cheng et al. 2007; tional constraints on music requires a rethinking of the Asada et al. 2009), our work does not attempt to prove or traditional machine musicianship concepts. Though pre- disprove one particular theory. Rather, we demonstrate why vious methods for generating music are still relevant and an algorithmic design inspired by embodied cognitive pro- can be useful resources, an integrated approach needs to cesses is more suited for robotic musicianship than disem- address a significant expansion of parameters and com- bodied cognitivist approaches. The general premise is that plexity. Specifically, the note-level stochastic methods embodied cognitive methods would enable a robotic musi- that are widely used may not be feasible for an integrated cian to find the most effective solution in conveying its mu- optimization method. sical goals given its physical constraints. Additionally, qualitative and anecdotal evidence suggests 2. What is the best way to represent the physicality of that some of the higher level musical semantics that describe the robot within the decision process? Some DoFs may a person’s style emerge as a result of that person’s physical need to be expressed individually (such as those