Methods to Support End User Design of Arrangement-Level Musical Decision Making
Total Page：16
File Type：pdf, Size：1020Kb
COPYRIGHT AND USE OF THIS THESIS This thesis must be used in accordance with the provisions of the Copyright Act 1968. Reproduction of material protected by copyright may be an infringement of copyright and copyright owners may be entitled to take legal action against persons who infringe their copyright. Section 51 (2) of the Copyright Act permits an authorized officer of a university library or archives to provide a copy (by communication or otherwise) of an unpublished thesis kept in the library or archives, to a person who satisfies the authorized officer that he or she requires the reproduction for the purposes of research or study. The Copyright Act grants the creator of a work a number of moral rights, specifically the right of attribution, the right against false attribution and the right of integrity. You may infringe the author’s moral rights if you: - fail to acknowledge the author of this thesis if you quote sections from the work - attribute this thesis to another author - subject this thesis to derogatory treatment which may prejudice the author’s reputation For further information contact the University’s Director of Copyright Services sydney.edu.au/copyright Methods to Support End User Design of Arrangement-Level Musical Decision Making Aengus Martin 2014 A thesis submitted in fulfilment of the requirements for the degree of Doctor of Philosophy Faculty of Engineering and Information Technologies Abstract This thesis is concerned with the study of methods and models to support the design of systems that perform music autonomously, by non-programming end users (typically musicians who are not proficient in conventional computer programming). Specifically, we address the design of the central decision making component of such systems. This component typically makes musical decisions on the time scale of a few seconds and we refer to it as a musical agent. We use the term arrangement-level musical decision making to refer to the activity performed by a musical agent. We develop and characterise three separate methods for supporting the design of musical agents by non-programming end users. The first is to use the partially observable Markov decision process (POMDP) which is an elegant model of the in- teraction between an agent and its environment. We show its potential for designing the responses of a musical agent to particular stimuli. For example, we demonstrate how, simply by adjusting the parameters of the model, an agent’s behaviour can be varied between ‘cautious’ and ‘risk-taking’, when there is uncertainty about a musical situation. However, we identify significant challenges with regard to the use of POMDPs more generally for designing musical agents. Specifically, it is unclear what representations of musical information should be used and how to design the reward function, which is an important parameter of the model. The second method is based on the paradigm of programming by example. We introduce the similarity-based musical agent which uses a novel instance-based machine learning algorithm to learn the style in a database of example performances. While the similarity-based musical agent shows good potential for emulating a iv particular style of decision making, it requires a lot of training data: around twenty example performances were needed to emulate two relatively simple behaviours accurately. We regard this as a critical issue, envisaging that a requirement to supply so many example performances in order to create even a simple musical agent, would constitute a significant barrier to the adoption of this system. The third method explored involves combining programming by example with a mechanism whereby a musician can embed musical knowledge into an agent. The main contribution of this work is a new piece of software called the Agent Designer Toolkit (ADTK) which supports this novel paradigm for designing musical agents. We show that the ADTK can be used to create agents that convincingly emulate styles of arrangement-level musical decision making in a wide variety of musical contexts, both mainstream and experimental, with small sets of example performances. Furthermore, the agents are often very quick to create. The ADTK defines a novel class of models comprising a combination of variable order Markov models, association rules and other user-defined relationships between variables. To use these models in music performance, a new method was developed for computing musical decisions represented as constraint satisfaction problems subject to real-time constraints. The method is based on binary decision diagrams and it may be applicable in a variety of other real-time computer music applications. The ADTK uniquely fulfils our goals in that it does not require the user to have any expertise in conventional computer programming. In addition, it can be seam- lessly embedded in two popular music software packages so that autonomous music systems can be created entirely inside a standard music production environment. While we identify certain usability issues with the software in its current incarnation, we show the promise of a number of strategies for mitigating them. For example, we provide support for the idea that a set of widely applicable preset configurations can be identified and these will make the software much more accessible. Finally, in addition to its use in music performance, we show the potential of the ADTK for a variety of other creative uses such as the generation of new musical ideas. Publications Following is a list of peer-reviewed publications written over the course of this thesis-work. Those arising directly from the research herein are marked with stars (?): A. Eigenfeldt, O. Bown, P. Pasquier and A. Martin, ‘The First Musical Metacre- ation Weekend: Towards a Taxonomy of Musical Metacreation,’ 2nd International Workshop on Musical Metacreation, Boston, MA (2013) (to appear) O. Bown, A. Eigenfeldt, P. Pasquier, B. Carey and A. Martin, ‘The Musical Metacreation Weekend: Challenges arising from the live presentation of musically metacreative systems,’ 2nd International Workshop on Musical Metacreation, Boston, MA (2013) (to appear) ? A. Martin and O. Bown, ‘The Agent Designer Toolkit,’ Demo at the 9th ACM Conference on Creativity and Cognition, Sydney, Australia (2013) ? O. Bown and A. Martin, ‘Backgammon,’ Live Performance with musical agents at the 9th ACM Conference on Creativity and Cognition, Sydney, Australia (2013) S. Ferguson, A. Johnston and A. Martin, ‘A Corpus-Based Method for Con- trolling Guitar Feedback,’ New Interfaces for Musical Expression, Seoul, Korean Republic (2013) O. Bown and A. Martin, ‘Autonomy in Music-Generating Systems,’ 1st Interna- tional Workshop on Musical Metacreation, Palo Alto, California (2012) ? A. Martin, C.T. Jin, B. Carey and O. Bown, ‘Creative Experiments Using a System for Learning High-Level Performance Structure in Ableton Live,’ Sound and Music Computing Conference, Copenhagen, Denmark (2012) vi ? A. Martin, C.T. Jin and O. Bown, ‘Implementation of a real-time musical decision-maker,’ Australasian Computer Music Conference, Brisbane, Australia (2012) K. Beilharz and A. Martin ‘The “Interface” in Site-Specific Sound Installation,’ New Interfaces for Musical Expression, Ann Arbour, Michigan (2012) ? A. Martin, C. Jin and O. Bown, ‘A Toolkit for Designing Interactive Musical Agents,’ Proceedings of OZCHI 2011, Canberra, Australia (2011) A. Martin and K. Beilharz, ‘Windtraces: Accessible Sonic Art,’ Proceedings of CreateWorld, Brisbane, Australia (2011) ? A. Martin, C. Jin, A. McEwan and W.L. Martens, ‘A Similarity Algorithm for Interactive Style Imitation,’ International Computer Music Conference, Huddersfield, UK (2011) A. Martin, S. Ferguson, K. Beilharz, ‘Mechanisms for Controlling Complex Sound Sources: Applications to Guitar Feedback Control,’ New Interfaces for Mu- sical Expression, Sydney (2010) ? A. Martin, C. Jin, A. van Schaik and W.L. Martens, ‘Partially Observable Markov Decision Processes for Interactive Music Systems,’ International Computer Music Conference, New York (2010) Acknowledgements There are a great many people without whom the construction of this thesis would have been (i) completely impossible, (ii) much more difficult, or (iii) far less tolerable. I will leave it to the individuals themselves to figure out in which category or categories they reside. First, I would like to thank my supervisor, Craig Jin, who from our first meeting at ICAD in Limerick, Ireland, gave me great encouragement, first to come and work as an audio programmer and then to pursue a PhD at the Computing and Audio Research Lab (CARLab) at Sydney University. Craig, along with André van Schaik and Jeffrey Shaw, was good enough to provide much of the support for said PhD from Australian Research Council Linkage Grant LP0669163, and for that I am especially grateful1. I would like to extend my heartfelt gratitude to Ollie Bown for becoming my associate supervisor and a hugely important influence on my work. His introduction of Ableton Live into the mix (no pun intended) was the perfect piece of the puzzle at the perfect time. My gratitude also goes to Bill Martens for his fantastic energy and understanding as Ollie’s predecessor and for seeing that Ollie would be an excellent fit. I extend my warmest high fives and embraces (embodied, of course, in sedate handshakes) to a great many students, postdocs, research assistants, staff, interlopers and individuals of nonspecific role, from CARLab and later also from the Computing 1I am additionally appreciative of the University of Sydney International Research Scholarship and the Norman I. Price scholarship from the School of Electrical and Information Engineering. viii Engineering Laboratory at Sydney University. In particular (but in no particular order), to Abhaya, Alan, David, Sean, Tara, Andrew, Tony, Alistair, Joubin, Roman, Nico, Pierre, Trang, Nick, Mike, Mahendra and Gaetano, I say thanks for the many discussions, laughs, beers and barbecues that we shared. At a number of points during the research reported herein, I cold-called re- searchers from fields unfamiliar to me (at the time), shamelessly looking for free expertise. Their positive responses amazed me and helped a great deal.