A Contemporary Approach to Expressiveness in the Design of Digital Musical Instruments Mathew Dalgleish BA(Hons), MA A thesis submitted in partial fulfilment of the requirements of the University of Wolverhampton for the degree of Doctor of Philosophy 2013 This work or any part thereof has not previously been presented in any form to the University or to any other body whether for the purposes of assessment, publication or for any other purpose (unless otherwise indicated). Save for any express acknowledgments, references and/or bibliographies cited in the work, I confirm that the intellectual content of the work is the result of my own efforts and of no other person. The right of Mathew Dalgleish to be identified as author of this work is asserted in accordance with ss.77 and 78 of the Copyright, Designs and Patents Act 1988. At this date copyright is owned by the author. Signature……………………………………….. Date…………………………………………….. Abstract Digital musical instruments pose a number of unique challenges for designers and performers. These issues stem primarily from the lack of innate physical connection between the performance interface and means of sound generation, for the latter is usually dematerialised. Thus, this relationship must instead be explicitly determined by the designer, and can be essentially any desired. However, many design issues and constraints remain poorly understood, from the nature of control to the provision of performer-instrument feedback. This practice-based research contends that while the digital and acoustic domains are so different as to be fundamentally incompatible, useful antecedents for digital musical instruments can be found in the histories of electronic music. Specifically, it argues that the live electronics of David Tudor are of particular prescience. His home-made circuits offer an electronic music paradigm quite antithetical to both the familiar keyboard interface and the electronic music studios that grew up in the years after World War II, and are seen to embody a number of aspirational qualities. These include performer-instrument interaction more akin to steering rather than fine control, the potential for musical outcomes that are unknown and unknowable in advance, and distinct instrumental character. This leads to the central contribution of this research; the development of a Tudor-inspired conceptual framework that can inform how digital musical instruments are designed, played, and evaluated. To enable more detailed and nuanced discussion, the framework is broken down into a series of sub-themes. These include both design issues such as nuance, plasticity and emergence, and human issues such as experience, expressiveness, skill, learning, and mastery. The notion of sketching in hardware and software is also developed in relation to the rapid iteration of multiple designs. Informed by this framework, seven new digital musical instruments are ii presented. These instruments are tested from two different perspectives, with the personal experiences of the author supplemented with data from a series of small- scale user studies. Particular emphasis is placed on how the instruments are played, the music they can produce, and their capacity to convey the musical intentions of the performer (i.e. their expressiveness). After the evaluation of the instruments, the Tudorian framework is revisited to form the basis of the conclusions. A number of modifications to the original framework are proposed, from the addition of a dialogical model of performer- instrument interaction, to the situation of digital musical instruments within a wider musical ecology. The thesis then closes with a suggestion of possibilities for future research. iii Acknowledgements I would like to thank my supervisory team of Lindsey Marshall, Mark Grimshaw, Matthew Cornford, and Faramarz Amiri for their support and guidance over the past years. I am also indebted to Simon Holland, Anders Bouwer, Rolf Gehlhaar, Colm Hewson, Andrew Lowe, Steve Carson, Tim Collins, Reiko Goto, and Euripredes Altinzoglou for their advice, collaborations, helpful efforts. Special thanks go to the staff and students of the Music Department at the University of Wolverhampton, and to the University of Wolverhampton CETL initiative for funding this programme of study. I would also like to express my gratitude to STEIM, Barry Parsons, Paul Stapleton, Robert Voisey, David Toop, Bill Buxton, Martin Iddon, Brad Garton, Kim Cascone, and the members of the microsound.org, NYC Resistor, RTCmix, and Pd mailing lists, among many others, who have kindly provided me with various opportunities, research materials, images, and other assistance. Special thanks to all those who have made images freely available by default (e.g. under Creative Commons licences), for this has made it possible to illustrate this thesis. Finally, thank you to the developers of the CNMAT, FTM-Gabor, Lobjects, Maxuino, Potpourri, and Soundhack software libraries, for without these tools this work would have been vastly more difficult. iv Contents Abstract.....................................................................................................................ii Acknowledgements..................................................................................................iv Contents.....................................................................................................................v List of Figures............................................................................................................x Glossary..................................................................................................................xiii 1. Introduction Personal Background and Motivations......................................................................1 Terminology...............................................................................................................2 Background to the Research......................................................................................5 The Hypothesis and Research Questions................................................................12 Overview of the Tudorian Framework....................................................................13 The Practice-Based Nature of the Research............................................................15 Roadmap of the Research Project...........................................................................17 • The Written Thesis............................................................................................17 • The Practical Portfolio......................................................................................19 2. Towards a Tudorian Framework for Digital Musical Instruments Overview..................................................................................................................23 Early Developments................................................................................................25 Temporal Machines.................................................................................................30 Live Electronics.......................................................................................................32 From Modular to MIDI Synthesisers......................................................................37 Interfaces for Computer Music................................................................................43 Digital Musical Instruments....................................................................................46 • Sensors..............................................................................................................49 v • Sound Generation..............................................................................................50 • Mapping............................................................................................................51 • Feedback and Sound Diffusion.........................................................................52 Post-Digital Systems................................................................................................54 A Tudorian Framework............................................................................................59 • Emergence.........................................................................................................61 • Nuance...............................................................................................................64 • Skill and Skilling...............................................................................................65 • Plasticity and Meta-plasticity............................................................................66 • Expression, Expressiveness, and Spectacle.......................................................69 • The Human Turn...............................................................................................76 • (The Nature of) Experience...............................................................................78 • Long-term Engagement, Learning, and Mastery...............................................81 Summary..................................................................................................................88 3. Methods and Methodologies Overview..................................................................................................................90 Towards a Digital Musical Instrument Methodology..............................................91 Design and Implementation.....................................................................................92