The Development of an Audio-Visual Language for Digital Music Performance by Masato Kakinoki Canterbury Christ Church University Thesis submitted for the Degree of Doctor of Philosophy Year 2017 0 Contents Acknowledgements .......................................................................................... iii Abstract ............................................................................................................. iv Introduction ....................................................................................................... 1 Visuals in digital music performance .............................................................. 6 Chapter 1: Using visuals in digital music performance ................................................... 6 Chapter 2: Key visual aspects ........................................................................................... 21 1. The performer’s physical body and her/his actions as visual information .................. 21 2. Theatrical actions......................................................................................................... 33 3. Parameters of mediatised visuals................................................................................. 36 4. The nature of the media used for mediatised visuals .................................................. 44 Liveness and causality of digital music performance .................................. 47 Chapter 3: Bodiness in digital music performance......................................................... 47 Chapter 4: Hardware devices ........................................................................................... 57 1. Mapping ....................................................................................................................... 59 2. Cockpit-type hardware ................................................................................................ 62 3. Muscle memory ........................................................................................................... 63 4. Unpredictability ........................................................................................................... 65 5. Mistakes ....................................................................................................................... 66 6. Laptop .......................................................................................................................... 67 Chapter 5: Fixed digital music performance and fluid digital music performance .... 71 1. Fluidity ........................................................................................................................ 73 2. Sequencer and pre-recorded materials in music performance ..................................... 77 3. Legibility ..................................................................................................................... 82 4. Simplicity and constraints ........................................................................................... 85 Chapter 6: Portfolio ........................................................................................................... 89 1. Design of audio engine ................................................................................................ 91 2. Design of control mapping engine ............................................................................ 114 3. Design of visual engine ............................................................................................. 116 4. Development of the performance system .................................................................. 123 Conclusion ...................................................................................................... 125 References ...................................................................................................... 129 Appendices ..................................................................................................... 145 1. Video examples ............................................................................................................. 145 2. Max patch, TouchDesigner project file and configuration file for MIDI Fighter Twister .............................................................................................................................. 145 i Table of figures Figure 1 ........................................................................................................................ 89 Figure 2 ........................................................................................................................ 90 Figure 3 ........................................................................................................................ 90 Figure 4 ........................................................................................................................ 91 Figure 5 ........................................................................................................................ 92 Figure 6 ........................................................................................................................ 94 Figure 7 ........................................................................................................................ 95 Figure 8 ........................................................................................................................ 96 Figure 9 ........................................................................................................................ 97 Figure 10 ...................................................................................................................... 98 Figure 11 ...................................................................................................................... 99 Figure 12 .................................................................................................................... 100 Figure 13 .................................................................................................................... 101 Figure 14 .................................................................................................................... 102 Figure 15 .................................................................................................................... 104 Figure 16 .................................................................................................................... 105 Figure 17 .................................................................................................................... 106 Figure 18 .................................................................................................................... 107 Figure 19 .................................................................................................................... 108 Figure 20 .................................................................................................................... 109 Figure 21 .................................................................................................................... 110 Figure 22 .................................................................................................................... 111 Figure 23 .................................................................................................................... 112 Figure 24 .................................................................................................................... 117 Figure 25 .................................................................................................................... 117 Figure 26 .................................................................................................................... 118 Figure 27 .................................................................................................................... 119 Figure 28 .................................................................................................................... 120 ii Acknowledgements This research was supported by Canterbury Christ Church University Graduate School, and partially supported by Canterbury Christ Church University School of Music and Performing Arts and Canterbury Christ Church University Faculty of Arts and Humanities. I would like to thank Akihiro Kubota and Markus Miessen for their generous gesture of sending me the PDF files of their books, which I could not obtain. The advice given by Yuko Izumi has been a great help to me in structuring and organising the thesis. The earliest practices of the performance system in front of an audience would not have happened without support from Nick Luscombe and Mark Rappolt. The research would not exist without the teachers I have had in the past: Muneya Kato, Takaaki Kumakura, Hiroshi Muto, Kiyoshi Yasutomi, Ed Hughes, Nick Collins, Tim Hopkins, and Sally-Jane Norman. I appreciate the support from Nicholas McKay, Robert Rawson, and the Graduate School at Canterbury Christ Church University. The development of the research would not have been possible without the critical advice and feedback given by my supervisors, Matt Wright and Panos Ghikas, who have always been passionate and encouraged me to keep improving on what I have achieved. Special thanks to my family and friends for believing in me throughout the journey. London, 30th October 2017 iii Abstract This practice-based PhD consists of a portfolio of creative work and a supporting commentary. The portfolio illustrates the design decisions relating to my