Canterbury Christ Church University’s repository of research outputs http://create.canterbury.ac.uk Copyright © and Moral Rights for this thesis are retained by the author and/or other copyright owners. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This thesis cannot be reproduced or quoted extensively from without first obtaining permission in writing from the copyright holder/s. The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the copyright holders. When referring to this work, full bibliographic details including the author, title, awarding institution and date of the thesis must be given e.g. Kakinoki, M. (2017) The development of an audio-visual language for digital music performance. Ph.D. thesis, Canterbury Christ Church University. Contact: [email protected] The Developet of a Audio-Visual Laguage for Digital Music Perforace by Masato Kakinoki, Canterbury Christ Church University Thesis submitted for the Degree of Doctor of Philosophy Year 2017 0 Contents Abstract ............................................................................................................ iii Introduction ....................................................................................................... 1 Visuals in digital music performance .............................................................. 6 Chapter 1: Using visuals in digital music performance ................................................... 6 Chapter 2: Key visual aspects ........................................................................................... 21 1. The performer’s physical body and her/his actions as visual information .................. 21 2. Theatrical actions......................................................................................................... 32 3. Parameters of mediatised visuals................................................................................. 36 4. The nature of the media used for mediatised visuals .................................................. 44 Liveness and causality of digital music performance .................................. 47 Chapter 3: Bodiness in digital music performance......................................................... 47 Chapter 4: Hardware devices ........................................................................................... 56 1. Mapping ....................................................................................................................... 58 2. Cockpit-type hardware ................................................................................................ 61 3. Muscle memory ........................................................................................................... 62 4. Unpredictability ........................................................................................................... 64 5. Mistakes ....................................................................................................................... 65 6. Laptop .......................................................................................................................... 66 Chapter 5: Fixed digital music performance and fluid digital music performance .... 70 1. Fluidity ........................................................................................................................ 72 2. Sequencer and pre-recorded materials in music performance ..................................... 75 3. Legibility ..................................................................................................................... 80 4. Simplicity and constraints ........................................................................................... 83 Chapter 6: Portfolio ........................................................................................................... 88 1. Design of audio engine ................................................................................................ 90 2. Design of control mapping engine ............................................................................ 113 3. Design of visual engine ............................................................................................. 115 4. Development of the performance system .................................................................. 122 Conclusion ...................................................................................................... 124 References ...................................................................................................... 128 Appendices ..................................................................................................... 144 1. Video examples ............................................................................................................. 144 2. Max patch, TouchDesigner project file and configuration file for MIDI Fighter Twister .............................................................................................................................. 144 i Table of figures Figure 1 ........................................................................................................................ 88 Figure 2 ........................................................................................................................ 89 Figure 3 ........................................................................................................................ 89 Figure 4 ........................................................................................................................ 90 Figure 5 ........................................................................................................................ 91 Figure 6 ........................................................................................................................ 93 Figure 7 ........................................................................................................................ 94 Figure 8 ........................................................................................................................ 95 Figure 9 ........................................................................................................................ 96 Figure 10 ...................................................................................................................... 97 Figure 11 ...................................................................................................................... 98 Figure 12 ...................................................................................................................... 99 Figure 13 .................................................................................................................... 100 Figure 14 .................................................................................................................... 101 Figure 15 .................................................................................................................... 103 Figure 16 .................................................................................................................... 104 Figure 17 .................................................................................................................... 105 Figure 18 .................................................................................................................... 106 Figure 19 .................................................................................................................... 107 Figure 20 .................................................................................................................... 108 Figure 21 .................................................................................................................... 109 Figure 22 .................................................................................................................... 110 Figure 23 .................................................................................................................... 111 Figure 24 .................................................................................................................... 116 Figure 25 .................................................................................................................... 116 Figure 26 .................................................................................................................... 117 Figure 27 .................................................................................................................... 118 Figure 28 .................................................................................................................... 119 ii Abstract This practice-based PhD consists of a portfolio of creative work and a supporting commentary. The portfolio illustrates the design decisions relating to my digital music performance system, and focuses upon the visibility and the fluidity of digital music performance. The goal of the design is to enhance the visibility without violating the audience’s auditory imagination unnecessarily, and to enhance the fluidity without relinquishing the unique fixed nature of digital music. The performance system consists of an audio engine, control mapping engine and visual engine. The audio engine and the control mapping engine were programmed with Cycling ‘74 Max. They let the performer deconstruct and reconstruct