Aesthetically Driven Design of Network Based Multi-User Instruments by Curtis McKinney A thesis submitted to the faculty of Bournemouth University in partial fulfilment of the requirements for the degree of Doctor of Philosophy in the School of Design, Engineering, and Computing July 2014 This copy of the thesis has been supplied on condition that anyone who consults it is understood to recognise that its copyright rests with its author and due acknowl- edgement must always be made of the use of any material contained in, or derived from, this thesis. I Aesthetically Driven Design of Network Based Multi-User Instruments Curtis McKinney Abstract Digital networking technologies open up a new world of possibilities for mu- sic making, allowing performers to collaborate in ways not possible before. Net- work based Multi-User Instruments (NMIs) are one novel method of musical collaboration that take advantage of networking technology. NMIs are digital musical instruments that exist as a single entity instantiated over several nodes in a network and are performed simultaneously by multiple musicians in real- time. This new avenue is exciting, but it begs the question of how does one design instruments for this new medium? This research explores the use of an aesthetically driven design process to guide the design, construction, rehearsal, and performance of a series of NMIs. This is an iterative process that makes use of a regularly rehearsing and performing ensemble which serves as a test-bed for new instruments, from conception, to design, to implementation, to performance. This research includes details of several NMIs constructed in accordance with this design process. These NMIs have been quantitatively analysed and empiri- cally tested for the presence of interconnectivity and group influence during per- formance as a method for measuring group collaboration. Furthermore qualita- tive analyses are applied which test for the perceived effectiveness of these in- struments during real-world performances in front of live audiences. The results of these analyses show that an aesthetically driven method of designing NMIs produces instruments that are interactive and collaborative. Furthermore results show that audiences perceive a measurable impression of interconnectivity and liveness in the ensemble even though most of the performers in the ensemble are not physically present. II Contents 1 Introduction 1 1.1 Research Questions . .3 1.2 Aims and Objectives . .4 1.3 Contribution to Knowledge . .5 2 Multi-User Musical Instruments 7 2.1 Definitions . .7 2.2 Structural Properties of Multi-User Instruments . .8 2.3 Models for Multi-User Instruments . .9 2.4 A Dimension Space for Collaboration . 11 2.5 A Survey of Multi-User Instruments . 12 2.5.1 Utilitarian Multi-User Instruments . 12 2.5.2 Extended Traditional Instruments . 13 2.5.3 Surface Instruments . 15 2.5.4 Interconnected Laptop Ensembles . 19 2.5.5 Cloud Instruments . 23 2.5.6 Kinetic Group Instruments . 27 2.5.7 Multiplayer Game Instruments . 30 2.6 A Final Word on Dimension Spaces . 35 2.7 Conclusions . 35 3 Aesthetically Driven Iterative Design Methodology 37 3.1 Linearity, Improvisation, and Aleatory . 38 3.2 Beauty, Play, and Viscerality . 41 3.3 Collaboration and Camaraderie . 43 3.4 Performance and Liveness . 45 3.5 Limitations of Approach . 47 3.6 Conclusion . 48 4 Design, Development, and Composition 49 4.1 Initialising the Design Space . 49 4.2 Strategies . 50 4.3 OSCthulhu . 52 4.3.1 NAT Hole-Punching and UDP multicasting . 52 4.3.2 Reality of the Internet: Packet Loss . 54 4.3.3 OSCthulhu 2.0 . 57 4.3.4 OSCthulhu and OSCgroups: A comparison . 60 4.4 Medusa . 60 4.4.1 Neuromedusae I . 62 III 4.4.2 Neuromedusae II . 64 4.5 Renditions . 67 4.5.1 Technical Overview . 68 4.5.2 Structure and Performance . 68 4.6 Curse of Yig . 69 4.7 Leech . 71 4.7.1 Technological Overview . 71 4.7.2 Mapping Data . 72 4.7.3 Artistic Considerations . 77 4.8 Flow . 78 4.9 Mutagen . 80 4.9.1 Networking a DAW . 80 4.9.2 Glitches....And Not The Good Kind . 81 4.10 Simulacra . 82 4.10.1 Failure and Reboot . 82 4.10.2 Composition and Sonic Infrastructure . 83 4.11 Azathoth . 86 4.11.1 Features . 86 5 Analysis of Work 90 5.1 Divergence Test of OSCthulhu . 90 5.1.1 Results . 91 5.1.2 Benefits of Convergence . 92 5.2 Anatomy of a Performance . 93 5.3 Collaborative Dimension Spaces . 112 6 Conclusion 118 6.1 Empirical Findings . 118 6.1.1 What is a multi-user instrument, and how is it defined? . 119 6.1.2 Are there distinct morphologies of multi-user instruments, and may a taxonomy be created to organise them? . 119 6.1.3 Is there a manner in which to examine the collaborative capabilities of a multi-user instrument? . 121 6.1.4 Is a methodology for designing NMI’s based on the aes- thetics of a specific ensemble able to generate NMI’s which satisfy the design requirements generated by that ensemble?122 6.1.5 May the issues of liveness and disembodiment inherent to NMI’s be addressed in some manner? . 126 IV 6.1.6 Are there better techniques for overcoming the technical difficulties involved with networking geographically dis- placed ensembles? . 127 6.1.7 Is there a way to streamline the development processes of creating NMI’s? . 127 6.2 Implications, Future Work, and Conclusion . 128 References 130 Appendices 139 A Simulacra Control Signal Comparisons 139 B Novelty Curve Peaks 151 C Texas A & M performance tweet logs 156 D Mute Magazine performance chat logs 159 V List of Figures 1 Collaborative dimension spaces for Utilitarian Multi-User Instru- ments. 13 2 Collaborative dimension spaces for Extended Traditional Instru- ments. 16 3 Collaborative dimension spaces for Surface Instruments. 18 4 Collaborative dimension spaces for Interconnected Laptop En- sembles. 22 5 Collaborative dimension spaces for Cloud Instruments. 26 6 Collaborative dimension spaces for kinetic group instruments. 31 7 Collaborative dimension spaces for multiplayer game instruments. ................................... 34 8 NAT hole punching process (Ford et al., 2005). 53 9 Screen capture of the online multiplayer game Unreal....... 55 10 Depiction of sending a setSyncArg message using OSCthulhu. 57 11 Screen capture of the SyncObject and Chat window in OSCthulhu 2.0 .................................. 59 12 Screen capture of a performance using the Medusa System . 61 13 Synthesis example from NeuroMedusae I.............. 63 14 Synthesis example from NeuroMedusae II............. 66 15 Sound visualisation during a performance of Renditions...... 67 16 Yig, a feedback based network music instrument. 69 17 Yig in performance at the Network Music Festival 2012. 70 18 Visualisation of network data in Leech ............... 73 19 File transfer sonification code in SuperCollider. 74 20 Packet capture sonification code in SuperCollider. 76 21 Pirated MP3 playback code in SuperCollider. 76 22 Public performance of Flow at the Public Domain arts festival. 79 23 Screen capture of the Mutagen sequencer. 81 24 Visuals in Simulacra ........................ 82 25 Performance of Simulacra at the Network Music Festival 2013 . 84 26 Synthesis example from Simulacra................. 85 27 OscGroups divergence over time. 91 28 OSCthulhu divergence over time. 92 29 Timeline of a live performance of Simulacra in London, England. 95 30 User connectome in Simulacra................... 97 31 Percentage of connectivity participation in Simulacra. Connec- tions to others (red outlines) vs self (no outline). 98 32 Percentage of bandwidth usage in Simulacra ........... 99 VI 33 Object creation and destruction activity in Simulacra ....... 100 34 Self-similarity matrix of a live performance of Simulacra ..... 102 35 Self-similarity matrix for casiosk1’s audio signal in Simulacra .. 103 36 Self-similarity matrix for casiosk1’s control signal in Simulacra . 104 37 Comparison of casiosk1’s raw control signal (top), and it’s com- puted novelty(bottom) over time in Simulacra ........... 104 38 Self-similarity matrix for octopian’s audio signal in Simulacra .. 105 39 Self-similarity matrix for octopian’s control signal in Simulacra . 106 40 Comparison of octopian’s raw control signal (top), and it’s com- puted novelty(bottom) over time in Simulacra ........... 106 41 Self-similarity matrix for 55hz’s audio signal in Simulacra .... 107 42 Self-similarity matrix for 55hz’s control signal in Simulacra ... 108 43 Comparison of 55hz’s raw control signal (top), and it’s computed novelty(bottom) over time in Simulacra .............. 108 44 Collaborative dimension spaces for NeuroMedusae I ....... 112 45 Collaborative dimension spaces for NeuroMedusae II ....... 113 46 Collaborative dimension spaces for Curse of Yig ......... 114 47 Collaborative dimension spaces for Leech ............. 114 48 Collaborative dimension spaces for Simulacra ........... 115 49 Similarity matrix comparing randomly generated noise with ca- siosk1’s audio stream in Simulacra ................ 139 50 Similarity of randomly generated noise with casiosk1’s audio stream in Simulacra plotted over time. 139 51 Similarity matrix comparing casiosk1’s control stream with ca- siosk1’s audio stream in Simulacra ................ 140 52 Similarity of casiosk1’s control stream with casiosk1’s audio stream in Simulacra plotted over time. 140 53 Similarity matrix comparing octopian’s control stream with ca- siosk1’s audio stream in Simulacra ................ 141 54 Similarity of octopian’s control stream with casiosk1’s audio stream in Simulacra plotted over time. 141 55 Similarity matrix comparing 55hz’s control stream with casiosk1’s audio stream in Simulacra ..................... 142 56 Similarity of 55hz’s control stream with casiosk1’s audio stream in Simulacra plotted over time. 143 57 Similarity matrix comparing randomly generated noise with oc- topian’s audio stream in Simulacra ................. 144 58 Similarity of randomly generated noise with octopian’s audio stream in Simulacra plotted over time.