Studio Composition Live audiovisualisation using emergent generative systems Ryo Ikeshiro Thesis submitted for the degree of Doctor of Philosophy Goldsmiths, University of London August 2013 Declaration I hereby declare that the work presented in this thesis is my own. Signature: Ryo Ikeshiro 2 Abstract The aim of this research is to explore new creative outcomes of the use of emergent generative systems as material in audio and audiovisual experimental art. The portfolio includes the following compositions: Construction in Self, Construction in Zhuangzi and Construction in Kneading, the last two each comprising of three separate pieces. The mathematical systems used are the Lorenz dynamical system and the Mandelbox fractal. Through considering their potential for emergence, the aesthetic possibilities that generative systems offer in the context of experimental computational art are explored. The approach is initially investigated in the audio domain in CiS, a generative electronic music work. CiZ and CiK, the two main sets of works that comprise the portfolio, then explore the technique of “live audiovisualisation”: the simultaneous sonification and visualisation of the same source of data in real-time. Aesthetic considerations of the use of data as sound and moving image and their combination is discussed with reference to research into auditory displays, experimental film and perception. The techniques used include my approach of “self-similar sonification”: the presentation of data as audio at multiple time- scales, including at audio rate by means of non-standard synthesis or audification. All the works are implemented in the programming environment Max/MSP/Jitter. 3 Acknowledgements I would like to thank my supervisors, Michael Young and Mick Grierson, for their support and guidance during my time at Goldsmiths. I would also like to thank the various teachers, colleagues and friends who have helped me along the way. A special thank you goes to my family and Tim and Kit Kemp for their generous support that made this PhD possible. 4 Contents List of tables and figures!...............................................................9 Notes on Movie examples!...........................................................10 Notes on Sound examples!..........................................................11 Research outputs!....................................................12 Creative work!..............................................................................12 Peer-reviewed papers!.................................................................13 Introduction!..............................................................15 Part I: Theory!.........................................................17 Computational Art!....................................................17 Separation of technique and aesthetics!......................................17 Institutional and underground!.....................................................19 Software art and generative art!...................................................20 Concept and conceptual!.............................................................22 Complexity!..................................................................................23 Emergence!..............................................................24 History of emergence!..................................................................24 Strong and weak emergence!......................................................25 Noise!...........................................................................................27 Aesthetics!................................................................28 Artificial Life!................................................................................28 Art-as-it-could-be!........................................................................29 5 Experimental art!..........................................................................31 Machinic!......................................................................................33 Cyborg!........................................................................................34 (New) Aesthetic!...........................................................................35 Alien phenomenology!.................................................................36 Alien music!..................................................................................37 Systems!..................................................................39 The Lorenz system!......................................................................39 Sensitive dependence on initial conditions!.................................40 Attractors and basins of attraction!..............................................41 Arrow of time!...............................................................................42 Vector fields, perturbations, phase transitions!............................43 Ordinary and singular events!......................................................43 Second type of first-order emergence!.........................................44 Mandelbox!...................................................................................44 Dimensions!.................................................................................45 The baker’s map!.........................................................................46 Squares and circles!....................................................................47 Part II: Techniques!................................................48 Data as Audio!..........................................................48 Auditory display and music!.........................................................48 Self-similar sonification!...............................................................50 Self-similarity!...............................................................................50 Audio rate!....................................................................................51 6 Audification!.................................................................................52 Non-standard synthesis!..............................................................53 Control rate!.................................................................................55 Inflection point triggering!.............................................................56 Elegance!.....................................................................................57 Live Audiovisualisation!............................................58 Integration of sound and image!..................................................59 Structural/Materialist film!.............................................................60 Recent examples!........................................................................62 Musical texture metaphors!..........................................................63 Auditory displays and perception!................................................65 Cross-modal phenomena!............................................................66 Added value and synchresis!.......................................................68 Superfield!....................................................................................69 Camera position!..........................................................................70 Synaesthesia!..............................................................................70 Live performance!........................................................................71 Part III: Practice!.....................................................72 Construction in Self!.................................................72 Introduction!.................................................................................72 Key concepts!..............................................................................72 Form!............................................................................................73 Calculation!..................................................................................74 Data as audio!.............................................................................74 7 Panning!.......................................................................................78 Structure!.....................................................................................79 Implementation!...........................................................................80 Documentation!............................................................................80 Evaluation!...................................................................................80 Construction in Zhuangzi (Part 1 to 3)!.....................83 Introduction!.................................................................................83 Key concepts!..............................................................................83 Modified Lorenz system!..............................................................84 Trajectories!.................................................................................84 Audiovisualisation!.......................................................................85 Implementation!...........................................................................86 Evaluation!...................................................................................87 Construction in Kneading (Part 1 to 3)!....................88 Introduction!.................................................................................88 Key concepts!..............................................................................88