UNIVERSITY OF CALIFORNIA, SANTA BARBARA CHON: From physical simulation to musical gesture by Rodney DuPlessis Proje ! do umen!#!ion submi!!ed in $#r!i#l ful&llmen! %or !'e de(ree o% )#s!er of S ien e in )edi# Ar!s #nd Te 'nolo(y Co""i!!ee* Professor Cur!is Ro#ds, Ch#ir Professor +oAnn ,u 'er#-)orin Dr. ,#rl Yerkes April 0102 CHON* From $'ysi #l simul#!ion !o musi #l (es!ure Co$yri('! © 0102 by Rodney DuPlessis ACKNOWLEDGMENTS I 5ould like !o !'#nk my ommi!!ee for !'eir su$$or! #nd (uid#n e over !'e ourse of my !ime in !'e )edi# Ar!s 7 Te 'nolo(y $ro(r#". I #lso !'#nk Andres Cabrer# #nd !'e Allos$'ere Rese#r ' 8rou$ for !'eir 5ork on Allolib, 5'i ' $rovided # solid found#!ion for C3ON. I #m (r#!eful !o my o'or! #nd olle#(ues for our oun!less s!imul#!in( onvers#!ions #nd for 'oldin( s$# e for ins$ir#!ion #nd re#!i6i!y !o 9ouris'. I 5#n! !o e:$ress my dee$ (r#!i!ude !o my 5ife for su$$or!in( me !'rou(' !'e lon( d#ys #nd l#!e ni('!s of %ren;ied odin( #nd e:$erimen!#!ion. Fin#lly, I !'#nk Suki #nd A$$# %or !'eir unders!#!ed "or#l su$$or!. ABSTRACT CHON* From $'ysi #l simul#!ion !o musi #l (es!ure by Rodney DuPlessis Physi #l me!#$'or $rovides # vis er#l #nd univers#l lo(i #l fr#me5ork for o"$osin( musi #l (es!ures. Physi #l simul#!ions #n #id !'e o"$oser !o re#!e musi #l (es!ures b#sed in o"$lex $'ysi #l me!#$'ors. CHON <Cou$led 3#rmoni Os ill#!or Ne!wor/= is # ross-$l#!%orm #$$li #!ion for simul#!in( "#ss-s$rin( ne!works #nd sonifyin( !'e mo!ion of indi6idu#l $#r!i les. CHON is #n in!er# !ive ins!rumen! !'#! #n $rovide o"$lex, ye! !#n(ible #nd $'ysi #lly b#sed, on!rol d#!# for syn!'esis, sound $ro essin(, #nd musi #l s ore (ener#!ion. This sys!e" builds on !'e ide# of !'e !r#di!ion#l LFO by ou$lin( !'e movemen! of mul!i$le on!rol si(n#ls usin( $'ysi #l $rin i$les. Contents 1 Introduction 1 1.1 Overview . .1 1.2 Related work . .2 2 Physical metaphor in music 6 2.1 Logical Framework . .6 2.2 Universal . .8 2.3 Visceral . 10 2.4 Physical Metaphor in Music . 10 3 Physics 12 4 Software implementation 19 4.1 Libraries . 19 4.2 Physics computation . 20 4.3 Sound engine . 21 4.4 OSC . 23 5 Interface 24 5.1 Visual interface . 24 5.2 GUI . 26 5.3 Interaction . 31 6 Applications and Limitations 33 6.1 Applications . 33 6.1.1 Continuous control data . 33 6.1.2 Trigger control data . 34 6.1.3 Generating musical scores . 34 6.2 Limitations . 35 7 Future directions 38 Bibliography 40 Appendix A: Timeline 43 Chapter 1 Introduction 1.1 Overview Coupled Harmonic Oscillator Network (CHON) is a real-time, interactive application for generating sonic gestures and textures using a simulation of a physical dynamical system as a musical interface. The physical system is a network of particles connected by a spring-like force. The user sets the system into motion by displacing a particle, which causes a chain reaction governed by Newtonian mechanics. The user can intervene in the evolving system, dragging and moving particles at will. The system generates complex yet tangible control data that can be used to drive parameters for sound synthesis and other purposes. CHON is part of a larger interest of mine in using physical metaphor in music. I have previously used intuitive and algorithmic methods to express physical metaphor in my music. CHON represents a hybrid approach, allowing intuitive interaction with the algorithmically generated simulation in real-time. The musical gestures that CHON can create are chaotic, but they seem nonetheless intelligible because of their connection to the physical world. The coupled harmonic oscillator is a fundamental model in physics that has applications in many areas of research. It is an extension of the simple harmonic oscillator, which can be represented by a mass on a spring. The simple harmonic oscillator moves sinusoidally, but when coupled via a spring-like force to another harmonic oscillator (or many), complex superpositions of sinusoidal waves emerge. CHON goes further, allowing the masses in the coupled harmonic oscillator network to move in 3 dimensions and to be arranged in a 1 2-dimensional grid. The application is written in C++ and uses the Allolib framework extensively. It is open source1, under a GPLv3 license, and runs on Linux, MacOS, and Windows. It uses an explicit numerical method to solve the discretized equations of motion of the particles. The visual interface is a 3D rendering of the particle system. The user interacts directly with the particles in the visual simulation using a computer mouse. A 2D graph can also be displayed which visualizes the displacement of each particle along a given axis. CHON has an internal synthesis engine that allows the user to sonify the movement of the particles. The instrument also generates a stream of OSC data from each particle, making it a versatile tool for generating up to 100 control signals that are linked by physical laws. I have used CHON to generate sounds using the internal sound engine as well as using the OSC data it broadcasts to control external synthesis and notation software. For example, I used CHON to write a score for two pianists. The piece, Pandæmonium, was premiered by HOCKET in August 2020. I am still exploring the musical possibilities afforded by CHON, and the possibilities of extending and improving CHON. I will discuss some of these future directions at the end of this paper. 1.2 Related work Physical modeling is a massive interdisciplinary field of research spanning industrial, scientific, and artistic applications. In music, the vast majority of physical modeling is used for re-synthesizing believable facsimiles of real instruments[1][2]. The same technology that allows these recreations has also been applied to create fantastical virtual instruments of unreal proportions or configurations[3][4]. The first example of physical modeling synthesis of instruments was demonstrated in 1971 by Hiller and Ruiz[5]. Today, physical modeling synthesis, driven by increasingly powerful computers, can achieve very convincing results. On the other hand, physical modeling interfaces often suffer from problems of control and complexity; the high number of parameters in a given simulation may be overwhelming for performance purposes[6, p. 288][7]. There are many approaches to the design of systems for physical modeling synthesis including modal synthesis, waveguide schemes, source-filter models, and mass-spring 1The source code can be accessed at https://github.com/rodneydup/CHON 2 methods[8]. CHON falls into the realm of mass-spring methods. Mass-spring methods use Newtonian mechanics to simulate the dynamics of particles with mass and inertia connected by spring-like forces[6, p. 271]. This is the type of simulation that drives CHON. The goal of most mass-spring applications is to simulate a physical body that can be described as a network of coupled masses. This, like every physical model, is a simplification of reality. Nevertheless, many types of physical bodies fit well into this model. The mass-spring model does a good job at representing the way that waves propagate through solid media such as strings and vibrating surfaces (like drums). The mass-spring model suffers from scalability issues, however. The cost of computation can increase quickly as more particles are used in a given simulation. Considering that physical modeling synthesis often needs to run at audio rate (updating 44,100 times per second) and that higher particle density generally results in a more realistic simulation, this is a difficult problem. The focus of most mass-spring simulations is on the object that the system simulates. In other words, the aim is that the gestalt of the movements of the particles generates sound that resembles the sound of the object being modeled. This is where CHON differs from other mass-spring applications (and indeed the majority of physical modeling synthesis projects in general). Rather than create a sound object from an amalgam of particle interactions, CHON was designed with the goal of exploring interconnected particles as sound objects unto themselves. I am also interested in larger-scale gestures, phrases, and forms that can arise from the overlapping undulations of the coupled particles, but the foundation of CHON remains the individual sound-particles themselves. Claude Cadoz and others at ACROE (Association pour la Création et la Recherche sur les Outils d’Expression) in Grenoble pioneered the use of the mass-spring model in physical modeling synthesis and musical creation. They generalized mass-spring models into what they call a mass-interaction scheme, where, in addition to more standard mass-spring situations, particles may be lumped into strange configurations and the forces connecting them may be non-linear[9]. ACROE’s CORDIS-ANIMA system was one of the first digital synthesis applications for physical modeling, paving the way for future research. CORDIS-ANIMA is a sophisticated and technical modular system for simulation and sonification/visualization of mass-interaction models[9]. In CORDIS-ANIMA, the user can specify many physical parameters of each particle, including how and which particles are coupled to each other, their initial position and velocity, their mass, whether they are fixed or can move.