NNdef: Livecoding Digital Musical Instruments in SuperCollider using Functional Reactive Programming Miguel Cerdeira Negrão School of Technology and Management Polytechnic Institute of Leiria Portugal [email protected] Abstract ACM Reference Format: The SuperCollider audio synthesis environment allows the Miguel Cerdeira Negrão. 2018. NNdef: Livecoding Digital Musical definition of Synths, digital instruments which generate sound Instruments in SuperCollider using Functional Reactive Program- ming. In Proceedings of the 6th ACM SIGPLAN International Work- using a graph of interconnected unit generators. In SuperCol- shop on Functional Art, Music, Modeling, and Design (FARM ’18), lider the definition of a Synth is mostly declarative, onthe September 29, 2018, St. Louis, MO, USA. ACM, New York, NY, USA, other hand the logic for controlling parameters of a Synth us- 8 pages. https://doi.org/10.1145/3242903.3242905 ing musical controllers is usually implemented in a different context using callbacks and explicit state. This paper presents a different approach where functional 1 Introduction reactive programming (FRP) is used to define the control Algorithmic computer music sound synthesis environments logic of the instrument, taking inputs from musical con- enable the creation of networks of unit generators (UGens), trollers, mobile apps or graphical user interface (GUI) wid- which define an instrument or synthesizer, as well asthe gets and sending outputs to the audio graph. Both audio and scheduling of sound events, using a programming language. FRP graphs are defined in the same context and compiled Unit generators synthesize or transform sound and can be simultaneously avoiding a hard division between audio and connected to form complex audio instruments. In a textual control logic. programming language instruments and sound events are An FRP implementation is used in the NNdef class to defined using segments of code which are sometimes called enable livecoding of both audio and FRP code, with hot- "patches". SuperCollider is one well known example of a swap allowing an interactive workflow. Also included isa computer music synthesis environment with an interpreted system to persist the state in the FRP network in order to high-level general-purpose programming language. save and recall the instrument at a later time. An important development in this area is livecoding, where snippets of code are evaluated causing changes in ongoing CCS Concepts • Applied computing → Sound and mu- processes. Livecoding allows quick experimentation and in- sic computing; cremental construction of patches, enabling a playful explo- ration of compositional possibilities. It also makes possible Keywords livecoding, functional reactive programming, the on-stage, simultaneous creation and manipulation of digital musical instrument patches during a live performance for an audience. Another relevant development is the use of musical con- trollers for performing a digital instrument. Different sorts of devices which capture physical motion into a data stream Permission to make digital or hard copies of all or part of this work for have been used to directly control parameters of a patch. personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that Notable examples are commercial Musical Instrument Digi- copies bear this notice and the full citation on the first page. Copyrights tal Interface (MIDI) controllers, smartphone and tablet apps for components of this work owned by others than the author(s) must communicating via Open Sound Control (OSC) [20], and be honored. Abstracting with credit is permitted. To copy otherwise, or custom-made sensor microcontrollers and microcomputers republish, to post on servers or to redistribute to lists, requires prior specific such as the Arduino board [15] and the Raspberry Pi com- permission and/or a fee. Request permissions from [email protected]. puter [1]. FARM ’18, September 29, 2018, St. Louis, MO, USA © 2018 Copyright held by the owner/author(s). Publication rights licensed Functional Reactive Programming (FRP) deals with events to ACM. which appear at unpredictable times and as such it is well ACM ISBN 978-1-4503-5856-9/18/09...$15.00 suited to deal with incoming data from electronic musical https://doi.org/10.1145/3242903.3242905 controllers. An FRP network can therefore be connected to FARM ’18, September 29, 2018, St. Louis, MO, USA Miguel Cerdeira Negrão an audio network in order to create a unified audio synthesis- to the proxy, the previous Synth is stopped and a new one processing instrument which can be played with physical starts, usually using a cross-fade. The Ndef class2 of JITLib controllers. associates a unique name with a proxy, enabling quick access, When using a callback approach the definition of the UGen which is useful when livecoding. graph is declarative while the control logic is not, creating The NNdef class3, an extension of JITLib’s Ndef, is the a mismatch. When using a general-purpose FRP library to main focus of this paper. It is an extension of JITLib using control an audio network it is necessary to create some boil- FRP for processing incoming data streams from physical erplate code and the two networks must be defined and controllers and other sources. managed separately. Although SuperCollider is not a pure functional language, This paper presents a specific integration of an FRP sys- and is therefore not the best candidate for using FRP, given tem within an audio livecoding library, implemented in the that it has a vast amount of functionality relating to com- SuperCollider synthesis environment which attempts to sim- puter music readily available, it was deemed a worthwhile plify the creation of digital musical instruments. The system experiment seeing what could be achieved with FRP in such is interfaced through the NNdef class whose features will be a language. explained in detail. The NNdef class is part of FPLib, a library for functional The specific contributions are: programming in SuperCollider created by the author. NNdef 4 • Integration between an FRP network and a constant- integrates an FRP network into the audio Ugen network rate high-efficiency audio network. contained in a proxy in JITLib, with the aim of allowing the • Simultaneous definition and compilation of both net- creation, through livecoding, of digital musical instruments. works. The audio and event networks connect at bridging ports • Hot-swapping and persistence of the FRP network where an outgoing event from the FRP network becomes a 5 enabling livecoding digital instruments which use mu- continuous audio/control signal . Inputs to the FRP network sical controllers. are obtained from MIDI controllers via the Modality toolkit [4], OSC messages and GUI widgets. 2 SuperCollider and Livecoding Using The main goal when developing NNdef was enabling the JITLib livecoding of digital instruments which use musical con- trollers, either for live performance or for composition6. Of- SuperCollider [12–14] is a programming language for real- ten creating a digital instrument with a physical controller is time audio synthesis and algorithmic composition, with an a trial and error process where the instrument undergoes sev- efficient sound synthesis engine (scsynth) designed specifi- eral iterations until arriving at a final design. The synthesis cally for music composition and performance. Coming from and control logic are developed iteratively, with the feedback the tradition of MUSIC-N, it allows the interconnection of between physical motion and sonic result being continuously UGens to form larger digital signal processing (DSP) net- evaluated, and the corresponding patch updated. Taking this works. The SuperCollider language (sclang), is a general- into account it was desirable that the livecoding interface purpose object-oriented language (OOP) with some features would allow defining the synthesis and control logic together of functional programming. Blocks of code are evaluated in the same context. in real-time by the sclang interpreter. The SuperCollider It was also desirable when livecoding an instrument that standard class library includes, besides the general purpose its current state would not be lost when switching over to a abstractions such as collections, powerful procedures for new definition of the instrument, particularly if the instru- algorithmic creation of UGen graphs and rich abstractions ment has modal control logic7 which goes beyond a direct for musical events (routines, patterns, etc.). The synthesis en- gine (scsynth), written in C/C++, combines UGens according 2Ndef stands for node proxy definition. to pre-made declarative descriptions, SynthDefs, instantiat- 3The name NNdef is a playful derivation of Ndef: since Ndef has one type ing them as Synths, which can be dynamically created and of graph, and NNdef has two (audio and FRP), it uses two Ns. 4More precisely, like Ndef, NNdef can have multiple Synths and each Synth destroyed. scsynth is a separate process and communicates has its own associated FRP graph. with sclang via OSC messages. 5In SuperCollider UGens can operate at audio-rate or at control-rate, which A Synth in SuperCollider consists of a network of UGens is slower. The later is used with low-frequency signals in order to decrease whose shape cannot be changed once it has started playing. CPU usage. 6 Although this makes for a very efficient design, it limits A broad view