LV2 Atoms: A Data Model for Real-Time Audio Plugins David E. Robillard School of Computer Science, Carleton University 1125 Colonel By Drive Ottawa ON K1S 5B6 Canada [email protected] Abstract around commands, which limits their applicability This paper introduces the LV2 Atom extension, a simple and ability to express structured data. yet powerful data model designed for advanced control This paper introduces the LV2 Atom extension of audio plugins or other real-time applications. At the [Robillard, 2012b], a simple yet powerful data most basic level, an atom is a standard header followed model designed for advanced control of LV2 plu- by a sequence of bytes. A standard type model can be gins or other real-time applications. Atoms serve used for representing structured data which is meaning- both as a model for representing state, and a proto- ful across projects. This model is currently used by sev- col for accessing or manipulating it. This includes eral projects for various applications including state per- primitive values like numeric controls or file names, sistence, time synchronisation, and network-transparent plugin control. Atoms are intended to form the basis but the model-based approach allows developers to of future standard protocols to increase the power of work with more sophisticated data as well. host:plugin, plugin:plugin, and UI:plugin interfaces. The key distinction between MIDI or OSC mes- sages and atoms is that atoms are not just com- Keywords mands, but a general data format. This paper aims to show that building on the foundation of a solid data LV2, plugin, data, state, protocol model is more elegant and powerful than command- based protocols. The idea is conceptually similar 1 Introduction to the popular use of JSON [Crawford, 2006] in the LV2 is a portable plugin standard for audio systems, web community: define a data model for represent- similar in scope to LADSPA, VST, AU, and others. ing arbitrary information, then construct messages It defines a C API for code and a format for data within that data model. files which collectively describe a plugin. The core However, atoms are not introduced to the exclu- LV2 API is similar in power to LADSPA, but exten- sion of other protocols. In fact, MIDI messages are sions can support more advanced functionality. This transmitted to and from plugins as a particular type allows the interface to evolve and accommodate the of atom. At the lowest level, atoms are a sequence of needs of real software as they arise. bytes (or a chunk) with a standard header. On top of LV2 is supported by many applications, in- this, a type model is defined which allows complex cluding Digital Audio Workstations like Ardour structures to be built from a few standard primitive [Davis and others, 2014], hardware effects proces- and container types. This model has several advan- sors like MOD [Ceccolini and Germani, 2013], and tages, including extensibility, support for round-trip signal processing languages like Faust [Graf,¨ 2013]. portable serialisation, and natural expression in plu- One key piece of functionality LV2 adds to gin data files. LADSPA is the ability to transmit events. This There are two aspects to the LV2 atom specifica- is most commonly used to communicate via MIDI tion: the low-level mechanics (Section 2) define the [MID, 1983] for playing notes, selecting programs, binary format of atoms and how they may be used, etc. MIDI is nearly ubiquitous in musical equip- while the high-level semantics (Section 3) define a ment, but has significant limitations [Moore, 1988]. type model built upon this binary format. Using Many applications require a more powerful model this model, projects can communicate meaningful to express and manipulate state. For example, structures at a conceptually high level, while the ac- “load sample /media/bonk.wav”, a typical op- tual mechanics involved are simply the copying of eration in some audio software, can not be ex- small chunks. This approach to plugin control has pressed in standard MIDI. Other protocols like OSC many applications (Section 4) in current projects, [Wright, 1997] are more powerful, but still designed which typically use the provided convenience APIs for reading and writing atoms (Section 5) with ease. Input logistics are straightforward: the host con- Ultimately, atoms are intended to form the basis of nects the input to an atom before calling the plugin’s future work (Section 6) designing standard proto- process() method. cols for advanced plugin control. Outputs are slightly trickier since the plugin must know how much space is available for writing, but 2 Mechanics atom types may have variable size. To resolve 2.1 Atom Definition this, the host initialises the size field in the output An LV2 atom is a 64-bit aligned chunk of memory buffer to the amount of available space before call- that begins with a 32-bit size and type: ing process(). Plugins read this value, then write a complete atom (including size and type) to the typedef struct{ buffer before returning. For real-time support, as uint32_t size; with audio, output buffer space is made available by uint32_t type; the host before calling process(). By default, out- } LV2_Atom; puts are given the same amount of space as inputs of the same type, but plugins that require more space This atom header is immediately followed by the can request larger output buffers ahead of time. body which is size bytes long. Atoms are, by defi- nition, Plain Old Data (POD): contiguous chunks of Thus far, atoms have been described without re- memory that can safely be copied bytewise.1 At the ferring to specific types. An AtomPort can be con- most basic level, this is all there is to atoms. nected to any value, but since plugins process sig- Types are assigned dynamically and not restricted nals over a block of time, it is usually more use- to any fixed set. Developers can define new atom ful for ports to contain many time-stamped atoms, events types, though all types are required to be POD. or . To achieve this, ports are connected to a Any atom can thus be copied or stored, even by Sequence atom. This is the mechanism commonly an implementation which does not understand its used by LV2 plugins to process streams of sample- type. Among other advantages, this makes it pos- accurate events (including MIDI) alongside audio. sible for hosts to transmit atoms between plugins The following section describes the set of standard without explicitly supporting each type used. Sim- atom types, which includes primitives like Int and ilarly, generic plugins like event routers, multiplex- containers like Sequence. ers, or delays, can work with any atom. Developers 3 Semantics are free to send complex data between plugins, or between UIs and plugins, without being held back 3.1 Atom Types by lacking standards or host support. Section 3.3 The structure of the atom type model is explains in detail how this decentralised extensibil- similar to JSON values or ERLANG terms ity is achieved. [Virding et al., 1996]: a few primitive types, and Note, however, that atoms are only POD by defi- collections which can be used to build larger nition, not necessarily portable: atoms may contain structures. The hierarchy of standard types defined architecture-specific data like integers with native in the atom extension2 is shown in Figure 1. endianness. The atom specification includes a set Primitives represent a single value, and do not of standard types which should be used where per- contain other atoms. The simplest types are prim- sistence or interoperability are important (see Sec- itives with fixed size, like Int. These types have tion 3.1). a corresponding C struct defined in atom.h which 2.2 Communication via Ports completely describes their binary format, for exam- ple: Plugins can send or receive atoms via an AtomPort which (like any LV2 port) is either an input or an typedef struct{ output. An AtomPort is connected directly to an LV2_Atom atom; LV2 Atom (just as a standard LADSPA or LV2 con- int32_t body; trol port is connected directly to a float). } LV2_Atom_Int; An AtomPort can be used with any atom type. Plugins can specify which types are supported using The other Number types and Bool correspond to the atom:bufferType property in their data files. the C types with the same name, but have a precisely Several types may be supported by a single port. defined size on all platforms (32 and 64 bits). 1 Type 0 has been reserved for a special reference type, in 2 There is also a standard type for MIDI messages defined case a need for non-POD communication arises in the future. in the separate LV2 MIDI extension. Atom with a time stamp.3 A Vector is a series of fixed- length atoms with the same type and no headers, Bool making the vector body a regular C array. Sound is Chunk a descriptive type, identical in format to a Vector of float, but explicitly representing a sample of au- Literal dio. URID 3.2 Portable Serialisation Number In addition to a binary format, each atom type has a portable serialisation. This allows implementa- Double tions (typically hosts) to convert atoms to and from Float text for portable storage, network transmission, or human readability. This format is used to describe Int atoms in the following sections, but it is important to Long keep in mind that plugins work with atoms in their native binary form. String Most primitive types are associated with XSD [W3C, 2004b] datatypes which define their textual URI format. Table 1 shows this mapping along with an Path example string.