Ambisonics plug-in suite for production and performance usage Matthias KRONLACHNER Institute of Electronic Music and Acoustics, Graz (student) M.K. Ciurlionioˇ g. 5-15 LT - 03104 Vilnius [email protected] Abstract only) plug-in suite offers 3D encoders for 1st and nd th Ambisonics is a technique for the spatialization of 2 order as well as 2D encoders for 5 order. sound sources within a circular or spherical loud- For the plug-ins created in this work, speaker arrangement. This paper presents a suite of the C++ cross-platform programming library Ambisonics processors, compatible with most stan- JUCE5 [Storer, 2012] has been used to develop 1 dard DAW plug-in formats on Linux, Mac OS X audio plug-ins compatible to most DAWs on and Windows. Some considerations about usabil- Linux, Mac OS X and Windows. JUCE is be- ity did result in features of the user interface and automation possibilities, not available in other sur- ing developed and maintained by Julian Storer, round panning plug-ins. The encoder plug-in may be who used it as the base of the DAW Track- connected to a central program for visualisation and tion. JUCE is released under the GNU Public remote control purposes, displaying the current posi- Licence. A commercial license may be acquired tion and audio level of every single track in the DAW. for closed source projects. It is possible to build To enable monitoring of Ambisonics content without JUCE audio processors as LADSPA, VST, AU, an extensive loudspeaker setup, binaural decoders RTAS and AAX plug-ins or as Jack standalone for headphone playback have been implemented. applications. LV2 (LADSPA version 2) support Keywords currently has to be added manually from the separate project DISTRHO6 [Coelho and Ro- Ambisonics, Plug-ins, Digital Audio Workstations, drigues, 2012]. Binaural, Ardour Ambisonics suffers from different existing 1 Introduction standards concerning channel order and normal- ization. To overcome this problem, a conver- It turns out to be difficult finding platform inde- sion tool is included in the plug-in suite. En- pendent audio plug-ins for encoding and decod- coders, rotators and decoders from the authors ing Ambisonics. Following section should give a suite are designed for the ACN channel order brief overview of still maintained plug-ins. and SN3D normalization, proposed by [Nach- Fons Adriaensen's AMB-plugins2 offer en- rd bar et al., 2011]. Conversion between standards coders and rotators (yaw axis only) until 3 of the input and/or output format can be done order. LADSPA can be used with the DAW Ar- by the conversion plug-in. dour under Linux and Mac OS X. There is no Apart from platform compatibility, some con- Windows host supporting LADSPA. For decod- siderations about usability did result in features ing Ambisonics signals into loudspeaker feeds, of the user interface, not available in other sur- Adriaensen's Jack client application AmbDec round panning plug-ins. Continuously rotat- [Adriaensen, 2005] is often used. 3 ing a sound source results in a discontinuity of Bruce Wiggins offers his WigWare plug-ins the angular representation between −180◦ and in VST format for Windows and MacOS X. +180◦. This jump is also reflected when draw- These processors include 2D and 3D encoders rd st ing automation curves resulting in a mismatch until 3 order as well as 1 order decoders for between the visual representation and auditory several fixed loudspeaker arrangements. 4 perceived movement of a sound source. A solu- Daniel Courville's Audio Unit (Mac OS X tion allowing to define absolute starting points 1Digital Audio Workstation and angular velocities for relative movements is 2http://kokkinizita.linuxaudio.org 3http://www.brucewiggins.co.uk 5http://www.rawmaterialsoftware.com 4http://www.radio.uqam.ca/ambisonic/ 6http://distrho.sourceforge.net proposed. 2.1 Automation parameters For headphone monitoring several binaural Most current DAWs are limited to represent au- decoders have been implemented simulating the tomation parameters between 0:0 and 1:0 along Ambisonics half sphere of the medium sized a time line. Panning plug-ins usually map this IEM Cube with 24 speakers and the concert range for azimuth and elevation between −180◦ 7 hall Mumuth with 29 speakers in a elliptical and +180◦. A full circle rotation results in stretched half sphere. a mismatch between the visual representation For visualization and external control pur- (Fig. 2) of the automation curve and the per- poses, a bidirectional Open Sound Control ceived continuous rotation of the source. Ad- (OSC) communication layer has been imple- ditionally the plug-in host may interpolate be- mented in the encoder plug-in. An external pro- tween a jump from 1.0 to 0.0, resulting in a very gram is able to display the current position and fast audible jump. audio level of every track in the DAW. The visu- alization program may also take control over the sources. This can be very useful in performance situations while having a multitrack playback coming from the DAW and a central display to control the position of the individual tracks. Currently no audio plug-in format can handle dynamic input/output channel counts. There- fore all plug-ins may be compiled for fixed Am- bisonics orders. channel order/scaling L' alternative output Ambisonics Converter format B-Format Converter Figure 2: Automation curve for full circle rota- calibration Loudspeaker Loudspeaker- configuration head Ambisonics Head Tracking MicEnc impulse responses tion using Ardour and AMB plug-ins microphone left Mono sound Encoder L N source L Binaural Rotation Decoder filter Mono sound Encoder right To overcome this problem, automation pa- source azimuth, Loudspeaker Headphone elevation, signals signals rameters (Fig. 3) have been added for setting size Independent from Loudspeaker configuration start points (SetAzimuth, SetAzimuthRel) and M… Ambisonics Order L… Number of Ambisonics Channels angular velocities (tgl-rot-azimuth). The maxi- N… Number of Loudspeakers mum speed of the angular velocity may be ad- justed between 0 and 360deg=sec by an addi- Figure 1: Ambisonics production and playback tional parameter (max-speed). This guarantees chain a wide range of adjustment and at the same time accuracy for the rotation speed. 2 Encoder The encoder plug-in distributes a mono source signal into Ambisonics channels (derived from spherical harmonic functions), according to az- imuth and elevation settings, representing the coordinates on a spherical surface. An addi- tional parameter called size [0; 1] may be used to adjust the spatial directivity. Adjusting the size parameter from zero towards one results in a scaling of the higher order components. For a size setting of 1, all Ambisonics channels will be zero except the 0th order (also known as W Figure 3: Encoder automation parameters channel), resulting in an equally distributed sig- nal over all loudspeakers. 2.2 Remote control and visualization 7http://www.kug.ac.at/en/ Keeping track of all sound source positions studies-further-education/studies/ within a DAW may be a difficult task for the infrastructure/the-mumuth.html mixing engineer. To allow a better overview and control of the spatial scenery, a coopera- movements during binaural playback. The ro- tive visualization and control unit has been im- tation plug-in listens to an adjustable UDP port plemented. All encoder plug-ins are equipped for incoming OSC messages. This allows to by- with a bidirectional OSC layer (Fig. 4), send- pass host automation and controlling the rota- ing and receiving control and status parameters. tion directly from the head tracking software Currently a functional prototype has been im- (Sec. 4.1). plemented in Pd/GEM (Fig. 5) displaying all tracks (encoders) on a sphere, including visual- ization of their audio levels. Currently the audio level is represented by the variable length of the cylinder representing a source signal. This con- cept may be extended to a more sophisticated implementation. individual OSC/UDP connection from visualization to encoders azimuth, elevation, size Encoder central OSC/UDP Visualization Figure 6: Ambisonics Rotator Encoder azimuth, elevation, and size, audio level Remote Control ... 4 Binaural decoder Encoder The binaural decoder computes virtual loud- digital audio workstation speaker feeds as a linear combination of the Am- bisonics signals according to a given decoding Figure 4: Encoder OSC communication with re- matrix. These virtual loudspeaker signals are mote visualization and control convolved with their individual stereo impulse responses, modeling the transfer path from the loudspeaker position to the left and right ear of the listener (Fig. 1). 4.1 Head tracking Head tracking is a significant feature for virtual reality scenes and headphone playback. Small head movements change the relative position of a sound source in aspect to the listeners ears, making localization more easy and removing ambiguity. The KinectTM sensor as add-on for the gam- ing console XBox 360TM by Microsoft offers a low budget depth sensor. [Fanelli et al., 2011] developed a software to gather head orienta- Figure 5: Visualization and remote control of tion angles and the head position relative to the several encoders with Pd/GEM Kinect sensor (Fig. 7). The author extended this head pose estimation software about send- 3 Rotator ing OSC data to the Ambisonics rotator plug- The rotation plug-in (Fig. 6) may be used to in8. The Ambisonics sound field has to be ro- manipulate the orientation in the Ambisonics tated in opposite direction to keep the sound domain, as described in [Musil et al., 2003]. source positions fixed and suppress the rotation An optimized way to calculate rotation matri- with the listeners head. ces can be found in [Rumori, 2009]. Therefore ◦ yaw, pitch and roll can be adjusted by ±180 . 8http://github.com/kronihias/ This is very useful for the incorporation of head head-pose-estimation versatile loudspeaker setup that can be changed within a minute.