Towards usability of Higher Order Ambisonics in Digital Audio Workstations Matthias Kronlachner Institute of Electronic Music and Acoustics University of Music and Performing Arts, Graz 2nd International Conference MUSIC AND TECHNOLOGIES Kaunas University of Technology, 8-10 November 2012, Kaunas, Lithuania Osaka, 1970 Pischelsdorf (AT), 2011-?? 2 What is Ambisonic(s)? • Surround recording and playback technique, Michael Gerzon 1970s • Little commercial success so far, patents expired • Independent of playback loudspeaker configuration, scaleable • 2D and 3D sound-fields can be synthesized 3 What is Ambisonic(s)? • Not using phantom sources (VBAP), but trying to recreate original sound-field (WFS) Vector Based Amplitude Panning (VBAP) 4 Wave Field Synthesis (WFS) What is Ambisonic(s)? • Comparable to M/S stereo recording technique • A sound-field is “sampled” using spherical-harmonics MS Stereo recording First Order Ambisonics Left = M+S W (omni directional) Right = M-S X, Y, Z (figure of eight) 5 13 2.6 Analytische Simulation der Schallfeldreproduktionsverfahren Die im vorherigen Teil hergeleiteten Ansteuerungsfunktionen für WFS und HOA wurden in MATLAB implementiert. Diese Implementierung dient der Berechnung der Stimuli für den Hörversuch und der Simulation. Für den im Versuch verwendeten Fall einer ebenen Welle aus 0° Richtung werden nun Simulationen der reproduzierten monochromatischen Schallfelder und der Betragsfrequenzgänge für verschiedene Aufpunkte durchgeführt. Diese Positionen sind auch in der Darstellung der Schallfelder als Kreise eingezeichnet und entsprechen den verwendeten Hörpositionen (Center, Front, Side) im Versuch. 2.6.1 Reproduzierte Schallfelder HOA [SA08] 2 2 2 2 2 2 1 1 1 1 1 1 0 0 0 0 0 0 y [m] y [m] y [m] −1 −1 −1 −1 −1 −1 −2 −2 −2 −2 −2 −2 −2 −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 x [m] x [m] x [m] (a) HOA 56 500Hz (b) HOA 28 500Hz (c) HOA 14 500Hz What2 is Ambisonic(s)? 2 2 2 2 2 1 1 1 1 1 1 • 4 channel signal (W, X, Y, Z) is called B-Format 0 0 0 0 0 0 y [m] y [m] y [m] • B-Format−1 does not represent− 1a speaker−1 feed neither a −1 −1 −1 microphone signal −2 −2 −2 −2 −2 −2 −2 −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 x [m] x [m] x [m] • trying to recreate(d) HOA 56original 1000Hz sound-field(e) in HOA a “sweet-spot” 28 1000Hz (f) HOA 14 1000Hz 2 2 2 2 2 2 1 1 1 1 1 1 0 0 0 0 0 0 y [m] y [m] y [m] −1 −1 −1 −1 −1 −1 −2 −2 −2 −2 −2 −2 −2 −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 x [m] x [m] x [m] (g) HOA 56 2000Hz (h) HOA 28 2000Hz (i) HOA 14 2000Hz B-Format (1st order Ambisonics) 6 plane wave from 0° (M. Frank) Abbildung 6: HOA: Schalldruck des reproduziertes Schallfeldes für verschiedenen Lautsprecheranzahlen und Frequenzen (für den Fall einer virtuellen ebenen Welle aus 0± Richtung) Man erkennt deutlich die Bildung eines Sweetspots im Zentrum der Wiedergabeanordnung. Dieser Bereich verkleinert sich mit hohen Frequenzen und geringer Lautsprecheranzahl. Im Bereich der vorderen Hörposition ist die Bildung besonders starker Artefakte erkennbar. 3. Ambisonics 16 x LS1 LS2 LS8 LS7 φ LS3 y LS6 LS4 LS5 Abbildung 3.5: reguläre Anordnung von 8 Lautsprechern für 2- dimensionale Wiedergabe What is Ambisonic(s)? Winkelabständen für 2D oder eine Anordnung anhand der fünf platonischen • KörperLoudspeaker für 3D, ergibt feeds sich folgendeare a linear Enkodierungsvorschrift: combination of the B- Format signals (Decoder Matrix) e.g.: 1 1 pj = W ( )+X (cos φj cos ✓j)+Y (sin φj cos ✓j)+Z (sin ✓j) L · p2 · · · (3.12) Dabei ist (φj, ✓j) die Position des j-ten Lautsprechers. L stelltx die Anzahl derALL Ambisonics speakers Kanäle work dar. together Für eine 2-dimensionale Wiedergabe kann der • LS1 Z-Kanalto synthesize vernachlässigt the werden. sound-field LS2 LS8 3.3 Higher Order Ambisonics (HOA) LS7 φ LS3 y Ambisonics erster Ordnung bietet nur eine beschränke Genauigkeit in der Abbildung und Reproduktion von Schallfeldern. Anfang der 1990er Jahre LS6 wurde gezeigt, dass der Ambisonics Ansatz auf höhere Ordnungen erweitertLS4 werden kann. Dies führt zu einer Verbesserung in der RichtungsabbildungLS5 und zu einer Vergrößerung des Sweetspots2. Eine Herleitung für den Higher Order Ambisonics Ansatz, welcher in B-Formateiner (1st Reihenentwicklung order Ambisonics) in Sphärische HarmonischeRegular (Abb.2D loudspeaker 3.6) resultiert, placement kann in [10], [33], [25] und [45] gefunden7 werden. 2Der Sweetspot bezeichnet einen Raumbereich, indem das Schallfeld optimal reprodu- ziert werden kann und somit die Richtungsabbildung am besten funktioniert. What is Ambisonic(s)? Higher Order Ambisonics (HOA) extends this approach to a bigger set of spherical harmonics 0th order (W) 1st order (X, Z, Y) 2nd order 3rd order 4th order 5th order Spherical harmonics up to 5th order 3D 8 Higher Order Ambisonics • more accurate spatial resolution in a larger area (“sweet-spot”) • i need more transmission channels, loudspeakers 9 Ambisonics in practice mic. calibration data rotation, Loudspeaker configuration Ambisonics MicEnc zoom Microphone Mono Sound Encoder L L N Source Manipulation Ambisonics Loudspeaker signals (optional) Decoder Mono Sound Encoder Source azimuth, elevation N L Independent from Loudspeaker configuration ≥ 2 L3D =(M + 1) L2D =2M +1 M… Ambisonics Order L… Number of Ambisonics Channels N… Number of Loudspeakers Overview of an Ambisonics Playback System 10 Microphone arrays Eigenmike (32 microphone capsules) for Higher Order Recordings mic. calibration data rotation, Loudspeaker configuration Ambisonics MicEnc zoom Microphone Mono Sound Encoder L L N Source Manipulation Ambisonics Loudspeaker signals (optional) Decoder Mono Sound Encoder Source Core Sound Tetramic, Soundfield and azimuth, elevation N L DIY First Order Microphone (4 cardioids) Independent from Loudspeaker configuration ≥ 2 11 L3D =(M + 1) L2D =2M +1 M… Ambisonics Order L… Number of Ambisonics Channels N… Number of Loudspeakers Encoder (Panner) Mono Signal sources can be positioned defining their position with azimuth (φ) and elevation (θ). Channel Routing in Reaper - Mono Source Input - 3rd order 3D output: 2 L3D =(M + 1) = 16 L2D =2M +1=7 mic. calibration data rotation, Wigware VST Plug-Ins (Win, Mac OS) Loudspeaker configuration Ambisonics MicEnc zoom up to 3rd order 3D Microphone Mono Sound Encoder L L N Source Manipulation Ambisonics Loudspeaker signals (optional) Decoder Mono Sound Encoder Source azimuth, elevation N L Independent from Loudspeaker configuration ≥ 2 12 L3D =(M + 1) L2D =2M +1 M… Ambisonics Order L… Number of Ambisonics Channels N… Number of Loudspeakers Ambisonics Domain the recording or streaming of Ambisonics signals is independent from the final loudspeaker placement... mic. calibration data rotation, Loudspeaker configuration Ambisonics MicEnc zoom Microphone Mono Sound Encoder L L N Source Manipulation Ambisonics Loudspeaker signals (optional) Decoder Mono Sound Encoder Source azimuth, elevation N L Independent from Loudspeaker configuration ≥ 2 13 L3D =(M + 1) L2D =2M +1 M… Ambisonics Order L… Number of Ambisonics Channels N… Number of Loudspeakers Loudspeaker configuration 2D Loudspeaker Ring @ BBC mic. calibration data rotation, Loudspeaker configuration Ambisonics MicEnc zoom Microphone Mono Sound Encoder L L N Source Manipulation Ambisonics Loudspeaker signals (optional) Decoder Mono Sound Encoder Source azimuth, elevation 29 Loudspeakers in half sphere N L Independent from Loudspeaker configuration ≥ @ Mumuth Graz 14 2 L3D =(M + 1) L2D =2M +1 M… Ambisonics Order L… Number of Ambisonics Channels N… Number of Loudspeakers Decoder • Computing loudspeaker signals from B-Format L N loudspeaker signals B-Format Ambdec (Linux, Mac OS) Standalone Jack Application mic. calibration data rotation, Loudspeaker configuration Ambisonics MicEnc zoom Microphone Mono Sound Encoder L L N Source Manipulation Ambisonics Loudspeaker signals (optional) Decoder Mono Sound Encoder Source azimuth, elevation N L VST Plug-In Decoder (Windows, Mac OS) Independent from Loudspeaker configuration ≥ 15 2 L3D =(M + 1) L2D =2M +1 M… Ambisonics Order L… Number of Ambisonics Channels N… Number of Loudspeakers Problems with existing solutions • Operating system compatibility (Win, Mac OS, Linux) Plug-In Standards (VST, AU, AAX, RTAS, LV2) • Listening/producing at home? • jumps in angular representation between -180° and 180° (especially when encoding - panning) • Different standards about channel sequence and channel scaling 16 Ambisonics standards Problem: different standardization of channel order and normalization (scaling) of channels Furse-Malham Ordering ACN Order 1st order: W,X,Y,Z (Ambisonics Channel Numbering) 2nd order: R,S,T,U,V > should be new standard 3rd order: K,L,M,N,O,P,Q 17 de facto standard for order ≤ 3 Different standards adjust scaling and channel order in every stage mic. calibration data rotation, Loudspeaker configuration Ambisonics MicEnc zoom Microphone Mono Sound Encoder L L N Source Manipulation Ambisonics Loudspeaker signals (optional) Decoder Mono Sound Encoder Source azimuth, elevation N L Independent from Loudspeaker configuration ≥ 2 L3D =(M + 1) 18 L2D =2M +1 M… Ambisonics Order L… Number of Ambisonics Channels N… Number of Loudspeakers Jumping angular representation • Visual representation of automation curve not intuitive 19 Listening at home Listening to recordings without a large number of loudspeakers possible using headphones and headtracking head tracking virtual individual or general (Open Sound Control) loudspeaker configuration HRTFs left N HRTF L Soundfield L Ambisonics Ambisonics Convolution headphone B-Format Rotation Decoder signals Filter virtual right loudspeaker L… Number of Ambisonics Channels signals N… Number of Virtual Loudspeakers spk1 HRTF..spk8 Head Related Transfer Function F8 F1 Ambisonics Binaural Decoder spk2 spk7 F7 F2 mic. calibration data rotation, Loudspeaker configuration F3 Ambisonics MicEnc zoom F6 Microphone spk3 spk6 Mono Sound F4 Encoder L L N F5 Source Manipulation Ambisonics Loudspeaker signals (optional) Decoder Mono Sound Encoder Source spk4 spk5 azimuth, elevation N L loudspeakers - left ear transfer path Independent from Loudspeaker configuration ≥ 20 2 L3D =(M + 1) L2D =2M +1 M… Ambisonics Order L… Number of Ambisonics Channels N… Number of Loudspeakers Binaural Decoder L L Stereo 21 Things to do..