Overview and Status of Binaural Rendering in Browsers Michael Weitnauer 1 DAGA 2018 © IRT 2018 Audio in Browsers

• Flash (deprecated) • Plugins (deprecated) • HTML5 (no signal processing) • Web Audio API

2 DAGA 2018 © IRT 2018 Web Audio API

• Modular routings with „Nodes“ • Basic audio processing operations • ChannelMerger / ChannelSplitter • BiquadFilter, IIRFilter • Gain, DynamicsCompressor • Delay • Convolver • Panner • Oscillator, WaveShaper • JS control of C/C++ implementations • Sample accuracy & low latency Source: https://developer.mozilla.org • Widely supported

3 DAGA 2018 © IRT 2018 Web Audio API

Source: https://webaudio.github.io/web-audio-api/

4 DAGA 2018 © IRT 2018 Web Audio API Support

5 DAGA 2018 © IRT 2018 Browser Share Worldwide

6 DAGA 2018 © IRT 2018 Browser Share Germany

7 DAGA 2018 © IRT 2018 Web Audio API – Binaural Background

• Multiple options for binaural rendering: 1. ConvolverNode 2. PannerNode 3. ScriptProcessor (AudioWorklet) • Same HRTF Databases for Firefox and Chrome (IRCAM Listen HRTF Database in 15° resolution for azimuth and elevation) • PannerNode has settings for distance model, extent (width, depth, height), etc.

8 DAGA 2018 © IRT 2018 BinauralFIR (IRCAM)

• JS library for custom WAA Node • Usage of own HRTF • Convolution with dry signals • Can be used along with other Web Audio Nodes

• https://ircam-rnd.github.io/binauralFIR/examples/ • https://github.com/Ircam-RnD/binauralFIR

9 DAGA 2018 © IRT 2018 Personalised Media Consumption Examples

https://lab.irt.de/demos/object-based-audio/RadioDrama/

10 DAGA 2018 © IRT 2018 Personalised Media Consumption Examples

https://lab.irt.de/demos/object-based-audio/360/ 11 DAGA 2018 © IRT 2018 Personalised Media Consumption Examples

12 DAGA 2018 © IRT 2018 bogJS (IRT)

• JS library for object-based audio • Personalized audio • Uses PannerNode

• https://github.com/IRT-Open-Source/bogJS

13 DAGA 2018 © IRT 2018 Songbird / Resonance Audio (Google)

• Libraries for different platforms • Simple room model features surfaces and dimensions • Ambisonics encoding • Export for Omnitone

• https://cdn.rawgit.com/resonance-audio/resonance-audio-web- sdk/master/examples/room-models.html • https://developers.google.com/resonance-audio/

• JS library • Ambisonics decoding to binaural (up to 3rd order) • HRTF convolution with decoded Ambisonics signal

• https://rawgit.com/GoogleChrome/omnitone/master/examples/hoa- renderer.html • https://www.youtube.com/watch?v=jK69EXqtRDo

• https://googlechrome.github.io/omnitone/

• No official v1 release yet • Not yet supported in ALL browsers on ALL devices • Different behaviour / implementations (codecs, channel orderings, HRTFs,...) • Complexity issues ! 16 DAGA 2018 © IRT 2018 Further development of Web Audio API

• v1 will be released within the next months • Usage of own HRTFs planned in v2 • AudioWorklet implementations are almost there

Experts in audio-visual media

Michael Weitnauer AV-Technologies [email protected] @_mweitnauer @IRTPresse @IRT_Lab www.irt.de

Die Folien/Dokumente sind durch das Urheberrecht geschützt. Eine Vervielfältigung ist nur mit Genehmigung des Verfassers gestattet. Dieser Urheberrechtshinweis darf nicht entfernt werden.