From Faust to Web Audio: Compiling Faust to Javascript Using Emscripten
Total Page:16
File Type:pdf, Size:1020Kb
From Faust to Web Audio: Compiling Faust to JavaScript using Emscripten Myles Borins Center For Computer Research in Music and Acoustics Stanford University Stanford, California United States, [email protected] Abstract 2 Some Context The Web Audio API is a platform for doing audio 2.1 WAAX and Flocking synthesis in the browser. Currently it has a number There are a number of projects that are in de- of natively compiled audio nodes capable of doing velopment abstracting over top of the Web Au- advanced synthesis. One of the available nodes the dio API in order to extend its capabilities, cre- "ScriptProcessorNode" allows individuals to create their own custom unit generators in pure JavaScript. ate more complicated unit generators, and al- The Faust project, developed at Grame CNCM, con- low for a more intuitive syntax. Projects such sists of both a language and a compiler and allows as WAAX (Web Audio API eXtension)2 by individuals to deploy a signal processor to various Hongchan Choi do so while using only the na- languages and platforms. This paper examines a tively compiled nodes in order to ensure opti- technology stack that allows for Faust to be com- mum efficiency.[H. Choi and J.Berger, 2013] piled to highly optimized JavaScript unit generators While these projects offer a wide variety of that synthesize sound using the Web Audio API. unit generators and synthesis modules, they cannot be used to implement all cutting edge Keywords techniques. For example the delay node inter- WebAudio, Faust, Emscripten, JavaScript, asm.js face does not offer a tap in or tap out function, making wave guide models impossible to imple- 1 Introduction ment.3 The Web Audio API, released in 2011, is \a The Flocking audio synthesis toolkit4 by high-level JavaScript API for processing and Colin Clark offers a unique declarative model synthesizing audio in web applications."1 Cur- for doing signal processing within the browser. rently there are a number of natively compiled Unlike WAAX, Flocking has opted to internally audio nodes within the API capable of do- manage all signal generation and using a sin- ing various forms of synthesis and digital sig- gle \ScriptProcessorNode" to hand off precom- nal processing. One of the available nodes, puted buffers of samples. the \ScriptProcessorNode", allows individuals WAAX and Flocking offer two very different to create their own custom unit generators in approaches to Web Audio. WAAX offers effi- pure JavaScript, extending the Web Audio API. ciency, whereas Flocking offers an extensible ar- While the concept of making interactive chitecture and declarative syntax in which web sound synthesis environments in the browser developers can write their own first-class custom is quite exciting, many factors stop individu- unit generators. That being said both projects als from investing time into the Web Audio suffer from the same problem, a lack of man platform. Ignoring the constraints of a single hours. There are only so many individuals who threaded environment there appear to be two have the time and domain specific knowledge primary limitations when working with web au- necessary to contribute to their development. dio: There has not yet been enough Signal Pro- 2.2 Introduction to Faust cessing related JavaScript code written yet, and The Faust project offers a unique solution to some signal processing concepts prove difficult this problem; rather than write code, generate to implement efficiently in a loosely typed lan- guage with no memory management. 2https://github.com/hoch/waax 3https://dvcs.w3.org/hg/audio/raw-file/tip/ 1https://dvcs.w3.org/hg/audio/raw-file/tip/ WebAudio/specification.html#DelayNode-section WebAudio/specification.html 4http://flockingjs.org/ it. Faust, developed at Grame CNCM,\is a pro- extend the current WebAudio architecture to gramming language that provides a purely func- utilize the asm.js susbset, it would require quite tional approach to signal processing while offer- a bit of overhead. Not only would integer and ing a high level of performance." [Orlarey et al., floating point specific interpretation need to be 2009] The project is both a language and a com- implemented, but a functional virtual machine piler, offering the ability to write code once and would need to have been developed in order to deploy to many different signal processing envi- take advantage of asm.js. ronments. Further, we would not see any of the opti- Faust also has a community of scientists and mization benefits that one would get from a developers who have contributed a large amount modern compiler such as gcc or clang. In the of code waiting to be compiled to other plat- spirit of this project, a search was done to find a forms. For example Julius Smith has done a way to automate away the need to worry about substantial amount of research using Faust to all of these complications. implement wave guide synthesis models[Smith et al., 2010] and Romain Michon has ported the 2.5 Introduction to Emscripten entire STK to Faust.[Michon and Smith, 2011] Emscripten is a project started by Alon Za- Creating an efficient compile path from Faust kai from Mozilla that compiles LLVM(Low to the Web Audio API would allow for all of the Level Virtual Machine) assembler to JavaScript, available Faust code to immediately be able to specifically asm.js.[Zakai, 2011] The platform is run in the browser. Further, using the archi- both a compiler and a virtual machine capable tecture compilation model that Faust is famous of running C and C++ code in the browser. for we would be able to wrap the compiled Web Emscripten gives you an interface to break Audio code to be compatible with all current out C functions so that they can be called us- libraries and frameworks such as WAAX and ing JavaScript. It also provides functions for Flocking. managing memory in the virtual machine your C code is running. These functions allow you to 2.3 Current Web Audio allocate new memory to be operated on (in the Implementation case of sound buffers), and the ability to manip- Currently there is an implementation done by ulate memory in the heap (in order to change St´ephaneLetz to compile Faust to Web Audio parameters). directly from the Faust Intermediate Represen- Currently Faust is able to compile to a C++ 5 tation . While the implementation is elegant, file using the minimal.cpp architecture file, the any algorithms relying on integer arithmetic are resulting file can painlessly be compiled to currently broken due to JavaScript representing asm.js with Emscripten. The upstream Faust2 all Numbers as 32-bit floating point at a binary branch can compile Faust to LLVM byte-code level. which offers another potential compilation path. 2.4 Introduction to asm.js 3 Making Noise One way to do integer arithmetic with cross- browser support is asm.js. The asm.js specifica- A first approach to automating the compila- tion6 outlines a `strict subset' of JavaScript that tion process from Faust to Web Audio involves offers a unique programming model. Through manually implementing each step. The Faust the use of typed arrays7 it is possible to do inte- code needs to be compiled to C++ and have ger and floating-point arithmetic. This is done the resulting dsp class wrapped in order to al- with a virtual machine that gives developers ac- low internal data and member functions to be cess to a heap and functions to be used to man- accessed once compiled to JavaScript. The re- age memory and perform arithmetic operations. sulting C++ file then needs to be compiled While it would have been possible to use by Emscripten to asm.js. The asm.js needs to St´ephaneLetz's work as a starting point and once again be wrapped in order to provide an intuitive JavaScript interface that will operate 5http://faust.grame.fr/index.php/7-news/ on the dsp object running in the Emscripten 73-faust-web-art virtual machine. Finally an interface between 6http://asmjs.org/spec/latest/ 7https://developer.mozilla.org/en-US/docs/ the Emscripten virtual machine and WebAudio Web/JavaScript/Typed_arrays?redirectlocale= needs to be made to hand off samples that need en-US&redirectslug=JavaScript/Typed_arrays to be sonified. As an initial proof of concept it was at- This tells faust to compile the above code us- tempted to compile the example noise.dsp that ing the minimal.cpp architecture file, to call the comes shipped with Faust to JavaScript by way object being created Noise, and to include all of Emscripten. Noise was a prime candidate for necessary header files and dependencies. these initial tests due to the integer specific cal- The resulting C++ code need to be wrapped culations used in its algorithm. with a series of meta functions that can be called The below sections will describe the pro- to operate on objects living in the virtual ma- cess used to manually implement noise in the chine. A constructor and destructor are imple- browser starting from a faust dsp file, and end- mented in order to create objects and properly ing with a working Web Audio API JavaScript clean them up, and a compute function is then Object. used to grab the latest frame of samples from the unit generator. In order to change the state of the unit generator after its instantiated in the heap a number of other functions are available to create a map of the ugen's parameters, and get / set values. 3.2 Emscripten & asm.js Once the wrapper has been concatenated with the Faust compiled C++ it can then be com- piled by Emscripten to asm.js.