Extending Snd with Eval-C and Snd-Rt

Kjetil Matheussen Norwegian network for Technology, Acoustic and Music. (NOTAM). Nedre Gate 5, 0551 OSLO, Norway, [email protected]

Abstract Language features

This paper presents two domain specific program- Guile Snd-Rt Eval-C ming environments made for extending Snd: Eval- C and Snd-Rt. Eval-C and Snd-Rt add another Interpreted x set of pragmatic and practical solutions for work- Compiled x x ing with music programming in Snd, from very low Closures x level to very high level, and both in realtime and Nested functions x x non-realtime. Functions as arguments x x x The software synthesizer program “San-Dysth” is Lisp lowlevel macros x x x also presented, since its programmed using Snd-Rt. Dynamic typing x And finally, Snd/Pd is presented, which makes Snd’s Static typing x x and Snd-Rt’s features available as a Pure Data (Pd) Type inference x external. Heap allocation x x Stack allocation x x Garbage collection x Keywords Runtime error checking x x1 Music programming, Snd, Programming languages, Can segfault x Lisp, Pure Data Speed 1 9 10 Hard realtime 0 10 5 Software overview Soft realtime 5 10 9 C library access 8 8 10 Name Description Vector read access 10 10 5 Vector write access 10 0 5 Snd A sound editor written by Bill List read access 10 10 5 Schottstaedt. List write access 10 0 5 Vct read access 10 10 5 Snd-ls A distribution of Snd. Vct write access 10 10 5 CLM A music synthesis and signal Signal bus access 0 10 2 processing package (Common CLM access 10 8 4 Lisp Music) written by Bill Schottstaedth. 1 Snd, Eval-C and Snd-Rt Snd-Rt A realtime music program- Eval-C and Snd-Rt are two domain specific ming environment running in- programming environments made for extending side Snd. Snd. Eval-C and Snd-Rt are implemented as Eval-C The programming language lisp low level macros running inside the Guile large parts of Snd-Rt is writ- Scheme implementation, and Guile again runs ten in. inside the Snd soundeditor. This means that San-Dysth A softsynth written in Snd- the three languages, Scheme, Snd-Rt and Eval- Rt. C, can be written and evaluated in the same source to provide an integrated and interactive Snd/Pd Snd running as a Pd external. environment. By combining Guile, Snd-Rt and Guile An R5RS Scheme interpreter. Eval-C, programmers get three levels of versatil- A community effort based on ity and execution speed which can all be used in the SCM Scheme implementa- 1Can be turned off, with the consequence of making tion by Aubrey Jaffer. Snd-Rt segfaultable and somewhat faster. the same source files. Snd/Pd, presented later, is programmed using all three languages.

2 Snd

Figure 2: Snd-ls V0.9.8.3

Snd-ls has enabled various features which are Figure 1: Snd 9.4, vanilla not by default enabled in vanilla Snd. Snd-ls is also configured to run Snd-Rt and Eval-C code. Snd is an extendable sound editor made by Bill Schottstaedt. Here is a quote from the Snd 3 Eval-C manual: Eval-C is a language based on C, but using S-expressions as its syntax format. Using S- Snd is a highly customizable, extensi- expressions makes it possible to mix Scheme and ble program. I’ve tried to bring out Eval-C in the same source files and easily add to the extension language nearly ev- advanced syntactic features to the C language ery portion of Snd, both the signal- such as object oriented programming and eval- processing functions and much of the uation on the fly to support interactive develop- user interface. You can, for example, ment of programs. Evaluating and re-evaluating add your own menu choices, editing code on the fly makes it very convenient to add operations, or graphing alternatives. low-level features to Snd and Snd-Rt since its Nearly everything in Snd can be set not necessary to recompile and restart Snd while in an initialization file, loaded at any developing. time from a text file of program code, or specified in a saved state file. It can 3.1 Some features provided by Eval-C also be set via inter-process commu- • Integration of C-code from within Lisp. nication or from stdin from any other Scheme and C-code is mixed in the same program (CLM and Emacs in particu- source, and the two languages can interact lar), embedded in a keyboard macro, directly. or typed in the listener. • Direct access to C libraries Virtually everything in Snd can be cus- • Lisp-macros. tomized via extension languages. Currently, Snd supports the following implementations • Generate, compile, link, run and redefine as being extension language: Guile (Scheme), C-code on the fly in Emacs or other Lisp Gauche (Scheme), Ruby (Ruby) and FTH programming environments (Forth). • Same efficiency as C Snd also provides features such as interac- tion with Common Music (CM) and bindings • Same low-level features as C for CLM, Gtk, Motif, Xlib, OpenGL, etc. Snd • Extra macros to support features such as runs in Linux, Freebsd, MacOSx, Solaris, Win- arrays, shared structures, classes, and au- dows and probably many other Unix dialects. tomatic wrapping of C library functions. 2.1 Snd-ls 3.2 Hello world! Snd-ls2 is a distribution of Snd.

2Snd-ls stands for Snd-Non Lisper. I accidentally (eval-c "" wrote ls instead of nl in the beginning of the develop- (run-now ment (printf (string "Hello world!\\n")))) First argument to eval-c contains extra op- By evaluating the above code, a structure tions sent to the C compiler. called struct name will be available from guile. The rest of the arguments are Eval-C code. It can be used like this: Strings containing pure C-code can be used (define test ( :one 1 anywhere in Eval-C code, which is very handy :twos ’(2) when writing macros and providing C features. :three "three")) For example, the above example can also be (-> test one) => 1 written like this: (-> test one 90) (eval-c "" (-> test one) => 90 (run-now "printf(\"Hello World!\\n\")")) (-> test twos) => (2.0) To make an actual null-terminated string for (-> test twos ’(4 5 6)) C, the string special form must be used instead, (-> test twos) => (4.0 5.0 6.0) as in the first version. 3.3 The Fibonacci function (-> test three) => "three" The simplest Fibonacci function: (-> test three "four") > > (define-c (fib (( n)) (- test three) = "four" < (if ( n 2) (-> test scm) => #f (return n) (return (-> test scm (lambda (x) x)) (+ (fib (− n 1)) (-> test scm) => # (fib (− n 2))))))) (fib 30) (-> test get-size) => 16 => 832040 (-> test get-c-object) define-c is a macro transforming its argu- => ("A POINTER" 147502592) ments into a call to eval-c. The function fib > above is transformed into: (- test destructor) ;;free it Modifiers and accessors are available for all (eval-c "" 4 slots. Known slot-types are SCM , char, int, (public float, double and char ∗, and arrays of those. C ( fib modifiers such as unsigned are treated appropri- (lambda (( n)) ately. There are also functions to add new types (if (< n 2) which can be mapped to the known types. Slots (return n) with unknown types keep their names, but are (return (+ (fib (− n 1)) treated as pointers by Eval-C. (fib (− n 2))))))))) Ec-structs are extremely convenient when When evaluating the above block of code, the communicating with C programs since they con- eval-c macro uses gcc or icc to compile and link tain actual C structures. the code. The C object must unfortunately be freed manually5. This was in the example done by 3.4 Structures evaluating (-> test destructor). The macro define-ec-struct creates structures 4 3 SCM is the Guile datatype, which can hold any type which are shared between Eval-C, C and Guile : of Scheme object. However, care must in some situations be taken using SCM types in ec-structres, since Guile’s (define-ec-struct garbage collector doesn’t scan ec-structure objects. The one objects are also kept in the Scheme object though (ie. <float-∗> twos the variable test in the example), so usually this should not be a problem. three 5 < > Either using a general garbage collector for C, such SCM scm) as the one made by Hans Boehm, or use a Guile SMOB around the structures, would solve this problem. How- 3And hopefully Snd-Rt sometime into to future as ever, freeing ec-structures manually hasn’t been a big well issue yet. 4 Snd-Rt • objects are started and Snd-Rt, or the RT extension for Snd, consists of stopped in realtime without sound two parts: glitches. The RT Engine - An engine for doing real- • Buses can be rerouted in realtime without time signal processing. sound glitches. The RT Compiler - A compiler for a Scheme- • Frame-accurate timing and scheduling like programming language to generate • Protection mechanism to avoid locking up realtime-safe code understood by the RT the computer if using too much CPU. Engine. • Controlled entirely from Scheme using The entire realtime engine and large parts of Guile, and, to a certain degree, from inside the compiler for Snd-Rt is written in Eval-C. the realtime objects themselves Nothing in the realtime engine or the RT lan- guage is written directly in C. Wrapper func- 4.4 The RT Compiler tions for libjack and liblrdf are created auto- matically using Eval-C as well. The RT compiler works by first translating the Scheme-like input-code into Eval-C code, and 4.1 The Fibonacci function then calling eval-c to create machine code. (define-rt2 (fib n) The RT compiler translates in several stages, (if (< n 2) which includes macro expansion, lambda lifting, n name mangling, type inference, syntax check- (+ (fib (− n 1)) ing, etc. (fib (− n 2))))) 4.4.1 RT Compiler features • Compilation of simple Lisp functions into (fib 30) machine code. => 832040.0 6 4.2 A sound • Generation of hard realtime safe code. (let ((osc (make-oscil))) • The compiled code should ideally be close ( 0 3 to C in efficiency. (lambda () (out (∗ 0.8 4.5 The RT programming language (oscil osc)))))) The Snd-Rt programming language (RT) is co- To run the example above, paste the 5 lines incidentally quite similar to the Lisp dialect into the terminal Snd-ls was started. After that, PreScheme [1]. But while PreScheme is a gen- a 440Hz pure tone should be played for exactly eral programming language standing on its own three seconds. feet, RT is a domain specific language special- 4.3 The RT Engine ized for music, sound and realtime use running inside the Guile Scheme interpreter. The main purpose of the RT Engine is to receive The RT language is designed to be a prag- objects created by the RT Com- matic language. It is not always as fast as C, piler, and provide ways to control exactly when and it is not as practical and feature rich as to run those objects. Scheme or Common Lisp, but it fits nicely in The API for doing this is hidden from the between and can provide almost-as-fast-as-C of user, and is accessed by instead calling methods usually-high-level-enough code. provided by the class. The RT language is mostly imperative, but 4.3.1 RT Engine features since it also has fairly good support for higher • Hard Realtime safe level functions, the programmer is often not re- • Jack and Pd driver stricted to one programming paradigm, but has the choice between coding semi-functionally or • Realtime scheduler with a properly made imperatively. priority queue to ensure a stable CPU load when adding and removing realtime objects 6Unless the RT code contains non-realtime safe func- to and from the engine. tions such as printf. 4.5.1 RT programming language chances are that it is easier to write an exter- features nal in C instead. However, in CLM, doing this • Ladspa kind of operation is both an easy and a straight forward task for a programmer. An example • Alsa midi of such a generator is the San Dysth softsynth • Lisp Macros described in the next section. • Automatic read access to Guile numbers, 4.7 Why a compiler? lists, vectors and numbers. (seamless inte- Most other music programming languages, like gration) PD, CSound and the SuperCollider server, are 7 • Automatic read- and write access to VCTs interpreters since the speed of the language con- • Guile has read- and write access to all of trolling the MusicV graph doesn’t matter that RT’s variables much. In those languages, the generators process • Most of CLM [2] is provided as well as blocks of audio samples, and most of the CPU is many functions provided by Snd, Sndlib therefore spent inside those generators and not and Guile. in the language controlling the graph. • Static typing and type inference However, in CLM, most of the time is of- ten spent in the language controlling the graph, • The types for numeric variables can be and therefore the difference in speed between specified for optimization interpreting and running compiled code is quite 9 • Runtime error checking and crash insur- significant. This is especially important when ance generating sound in realtime, since we don’t want to hear interruptions in the sound. • Buses • Realtime-safe ring-buffers between Guile 4.8 Dynamic memory allocation and RT Some important features are missing from the RT programming language, such as functions to • Lisp structures. Accessible from, and using the same syntax as, both Guile and RT.8 dynamically allocate memory on the heap for creating lists and closures. In practice, these • Access to external libraries can easily be features will probably not very often be used added on the fly by using Eval-C since its more natural to create lists in Guile instead of Snd-Rt, ie. preallocating the required 4.6 No control rate lists. And so far, I have not had any use for A liberating feature of CLM and Snd-Rt is that closures. there is no control rate, and that every sound However, since it hasn’t been possible to cre- frame is automatically exposed to the program- ate lists or closures in Snd-Rt so far, I can not mer. Pd, for example, generates 64 frames at be entirely sure its not very useful either. each control rate tick, while CLM generates only one. 5 San Dysth This creates a simpler interface for the user, San Dysth is a standalone realtime soft-synth and removes the need to create generators in written in Snd using the Snd-Rt language. a different language, usually C or C++. The San Dysth has controls to generate various downside is that its harder to make the signal kinds of sounds in between white noise and pure processing algorithms run fast this way, espe- tones. It also provides controllers to disturb the cially for interpreters. generated sound by using a ”period counter” to Exposing every sound frame to the program- extend the variety of the generated output. mer is also more powerful. For example, a new Common usage for San Dysth is organ-like type of filter can perhaps be made in Pd by us- sound, organic-like sound, alien-like sounds, ing the z∼ external or similar mechanisms, but water-like sounds, and various kinds of noise. 7Vct is Snd’s data type for holding arrays of sound Noise artists could find this softsynth most use- data. It is similar in functionality to Scheme’s vector ful. type, but vct’s can only contain sounds, and are opti- mized to do so. 9If we assume that compiled code runs faster than 8Not the same kind of structure as ec-structures interpreted code, which has been a common observation. ;; The following block creates and plays a ;; realtime object. The variables max-add-val, ;; max-drunk-change, period, das-filter, vol and ;; rate are parameters which later can be adjusted ;; by the user. Notice the use of dynamic scoping ;; when Snd-Rt use variables created by Guile. ;; (let∗ ((val 0) (addval 0) (max-add-val 0.01) (max-drunk-change 0.00005) (period 400) (periodcounter period) (inc-addval #f) (sr (make-src :srate 0 :width 20)) (rate 10) (vol 20) (das-filter (make-filter 4 (vct 1 −0.474 −0.723 −0.426) (vct 1 0.698 0 0)))) Figure 3: San Dysth ( (lambda () (out (∗ vol San Dysth’s synthesis technique works by us- (src sr rate san-dysth-dsp)))))) ing a set of rules which change state for each 6 Snd/Pd sample: Snd can be configured to work as a Pd external. ;; This function provides the main synthesis 6.1 FM synthesis example ;; routine for San-dysth. The function ;; returns one sample per call. ;; (define-rt (san-dysth-dsp direction) (cond ((<= val −1) (set! inc-addval #t)) ((>= val 1) (set! inc-addval #f)) ((> addval max-add-val) (set! periodcounter period) (set! inc-addval #f)) ((< addval (− max-add-val)) Figure 4: Snd/Pd doing FM synthesis (set! periodcounter period) (set! inc-addval #t)) The file pd-fm.scm used in figure 4 looks like ((= 0 (inc! periodcounter −1)) this: (set! periodcounter period) (let ((amp 0.6) (set! inc-addval (not inc-addval)))) (mc-ratio 2) (define drunk-change (index 4) (random max-drunk-change)) (freq 300)) (set! addval (define fm (filter das-filter (make-oscil (∗ freq mc-ratio) (if inc-addval :initial-phase (/ 3.14159 2.0))) drunk-change (define carrier (make-oscil freq)) (− drunk-change)))) (define fm index (∗ (hz->radians freq) (inc! val addval))))))))) mc-ratio index)) (pd-inlet inlet-num type func) (define instrument func is a function which is called when the ( object receives a message to inlet number (lambda () inlet-num of type type. Common values for (out 0 (∗ amp type are ’float, ’list and ’bang. (oscil carrier (pd-outlet outlet-num arg0 arg1 ...) (∗ fm index Sends one or more values to outlet outlet- (oscil fm))))))))) num. The arguments can be of any type. (pd-inlet 0 ’mc-ratio (lambda (val) (pd-bind symbol func) (set! mc-ratio val))) Pd messages sent to symbol arrive at the (pd-inlet 0 ’Fm-Frequency func function. (lambda (val) (set! (mus-frequency fm) (pd-unbind symbol) (∗ mc-ratio val)))) Stop receiving messages sent to symbol. (pd-inlet 0 ’Carrier-Frequency (pd-send symbol arg0 arg1 ...) (lambda (val) Sends one or more values to receivers for (set! (mus-frequency carrier) symbol. symbol can either be a scheme sym- val))) bol or a pd symbol. (pd-inlet 0 ’Index (lambda (val) (pd-get-symbol symbol) (set! (-> instrument fm index) Returns the pd symbol for the scheme sym- (∗ (hz->radians freq) bol symbol. pd-send works faster when a pd mc-ratio symbol is used instead of a scheme symbol. val)))) (pd-set-destroy-func thunk) (pd-inlet 0 ’Amplitude thunk is called before the object is de- (lambda (val) stroyed or reloaded. (set! (-> instrument amp) val)))) 6.5 API for signal processing 6.2 Threading The API for doing signal processing is simple: (in 0) is by default mapped to inlet 1, (in 1) is When Snd runs as a Pd external, it also runs mapped to inlet 2, and so on. (out 0) is by de- in its own dedicated thread. This ensures fault mapped to outlet 1, (out 1) is by mapped that Guile’s garbage collector doesn’t interrupt to outlet 2, and so on. sound processing. Communication between Pd and Snd happens by sending messages back and 7 Future work / Soft realtime using forth on two ringbuffer. Snd-Rt’s signal process- Guile ing thread, on the other hand, runs directly in Pd’s main thread, like all other signal process- There are situations where its more convenient ing code in Pd. to start new RT instances based on input either from a graphical user interface, midi messages 6.3 Interactive development or OSC messages, rather than to start many The Snd external also uses Snd’s code to read RT instances in advance and then let those in- Scheme code from stdin. This makes it possible stances handle input. to do interactive development inside a Lisp envi- Too much resources may also be used when ronment like Emacs by starting Pd as a Scheme forced to start many new RT instances in sub-process. The external also uses Snd’s code advance, instead of only starting them when to properly catch errors and display relevant needed. error messages, making debugging very conve- To schedule RT instances instantly, which in nient. this case means that scheduling an event takes less time than Just Noticable Difference (JND), 6.4 API for data processing Guile needs to support soft realtime. By inter- Below is a brief description of all functions pro- preting studies by M¨aki-Patola and H¨am¨al¨ainen vided for the Snd/Pd interface. A more com- on continuous sound instruments without tac- plete description is available in the Snd manual. tile feedback. [3], and Adelstein, Begault, An- derson and Wenze [4] on haptic-audio asyn- in Guile, although both options are available. chronity, two studies which measure two very different kinds of JNDs, it seems probable that 8 Acknowledgements a JND of less than 20-30ms might be sufficient This work has partly been funded by the Art for almost all situations. But its not unlikely Council Norway, Intravision Group13 and No- that more than 20-30ms could be tolerant for tam. most kind of use as well. Thanks to the following persons for support Soft realtime is also needed to do proper in- and help on this work: Chris Chaffe, Alexander teractive data handling when running Snd as a Refsum Jensenius, Bjarne Kvinnesland, Jose Pd external. Rio-Pareja, Jøran Rudi, Bill Shottstaedt, Siren A large problem for Guile to support soft Tjøtta, Anders Vinjar and Sook Young Won. realtime is Guile’s garbage collector. Guile’s garbage collector seems to stall scheduling of Web links 10 RT-objects for as much as 100-500ms now and CLM http://ccrma.stanford.edu/software/clm/ Common Music http://commonmusic.sourceforge.net/doc/cm.html then. Pd http://crca.ucsd.edu/∼msp/software.html Guile http://www.gnu.org/software/guile/guile.html Preliminary tests using Han-Wen Nienhuys HBGC http://www.hpl.hp.com/personal/Hans\ Boehm/gc/ and Ludovic Court`es’ experimental patches for San-Dysth http://www.notam02.no/∼kjetism/sandysth/ Snd http://ccrma.stanford.edu/software/snd/ Guile to let Guile use Hans Boehms garbage col- Snd-Rt http://www.notam02.no/arkiv/doc/snd-rt/ lector (HBGC) [5], and further let Guile receive midi to schedule appropriate RT objects in re- References altime, has shown promising results.11 But this needs more testing. [1] R. Kelsey. Pre-scheme: A scheme dialect for So far I have unfortunately not succeeded systems programming, 1997. running Guile with the HBGC in incremental [2] William Shottstaedt. Machine tongues xvii: mode. Running the HBGC in incremental mode Clm: Music v meets common lisp. Computer could make Guile able to guarantee soft real- Music Journal, 18(2):30–37, 1994. time performance.12 I have not yet investegated [3] T M¨aki-Patola and P H¨am¨al¨ainen. Latency reasons for the failure though, just observed it tolerance for gesture controlled continuous crashing. sound instrument without tactile feedback, In addition, patches to at least let Guile do 2004. realtime safe memory allocations might be nec- essary for Guile to run using SCHED FIFO or [4] Bernard D. Adelstein, Durand R. Begault, SCHED RR [6] realtime priority without risk- Mark R. Anderson, and Elizabeth M. Wen- ing priority inversion. zel. Sensitivity to haptic-audio asynchrony, Simply turning off the garbage collector is 2003. not an option, especially since memory usage [5] H.-J. Boehm. A garbage collector for c and in Guile tends to surpass the size of physical c++. memory surprisingly quickly. http://www.hpl.hp.com/personal/Hans Boehm/ Using a different Scheme implementation gc/. other than Guile is also an alternative. RScheme [7] is currently the only known larger [6] M. Harbour. Real-time posix: an overview, scheme implementation who has, or at least ear- 1993. lier claimed to have, realtime support. It might [7] Donovan Kolbly. Rscheme, 1997. be worth investigating the possibility of using RScheme as an extension language for Snd. But for now, the safe option is to receive ex- ternal input, such as MIDI, in Snd-Rt, and not

10I have not measured the garbage collector latency, but sometime it feels like around 500ms on my machine. 11http://lists.gnu.org/archive/html/guile-devel/2007- 06/msg00002.html 12Hans Boehm has indicated on a mailing list post- ing that a garbage collection time of 20-50ms could be achievable in incremental mode. http://gcc.gnu.org/ ml/java/2006-04/msg00072.html 13For the Windows port