PulseAudio on Mac OS X

Daniel Mack [email protected]

Abstract • IOAudioEngines are required to report PulseAudio is becoming the standard audio envi- their sample rate by delivering exact times- ronment on many Linux dektops nowadays. As it tamps whenever their internal ring buffer offers network transparency as well as other inter- rolls over. The more precise, the better, esting features OS X won’t offer its users natively, as its userspace counterpart (the HAL, see it’s time to have a closer look on possibilities on how below) can do better estimation of the de- to port this piece of software over to yet another vice’s speed and approximate closer to the platform. actual hardware sample pointer positions, In the recent months, I put some effort into at- resulting in smaller latencies. tempts to port PulseAudio to Mac OS X, aiming for cross-platform interoperability between hetero- 1.2 HAL gen audio host networks and enabling other features PulseAudio offers such as uPnP/AV streaming. The HAL is part of the CoreAudio framework This paper will first give a short overview about and is automatically instanciated within the how CoreAudio, the native audio system on Mac OS process image of each CoreAudio client appli- X, is structured, the various ways that can be used cation. During its startup, it scans for plugins to plug into this system, and then focus on the steps in /Library/Audio/Plugins/HAL and this way it takes to port PulseAudio to Mac OS X. offers the possibility of loading userspace imple- Keywords mentations of audio drivers. The HAL is also in charge of interfacing to the IOAudio based PulseAudio, Mac OS X, network audio, virtual audio kernel drivers and hence acts as their bridge to driver, portability userspace clients. 1 CoreAudio essentials 1.3 AudioHardwarePlugins for HAL 1.1 IOAudio Kernel extensions Automatically loaded by the HAL code upon • The Darwin kernel is in charge of handling creation of an audio client, AudioHardwarePlu- hardware drivers, abstracted via the IOKit gins are instanciated via the standard CFBundle API framework. load mechanisms. An interface must be im- plemented to provide the hooks needed by the • The kernel’s representation of an audio de- HAL, and a full-fledged infrastructure of APIs vice is an object derived from the IOAu- for adding audio devices, streams and controls dioDevice base class, which holds a refer- are available. Unlike kernel drivers, virtual ence of an IOAudioEngine (or a derived drivers implemented as HAL plugin are working type thereof). on a per-client base, so their implementations • The kernel’s way of adding audio streams must care for mixing and inter-client operabil- to a device is attaching objects of type ity themselves. IOAudioStream to an IOAudioEngine. 1.4 System Sound Server • The kernel’s API is only one way to provide This daemon is in charge for handling system- an audio device to the system; the other is internal sound requests such as interface and a plugin for the HAL (see above). alert sounds. • Sample material is organized in ring buffers 1.5 coreaudiod which are usually shared with the hard- coreaudiod is a system-wide daemon that ware. gives home to the System Sound Server and provides the AudioHardwareServices API for re-route audio differentely. An example of this querying parameters of available audio drivers. approach is the closed-source shareware utility The daemon also handles the default sound in- AudioHijack2. More research is needed in or- terface configuration on a per-user level1. der to find out whether this approach is also feasable for PulseAudio sound re-routing. At 1.6 AudioUnits the time of writing, this option is not being in- AudioUnits are Mac OS X typical CFBundles vestigated on. which can be installed user-wide or system-wide to fixed locations in the file system and which 3 PulseAudio on OS X can be accessed by arbitrary applications with In order to bring PulseAudio to Mac OS X, an standarized API for audio processing. They some tweaks are needed to the core system, can also offer a graphical representation for pa- and some parts have to be re-developed from rameter control and visualization. The two sup- scratch. ported types of AudioUnit plugins are effect processors and virtual instruments. 3.1 pulseaudiod Porting the daemon is of course the main part 2 Possible audio hooks of the work as it is the heart of the whole sys- The purpose of this project is to be able to hook tem other pieces connect to. Since a couple of into the transport channels of all audio applica- versions, pulseaudiod, along with a selection of tions - including system sounds, if desired - and its essential modules, builds fine on OS X. Some re-route audio through an either local or remote adoptions were neccessary to make this happen. PulseAudio server connection. Mac OS X officially offers a number of ways • poll() is broken since Mac OS X 10.3, dis- to access the audio material: respecting the timeout argument and re- turning immediately if no file descriptor • A virtual sound card interface implemented has any pending event. This was circum- as kernel driver which can either be con- vented by using the select() syscall, just like figured as standard sound interface for all PulseAudio does for Windows. appliactions and/or system sounds. Appli- cations may let the user decide which sound • recv() with MSG PEEK does in fact eat up card to use for input and output sound data from the given file descriptor. The rendering, but for those which don’t (like workaround was to use a different ioctl() iTunes, QuicktimePlayer, iChat, ...), set- for this purpose. ting the system-wide default is the only op- • OS X lacks a proper implementation of tion. POSIX locks but implements its own thing • A virtual sound card interface implemented as defined in Multiprocessing.h. A ver- as AudioHardwarePlugin for the HAL. The sion which uses them internally for the same rules as for the kernel versions apply: PulseAudio daemon was needed. if an application doesn’t allow its user to • clock functions work differently than on choose the device for audio output, the sys- Linux, so a specialized version for the clock tem falls back to the configured default. wrapper functions in PulseAudio was also • An AudioUnit which is loaded by more ad- neccessary. vanced applications such as Logic. For ap- • Mac OS X offers a powerful API to give plication which don’t use this plugin inter- userland tasks high priority. This is es- face, this is no option. sential for real-time applications just like PulseAudio, so an implementation using Another possible way of interaction is unof- this API was added to the daemon. ficial, somewhat hackish and based on the idea of library pre-loading for selected applications. • Some library PulseAudio uses are not suit- Binaries are relaunched with their CoreAudio li- able for OS X. Work on the build system braries temporarily replaced by versions which was done to build some parts of the suite conditionally. 1http://lists.apple.com/archives/coreaudio- api/2007/Nov/msg00068.html 2http://rogueamoeba.com/audiohijackpro/ 3.2 CoreAudio device detection module dependencies for others are copied to the frame- In order to make use of audio input and work bundle and the tool install name tool output devices CoreAudio knows about, a which ships with XCode is called to remap the new pulseaudiod module was written which path locations recursively. uses the CoreAudio specific callback mecha- 3.6 PulseConsole nisms to detect hotplugged devices. For each PulseConsole is a Cocoa based GUI applica- detected device, a new module instance of tion written in Objective-C that aims to be a module-coreaudio-device is loaded, and un- comfortable configuration tool for PulseAudio loaded on device removal, accordingly. servers, both local and remote instances. It of- This module is part of the offcial PulseAu- fers a way to inspect and possibly modify details dio sources since some months and is called and parameters and a nice GUI for per-stream module-coreaudio-detect. mixer controls and routing settings. 3.3 CoreAudio source/sink module The plan is to make this tool as convenient as possible, also with GUIs for mixer controls, Loaded and unloaded by the house-keeping detailed server inspection and all the like. This module module-coreaudio-detect, this mod- will need some time to finish, but is actively ule accesses the actual CoreAudio device, developed already. queries its properties and acts as translation layer between CoreAudio and PulseAudio. An 3.7 AudioHardwarePlugin for HAL important implementation detail is that code in CoreAudio allows to add software plugins to this module has to cope with the fact that audio register virtual sound interfaces. Such a plugin is exchanged between different threads. was developed for PulseAudio, with the follow- This module is part of the offcial PulseAu- ing key features. dio sources since many months and is called module-coreaudio-device. • Allows audio routing to both the local and any remote server instances. 3.4 Bonjour/ZeroConf service • Multiple plugin instances communicate discovery module with each other over a distributed notifi- Porting the dependency chain for Avahi (dbus, cation center. This is essential for sharing ...) wasn’t an easy and straight-forward task to stream volume information. do, and given the fact that Mac OS X features a convenient API for the same task, a new module • Each plugin instance announces itself to a for mDNS service notification was written. The system-wide message bus and can receive code for this module purely uses Apple’s own setup controls. This way, an existing con- API for announcing services to members of a nection to a sound server can be changed local network. to some other server instance. This module is also part of the official • The plugin is capable of creating multiple PulseAudio source tree since a while and is virtual sound interfaces. This can be help- called module-bonjour-publish. ful to cope with more than the standard stereo channel mapping. The configura- 3.5 Framework tion of which interfaces are created is con- On Mac OS X, libraries, headers and as- trolled by the Preference Pane implemen- sociated resources are bundled in frame- tation (see below). work bundles. As PulseAudio libraries and the libraries they are linked against are 3.8 PulseAudio AudioUnits shared amongst several components for this For a more fine-grained way of routing specific project, they are all put in one single location audio pathes through the PulseAudio daemon, (/Library/Frameworks/pulse.framework). AudioUnit plugins were developed. They con- This path was passed to the configure script nect to the local audio daemon and act as sound as --prefix= directive when PulseAudio source and sound sink, respectively. All audio was built. A script (fixupFramework.sh) hosts that are capable of dealing with this type is in charge to resolve libraries dependencies of plugin interface (ie, Apple Logic) can use which are not part of a standard Mac OS X this way of connecting specific sound pathes to installation. All libraries that are found to be PulseAudio. 3.9 Virtual audio driver (kext) More information can be found in the repos- Another way of adding an audio device driver to itory at github.com. a system is hooking up a kernel driver for a vir- 3.10 virtual audio driver adapter tual device and communicating with this driver module from user space to access the audio material. This is what the virtual audio driver does. Acting as counterpart of the virtual audio driver This part of the project mostly ex- kernel module, a special purpose module for ists for historical reasons, before the pulseaudiod takes notice of added and re- AudioHardwarePlugin approach was fol- moved virtual sound card instances, maps the lowed, which turned out to be much more shared memory offered by the kernel and creates interesting and feasible for the purpose. The stream instances inside the PulseAudio daemon. code is still left in the source tree for reference The name for these streams are taken from the and as proof-of-concept which might act as kernel space interface. As the kernel extension reference in the future. is not currently used anymore, this part of the Some of its key features include: source tree is also considered legacy. 3.11 Preference pane • support for any number of interfaces, fea- turing a configurable number of input and The PulseAudio preference pane hooks itself output channels each. into the standard Mac OS X system preferences and offers the following features: • userspace interface to control creation and deletion of interfaces. • control the startup behaviour of the • usage of shared memory between userspace PulseAudio daemon and kernel space, organized as ring buffer. • configure authentication settings for net- • infrastructure to register a stream to work connections userspace for each client that is connected • GUI for adding and deleting virtual sound to the interface. The framework for this interfaces code exists, but all attempts to actually make it work failed so far. 3.12 Component locations The concept of the driver model is to have Mac OS X organizes its file system contents in one abstract IOService object (instance of a quite different way than Linux installations. PADriver) which is the root node for all other As described above, a framework is built in or- objects. Upon creation (at load time of the der to share the PulseAudio libraries amongst driver), the PADriver will be announced to the the various components. Components linking userspace. to the PulseAudio libraries have their linker set- A IOUserClient class named PADriverUser- tings configured to this path. Hence, the dae- Client can be instanciated by user space, and mon and command line utilitily binaries as well commands can be issued to create new and as the loadable modules are found at the frame- delete instances of PADevices. A PADevice is work location as well, and if you want to ac- derived from IOAudioDevice and acts as a vir- cess the PulseAudio command line tools (pacmd, tual audio device. To export audio functions, it paplay, ...) in the shell, the $PATH environment has to have an PAEngine (derived from IOAu- variable needs tweaking. dioEngine). Apart from that, the other components are Depending on the type of audio engine (one expected to be installed into specific locations for the mixed audio stream or one for each indi- so they can be found by the system. There will vidual user client), the PAEngine can have one be documentation in the source tree to describe or many references to PAVirtualDevices, respec- the exacte pathes. tively. Once a PAVirtualDevice is created, it is an- 3.13 Installer and packaging nounced to the userspace, just like a PADriver. A PackageMaker receipt has been created to A userclient will create an object of type PAVir- generate installer packages that can be pro- tualDeviceUserClient which can be used to issue cessed by the standard Mac OS X package in- commands specific to a PAVirtualDevice. staller, giving the user the general look and feel and procedure as most OS X add-ons. Depend- toolchain fully usable. In particular, the follow- ing on Apple’s policy for such tool suites, at- ing topics need to be addressed. tempts might be made to publish the package via Apple’s application store. • Get the latency down. There are currently problems with untight scheduling in the 3.14 License and source code PulseAudio client implementation, and too All parts of this suite are licensed under the big buffer sizes. GNU General Public License in version 2 (GPLv2). • Considerations for multi-architecture li- The source code is accessible in the public braries and binaries. XCode is not the git repository found at https://github.com/ problem in this regard, but the auto- zonque/PulseAudioOSX conf/automake build system is. • The clocking model is subject to reconsid- 4 Possible scenarios eration. While things are comparitively Once the whole suite is developed as described easy in scenarios dealing with real hard- and stable to a acceptable level, interesting au- ware soundcards, it becomes more obfus- dio routing scenarios are imaginable. cated in this virtual case as the PulseAudio daemon is the driving part for all clocks. • Sound played back by iTunes can be That means that if audio is actually routed routed through the virtual PulseAudio into a null-sink on the PulseAudio side, the sound interface and from there be sent to virtual sound card will play at high speed, an uPnP/AV audio sink. which might cause problems with audio ap- • Sound played back by iDVD can be routed plications that assume real-time playing. through the virtual PulseAudio sound in- • Cosmetic work on the GUI tools to give terface and then be sent to an Airport Ex- them the look of a nice tool users want to press using PulseAudio’s ROAP module. accept as part of their system. Currently, Mac OS X can not natively do that. they look like debug tools for developers. • A LADSPA proxy plugin could be developed • Testing. Of course. The whole project is to communicate with PulseAudio directly rather fresh, so it hasn’t seen a lot of testers on Linux hosts. The stream for this plugin yet. could be re-routed to a network host run- ning PulseAudio on Mac OS X, and there 6 Trademarks be used as virtual input stream in , Logic Mac, and Mac OS, Mac OS X, iTunes, iDVD, hence allowing virtual instruments and ef- Logic, Airport and Cocoa are trademarks of Ap- fect plugins on Mac OS X to be used in ple Inc., registered in the U.S. and other coun- environments. LADSPA tries. • Without any network interaction, sim- ply routing all audio through the virtual 7 Acknowledgements PulseAudio sound interface allows users My thanks go to the world-wide Linux audio to control volumes of all connected audio community for providing ALSA and PulseAudio clients individually (eg, silence annoying as sophisticated audio layers on Linux, making flash player in your browser, leveling au- this project possible at all. dio applications that don’t offer a way to do this natively, etc). • Soundcards that are not supported by ALSA driver can be accessed from Linux over the network, using a Mac OS X audio host. 5 Challenges and TODOs This project is considered work in progress and is not yet finished. There are many details that need to be refined in order to make this