Multimedia Driver Supp ort in the FreeBSD Op erating System Luigi Rizzo James S Lowe Universita di Pisa University of WisconsinMilwaukee Dip di Ingegneria del lInformazione Col lege of Engineering and Applied Science lrizzoietunipiit jamescsuwmedu httpwwwietunipiitluigi httpwwwuwmedujames Abstract applications which deliver multimedia streams re quiring synchronization b etween the application and the hardware In this pap er we will discuss the motivation de Acquisition and rendering of multimedia streams re sign implementation issues and applications for the quires sp ecial p eripherals such as audio CODECs audio and the video acquisition drivers which are and frame grabb ers as well as adequate supp ort in available in FreeBSD Both systems have b een the op erating system In fact CODECs and frame designed from scratch with sp ecial attention paid grabb ers are relatively simple devices and in prin to the eective transfer and synchronization of data ciple they could b e easily supp orted in an op erating between the hardware and adv anced applications system using the conventional read and write system calls plus a small number of ioctl calls to The main fo cus in the design of the audio driver congure the sp ecial features of the devices How was for supp orting multimedia applications which ever synchronization is a fundamental requirement often require full duplex op eration and have strict with many advanced applications so the internal synchronization requirements This suggested the structure of the driver and the interface it exp orts denition of a new software interface simpler to use must supp ort these requirements in a exible and ef than the previously existing OSS Voxware driver cientway Furthermore video streams are highly The driver is backward compatible with the OSS demanding in terms of bandwidth so the conven API due to the large base of applications that use tional readwrite interface might not be the this API most ecient or eectiveway to transfer data The video acquisition driver fo cuses on providing In recentyears multimedia supp ort in the FreeBSD exible access to the acquisition device and the abil op erating system has received a lot of attention ity to suit the needs of dierent applications from namely the addition of supp ort for full duplex au simple TV viewers to video conferencing programs dio cards and high p erformance video grabb ers com The driver provides memory mapp ed access to the municating through the PCI bus Most of these capture buer exible supp ort for synchronization systems have b een designed from scratch In this with the application and supp orts the various cap pro cess we had the signicantadvantage of already ture formats and scaling capabilities that are avail knowing which applications would likely use these able from the video acquisition hardware devices and what features were required As a con sequence we could use this information to provide ecient supp ort for the applications without miss ing some imp ortant features and without cluttering Intro duction the design with features that are of no practical use For the video acquisition driver wewere essentially Multimedia applications are b ecoming more and free to design our software interface from scratch to the increasing per more p opular everyday due For the audio driver the existence of a relatively formance of workstations and networks The ability to run highly demanding compression algorithms in The term CODEC comes from a hybrid of the words software has promoted the developmentofadvanced COmpressor and DECompressor large base of applications forced us to implement compatibility interface We decided to lo ok at a Record buffer Playback buffer compatibility issues only after the architecture of Input Output er was designed so that these issues did not the driv Mixer Mixer RDY RDY DMA FREE DMA FREE inuence our design AUX ADC DAC pap er is structured as follows In Section The CD CD we present the audio driver starting with a brief description of audio hardware followed by a dis cussion of the various features implemented bythe Figure A mo del of a typical full duplex audio device driver and a brief discussion of our design choices On the left is the record or input section on the right We discuss the same topics in Section for video the playback or output section acquisition supp ort Finally Section discusses the implementation status of the drivers presented in systems mainboard as in the case of Intelbased this pap er and Section contains our conclusions PCs In some other cases the audio device has an internal DMA engine which acquires control of the bus and p erforms the data transfer devices on a PCI bus generally op erate in this way As an al Audio ternative the audio device could havesomeinternal memory where data is buered eg under control of a sp ecialized pro cessor and the main pro cessor The structure of a typical audio device is shown in has only to transfer blo cks of data at given intervals Figure with two logically indep endent systems p ossibly using programmed IO providing supp ort for audio record and playback In some cases the audio device has an on b oard Digi In the record section analog sources are routed tal Signal Pro cessor DSP which can b e used to run through a mixer to the Analog to Digital Con data compression algorithms ooading the main verter ADC whose output is stored into a buer pro cessor from this task This is in general a useful The hardware dictates the native resolution or b ecause some algorithms eg GSM or MPEG en bits data formats linear or law channels co dingdeco ding are exp ensive and it is cheap er to monostereo and sampling rate The software im run them on a DSP which has b ecome a commo d plements the buer and may additionally implement ity device b ecause it is extensively used in cellular some data format conversion b efore passing data to phones and other digital audio devices the application program There are large variations in the capabilities of the The playback section uses a buer to hold data that mixers as well Some simple devices just have one is sent to the Digital to Analog Converter DAC input and one output channel directly connected to The output of the DAC is fed to a mixer which the ADC and the DAC resp ectively with no vol combines together other analog sources and sends ume con trols The ma jority of audio devices for them to the appropriate outputs Similar to record PCs have a simple multiplexer with only a master ing the op erating system software implements the volume control on the input section and a full fea buering strategy and p ossible format conversions tured mixer with indep endent volume controls for to supplement the native features supplied by the the various channels on the output section Newer hardware devices have separate fullfeatured mixers on b oth the input and output paths containing a large se The DAC and the ADC are usually part of the lection of sources and destinations along with the same physical device referred to as the CODEC ability of p erforming miscellaneous functions such The transfer of samples between the CODEC and as swapping left and rightchannels muting sources the buers supplied by the op erating system can b e etc done in several ways Most of the time the audio device uses a DMA channel which is presentonthe Except for cases where hardware limitations makethe system share resources thus preventing full duplex op eration or o dd mixer characteristics ally simple b ecause the requested op eration eg setting a volume or selecting an input source gener Appl. Appl. Appl. ally takes place in real time and the only signicant veaninterface which can ease the p ort Server issue is to ha ware to dierent systems Library ing of soft Device driver Hardware (CODEC, DSP, ...) Audio Access Granularity Figure Possible structure of applications using the The natural representation of sound is a stream of audio device Application access to the driver can be data Our driver emulates this natural representa direct via a library or through a server tion by supplying the application with the appropri ate to ols to manipulate a circular queue rather than Audio Device Initialization requiring the application to collect and manipulate the data in blo cks The device driver layer in Figure is resp onsible The audio driver has two mo des of op eration char for the transfer of data b etween the applications and acter and block mo de In character mo de the device the audio hardware Prior to using the audio device pro duces a stream of bytes and select returns for data transfer applications will need to acquire when one byte can b e transferred Toenter the block the capabilities of the device supp orted sampling mode not to b e confused with blocking mode which rates data formats number of channels full du is an orthogonal feature the AIOSSIZE ioctl is plex op eration and any other devicesp ecic fea used to sp ecify the granularity to be used for the tures and set the desired data formats These select op eration The latter will return success op erations are usually done by means of ioctl fully only when at least a whole blo ck of data can b e calls whose name and format change from system transferred AIOSSIZE ioctl call can mo dify the to system Although some approaches eg those requested value if it is not an acceptable one and it where all parameters are read or set at once using returns the blo cksizeinuse Ablock size of or a single call app ear to b e more elegant and ecient will set the device driver into character