The GStreamer Multimedia What is GStreamer Architecture A for building multimedia applications

Allows complex graphs to be built from simple elements

Steve Baker Supports any data-type for which elements exist

[email protected] Plugin support allows media specific extensions

Main core dependency is Glib

Runs on (x86 & PPC), Solaris and FreeBSD

What is GStreamer Why create GStreamer

Uses 3rd-party libs where possible Linux (and other *nixes) have bad media

¡ Code re-use is good handling

GStreamer is not a sound server Many hacked single-purpose apps and libs exist

¡ But you could build one with it Other platforms have multimedia frameworks

Associated with GNOME but is graphics-toolkit ¡ DirectShow

agnostic ¡ QuickTime ¡ ¡ KDE may adopt GStreamer BeMedia

¡ RealMedia SDK Applications using GStreamer Intellectual property issues

Rhythmbox Core library is licensed as LGPL

Totem, Gst-player Apps are becoming LGPL to avoid GPL patent

Gnome-Sound- issues Recorder, gst-mixer Plugins allow distributions to ship core

Sound-juicer, Marlin GStreamer but leave out patent-encumbered libraries Video-whale ¡ For example , Sorenson, etc

Gnonlin, VDV Third party vendors could distribute binary- Your App Here... only GStreamer plugins (but it hasn't happened yet)

Language Bindings Tools and utilities

Core library is written in

Python, , Ruby, Guile

Java, C#, C++

Gst-editor – graphical pipeline building style C++ (for KDE)

Gst-launch – command-line pipeline building Not all bindings are as mature as others ¡ filesrc location=”britney.mp3” ! mad ! osssink Gst-register – updates registry of plugin features

Gst-inspect – gives info about installed plugins Pipelines and elements Pads and capabilities

Elements are joined together to form complete Elements have pads which can be connected

pipelines Data and events flow through pads

Each element performs a single function Before data can flow pads must negotiate Data and events flow from left to right mutual capabilities

Some pads only appear on-demand

Buffers and memory usage Chain, get and loop elements

Data travels in objects called buffers “Chain” elements are the simplest and fastest –

Data might not always be writeable one buffer in, one buffer out

Allocating and copying memory is avoided “Loop” elements are required for more than one where possible source or sink, or if number of buffers going in differs from those coming out Optimisations are possible without all elements

being aware “Get” elements only have source pads Input, output, Core elements and filter plugins Inputs – file, GnomeVFS, UDP, DVD

Outputs – file, sound card, X display, SDF,

network Some elements contain other elements (Bins)

Codecs – mp3, , The “thread” element is a bin which runs its children in a separate thread Filters – LADSPA, effectTv

The “queue” element is placed between threads Plus many others to decouple the data flow

The “tee” element splits one input into more then one output

Clocks and synchronisation Schedulers and cothreads

Schedulers decide when each element needs to process data

Cothreads are user-space threads which are simpler to use and faster than real threads

For A/V playback, always sync to the sound Original scheduler created a cothread for each card element

Sound card clocks are precise but not accurate Current default scheduler doesn't need Any real-time elements synchronise to the cothreads, so it runs on more platforms but has “default” clock higher latency

Non real-time apps go as fast as they can Autoplugging with spider New features in development

Each media type has its own type-finding Rewritten subsystems for content negotiation, function metadata, capabilities

Spider finds the media type of a stream, then Element interfaces allow full interaction finds an element to decode it between apps and elements

Elements are added until a complete pipeline Current interfaces include

can be built ¡ Audio mixer (used by gst-mixer in -media)

¡ Video overlay

¡ Property probing (to find hardware device names)

¡ Interactivity for DVD menus and Flash

More Information

http://gstreamer.net/

#gstreamer on irc.freenode.net

Any questions?