PipeWire: The New Multimedia Service Now Ready for #utomotive Julian Bouzas Software Engineer Open First $i %&m Julian ● Multimedia Team at Collabora since 2019 ● PipeWire and Wireplumber Developer ● julian bou!as@collabora com 2 What is PipeWire' (resh multimedia service for )inu* ● Originall% meant for video onl%: Pulse(udio &or )ideo *PulseVideo) ● No- generic multimedia service for both )ideo and (udio ● )ideo Capture' – Cameras – /raphic Sources *Wa%land1 Vulkan, OpenG3 + ● (udio Playbac2 and Capture' – Microphone and Speakers – 4luetooth devices # Wh" do we need PipeWire' ● 6ni7es bot. Pulse(udio and 8ac2 audio servers1 $reatl% simpli&%in$ t.e 3inu9 Multimedia 0tac2 ● Permissions' 0upports containers li2e Flatpa2 and does not rel% on video and audio user $roups ● 3o- latenc%' can .andle ver% small bu:ers si!es *e $ ;2 samples+ ● Flexible' e9ternal 0ession Mana$er adaptable to an% use cases 5 Stac+ < Compati-ilit" #P%s on top of PipeWire ● (30( applications – PipeWire PCM plugin ● Pulse(udio applications – >eplacement for libpulse.so and libpulse-mainloop?glib.so ● 8(C@ applications – >eplacement for libjac2 so = #rchitecture and .esign ● Modular wit. Plu$ins ● Grap. based li2e G0treamer' Nodes, Ports and Links ● Multi-Process' – Daemon processes most o& t.e data *nodes can also run in the clients to avoid stalling+ – Bxternal session mana$er con7gures and links the nodes ● Fully based on its internal and 0imple Plu$in API library (SP(+ – Bxtremely simple and li$.t-ei$.t $eneric purpose multimedia librar% – Mostl% header?only C library -ith no other dependencies *$lib1 /0treamer1 etc. + A Performance and E/cienc" ● 0tatic code desi$n approach – (lmost no mallocs ● 6ses modern 3inux (PCs – mem&d & dmabu& to !ero?cop% device bu:ers – event&d & timer&d &or schedulin$ ● 3o- CP6 Usa$e and lo-?latency real-time capable 9 ,P0 0sage against Pulse#udio ● Eard-are' Cntel(>+ Core*TM+ i7?#==0 CPU @ ; #0/E! ● 0ource' PipeWire ● PipeWire .andles bu:er si!es o& ;2 samples1 Pulse(udio underruns 10 ,P0 0sage against J#,1 ● Eard-are' Cntel(>+ Core*TM+ i7?#==0 CPU @ ; #0/E! ● 0ource' PipeWire ● PipeWire .andles bu:er si!es o& ;2 samples1 8(C@ underruns 11 Securit" ● B9ternal 0ession Mana$er $rants permissions to applications ● ,odes can be onl% visible &or some applications ● Type o& Permissions' – *>+ >ead: )isible, Capture data – *W+ Write: Play data – *F+ E9ecute: allo- e9ecuting methods on objects *e $. 0etup &ormat on nodes) 12 E*ternal Session Manager ● ,ot included in t.e PipeWire project ● Creates and Con7$ures Devices to emit ne- ,odes ● 0ets up ,odes *Format1 Ports1 etc + ● Creates lin2s based on its polic% lo$ic -.en a client connects ● /rants securit% and access control to clients *applications+ ● 3aunc.ed b% t.e PipeWire daemon at startup 1; ,urrent Status ● )ersion 0 ; 5 released in May 2020 and distributed in Fedora ;2 – Plenty o& 8(CK applications alread% -orkin$ -it. PipeWire – Man% Pulse(udio apps -ork as -ell – 4luetoot. startin$ to be &ully supported1 needs more testin$ – Musical Cnstrument Di$ital Cnter&ace *MCDC+ -or2s – Plans to replace Pulse(udio soon – )ideo capture &rom )#32 devices -or2s -ell – Wa%land screencastin$ &rom -eston1 $nome?s.ell1 and -lroots supported ● (dopted b% (/3 *(utomotive /rade 3inu9+ as t.e core audio &rame-or2 1# Who started this ● Author: Wim Taymans – Well-kno-n old /Streamer developer & e9-maintainer – Sponsored b%' >ed Hat ● Embraced by PulseAudio developers – Seen as the ne9t $eneration of PulseAudio ● Welcomed by ALSA and JACK developers ● License: MIT PipeWire .2T tool ● Generates DOT $rap.s ● 6sa$e' – G p--dot p- dot Hoptions] ● Options' – ??all (sho-s all object types' nodes1 devices1 ports1 clients1 etc + – ?-smart *onl% sho-s lin2ed objects) – ?-details *sho- all object properties) 1< PipeWire in the #utomotive Industr" Wh" PipeWire suits #utomotive ● Problem: Device handlin$ in connected cars is comple9' – Plent% of speakers and cameras – Multiple audio streams' >adio1 Bmer$enc%1 ,avi$ation, Communication. – Phone calls over 4luetoot. ● 0olution: PipeWire wit. a flexible external 0ession Mana$er – Custom polic% logic – Custom hard-are pipelines – Eard-are control abstraction – 0ecurit% 1A WirePlum-er ● First e9ternal session mana$er implementation &or PipeWire ● Ori$inall% planned &or Bmbedded only *(utomotive+ ● Now $eneric and &ull% &eatured 0ession Mana$er &or bot. Bmbedded and Desktop ● 4ased on /Object to support -ritin$ bindin$s in ot.er lan$ua$es' >ust1 P%t.on1 36( 19 %ntroduction of new 2-3ects ● Bndpoint' object t.at .andles PipeWire nodes – (udio Soft-are DSP Endpoint – Simple Node Endpoint ● 0tream' connection points o& an Bndpoint ● 0ession' set o& Bndpoints – )ideo Session – (udio Session 20 Software .SP e*ample 21 $ardware .SP Example 22 WirePlum-er .esign ● Provides an lib-ireplumber (PC t.at ma2es eas% -ritin$ WirePlumber modules and even ot.er session mana$ers ● Modular desi$n 2; WirePlum-er Modules ● Monitor – monitors devices and creates nodes -.en enabled ● ClientPermissions – $rants permissions to clients -.en connected ● Con7$Bndpoint – creates di:erent endpoints per nodes based on con7$uration 7les ● Con7$Polic% – lin2s endpoints based on con7$uration 7les 2# Bindings e*ample usages ● Pyt.onK>ust session mana$er to avoid use o& low level PipeWire (PC and objects ● WirePlumber Module t.at interprets 36( files &or Luic2 and easy custom polic% lo$ic 25 WirePlum-er 4ersions ● v0 1 0 *Jul 2019+' used in (/3 Eapp% Ealibut A 0 0 ● v0 1 2 *Oct 2019+' used in (/3 Eapp% Ealibut A 0 2 ● v0 2 0 *Dec 2019+' used in bot. (/3 Eapp% Ealibut A 0 5 and (/3 Ctc.y Cce7sh 9 0 0 ● v0 ; 0 *June 2020+' First version -it. Des2top 0upport 2< (uture !elease ● 0upport &or 4indin$s in ot.er lan$ua$es ● Clean t.e (PC and ma2e it stable *almost t.ere+ ● Cmprove Documentation ● More unit tests and e9amples 2= Who started this ● Author: George Kiagiadakis – 0ponsored b%' Collabora ● Welcomed by PipeWire developers ● Git repository: – .ttps'KK$itlab &reedes2top or$KPipeWireK-ireplumber ● Documentation: – .ttps'KKPipeWire pa$es &reedes2top or$K-ireplumber ● License: MIT Showtime Than+ "ou for watching ● 8oin us on I>C at Mpipewire on Freenode ● .ttps'KK$itlab freedes2top.or$/pipe-ire ● julian bou!as"collabora.com ;0.