Vulkan, OpenGL and/or Zerocopy Matthew Waters (ystreet00) GStreamer conference 20"# "0 October 2016 Who Am & ● %ustralian ● Work ( )entricular ● Graph'cs * OpenGL, Vulkan ● Mult'med'a +u'ck &ntroduct'on * OpenGL ● OpenGL $orn rom SG& in ",,0-s ● )ross(plat orm .D %0& – 1"", wayland, OS 1, 'OS, %ndro'd, Windows, 2m$edded Linu3 +u'ck &ntroduct'on * GStreamer OpenGL/2S ● Minimum tar4et OpenGL 2S !50 – 2ssentially the $e4innin4 o GLSL support ● Versions supported * OpenGL 2S !50/.53 /esktop !53/.53/653 ● 0lat orms supported * L'nu3 (1"" + Wayland), OS 1, W'ndows, 'OS, %ndroid, 2m$edded Linu3 ● Various elements a8aila$le * 4lima4es'nk, 4lcolorcon8ert, 4l8ideom'3er, 4ltrans ormat'on, 4lo8erlay, 4le9ects:;, etc What-s <ew= - l'$4st4l ● 4l8iewcon8ert now supported on OpenGL 2S !50 plat orms (stereo elements as well) ● 4lcolorcon8ert now allows con8ert'n4 to mult'(planar colorspaces 'n OpenGL 2S .53 ● 4ldownload %0& remo8ed rom the library ● dma($u GL uploader ● GL >ueries ● /elayed GStreamer de$u44in4 What-s <ew= * l'$4st4l GL memory ● <ew GL $u9er $ased GstMemory ● GstGLMemory0?O * GL te3tures w'th 0'3el ?u9er O$@ects ● GstGLMemory2GL * GL te3tures w'th 2GL&ma4eAs ● <ew GL render$u9er $ased GstMemory ● GstGLBrame$u9er +u'ck &ntroduct'on * Vulkan ● Vulkan released Be$ruary !0"# ● )ross(plat orm .D %0& – 1"", wayland, %ndro'd, Windows, 2m$edded Linu3 ● %ims to $e a $etter Ct or modern G0Us ● More control o8er synchron'Eation Vulkan 'n GStreamer ● 8ulkansink and vulkanupload elements ● Only $asics 'mplemented – Modelled on l'$4st4l %0& ● Much more work needed to $e on par with OpenGL support Vulkan 'n GStreamer ● Somewhat sim'lar 'n rastructure to li$4st4l – Gst)onte3t – /'splay – Window – &nstance – /e8'ce ● Some thin4s are 8ery d'9erent – Synchron'Eat'on * semaphores, e8ents, ences – More applicat'on state Zerocopy - &ntroduct'on ● /efinit'on ran4es romF – <o copies at all – <o )0U per ormed copies ● Most common $etween decoder and renderer ($ut also occur $etween capturer/encoder) Zerocopy * Where= ● V%(%0& ● OpenM%1 &L ● %ndro'dAs Media)odec ● iOS/macOS V'deoGool$o3 ● V/0%D ● 18MC/18?% ● /1V% Zerocopy * How= ● ?ind – VM MMD – Synchron'Eat'on ● Dse ● Dn$'nd – Synchron'Eat'on Zerocopy * dma($u ● Dsed $y 86l!, (V%(%0& on 2GL) ● dma($u is con8erted to an 2GL&ma4e ● 2GL&ma4e is $ound to a GL te3ture with 4l2GL&ma4eGar4etGe3ture!D() Zerocopy * Andro'd Med'a)odec ● Hold the consumer end o a >ueue ● )an only pop o9 the end o the >ueue $ut donAt ha8e un'>ue rame handles ● Synchron'Eat'on 's all internal ● )an only $'nd to one GL conte3t at a time Zerocopy * OpenM%1 &L ● Dses 2GL&ma4eAs ● I0' has a separate om3 component * e4l_render ● 2GL&ma4e $acked $y a GL te3ture passed into OM1 ● )ustom GstGLMemory Zerocopy * V'deoGool$o3 - macOS ● Dses &OSur ace ● )GLGe3&ma4e&OSur ace!/() ● )ustom GstGLMemory su$class Zerocopy * V'deoGool$o3 - 'OS ● &OSur ace is a8a'la$le $ut not pu$lic ● Dses )VOpenGL2SGe3ture)ache instead – )VOpenGL2SGe3ture)ache)reateGe3tureBrom&ma4e() ● )ustom GstGLMemory su$class Zerocopy * 4streamer(8aap' ● GstVideoGLGe3tureUploadMeta – 1"" uses GL:21G:texture: rom:p'3map – Wayland/2GL uses dma($u What-s Happen'n4= ● OpenGL helper li$rary * mo8e to 4st(plu4'ns($ase ● OpenGL model viewer * st'll F() GhanksJ ystreet00 in #4streamer on freenode .