Modernizing Desktop Linux Development Christian Hergert Principle Software Engineer [email protected] @hergertme TM Desktop Linux Development • Mostly in C/C++ • Build systems from last millennium that are a mess • Every desktop deployment is bespoke and developers throw their hands up and close -EW!"#$%!RME • &pps are shipped on O$ release cycle' lucky if we get security updates at all after initial release • Debugging in the wild is nearly impossible • We value independence which can cause fragmentation TM Why Modernize? • )otential contributors skip past us after it*s too difficult to setup workstation for development • We need new lifeblood to keep desktop Linu- going and new contributors have more e-pectations • !ur current level of fragmentation makes it hard to write polished software and results in more bugs • .raining contributors is comple- and time consuming • Be more e+cient with our time so each of us can do more TM Can we… • Maintain independence while reducing fragmentation? • $eparate applications from the O$ without losing long- term stability? • Empower software vendors to ship better software to more users in less time/ • Be a playground for ambitious ideas without sacrificing stability of the platform/ TM Why is it hard to contribute? • ,acking or missing documentation • Dependency incompatibility for application vs platform or even between two separate applications • Wide-scale app distribution is hard and costly' so most don’t and certainly not across every distro • Workstation setup is a really high bar for newcomers • %ragmentation leaves contributors confused about which and what platforms to support and to what degree TM Why is it hard to contribute? • $hifting and different priorities between pro2ects • $ystems comple-ity and overlapping concerns between unrelated pro2ects • ,ack of training materials • .oo many technologies to learn at once TM What makes setup difficult? • New contributors often ask what Linu- distribution to use so that they match others setup, out of frustration • Dependencies for development outpace stable O$ • Bespoke O$ deployments result in many WFM bugs • 4etting working stack traces often puts people at odds with distribution compiler 5ags TM !as the ecosystem changed? • Continuous integration • Valgrind/&$an/.$an/7B$an more ubi8uitous • 4it' Gitlab' Github have by-and-large won • Containers are everywhere • )rogramming language diversity • Device and interface diversity (laptop, desktop' mobile' Io.; • Cross-toolchains generally work now • 4,/EGL/Vulkan can now be relied on TM What changes can we expect? • Immutable base !$ with reliable O$ updates (Chrome!$ and Silverblue already here' albeit different designs; • Containers for applications becomes almost necessary • $tricter sandbo-es for user privacy and security • Even more kinds of computing devices (possibly at the e-pense of generali<ed computing devices; • New models of privileged access 9less sudo' more implied access via work5ow) • More language diversity as O$ vendors are less involved TM So what is this Builder thing? • 4reat app with a terrible name (my fault' sorry) • ,ikely the 0rst container-native IDE • Memory conscientious (written in C' read-only mmap(; search inde-es' fancy data-structures when appropriate; • Every &)I of consequence is asynchronous by design • %ast b-tree and rope-based te-t editor with overview maps and buttery smooth scrolling' even on Hi()I systems • Integrated UI designer based on Glade TM So what is this Builder thing? • Integration points can be e-tended with plugins written in C/C++' Python, or Vala= Rust support is not far away= • 7nit test integration for build systems • Debugger integration (currently 2ust gdb' but language specific debugger plugins are welcomed; • )ro0ler based on the perf-based Sysprof pro0ler • Completion, diagnostics' fi--its • .erminal access to host' build environment' or runtime TM So what is this Builder thing? • >? build systems supported and counting • Easy installation of Builder via Flatpak in two clicks • Manage developer $(#s to simplify system setup – which can now be done in minutes rather than evenings • Code-inde-ers with lightening fast fu<<y-search • Non-opinionated on developer*s language choice • Multi-monitor support • $emantic indentation, code-formatting, highlighting TM So what is this Builder thing? • Code e-ecution abstractions to allow for cross-architecture e-ecution (qemu-user-static with flatpak supported natively; • Device abstractions to quickly setup cross-compilation • Deployment A):s for plugins to support e-ecution on non- local systems TM Improving Builder $mproves the Platform TM Is contributing easier? • )reserves independenceA bring your own distribution, thanks to %latpak • Easy install from gnome-software' 5athub=org, or an app market near you • 4etting a shared toolchain is automatically handled for you and your newcomers • !ne-click to clone common 43!ME apps= One more to build, run or debug • Reducing time to 0rst patch boosts contributor funnel TM What challenges come with a container native $DE? TM Challenges - P)* • $haring pseudo-terminals across pid namespaces is tricky • %( passing P.B outside of direct fork9;/e-ec9; requires e-tra setup for controlling P.B (.:!CSC..B ioctl) • $ome shells like C$H want to coordinate between instances and that breaks with multiple pty-namespaces • 4uessing the users preferred shell requires checking passwd/getent on the host • 7sers e-pect a shell for their host system' but also for the build container' runtime container' etc TM Challenges – FD Passing • %( passing is a convenient way to share information between processes' particularly when shared memory is not an option • If you cross pid/mount/network namespaces' you can e-ec(; processes and still have access to stdin/out/err • :)C with container for high-bandwidth communication • )odman recently added FD passing support for us TM Challenges – Path Translation • Di1erent tooling may need to access files outside of container (symbols' debugger sources' etc; • )aths inside the container and outside the container may differ (or even clash with host system; • $ome additional work could be re8uired to translate based on path such as /usr vs /app in Flatpak • .ooling that runs in the IDE may use di1erent paths than tooling that runs inside build container TM Challenges – Debugging • Currently we use gdb' but more backends will be supported in the future= That re8uires some abstraction from the start= • )assing F( for ).B to be used by inferior • Control gdb using primary ).B (using gdbwire+MI2; • But where does gdb binary come from • Where do application symbols come from' and does the ( &"% data (including absolute/relative paths match/; • How about __FILE__ and other paths to sources/ TM Challenges – Profiling • Currently we use Sysprof' another pro2ect of mine based on ,inu--kernel perf • $ymbol access has similar problems to gdb' we have to resolve that based on pro2ect information • Resolving functions based on instruction-pointer+EL% still requires further resolution when symbols in e-ternal file • Containers usually have P."&CE/perf disabled' so we need coordiantion from the host (sysprofd in our case; • $omeday' we want to provide sysprofd from Builder flatpak TM Challenges – Execution • 7ser namespaces should be usable' but lots of CVE means that it*s often disabled by default on major distributions • $uid helpers can help work around this (bubblewrap) but not after userns capability is dropped • %latpak session helper allows Builder to e-ecute programs available on the host (bash' getent' 5atpak' etc; • Can chain features to also get podman-e-ec' jhbuild-run, and whatever is ne-t TM Language Servers TM Language Servers – The Good • Convenient way to share code between IDEs and Editors • &llows tooling to run in container matching the build environment which can simplify server greatly at cost of complicating build environment • When combined with Flatpak S(# e-tensions' we can get automated setup of comple- languages out-of-the-bo-= Relying on user-setup would basically guarantee nobody will use them= • .he protocol itself is reasonably good TM Language Servers – The Bad • F$!3 is laughably ine+cient when dealing with large data sets like clang completion results' especially when client- side 0ltering is required • F$!3 parsers can result in memory fragmentation unless you are very careful, resulting in lots of small strings on the heap or more fragmentation w/ unpredictable lifetimes • Builder*s internal language servers optionally use GVariant for zero-parse structures and good memory slice reuse= &lso can reference strings inline from the message • Many of the language servers out there are brittle at best TM SDKs TM SDK Management • %latpak and !$.ree based • $(# e-tensions allow for e-tra build components that are useful' but specialized' thereby keeping base $(# smaller • Containerized 9using flatpak/bubblewrap) • .oolchains' libraries' debuggers' build systems' all shared by your development team' completely eliminating comple- system setup • Incremental S(# updates using O$.ree diffs/static-deltas • 43!ME Software can keep S(#s up-to-date too TM 0")ree • ,ike git for binaries • Incremental updates by diff of 2 versions • Gstatic deltasH provide optimized, pre-compiled diff between two versions for very fast downloads= Typically generated for N->' N-D releases • Content addressed for automatic deduplication • Hard-link farm design means low-overhead and works on any P!$:I compliant file-system (but better