Bitrig ports: BSD ports, packages, and Uncommon Operating Systems John C. Vernaleo, Ph.D. [email protected] Bitrig Abstract src and ports for any system and count distinct contrib- utors). The BSD operating systems strive to provide a com- There are a few challenges with third party software. plete, usable system in one coherent place. In theory For starters, we largely live in a Linux world (as free this works well, but in practice, most people depend Unixes go) or even worse an OSX or Windows world. on a variety of third party software for their computing This means that most software was probably not built needs. This is where ports and packages come in. We for the BSD of our choice. If everyone followed stan- will discuss some issues that come up in dealing with dards, this wouldn’t be a problem. That’s what POSIX packaging third party software on an uncommon oper- is for you might say. Unfortunately, that does not get ating system (Bitrig) along with how this can help us to us very far. POSIX just gets us a mostly compatible set improve software portability for all the BSD operating of system calls but does not get us everything. If it did, systems (and other UNIX-like systems). we probably would not need the configure step when building software. 1 Introduction The problem gets even worse when you are not us- ing one of the largest BSDs. Even more exotic oper- The various BSD operating systems each strive to pro- ating systems (Plan 9, GNU/HURD, etc.) likely have vide a complete, usable system in one coherent place problems beyond even that. This paper will discuss with varying focuses of their own. While they mostly some of the problems from the perspective of one of succeed at this goal, in practice it is rare for most peo- the less common BSDs (Bitrig) and some of the ways ple to use a UNIX-like system without a variety of we work to deal with it. Hopefully these solutions can third party software. Third party software is arguable help other operating systems along with pushing us all the reason UNIX rose to prominence (and for better or towards more compatible and cross-platform software. worse that is a large part of why GNU choose to mostly follow UNIX). One could download, extract, patch, configure, com- 2 Bitrig pile, and install each program one needs separately. For small programs this even works. But for larger pro- The work for this paper was primarily done on the Bi- grams (web browsers, editors, etc.) or a large number trig operating system so a short introduction to Bitrig of programs, this gets very cumbersome, very quickly. is useful. Bitrig is a free, fast, and secure Unix-like It is also almost impossible to stay up to date with soft- Open Source operating system [6]. It started as a fork ware when dealing with each piece individually. of OpenBSD but with a focus on modern platforms The solution is to have some system of packages and a modernized tool chain, development tools, and (or ports in OpenBSD speak) that is distributed by the development practices. The supported platforms are operating system vendor to make third party software amd64 and arm (with experimental work on armd64 easy to install and manage. While not strictly part (aarch64)). of the BSD system (OpenBSD for example makes it In terms of modernizing tools and practices, Bitrig very clear that ports are not audited or held to the same uses clang as the base compiler and git for version con- standards as things in base [4]), many users will spend trol. Github.com is used for hosting and dealing with most of their time using the third party software. Third patches (pull requests really) and bugs reports from party software also ends up requiring more developers users. Through the use of modern software develop- than the base system (compare commit logs between ment methods we hope to make Bitrig easier for new- comers to contribute to. the ports with a simple find and replace and is really There are a number of other features to Bitrig but just specify to the ports infrastructure. Other changed they are already described elsewhere [10] and are not libraries are handled similarly. specifically relevant here. The most common problem is simply that Bitrig is not named OpenBSD. Most nontrivial software (at 3 Ports and Packages least the traditional compiled ones) use some com- bination of configure, automake, and other tools. 3.1 Bitrig Ports These were meant to deal with multiple platforms (largely stemming from the days when there were more Here we will briefly describe the Bitrig ports sys- UNIXes than today). The common mantra is to test for tem [1]. Bitrig ports are based on OpenBSD ports. We capabilities not names. Unfortunately, no one seems to have made some minor changes, but for the most part do that. The vast majority of software seems to test the systems remain very similar. Ports is a set of Make- for OS name (and sometimes version!) during the con- files to download, configure, compile, and package figure step. This is a major problem for any uncom- third party software. The package are then installed, mon system. In the case of Bitrig one can usually just removed, or otherwise dealt with using the pkg add modify the config scripts by adding a test for Bitrig in tools which are a set of Perl scripts also inherited from the same place as the OpenBSD test. It is unfortunate OpenBSD. Ports also includes any patches or other that every new OS needs to do this, but without people files needed to build or install the software. OpenBSD getting significantly more skilled with configure, we assumes ’normal’ users will only interact with pack- are stuck with it. To avoid this in their own software, ages and leave ports to developers [4]. Bitrig makes no developers should either test for capabilities with con- such assumption. figure or at least have sensible defaults for unknown From the point of view of most third party software, operating systems. Bitrig should seem pretty much the same as OpenBSD so one might expect to be able to just build the ports 3.2 Other Ports/Package systems tree and have everything work. Unfortunately, this goes wrong in a few ways. It has been suggested by several parties that Bitrig The first way is because Bitrig is not (quite) should switch to a different ports and package system. OpenBSD. We’ve made some changes that matter to FreeBSD has pkgng [5] and NetBSD has pkgsrc [3]. some ports. For starters, instead of using the GNU Both of these have a number of positive features and compiler as the system compile, we use clang. Mod- pkgsrc does work on Bitrig at this time, but they don’t ern versions of clang are very good and compile al- solver the fundamental problem. Ultimately, no pack- most anything the GNU compilers can handle (for c ages system, no matter how good it is will make soft- and c++, FORTRAN and other languages are a differ- ware build on something if the code does not support ent story). Unfortunately, almost is not good enough. it without patching the code. There is still a ton of gcc specific code in the wild. In That being said, there are benefits to using a pack- some cases this could be a failure on clang’s part but in aging system shared by other OSes. Shared packaging every case we’ve actually hit, it is non-standard C that systems allow systems to share effort. For smaller sys- gcc accepts. tems, this can be a huge benefit. In the case of Bitrig The best solution here is to fix the code to work with ports it would eliminate the time consuming job of im- clang (assuming clang is correct). If that is not pos- porting changes from the OpenBSD ports repository. sible (some software such as emacs pretty thoroughly This is potentially a big win but there is a price. No assume GNU compilers) the program can still be built system will hold back the larger (and frequently hard- with gcc. The ports system can normally handle this est to update) packages such as Firefox and Chromium with a simple variable (more or less setting CC tem- for one of the less common operating systems they porarily) but in some cases, software will hardcode gcc support. It wouldn’t make sense for their other users. in the Makefiles (cmus is an example I recently came This means that packages will end up broken and since across). This requires patching the Makefile and is the packaging system is shared, there is no way for a something people should not do if they want things to operating system such as Bitrig to stay on an older ver- be portable at all. FORTRAN developers have known sion. This leaves developers to either live with poten- that hardcoding a specific f77 compilers is a problem tial broken packages or with a rush to test or fix things. for portability for a long time, but in the C/C++ world, This is not a failure of those packaging systems, it is gcc has been dominant for long enough that people do just the reality that there is a price to pay for shared hard code it. package systems despite the benefits of them which a Bitrig uses a different C++ library (libc++ and large part of why Bitrig has not completely adopted libc++abi instead of libstdc++).