On submitting kernel patches Andi Kleen Intel Open Source Technology Center [email protected] Abstract 2 Why submit kernel patches to mainline? A lot of groups and individual developers work on im- There are many good reasons not to keep changes pri- proving the Linux kernel. Many innovative new features vate but to submit them to Linus Torvalds’ mainline are developed all the time. The best and smoothest way source tree. to distribute and maintain new kernel features is to in- corporate them into the standard mainline source tree. This involves a review process and some standard con- • A common approach for companies new to Linux ventions. Unfortunately actually getting innovative new is to take a snapshot of one kernel version (and its features through review can be a rough ride and some- associated userland) and try to standardize on that times they don’t make it in at all. This paper examines version forever. But freezing on old versions for a some common problems for submitting larger changes long time is not a good idea. New features and bug and some strategies to avoid problems. fixes are constantly being added to mainline and some of them will be eventually needed. Freezing on a old version cuts off from a lot of free develop- 1 Introduction ment. While some changes from mainline can be rela- Many people and groups want to contribute to the Linux tively easily backported to older source trees, many kernel. others (and that will likely be the bug fix or feature you want) can be very difficult to backport. The Sometimes it can be difficult to get larger changes into Linux kernel infrastructure is constantly evolving the mainline1 sources, especially for new contributors. and new changes often rely on newer infrastructure This paper examines some of the challenges in sub- which is not available in old kernels. And even mitting larger patches, and outlines some solutions for relatively simple backports tend to eventually be- them. come a maintenance problem when they add up in numbers because such a tree will diverge quite a lot This paper assumes that the reader already knows the from a standard kernel and invalidate previous test- basics of patch submissions. These are covered in de- ing. Then there are also security fixes that will be tail in various documents in the Linux kernel source tree missed in trees that are frozen too long. It is pos- (see [4], [3], [5]). Instead of repeating the material in sible to keep up with the security changes in main- there this paper covers some strategic higher level points line, but it’s quite expensive requiring constant ef- in patch submission. fort. And missed security fixes can be very costly to fix later. Some of the procedures suggested here are also only At some point usually, it is required to resync the applicable to complex or controversal changes. Linux code base with mainline when updating to a new kernel development isn’t (yet) a bureaucracy with fixed version. Forward porting private changes during complicated procedures that cannot be diverged from such a version update tend to be hard, especially and there is quite some flexibility in the process. if they are complicated or affect many parts of the 1“Mainline” refers to the kernel.org tree as maintained by kernel. To avoid this problem submit changes to Linus Torvalds. mainline relatively quickly after developing them, • 253 • 254 • On submitting kernel patches then they will be part of the standard code base and Linus Torvalds still be there after version updates. This will also give additional code review, additional improve- ments and bug fixing and a lot of testing for free. Maintainer 2 Maintainer 3 Andrew Morton Maintainer 1 • For changes and redesigns done in mainline, usu- Sub Maintainer 2 Developer 1 Developer 2 Sub Maintainer 1 ally only the requirements of in-tree code are con- sidered. So, even if an enhancement works first externally with just standard exported kernel sym- Figure 1: Kernel maintainer hierarchy (example) and bols, these might change or be taken away at any patch flow. Andrew Morton is the fallback maintainer time. The only sure way to avoid that or at least taking care of areas with no own maintainer or of guarantee an equivalent replacement interface to patches crossing many subsystems prevent breaking your code is to merge into main- line. problems. One important part of patch submission is to When the code is in mainline, it will be updated make sure these bugs will be handled adequately. to any interface changes directly or in consultation The mainline kernel changes at a very high rate (see [7] with the maintainer. And in mainline, the user base for detailed numbers), and it is very important for the will also test on the code and provide free quality- overall quality of the Linux kernel to keep the bug rate assurance. under control. See [2] for details on the mechanics of QA and bug reporting in mainline Linux. Because new • Writing a driver for some device and getting it into code often has more bugs than old code, the maintainers mainline means that all the distributions will auto- tend to use various procedures to make sure the bugs matically pick it up and support that device. That in the new code are minimized early on and handled makes it much easier for users to use in the end quickly. One common way to do this is extensive code because installing external drivers tends to be com- review. plicated and clumsy. All new kernel code has to go through standard code review to make sure it follows the kernel coding style For another perspective on the why to submit changes to [6] and avoids deprecated kernel interfaces and some mainline, see also Andrew Morton’s presentation [1]. common mistakes. Code review will also look for other functional problems (although that is not the main fo- cus) and include a design review. Coding style is already 3 Basics of maintenance covered extensively in [6], and I won’t cover it in detail in this paper. In the end, coding style makes only a small All code in the kernel has some maintenance overhead. part of a successful piece of kernel code anyways and is Once code is submitted the kernel maintainers commit commonly overrated. Still it is a good idea to follow themselves to keeping it functional for a long time.2 It is the coding style in the initial posting so that discussions usually expected that the person who submits new code about white space and variable naming do not distract does most of the maintenance for that code at least ini- from the actual functionality of the change. tially. Some of the procedures described here are actu- ally to demonstrate that the patch submitter is trustwor- This paper takes a higher level look on the mechanics thy enough and they they not just plan to “throw code of merging larger functionality, and assumes the basic over the fence.” Linux code requirements (like coding style, using stan- dard kernel interfaces etc.) are already covered. All code has bugs, so initially when code is submitted, it is assumed contain new defects. Exposing code to 4 Types of submissions mainline also tends to generate a lot of new testing in new unexpected circumstances, which will expose new 4.1 Easy cases 2There is a procedure to deprecate functionality too, but it is • The easiest case is a clear bug fix. The need for rarely used and only for very strong reasons. a bug fix is obvious, and the only argument might 2008 Linux Symposium, Volume One • 255 be how the bug is actually fixed. But getting clear support it and the driver (if correctly written) will in bug fixes merged is usually no problem. Some- most cases not impact any other code. times, the maintainers might want to fix a particu- lar bug differently than with the original patch, but The increasing the bug rate argument in Section 3 is for- then the bug will usually be fixed in a different way tunately not a serious problem in this special case If the by someone else. The end result is that the bug is hardware is not there, the driver will not be used, and 5 fixed. can then also not cause bugs. Luckily, this means that because most of the kernel source base are drivers, the Occasionally, there can be differences on what is effective bug rate is not raising as quickly as you would considered to be a bug and what is not. In this case, expect from the raw source size growth. Still, this is the submitter has to argue for its case in the review a problem maintainers are concerned about, especially mail thread. for core kernel code that is used on nearly6 all hard- • Then there are cleanups. Cleanups can range from ware and on drivers for hardware used very widely in very simple as in fixing coding style or making the user-base. code sparse3 clean to removing unused code to refactoring an outdated subsystem. Getting such There are well-established procedures to get new drivers cleanups in is not normally a problem, but they in, and doing so is normally not a problem. In some have to be timed right to not conflict with other cases, depending on the code quality of the driver, it can higher priority changes during the merge win- take a long time with many iterations of reviews and dows.4 The maintainers can normally coordinate fixes.