On Submitting Kernel Patches

Total Page:16

File Type:pdf, Size:1020Kb

On Submitting Kernel Patches On submitting kernel patches Andi Kleen Intel Open Source Technology Center [email protected] Abstract 2 Why submit kernel patches to mainline? A lot of groups and individual developers work on im- There are many good reasons not to keep changes pri- proving the Linux kernel. Many innovative new features vate but to submit them to Linus Torvalds’ mainline are developed all the time. The best and smoothest way source tree. to distribute and maintain new kernel features is to in- corporate them into the standard mainline source tree. This involves a review process and some standard con- • A common approach for companies new to Linux ventions. Unfortunately actually getting innovative new is to take a snapshot of one kernel version (and its features through review can be a rough ride and some- associated userland) and try to standardize on that times they don’t make it in at all. This paper examines version forever. But freezing on old versions for a some common problems for submitting larger changes long time is not a good idea. New features and bug and some strategies to avoid problems. fixes are constantly being added to mainline and some of them will be eventually needed. Freezing on a old version cuts off from a lot of free develop- 1 Introduction ment. While some changes from mainline can be rela- Many people and groups want to contribute to the Linux tively easily backported to older source trees, many kernel. others (and that will likely be the bug fix or feature you want) can be very difficult to backport. The Sometimes it can be difficult to get larger changes into Linux kernel infrastructure is constantly evolving the mainline1 sources, especially for new contributors. and new changes often rely on newer infrastructure This paper examines some of the challenges in sub- which is not available in old kernels. And even mitting larger patches, and outlines some solutions for relatively simple backports tend to eventually be- them. come a maintenance problem when they add up in numbers because such a tree will diverge quite a lot This paper assumes that the reader already knows the from a standard kernel and invalidate previous test- basics of patch submissions. These are covered in de- ing. Then there are also security fixes that will be tail in various documents in the Linux kernel source tree missed in trees that are frozen too long. It is pos- (see [4], [3], [5]). Instead of repeating the material in sible to keep up with the security changes in main- there this paper covers some strategic higher level points line, but it’s quite expensive requiring constant ef- in patch submission. fort. And missed security fixes can be very costly to fix later. Some of the procedures suggested here are also only At some point usually, it is required to resync the applicable to complex or controversal changes. Linux code base with mainline when updating to a new kernel development isn’t (yet) a bureaucracy with fixed version. Forward porting private changes during complicated procedures that cannot be diverged from such a version update tend to be hard, especially and there is quite some flexibility in the process. if they are complicated or affect many parts of the 1“Mainline” refers to the kernel.org tree as maintained by kernel. To avoid this problem submit changes to Linus Torvalds. mainline relatively quickly after developing them, • 253 • 254 • On submitting kernel patches then they will be part of the standard code base and Linus Torvalds still be there after version updates. This will also give additional code review, additional improve- ments and bug fixing and a lot of testing for free. Maintainer 2 Maintainer 3 Andrew Morton Maintainer 1 • For changes and redesigns done in mainline, usu- Sub Maintainer 2 Developer 1 Developer 2 Sub Maintainer 1 ally only the requirements of in-tree code are con- sidered. So, even if an enhancement works first externally with just standard exported kernel sym- Figure 1: Kernel maintainer hierarchy (example) and bols, these might change or be taken away at any patch flow. Andrew Morton is the fallback maintainer time. The only sure way to avoid that or at least taking care of areas with no own maintainer or of guarantee an equivalent replacement interface to patches crossing many subsystems prevent breaking your code is to merge into main- line. problems. One important part of patch submission is to When the code is in mainline, it will be updated make sure these bugs will be handled adequately. to any interface changes directly or in consultation The mainline kernel changes at a very high rate (see [7] with the maintainer. And in mainline, the user base for detailed numbers), and it is very important for the will also test on the code and provide free quality- overall quality of the Linux kernel to keep the bug rate assurance. under control. See [2] for details on the mechanics of QA and bug reporting in mainline Linux. Because new • Writing a driver for some device and getting it into code often has more bugs than old code, the maintainers mainline means that all the distributions will auto- tend to use various procedures to make sure the bugs matically pick it up and support that device. That in the new code are minimized early on and handled makes it much easier for users to use in the end quickly. One common way to do this is extensive code because installing external drivers tends to be com- review. plicated and clumsy. All new kernel code has to go through standard code review to make sure it follows the kernel coding style For another perspective on the why to submit changes to [6] and avoids deprecated kernel interfaces and some mainline, see also Andrew Morton’s presentation [1]. common mistakes. Code review will also look for other functional problems (although that is not the main fo- cus) and include a design review. Coding style is already 3 Basics of maintenance covered extensively in [6], and I won’t cover it in detail in this paper. In the end, coding style makes only a small All code in the kernel has some maintenance overhead. part of a successful piece of kernel code anyways and is Once code is submitted the kernel maintainers commit commonly overrated. Still it is a good idea to follow themselves to keeping it functional for a long time.2 It is the coding style in the initial posting so that discussions usually expected that the person who submits new code about white space and variable naming do not distract does most of the maintenance for that code at least ini- from the actual functionality of the change. tially. Some of the procedures described here are actu- ally to demonstrate that the patch submitter is trustwor- This paper takes a higher level look on the mechanics thy enough and they they not just plan to “throw code of merging larger functionality, and assumes the basic over the fence.” Linux code requirements (like coding style, using stan- dard kernel interfaces etc.) are already covered. All code has bugs, so initially when code is submitted, it is assumed contain new defects. Exposing code to 4 Types of submissions mainline also tends to generate a lot of new testing in new unexpected circumstances, which will expose new 4.1 Easy cases 2There is a procedure to deprecate functionality too, but it is • The easiest case is a clear bug fix. The need for rarely used and only for very strong reasons. a bug fix is obvious, and the only argument might 2008 Linux Symposium, Volume One • 255 be how the bug is actually fixed. But getting clear support it and the driver (if correctly written) will in bug fixes merged is usually no problem. Some- most cases not impact any other code. times, the maintainers might want to fix a particu- lar bug differently than with the original patch, but The increasing the bug rate argument in Section 3 is for- then the bug will usually be fixed in a different way tunately not a serious problem in this special case If the by someone else. The end result is that the bug is hardware is not there, the driver will not be used, and 5 fixed. can then also not cause bugs. Luckily, this means that because most of the kernel source base are drivers, the Occasionally, there can be differences on what is effective bug rate is not raising as quickly as you would considered to be a bug and what is not. In this case, expect from the raw source size growth. Still, this is the submitter has to argue for its case in the review a problem maintainers are concerned about, especially mail thread. for core kernel code that is used on nearly6 all hard- • Then there are cleanups. Cleanups can range from ware and on drivers for hardware used very widely in very simple as in fixing coding style or making the user-base. code sparse3 clean to removing unused code to refactoring an outdated subsystem. Getting such There are well-established procedures to get new drivers cleanups in is not normally a problem, but they in, and doing so is normally not a problem. In some have to be timed right to not conflict with other cases, depending on the code quality of the driver, it can higher priority changes during the merge win- take a long time with many iterations of reviews and dows.4 The maintainers can normally coordinate fixes.
Recommended publications
  • Debian Developer's Reference Version 12.0, Released on 2021-09-01
    Debian Developer’s Reference Release 12.0 Developer’s Reference Team 2021-09-01 CONTENTS 1 Scope of This Document 3 2 Applying to Become a Member5 2.1 Getting started..............................................5 2.2 Debian mentors and sponsors......................................6 2.3 Registering as a Debian member.....................................6 3 Debian Developer's Duties 9 3.1 Package Maintainer's Duties.......................................9 3.1.1 Work towards the next stable release............................9 3.1.2 Maintain packages in stable .................................9 3.1.3 Manage release-critical bugs.................................. 10 3.1.4 Coordination with upstream developers............................ 10 3.2 Administrative Duties.......................................... 10 3.2.1 Maintaining your Debian information............................. 11 3.2.2 Maintaining your public key.................................. 11 3.2.3 Voting.............................................. 11 3.2.4 Going on vacation gracefully.................................. 12 3.2.5 Retiring............................................. 12 3.2.6 Returning after retirement................................... 13 4 Resources for Debian Members 15 4.1 Mailing lists............................................... 15 4.1.1 Basic rules for use....................................... 15 4.1.2 Core development mailing lists................................. 15 4.1.3 Special lists........................................... 16 4.1.4 Requesting new
    [Show full text]
  • Beginning Portable Shell Scripting from Novice to Professional
    Beginning Portable Shell Scripting From Novice to Professional Peter Seebach 10436fmfinal 1 10/23/08 10:40:24 PM Beginning Portable Shell Scripting: From Novice to Professional Copyright © 2008 by Peter Seebach All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN-13 (pbk): 978-1-4302-1043-6 ISBN-10 (pbk): 1-4302-1043-5 ISBN-13 (electronic): 978-1-4302-1044-3 ISBN-10 (electronic): 1-4302-1044-3 Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1 Trademarked names may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, we use the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. Lead Editor: Frank Pohlmann Technical Reviewer: Gary V. Vaughan Editorial Board: Clay Andres, Steve Anglin, Ewan Buckingham, Tony Campbell, Gary Cornell, Jonathan Gennick, Michelle Lowman, Matthew Moodie, Jeffrey Pepper, Frank Pohlmann, Ben Renow-Clarke, Dominic Shakeshaft, Matt Wade, Tom Welsh Project Manager: Richard Dal Porto Copy Editor: Kim Benbow Associate Production Director: Kari Brooks-Copony Production Editor: Katie Stence Compositor: Linda Weidemann, Wolf Creek Press Proofreader: Dan Shaw Indexer: Broccoli Information Management Cover Designer: Kurt Krames Manufacturing Director: Tom Debolski Distributed to the book trade worldwide by Springer-Verlag New York, Inc., 233 Spring Street, 6th Floor, New York, NY 10013.
    [Show full text]
  • Leitfaden Für Debian-Betreuer
    Leitfaden für Debian­Betreuer Osamu Aoki, Helge Kreutzmann, and Mechtilde Stehmann August 27, 2021 Leitfaden für Debian­Betreuer by Osamu Aoki, Helge Kreutzmann, and Mechtilde Stehmann Copyright © 2014­2020 Osamu Aoki Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the ”Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED ”AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IM­ PLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. Diese Anleitung wurde mit den nachfolgenden Dokumenten als Referenz erstellt: • »Making a Debian Package (AKA the Debmake Manual)«, Copyright © 1997 Jaldhar Vyas. • »The New­Maintainer’s Debian Packaging Howto«, Copyright © 1997 Will Lowe. • »Debian­Leitfaden für Neue Paketbetreuer«, Copyright © 1998­2002 Josip Rodin, 2005­2017 Osamu Aoki, 2010 Craig Small und 2010 Raphaël Hertzog. Die neuste Version dieser Anleitung sollte • im Paket debmake­doc und • auf der Debian­Dokumentations­Website verfügbar sein.
    [Show full text]
  • Compiling C Programs
    CSCI 2132: Software Development Lab 6: Exploring bash and C Compilation Synopsis In this lab, you will: • Customize the behaviour your bash shell • Write and compile some simple C programs • Practice the edit-compile-fix cycle using emacs and the shell • Learn to patch programs using diff and patch Contents Overview......................................................................................................2 Step 1: Login and lab setup.................................................................................3 Step 2: The name of your shell.............................................................................3 Step 3: The .bashrc file.....................................................................................3 Step 4: Customize rm ........................................................................................4 Optional Step: Install bashrc.new as your new .bashrc file............................................7 Step 5: Edit .profile .......................................................................................8 Optional Step: Install profile.new as your new .profile file.......................................... 10 Step 6: Write a simple C program.......................................................................... 11 Step 7: Explore exit codes................................................................................... 12 Step 8: diff .................................................................................................. 14 Step 9: patch................................................................................................
    [Show full text]
  • Distributed Revision Control with Mercurial
    Distributed revision control with Mercurial Bryan O’Sullivan Copyright c 2006, 2007 Bryan O’Sullivan. This material may be distributed only subject to the terms and conditions set forth in version 1.0 of the Open Publication License. Please refer to Appendix D for the license text. This book was prepared from rev 028543f67bea, dated 2008-08-20 15:27 -0700, using rev a58a611c320f of Mercurial. Contents Contents i Preface 2 0.1 This book is a work in progress ...................................... 2 0.2 About the examples in this book ..................................... 2 0.3 Colophon—this book is Free ....................................... 2 1 Introduction 3 1.1 About revision control .......................................... 3 1.1.1 Why use revision control? .................................... 3 1.1.2 The many names of revision control ............................... 4 1.2 A short history of revision control .................................... 4 1.3 Trends in revision control ......................................... 5 1.4 A few of the advantages of distributed revision control ......................... 5 1.4.1 Advantages for open source projects ............................... 6 1.4.2 Advantages for commercial projects ............................... 6 1.5 Why choose Mercurial? .......................................... 7 1.6 Mercurial compared with other tools ................................... 7 1.6.1 Subversion ............................................ 7 1.6.2 Git ................................................ 8 1.6.3
    [Show full text]
  • Debianization of Predictprotein
    debian-unstable package-building git-repository lintian and quilt sources Programming challenge 10: Debianization of PredictProtein Jens Preußner Supervisor: Laszlo Kajan The Bioinformatics Lab 10th of July, 2012 debian-unstable package-building git-repository lintian and quilt sources Table of contents 1 debian-unstable Installation 2 package-building Structure of directory Building 3 git-repository Local git repository Remote git repository Initial push 4 lintian and quilt Fixing lintian warnings Working with quilt 5 References Now perform as root: $ chroot /srv/unstable/ $ adduser --home /home/<user> --uid <uid> <user> $ su - <user> debian-unstable package-building git-repository lintian and quilt sources Installing debian-unstable in L1 Required packages: debootstrap, dpkg-dev - easy Bind your home and proc, see Network filesystems debian-unstable package-building git-repository lintian and quilt sources Installing debian-unstable in L1 Required packages: debootstrap, dpkg-dev - easy Bind your home and proc, see Network filesystems Now perform as root: $ chroot /srv/unstable/ $ adduser --home /home/<user> --uid <uid> <user> $ su - <user> debian-unstable package-building git-repository lintian and quilt sources Installing debian-unstable in L1 To know for sure, that you are in wheezy/sid In new environment: $ cat /etc/debian version wheezy/sid instead of 6.0.4 debian-unstable package-building git-repository lintian and quilt sources My special case I had a perl module for uploading The module was not contained in CPAN Naming conventions:
    [Show full text]
  • C/C++ Compile Guide
    WizFi630S Guide C/C++ Compile Guide (Version 1.0.0) © 2019 WIZnet Co., Ltd. All Rights Reserved. For more information, please visit our website at http://www.wiznet.io/ © Copyright 2019 WIZnet Co., Ltd. All rights reserved. 1 WizFi630S Guide Document Revision History Date Revision Changes 2019-11-25 1.0 Release © Copyright 2019 WIZnet Co., Ltd. All rights reserved. 2 WizFi630S Guide Contents 1. Overview ................................................................................................................. 4 2. Download ................................................................................................................ 4 2.1 Prerequisites .................................................................................................. 4 2.2 Packages for Building Environment .......................................................... 4 2.3 OpenWRT Firmware Repository................................................................. 6 2.4 Menuconfig .................................................................................................... 7 3. Write C Code........................................................................................................... 7 3.1 Helloworld ...................................................................................................... 7 3.2 Make the Environment Script .................................................................... 8 4. Cross Compile ......................................................................................................... 8 4.1
    [Show full text]
  • Buildroot Training Buildroot Training
    Buildroot Training Buildroot Training © Copyright 2004-2021, Bootlin. Creative Commons BY-SA 3.0 license. Latest update: October 6, 2021. Document updates and sources: https://bootlin.com/doc/training/buildroot Corrections, suggestions, contributions and translations are welcome! embedded Linux and kernel engineering Send them to [email protected] - Kernel, drivers and embedded Linux - Development, consulting, training and support - https://bootlin.com 1/353 Rights to copy © Copyright 2004-2021, Bootlin License: Creative Commons Attribution - Share Alike 3.0 https://creativecommons.org/licenses/by-sa/3.0/legalcode You are free: I to copy, distribute, display, and perform the work I to make derivative works I to make commercial use of the work Under the following conditions: I Attribution. You must give the original author credit. I Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under a license identical to this one. I For any reuse or distribution, you must make clear to others the license terms of this work. I Any of these conditions can be waived if you get permission from the copyright holder. Your fair use and other rights are in no way affected by the above. Document sources: https://github.com/bootlin/training-materials/ - Kernel, drivers and embedded Linux - Development, consulting, training and support - https://bootlin.com 2/353 Hyperlinks in the document There are many hyperlinks in the document I Regular hyperlinks: https://kernel.org/ I Kernel documentation links: dev-tools/kasan I Links to kernel source files and directories: drivers/input/ include/linux/fb.h I Links to the declarations, definitions and instances of kernel symbols (functions, types, data, structures): platform_get_irq() GFP_KERNEL struct file_operations - Kernel, drivers and embedded Linux - Development, consulting, training and support - https://bootlin.com 3/353 Company at a glance I Engineering company created in 2004, named ”Free Electrons” until Feb.
    [Show full text]
  • Debian Packaging Tutorial
    Debian Packaging Tutorial Lucas Nussbaum [email protected] version 0.27 – 2021-01-08 Debian Packaging Tutorial 1 / 89 About this tutorial I Goal: tell you what you really need to know about Debian packaging I Modify existing packages I Create your own packages I Interact with the Debian community I Become a Debian power-user I Covers the most important points, but is not complete I You will need to read more documentation I Most of the content also applies to Debian derivative distributions I That includes Ubuntu Debian Packaging Tutorial 2 / 89 Outline 1 Introduction 2 Creating source packages 3 Building and testing packages 4 Practical session 1: modifying the grep package 5 Advanced packaging topics 6 Maintaining packages in Debian 7 Conclusions 8 Additional practical sessions 9 Answers to practical sessions Debian Packaging Tutorial 3 / 89 Outline 1 Introduction 2 Creating source packages 3 Building and testing packages 4 Practical session 1: modifying the grep package 5 Advanced packaging topics 6 Maintaining packages in Debian 7 Conclusions 8 Additional practical sessions 9 Answers to practical sessions Debian Packaging Tutorial 4 / 89 Debian I GNU/Linux distribution I 1st major distro developed “openly in the spirit of GNU” I Non-commercial, built collaboratively by over 1,000 volunteers I 3 main features: I Quality – culture of technical excellence We release when it’s ready I Freedom – devs and users bound by the Social Contract Promoting the culture of Free Software since 1993 I Independence – no (single)
    [Show full text]
  • Debian Packaging with Git WorkOws and Tools
    Debian Packaging with Git Workows and Tools David Bremner 26 July 2011 Introduction Importing Upstream Source Importing Repacking ./debian Handling patches Simplest approaches Use git for patches Rebased patch branches making a source package (in)Conclusions Advanced Topics/Brainstorming Goals for the talk I take away some of the mystery I know what is possible I have some discussion about best practices Talk pre-reqs and goals Talk-Depends: basic-packaging (>=1.0), git-curious I You know what a source package is I You know what a git branch and tag is (or want to) Talk pre-reqs and goals Talk-Depends: basic-packaging (>=1.0), git-curious I You know what a source package is I You know what a git branch and tag is (or want to) Goals for the talk I take away some of the mystery I know what is possible I have some discussion about best practices The Packaging Cycle 1. start with upstream source 2. possibly remove some non-free or just useless things 3. create or update ./debian 4. possibly patch upstream source 5. make a source package Introduction Importing Upstream Source Importing Repacking ./debian Handling patches Simplest approaches Use git for patches Rebased patch branches making a source package (in)Conclusions Advanced Topics/Brainstorming Importing Upstream Source I at least one of import git import svn I get upstream $VCS history I unpack tarball I commit changes (including deletions) manual import I maybe save a copy using pristine-tar I maybe merge upstream in debian packaging branch I maybe tag pristine-tar I Allows checksum-identical regeneration of tarballs, small overhead pristine-tar commit foo.tar.gz [upstream-ref] pristine-tar checkout where-ever/foo.tar.gz I convenience feature, but for some rare tarballs, it doesn't work (well) Tracking upstream $VCS and tarballs I Get $VCS history on a branch I Import tarball on top: dis are contained in the import commit.
    [Show full text]
  • The Openwrt Embedded Development Framework
    The OpenWrt embedded development framework Florian Fainelli - fl[email protected] January 22, 2008 Abstract One of the biggest challenges to getting started with embedded devices is that you cannot just install a copy of Linux and expect to be able to compile a firmware. Even if you did remember to install a compiler and every development tool offered, you still would not have the basic set of tools needed to produce a firmware image. The embedded device represents an entirely new hardware platform, so in a process called cross compiling you need to produce a new compiler capable of generating code for your embedded platform, and then use it to compile a basic Linux distribution to run on your device. The process of creating a cross compiler can be tricky and in many cases when you are dealing with embedded devices you will be pro- vided with a binary copy of a compiler and basic libraries rather than instructions for creating your own. Likewise, it is also common to be provided with a patched copy of the Linux kernel from the board or chip vendor, but this is also dated and it can be difficult to spot ex- actly what has been modified to make the kernel run on the embedded platform. OpenWrt has different approach to building a firmware; down- loading, patching and compiling everything from scratch, including the cross compiler. OpenWrt does not contain any executables and only very few sources, it is an automated system for downloading the sources, patching them to work with the given platform and compil- ing them correctly for that platform.
    [Show full text]
  • Latest Features of Dpkg-Dev
    Latest features of dpkg-dev By Raphaël Hertzog <[email protected]> Mini-Debconf Paris 30-31 Oct 2010 Université Paris 7 What I'm going to cover ● New features of dpkg-dev in the last 2-3 years ● dpkg-dev: tools used to build debian packages ● In particular: ● Support of symbols files by dpkg-shlibdeps, dpkg-gensymbols ● Support of new source formats by dpkg-source ● Supplementary options for dpkg-source ● Cross distribution collaboration with dpkg-vendor ● Custom compilation flags with dpkg-buildflags ● Miscellaneous improvements to other tools Support of symbols files ● Before: shlibs files: library used → dependency with minimal version ● Now: symbols files: ● Library used → dependency template ● Symbol used → minimal version for dependency ● dpkg-shlibdeps can thus generate more accurate dependencies with relaxed requirements in many cases ● dpkg-gensymbols needed to generate the symbols files Example of shlibs/symbols files Shlibs file: libc 6 libc6 (>= 2.11) Symbols file: libc.so.6 libc6 #MINVER# | libc6 (>> 2.11), libc6 (<< 2.12) abs@GLIBC_2.0 2.0 accept4@GLIBC_2.10 2.10 duplocale@GLIBC_2.3 2.3 errno@GLIBC_PRIVATE 0 1 [...] More symbols goodness ● Ensuring generated dependency is as strong as the corresponding build-dependency ● Example: libgtk-x11-2.0.so.0 libgtk2.0-0 #MINVER# * Build-Depends-Package: libgtk2.0-dev gtk_about_dialog_get_artists@Base 2.8.0 [...] ● If a package build-depends on libgtk2.0-dev (>> 2.12), the dependency on libgtk2.0-0 will be at least as strict (i.e. >> 2.12). Adding symbols files in a source package
    [Show full text]