How to Collaborate on Linux Kernel Development

Mauro Carvalho Chehab

Linux Kernel Expert Samsung Research Brazil Samsung Open Source Group

Oct 17, 2014

Not to be used for commercial purpose without getting permission Open Source Group – Silicon Valley All information, opinions and ideas herein are exclusively the author's own opinion © 2014 SAMSUNG Electronics Co. Samsung Research Brazil Contributing to the Linux Kernel

● Authors have rights on the contents they produce: – Moral rights: the rights to claim its authorship on a work and to be against modified copies that would cause moral harm to the author; – Property rights: the ones that could generate profits (sell, distribute, publish, rent, copy, use). – Applies to intellectual work, like books, paintings, music, software

● Public domain – Something that was not copyrighted – No restrictions are imposed to its usage: anyone can copy, modify, use, etc

● A Copyright traditional model that doesn't help collaboration, as the holder of a copyright is typically the only one that benefits for its work.

● On the other hand, putting some work in public domain makes collaboration possible, but it would mean to loose all rights on such work.

● So, the Open Source movement came with a new concept, called Copyleft, where the author keep holding the rights on his work, while creating a hole community of developers helping to improve and to create new ways of using his work.

The real gain with Open Source is to build an eco system that will keep the products well maintained, well supported and allowing new ways of usage.

It is a trully win to win model.

● It is a variant of the Copyright: the rights belong to the authors

● The author gives the right (via a software license) that allows copy, distribute, modify and usage

● Modified copies should also be licensed under the same copyleft license

● A violation to a copyleft license is a copyright violation

Licensing

● The GNU General Public License is intended to guarantee your freedom to share and change free software – to make sure the software is free for all its users – Free software: freedom (not price)

● Freedom to distribute copies of free software – and charge for this service if you wish ● Receive source code or can get it if you want it ● Can change the software or use pieces of it in new free programs ● Rights remain the same after copied/modified: – If copies are distributed (whether gratis or for a fee): the recipients has all the rights that the distributor have; – The same license applies to distributed copies; 7

● Rights protected by GPLv2: – Software is copyrighted; – It gives legal permission to copy, distribute and/or modify the software. – Moral rights protection:

● If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations. – Patents: there are clauses stating that any patent must be licensed for everyone's free use or not licensed at all.

● Kernel is licensed under GPL version 2 only

● There's an extra clause at Kernel that explicitly allows running userspace using other licenses:

NOTE! This copyright does *not* cover user programs that use kernel services by normal system calls - this is merely considered normal use of the kernel, and does *not* fall under the heading of "derived work".

Also note that the GPL below is copyrighted by the Free Software Foundation, but the instance of code that it refers to (the Linux kernel) is copyrighted by me and others who actually wrote it.

Interacting with an upstream development Community

● Most of the Open Source discussions happen via email; ● RTFM: Read The F#$& Fine Manual – Before sending something to a public ML, you should check first if your questions/patches/etc weren't already sent to it; ● Be polite;

● Be prepared for critics: – That's part of the game; ● Don't be offended by emails;

● Don't feed the Mailing List trolls.

● There are several important rules when sending emails to a public ML:

– Always send emails in plain text:

● There are security issues on using html – most lists block it;

– Never do top-posting: answers should be below questions

● Just the opposite way that Outlook does

– Use an email client that won't mangle whitespaces

● Whitespace insert/removal would break patches

● Best option nowadays: use git send-email

● Upstream development is dictated by the community, not by someone or some company – If someone (and/or their company) wants a feature, submit a patch

● The community reviews the patch; ● If community is happy, it gets merged, otherwise, it is rejected ● The development communities are huge, informal and open – All discussions are via mailing lists

● On Kernel, most lists are hosted at vger.kernel.org; – IRC and physical meetings (Summits and/or BoF – Birds of Feather) help to get agreements on more polemic items

● BoFs are informal and often formed in an ad hoc manner. ● IRC channels are generally hosted at freenode.net (or oftc.net) – Examples at Freenode: #tizen, #v4l, #linuxtv, #gstreamer

Open Source Group – Silicon Valley © 2014 SAMSUNG Electronics Co. Samsung Research Brazil The Development Model (2) ● Contributions are open to everyone, and patches are publicly reviewed – Reviewing a patch internally inside a company is generally considered a bad habit, as the community won't give a single penny for such reviews made under black walls;

● On most packages, the version number indicates if the version is a review or a major change with API breakages, as specified at http://semver.org : – Given a version number MAJOR.MINOR.PATCH, increment the:

● MAJOR version when you make incompatible API changes, ● MINOR version when you add functionality in a backwards- compatible manner, ● PATCH version when you make backwards-compatible bug fixes. – NOTE: Not all projects follow it, including the Linux Kernel!

Interacting with the Kernel community

● The Linux Kernel version number is, currently: – 3.x – For released Kernels; – 3.x.y – For stable fixes for a released Kernel; – 3.x-rcy – For Release Candidate Kernels (one per week);

● A new Linux Kernel version is released on every ~2 months

– After a new release, 2 weeks are used to merge the features for the next one. This period of time is called: “merge window”;

– A variable period (typically 6-8 weeks) is used to review the code. After each week, a new release candidate is released

● So, there are 3.16-rc1 to 3.16-rc7, plus 3.16 (the final version).

● Subsystem maintainers are responsible to collect the accepted patches, do a final review and submit them to Linus Torvards (the creator of Linux) and main maintainer.

Maintainers are like editors in the Publishing industry.

- David Miller Linux Kernel Networking Maintainer

TIP: please listen Greg K. H. presentation: I don't want your code: http://www.cnx-software.com/2013/03/07/linaro-connect-hk-greg-kroah-hartman-keynote-i-dont-want-your-code/

● The Linux kernel community is very technically brutal when it comes to code review. – a particular format that need the patches to fit, and newbies often get it wrong; – You should look at the patch format of several different commits in the git history; – Very high standard for coding style and the patch submission process.

● Subsystem maintainers can also be particularly picky when patches come in from a newbie – Since they've never seen code from you before, they don't know what the newbie's skill level; – Often, new contributors are held to higher standards than the long- term ones.

(cont.) http://kernelnewbies.org/KernelDevViewpoint

● However, if you look at it from a maintainer's point of view, it makes sense. – The maintainer knows you're a new person, and has no idea if you will stick around and fix bugs in the code you contribute; – The maintainer is basically stuck with your code long-term;

● He need to both understand what your code is doing, and ensure it is correct; – When code comes in from a long-term contributor:

● there is much more discussion about whether the idea is correct; ● much less nit-picky code style review; ● the maintainer knows the contributor will be around to fix latter discovered issues.

http://kernelnewbies.org/KernelDevViewpoint

Source: https://github.com/gregkh/presentation-linux-maintainer/blob/master/maintainer.pdf?raw=true

Submitting a Patch

Subject: uvc: more buffers Patch summary: what it does Date: Fri, 09 Aug 2013 12:11:36 -0000 From: Oliver Neukum main author To: [email protected] , [email protected] Driver maintainer Mailing List This is necessary to let the new generation of cameras of the project/subsystem from LiteOn used in Haswell ULT notebook operate. Otherwise the images will be truncated. Patch description: Signed-off-by: Oliver Neukum why it is needed and Nacked-by: Laurent Pinchart how was done

diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h index 9e35982..9f1930b 100644 --- a/drivers/media/usb/uvc/uvcvideo.h Patch custody chain: +++ b/drivers/media/usb/uvc/uvcvideo.h SOB confirming that the patch @@ -114,9 +114,9 @@ /* Number of isochronous URBs. */ is GPL + acks, reviews, #define UVC_URBS 5 Tests, etc. (Kernel-specific) /* Maximum number of packets per URB. */ -#define UVC_MAX_PACKETS 32 +#define UVC_MAX_PACKETS 128 /* Maximum number of video buffers. */ -#define UVC_MAX_VIDEO_BUFFERS 32 Unified diff of +#define UVC_MAX_VIDEO_BUFFERS 128 the changes /* Maximum status buffer size in bytes of interrupt URB. */ #define UVC_MAX_STATUS_SIZE 16 Source: https://patchwork.linuxtv.org/patch/19656/ 23

Open Source Group – Silicon Valley © 2014 SAMSUNG Electronics Co. Samsung Research Brazil Patch actually applied ;-) Subject: [media] uvcvideo: Remove duplicate check for number of buffers in queue_setup From: Laurent Pinchart Date: Tue, 18 Feb 2014 10:02:50 -0300

videobuf2 already ensures that the number of buffers will not exceed VIDEO_MAX_FRAME, which is equal to our arbitraty limit of UVC_MAX_VIDEO_BUFFERS. Remove the duplicate check.

Signed-off-by: Laurent Pinchart Tested-by: Philipp Zabel Signed-off-by: Mauro Carvalho Chehab

--- a/drivers/media/usb/uvc/uvc_queue.c +++ b/drivers/media/usb/uvc/uvc_queue.c @@ -51,3 +50,0 @@ static int uvc_queue_setup(struct vb2_queue *vq, const struct v4l2_format *fmt, - if (*nbuffers > UVC_MAX_VIDEO_BUFFERS) - *nbuffers = UVC_MAX_VIDEO_BUFFERS; - --- a/drivers/media/usb/uvc/uvcvideo.h +++ b/drivers/media/usb/uvc/uvcvideo.h @@ -118,2 +117,0 @@ -/* Maximum number of video buffers. */ -#define UVC_MAX_VIDEO_BUFFERS 32 24

● 1) Get a copy of the latest development tree:

– http://git.linuxtv.org/cgit.cgi/media_tree.git/ (for a Kernel media patch)

● 2) Write and test the changes

● 3) generate the patch $ git commit -a – write a proper summary, descriptions and add your SOB (for Kernel patches) $ git show –pretty=email > my_changes.patch

● 3) Check if CodingStyle was not violated or if it doesn't have some common mistakes: $ ./script/checkpatch.pl my_changes.patch – Fix patch until checkpatch.pl is happy and test it again

● 4) Send it to the maintainers

– Use get_maintainers.pl to know to whom address the email $ ./script/get_maintainers.pl my_changes.patch

Open Source Group – Silicon Valley © 2014 SAMSUNG Electronics Co. Samsung Research Brazil Steps for writing/submitting (2) $ ./scripts/get_maintainer.pl /tmp/uvc-more-buffers.patch Laurent Pinchart (maintainer:USB VIDEO CLASS) Mauro Carvalho Chehab (maintainer:MEDIA INPUT INFRA...) [email protected] (open list:USB VIDEO CLASS) [email protected] (open list)

● In the specific case of media drivers, don't c/c the maintainer. If everything is ok in the patch, it will appear in a few at patchwork:

– https://patchwork.linuxtv.org/project/linux-media/list/

● Also, in general, don't copy LKML (linux-kernel). That's the last resort when there's no specific subsystem mailing list.

● Stuff not maintained and without any subsystem maintainer should be submitted to Andrew Morton .

● Wait for people to review it. It may take some time, as Kernel developers are generally busy with other stuff

– Everybody can review a patch, including you.

● If you find a problem, write an email describing what you noticed.

● If you tested the patch, reply to it with Tested-by: you ● If you reviewed the patch but didn't test, reply to it with Reviewed-by: you ● Driver/system maintainers can also reply with Acked-by if they're ok if someone else would review and apply that patch. – A patch with reviews/tests will likely flow quicker than one that would depend solely on the driver maintainer to test

● The driver maintainer will get your patch, test or review and either apply or ask someone else to apply it;

● The patch will be applied by a sub-maintainer or by a subsystem maintainer;

● The subsystem maintainer will send it to Linus on the next merge window.

Tools for kernel development: git, gitk, quilt, patch, diff, diffstat, kdiff3

● The diff utility creates a difference between two files/directories; – It supports several different formats – Kernel (and most projects) use the unified diff format. ● The patch utility allows applying a patch into a source code;

● The diffstat utility provides stats about the patch: files changed, number of lines added/deleted per file and total number of changed lines;

● Sometimes, it is useful to compare 2 files and see the differences graphically, generating a 3rd file with the merged difference. – The KDE utility kdiff3 allows doing that ● Managing a pile of patches is sometimes complex

– There's a tool that allows managing such patch stacks: quilt Open Source Group – Silicon Valley © 2014 SAMSUNG Electronics Co. Samsung Research Brazil Tools for open source development ● Git - source code management (SCM) system written by Linus in 2005 to replace BitKeeper; – Each tree copy (clone) is complete and independent from the other ones – no central repository is needed; – Currently, the most used SCM on Open Source, including on the Linux Kernel;

● Gitk – A graphical tool that comes together with git, to help see the tree history

● Example: Linux Kernel development: – All Kernel workflow is based on git trees via the several subsystem maintainer's trees;

● All relevant trees are merged at the linux-next tree, during development phase; ● Most trees are located at: http://git.kernel.org

For those interested on working with media devices (webcams, TV, radio), please take a look at: http://linuxtv.org/wiki/index.php/Developer_Section