Cover Intro: Kernel Tricks COVER STORY KERNELPromising projects from the Linux kernelTRICKS community Developers are constantly looking for new ways to interact with the versatile Linux kernel. This month we study some innovative projects leading deep into kernel space. BY JON MASTERS, JAN RÄHM, AND JOE CASAD ost Linux users know they can tion. At the heart of that distribution is is really doing, take a look at Figure 1. rely on the kernel without ever the Linux kernel itself – a complex col- The kernel appears in the center of the Mgiving it much thought. But if lection of software routines that takes diagram, surrounded by a sample of the you do look a little deeper, you’ll find control whenever a machine first boots operations it assists in performing. some evolving technologies that extend and periodically (many times each sec- On the left, hardware interrupts from the kernel in interesting ways. This ond) handles the resource requirements devices such as disks, network cards, month we cover some innovations at of applications, manages system hard- sound devices, graphics cards, and so on the edges of the Linux kernel. ware devices, and generally does a lot arrive at unpredictable times (data has Our first article describes the new of the low level dirty work. perhaps been requested, but there is no userspace driver model in upcoming ker- The Linux kernel is regarded as a fully way to know exactly when the device nel versions. In our second story, Klaus functional, complex, and well-docu- will actually deliver that data). In a simi- Knopper looks at block device compres- mented role model for open source soft- lar way, the system timer (on most sys- sion with the Cloop module, which lets ware. Everyone has access to the source Live CD developers put up to 2GBs on a code, which is updated daily; and ev- 700MB CD. We’ll also examine kernel- eryone can join in with dis- based virtualization with KVM, and cussions on the mailing we’ll end with a look at the Flash Trans- lists or submit lation Layer and the LogFS flash filesys- patches. tem. To under- We hope you enjoy this month’s stand what Kernel Tricks cover story collection. But the kernel first, read on for a brief introduction to the brain of Linux. The Linux Kernel The term Linux, in the strictest sense, does not refer to the shiny collection of software components and applications that is often known as a Linux distribu- COVER STORY Userspace Drivers .................... 28 Cloop .......................................... 32 KVM .......................................... 37 LogFS ........................................ 40 JANUARY 2008 ISSUE 86 21 021-026_coverintro.indd 21 16.11.2007 13:32:04 Uhr COVER STORY Cover Intro: Kernel Tricks opment, Linux 2.4 was released in the late 1990s. Linux 2.4 was significant because it was the first kernel release widely used by the masses – not just computer enthusiasts, but also those try- ing Linux for the first time on their desk- top computers, servers, and even embed- ded gadgets. Version 2.4 was also used in the new generation of “Enterprise” distributions from major Linux vendors. Once Linux 2.4 was safely out of the way, work began in earnest on a 2.5 de- velopment series, leading up to a major 2.6 kernel release a few years ago. Version 2.6 was a complete revolution for Linux. It had a rewritten scheduler (that can scale to many thousands of CPUs by now), a full device manage- Figure 1: The Linux kernel manages processes, hardware interrupts, and housekeeping tasks. ment subsystem with greatly improved and re-written dynamic device support, tems) fires regularly to allow Linux to kernel itself should be thought of more and literally countless other improve- perform periodic housekeeping opera- as a collection of software routines in a ments that came as a result of the grow- tions many times per second. privileged library. These specially privi- ing acceptance of Linux as a viable com- A kernel would not be particularly leged routines are always executed in mercial offering. By this point, features useful if it were not able to also service response to specific events, and at that weren’t coming in from enthusiasts at the user in the provision of support for time, whatever else was running is tem- nearly the same rate that they were running programs (called tasks, within porarily saved as the system switches being offered by well-known Linux the world of the Linux kernel). On the into kernel code. Most of the time, ker- hackers employed by Linux vendors and right side of the diagram are three tasks. nel code runs either as a result of timers companies like IBM. Each represents a different kind of appli- and hardware interrupts, or as the result cation. The first, at the bottom, is a regu- of a request from an application. It also Linux Is Not Just Linus lar user program – in this case, a Bash runs at startup and in various error con- Although it should be obvious by now, shell running on a user’s desktop. Most ditions. Linux is not the day-to-day product of programs don’t communicate with the one man’s efforts. Linus Torvalds is kernel directly. Instead, they use the A Little History widely recognized as the inventor and GNU C Library functions, which in turn The Linux operating system dates back head of the Linux kernel project, but call standard kernel interfaces in order to Linus Torvalds’ first experiments in Linus serves more as a highly skilled to provide required services. 1991. At that time, Linux was merely a project manager at this point (Figure 2). Sometimes, an application does not toy operating system alternative. It ran From a qualitative point of view, Linus use the C library but talks to the kernel only on Intel more directly instead. This is the case 386-compatible with certain low-level, statically linked processors; how- tools – for example, busybox (commonly ever, it was very used as a recovery tool, as well as in em- hackable, and it bedded gadgets such as the Nokia N810 wasn’t long be- Internet Tablet). In the diagram, you can fore Linux was see one of these special tools providing available for a the services of init. Finally, at times the wide variety of kernel runs special kernel code as if it different ma- were almost a regular program (but with chines – first privileges). came the more The key point of Figure 1 is that a ker- generic PC sup- nel is far from magical. Its behaviors and port, but thereaf- processes can be explained. Many peo- ter, developers ple believe that a kernel is somehow began to port “running” all the time and constantly Linux to other overseeing system operations. Although architectures. Figure 2: Linus Torvalds at the Kernel Summit 2007: the founding it is true that Linux does perform a vari- After almost a father of the Linux kernel still makes a major contribution to its devel- ety of system monitoring functions, the decade of devel- opment. 22 ISSUE 86 JANUARY 2008 021-026_coverintro.indd 22 16.11.2007 13:32:47 Uhr Anzeige Nokia 021-026_coverintro.indd 23 16.11.2007 13:33:07 Uhr COVER STORY Cover Intro: Kernel Tricks When you read the document, you may Linus’ git repository. Git is the Linux notice some gaps and that the column kernel source management utility writ- contains an s for orphaned status. These ten by Linus Torvalds. orphaned projects are projects that no- body is currently working on. Of course, Tree Structure the developer community is always The kernel tree structure consists of happy for somebody to step in and as- four main trees above the branches or sume responsibility for an orphaned de- subtrees: main 2.6.x, 2.6.x.y-stable, velopment sector, but if you are new to 2.6.x-git, and 2.6.x-mm. Linus Torvalds kernel development, you should proba- maintains the main 2.6.x tree. When a bly consider sticking to bug fixes and new kernel version is imminent, Tor- patches before attempting the main- valds opens a two-week window in Figure 3: Andrew Morton is the number two tainer role. which developers can send him their in kernel development. His -mm branch diffs. provides the impetus for new features in Linux Day-to-Day In nearly every case, changes destined the Linux kernel. Development for the main tree have been tested in the Linux kernel development takes place on -mm kernel over a period of days or may not be the main code developer; a minute-by-minute basis, around the weeks. Andrew Morton’s 2.6.x-mm tree however, he still coordinates releases clock, every day of the year. Central differs from Torvald’s Vanilla kernel in and work on the kernel, contributes to co-ordination happens using the Linux that it contains untested changes by mailing lists, and takes an active part Kernel Mailing List (LKML), an archive Morton himself or resulting from patches in discussions on new functions. of which you can find at lkml. org. reviewed by Morton. Morton integrates Below Linus Torvalds in the kernel Although general Linux kernel develop- changes to all subsystems and patches hierarchy are a number of maintainers ment discussion happens on the LKML, from the mailing list with his version. supervising work on different parts of literally dozens of other popular email- The -mm branch is regarded as a play- the kernel. A maintainer decides on the ing lists cover each of the many features ground for new developments and fea- development roadmap, typically in col- within Linux.