HelenOS project project documentation Contents Contents 2 1 Introduction 4 1.1 How to read this document ...................... 4 2 Project 5 2.1 Specification .............................. 5 2.2 Project meetings ........................... 6 2.3 Planning work ............................. 6 2.4 Kernel camps ............................. 6 2.5 Coding style .............................. 7 3 Developers 8 3.1 Jakub Jerm´aˇr ............................. 8 3.2 Ondˇrej Palkovsk´y ........................... 8 3.3 Martin Dˇeck´y ............................. 9 3.4 Jakub V´aˇna .............................. 9 3.5 Josef Cejkaˇ .............................. 9 3.6 Sergey Bondari ............................ 9 4 Software 10 4.1 Communication tools ......................... 10 4.2 Concurrent versions systems ..................... 11 4.3 Web tools ............................... 12 4.4 Third party components of HelenOS ................ 12 4.5 Build tools ............................... 13 4.6 Virtual environments ......................... 14 4.6.1 Bochs ............................. 15 4.6.2 GXemul ............................ 15 4.6.3 msim .............................. 15 4.6.4 PearPC ............................ 16 4.6.5 QEMU ............................. 16 4.6.6 Simics ............................. 16 2 4.6.7 Ski ............................... 17 4.6.8 VMware ............................ 17 A Goals and Achievements 19 A.1 Overall Conception .......................... 19 A.2 1a ................................... 20 A.3 Particular features .......................... 21 A.4 Implementation details ........................ 22 References 23 3 Chapter 1 Introduction The HelenOS project[1] is an effort to develop portable and general purpose operating system. Operating systems in general are very non-trivial pieces of software. It takes many people, many months and many tools to design and develop even medium size and feature-limited kernel and userspace layer. This report aims to document the development process of the HelenOS oper- ating system as it is specified in [2] and as it has been carried out by the original six developers (i.e. Jakub Jerm´aˇr,OndˇrejPalkovsk´y,Martin Dˇeck´y,Jakub V´aˇna, Josef Cejkaˇ and Sergey Bondari) in their work on Software project1 at Faculty of Mathematics and Physics at Charles University in Prague. Other aspects of the wider HelenOS project (e.g. master theses related to the topic) are not discussed here. 1.1 How to read this document Chapter 2 provides an insight into project’s timeline, planning, development. It also presents some statistic data related to the HelenOS project. Chapter 3 evaluates contributions and project dedication of each individual developer. Chapter 4 gives thorough coverage of the third-party software involved with HelenOS and also experience comming from using that software. 1Software project is the name of a subject at MFF UK. It is supposed to last two semesters at least. 4 Chapter 2 Project The HelenOS project was formed in late October 2004, when the six developers grouped and decided to adopt previous work of Jakub Jerm´aˇron the SPARTAN kernel1 as a foundation for their new operating system. 2.1 Specification The team had then worked on a specification[2] until March 8, 2005. The spec- ification was based on Martin Dˇeck´y’sdraft and incorporated many suggestions from other members of the team. The biggest part of the discussion was con- cerned about how many and what processor architectures we will support. At that time, the SPARTAN kernel supported ia32 and mips32 to the extent that kernel threads could be scheduled. The ia32 port could do some very basic virtual memory operations and was capable of SMP service. Moreover, the mips32 port ran only in the msim simulator. None of them supported userspace threads. We realized the need to support at least one 64-bit architecture and have long discussed whether it should be amd64 or ia64. We also considered ppc64. At the end, we decided to declare support for three new architectures, five architectures in total. Both amd64 and ia64 made it to the specifications, as well as PowerPC. As for PowerPC, the specification didn’t say whether ppc32 or ppc64 or both will be supported.2 It is worth noting that we wanted to be sure of access to respective hardware or at least simulator, prior to committing to support particular architecture. The decision to support almost all suggested architectures3 came after we had known for sure the above condition was satisfied. 1The SPARTAN kernel has been developed by Jakub Jerm´aˇrsince 2001. 2This has later proven a bit problematic because it is not very clear what ppc32 should be (i.e. the 32-bit G4 processor is not compatible with the 32-bit mode of the G5 processor. 3Namely, we didn’t declare support for sparc64, but it got supported anyway as part of Jakub Jerm´aˇr’smaster thesis. 5 We constructed our specification so that it contained a well defined set of mandatory features of the kernel and the userspace layer that had to be im- plemented. Besides the mandatory features, there was also an optional part comprising of three research or experimental topics. We hoped to eventually find time to work on them. 2.2 Project meetings After adopting our specification, we started to meet regularily every two weeks for the sake of consultations. The regular meetings were cancelled only during the exam periods and summer holiday. The first meeting took place on April 28, 2005. There were exactly twenty two project meetings before 0.2.0 release. The Faculty of Mathematics and Physics officially opened our project on June 10, 2005. However, serious collective work on the project, preceeded by individual efforts of some team members, began two months later. 2.3 Planning work In the beginning, we structured our work by creating three two-member teams, each dedicated to one new architecture (i.e. amd64, ia64 and ppc32). However, dividing into couples didn’t work out for the amd64 and ppc32 teams. In the end, both of those architectures were supported only with one member of respec- tive team. This might have been because of two factors. First, the collective responsibility for the project allowed the less motivated members to work less than others. Second, over the time, some developers profiled out to be good at specific tasks to which they later adhered and were forwarded more similar work. It was generally accepted within the team if one of the couple traded one architecure-specific task for another task on HelenOS. 2.4 Kernel camps There were two really important moments in our development process. Both of them took place in Harrachov, Czech Republic, where five team members moved two times, each time for a week of full-time intensive HelenOS development. These actions were called Kernel Camp 2005 and Winter Camp 2006. The former camp took place in August 2005 and was focused on getting all the architectures into our source tree and deepening their support. The latter camp took place in March 2006 and was dedicated to userspace support. In fact, we made the second camp the deadline for userspace milestone. With the exception of ppc32, all ports had some support for userspace prior to the second camp. Both of the camps moved the project miles ahead. 6 2.5 Coding style We have adopted common coding style specification in order to improve code readibility and maintainability. Even though the specification relates only to stylistic matters, following it has the potential to encourage and improve cooper- ation within the team and provide good preconditions for future project growth. 7 Chapter 3 Developers 3.1 Jakub Jerm´aˇr Jakub is the original author of the SPARTAN kernel and the initiator of the idea to start the HelenOS project. Once the team was composed, he implemented considerable parts of the ia64 code and he also worked on the mips32 memory management. On the generic front, Jakub designed the generic virtual address translation interface for the 4-level hierarchical page table translation mecha- nism as well as the global page hash table translation mechanism. He has been involved in the address space management functions and userspace synchroniza- tion through futexes. Other areas in which he contributed include the kernel console and the kernel ELF loader. Jakub is also the author of the generic buddy system framework as well as the B+tree implementation. His latest contribution is the userspace PCI driver. 3.2 Ondˇrej Palkovsk´y Ondˇrej has completely created the amd64 port and completed the mips32 port to the extent that it runs on a real MIPS machine1. Besides the architecture specific involvement, Ondˇrejprogrammed the slab allocator and modified the frame allocator to be self-contained which in turn let the old and very limited heap manager be removed from the kernel entirely. He also created architecture independent FPU lazy switching framework. Other example of his activity is the IPC subsystem and partial TLS2 support. Lastly, Ondˇrejequipped the kernel console with features found in userspace command shells (e.g. tab completion of commands and command history through keyboard arrows) and wrote the kernel configuration software. 1SGI Indy 2Thread local storage. 8 3.3 Martin Dˇeck´y Right from the beginning, Martin has cared about project’s code purity and readibility. He was the first developer to start writing Doxygen-style comments. He has promoted the proper use of C language const keywords and extensive typedefing. On the tools front, he has rewritten the initial build system and created all our toolchain build scripts. Martin worked and much improved the ia32 and amd64 kernel booting using the Grub bootloader and Multiboot specification. He also created specialized boot loaders for mips32 and ppc32 — architectures that don’t provide many other ways to load userspace init tasks. Finally, Martin bacame the sole author of the entire ppc32 port and has encountered partial success in booting ppc64 port to real hardware3. 3.4 Jakub V´aˇna Jakub has worked on ia32 and the ia64 FPU context switching and passive ia32 and active and passive ia64 console.