Master’s Thesis Nr. 141 Systems Group, Department of Computer Science, ETH Zurich Message Passing for Programming Languages and Operating Systems by Martynas Pumputis Supervised by Prof. Dr. Timothy Roscoe, Dr. Antonios Kornilios Kourtis, Dr. David Cock October 7, 2015 Abstract Message passing as a mean of communication has been gaining popu- larity within domains of concurrent programming languages and oper- ating systems. In this thesis, we discuss how message passing languages can be ap- plied in the context of operating systems which are heavily based on this form of communication. In particular, we port the Go program- ming language to the Barrelfish OS and integrate the Go communi- cation channels with the messaging infrastructure of Barrelfish. We show that the outcome of the porting and the integration allows us to implement OS services that can take advantage of the easy-to-use concurrency model of Go. Our evaluation based on LevelDB benchmarks shows comparable per- formance to the Linux port. Meanwhile, the overhead of the messag- ing integration causes the poor performance when compared to the native messaging of Barrelfish, but exposes an easier to use interface, as shown by the example code. i Acknowledgments First of all, I would like to thank Timothy Roscoe, Antonios Kornilios Kourtis and David Cock for giving the opportunity to work on the Barrelfish OS project, their supervision, inspirational thoughts and critique. Next, I would like to thank the Barrelfish team for the discussions and the help. In addition, I would like to thank Sebastian Wicki for the conversations we had during the entire period of my Master’s studies. Finally, I would like to thank my girlfriend and my family for their support which allowed me to follow my passion. ii Contents Acknowledgments ii Contents iii 1 Introduction1 2 Background3 2.1 Go Programming Language.................... 3 2.1.1 Types and Type System.................. 4 2.1.2 Concurrency......................... 5 2.2 Barrelfish Operating System.................... 7 2.2.1 Dispatchers......................... 8 2.2.2 Inter-Dispatcher Communication............. 9 2.3 Related Work ............................ 10 3 Porting Golang to the Barrelfish OS 12 3.1 Initial Port.............................. 12 3.1.1 Choosing a Go Compiler ................. 13 3.1.2 Run-Time Integration ................... 14 3.1.3 Running Goroutines.................... 17 3.2 Channel Integration......................... 19 3.2.1 Comparison of Message Passing in Barrelfish and Go . 19 3.2.2 Design Considerations................... 21 3.2.3 Implementation....................... 21 3.2.4 Discussion.......................... 26 3.3 Implementing OS services in Go................. 28 3.3.1 Foreign Function Interface ................ 28 3.3.2 Example Application: Nameservice........... 29 4 Evaluation 31 4.1 LevelDB ............................... 31 iii Contents 4.1.1 Suites............................. 32 4.1.2 Setup............................. 33 4.1.3 Results............................ 34 4.2 Concurrency............................. 35 4.2.1 Setup............................. 37 4.2.2 Results............................ 37 4.3 Channel Integration......................... 43 4.3.1 Setup............................. 47 4.3.2 Results............................ 47 4.3.3 Optimizations........................ 49 5 Conclusions and Future Work 54 5.1 Future Work............................. 54 5.1.1 Improving Send Semantics................ 54 5.1.2 Moving Event Dispatch to the Runtime......... 54 5.1.3 Multicore Support ..................... 55 5.1.4 Preventing Stack Overflow ................ 55 5.1.5 Hake Integration...................... 55 5.2 Conclusions ............................. 55 A Channel Integration Artifacts 57 B Nameservice Implementation in Go 65 Bibliography 68 iv Chapter 1 Introduction Recently, message passing as a form of communication is getting more at- tention within domains of programming languages and operating systems. Concurrent programming languages such as Go [14] and Erlang [6] make this form accessible as a first-class citizen and encourage synchronization among concurrent components by passing messages instead of sharing mem- ory. It is a common believe that the latter approach to concurrency is more difficult to reason and as a consequence, is more error-prone [22]. In addi- tion, the message passing systems can be formally described and verified by applying theories of process calculi [38] or the Actor model [34]. Meanwhile, increasing complexity, heterogeneity and ubiquity of multi-cores in the hardware landscape have led to research and development of operat- ing systems which resemble more distributed systems than regular operat- ing systems [17]. An example of such a tendency is the Barrelfish OS [16]. The OS is based on a multikernel approach which implies that each core is running a special isolated instance of a small kernel. One of the founding assumptions of the OS is that kernels do not share memory and instead, consistency and synchronization among kernels can only be achieved via channels-based message passing. The goal of this thesis is to explore how (or whether) a message passing lan- guage can be incorporated in the context of a messaging passing operating system. In particular, how channels of the Go programming language can be integrated with Barrelfish’s messaging infrastructure in order to provide the easy-to-use concurrency model of Go to the OS. In addition, how the integration can be employed for building Barrelfish OS services in Go. We will start with Chapter2 in which the relevant parts of Go and Barrelfish will be introduced. Following that, Chapter3 will present our effort towards porting Go to the Barrelfish OS, as well as integrating the Go channels with Barrelfish’s messaging infrastructure. In addition, the chapter will take a 1 look into implementing the OS services in Go. Next, in Chapter4 we will evaluate the port by performing series of benchmarks. Finally, Chapter5 will provide conclusions and directions for future work. 2 Chapter 2 Background In this chapter we present background information on two topics used throughout the thesis: the Go programming language and the Barrelfish operating system. The former is introduced in Section 2.1 and an overview of the latter is given in Section 2.2. 2.1 Go Programming Language Go (or golang) is a general-purpose open source programming language, with an emphasis on system programming. The language development ini- tially started with Rob Pike, Ken Thompson and Robert Griesemer at Google in 2007 [13]. The main motivation for conceiving the language was to address the prob- lems of a large-scale software development process at Google [45]. Firstly, it means to minimize build time by taking an alternative approach to de- pendency handling when compared to C/C++. Secondly, it aims to reduce a set of possible implementations for a single problem by providing a sim- ple programming model. Thirdly, the language should enable writing tools targeting the language itself. The simple specification and the programming model of Go is backed by one of the main principles behind the language - do less, but enable more [20]. The slogan is translated to a requirement to allow expressing solutions to a variety of problems despite the simplicity and the brevity of the language. Indeed, Go stands out by the size of its specification when compared to any other general-purpose programming language [14], e.g., Scala [48]. One can argue that the simplicity lead to increasing popularity and adoption of the language. For example, many emerging projects outside Google such as Docker [42] or CoreOS [19] picked Go as the main language. Besides being a compiled, statically-typed and garbage-collected language, 3 2.1. Go Programming Language Go brings to the table concurrency as a first-class citizen. The concurrency feature was the main influencing factor for choosing the language for this thesis project. An example of a Go application is shown in Figure 2.1. The application prints the first 10 Fibonacci numbers. Generation of the numbers is done by the fib coroutine which runs concurrently with the main function. The communication between main and fib happens over the Go channel c. 1 package main 2 3 import"fmt" 4 5 func fib(c chan int){ 6 for i, j := 0, 1; ; i, j = i+j, i { 7 c <- i 8 } 9 } 10 11 func main() { 12 c := make(chan int) 13 go fib(c) 14 for i := 0; i < 10; i++ { 15 fmt.Println(<-c) 16 } 17 } Figure 2.1: The Fibonnaci numbers generator in Go. The following sections give a brief overview of type system and concurrency primitives of Go. The overview should help the readers to better understand the upcoming parts of the thesis. 2.1.1 Types and Type System Go supports basic types such as integers, floats, chars, bytes, pointers, strings, arrays and dynamic arrays called slices. Their representation in memory re- sembles the one of C with a few exceptions, e.g., strings and arrays are stored together with their length [23], which allows to perform runtime checks against overflow. Besides the primitive types, Go includes built-in hash tables and C style structs for type composition. Instead of adopting class based inheritance, Go provides an alternative so- lution: interfaces. Interfaces are used to specify a list of required methods. A type satisfies an interface definition if it implements methods required by the interface. Go compiler is able to infer such information and does not require any additional information besides the interface declaration. In ad- 4 2.1. Go Programming Language dition, every object implements an empty interface. The closest equivalent of the interfaces is Type Classes of Haskell [41] or Traits in Scala [48]. Figure 2.2 presents declarations of the Stringer interface and the Animal type which satisfies the interface by implementing the required String method. 1 type Stringer interface{ 2 String() string 3 } 4 5 type Animal struct{ 6 Name string 7 } 8 9 func (a Animal) String() string{ 10 return a.Name 11 } Figure 2.2: An example of the interfaces in Go.