Fencing off Go: Liveness and Safety for Channel-Based Programming Julien Lange Nicholas Ng Bernardo Toninho Nobuko Yoshida Imperial College London, UK tifact r Comp f g * let * j.lange, nickng, b.toninho, n.yoshida @imperial.ac.uk A t e n * te A is W s E * e n l l C o L D C o P * * c u e m s O E u e e P n R t v e o d t * y * s E a a l d u e a t Abstract goroutines (i.e. lightweight threads), drawing heavily from process Go is a production-level statically typed programming language calculi such as Communicating Sequential Processes (CSP) [23]. whose design features explicit message-passing primitives and Concurrent programming in Go is mostly guided towards channel- lightweight threads, enabling (and encouraging) programmers to based communication as a way to exchange data between gorou- develop concurrent systems where components interact through tines, rather than more classical concurrency control mechanisms communication more so than by lock-based shared memory con- (e.g. locks). Channel-based concurrency in Go is lightweight, of- currency. Go can only detect global deadlocks at runtime, but pro- fering logically structured flows of messages in large systems pro- vides no compile-time protection against all too common commu- gramming [3, 54], instead of “messy chains of dozens of asyn- nication mismatches or partial deadlocks. chronous callbacks spread over tens of source files” [4, 39]. This work develops a static verification framework for bounded On the other hand, Go inherits most problems commonly found liveness and safety in Go programs, able to detect communication in concurrent message-passing programming such as communi- errors and partial deadlocks in a general class of realistic concur- cation mismatches and deadlocks, offering very little in terms of rent programs, including those with dynamic channel creation and compile-time assurances of correct structuring of communication. infinite recursion. Our approach infers from a Go program a faith- While the Go runtime includes a (sound) global deadlock detector, ful representation of its communication patterns as a behavioural it is ultimately inadequate for complex, large scale applications that type. By checking a syntactic restriction on channel usage, dubbed may easily be undermined by trivial mistakes or benign changes to fencing, we ensure that programs are made up of finitely many dif- the program structure [20, 40], nor can it detect deadlocks involving ferent communication patterns that may be repeated infinitely many only a strict subset of a program’s goroutines (partial deadlocks). times. This restriction allows us to implement bounded verification Liveness and Safety of Communication While Go’s type system procedures (akin to bounded model checking) to check for liveness does ensure that channels are used to communicate values of the and safety in types which in turn approximates liveness and safety appropriate type, it makes no static guarantees about the liveness in Go programs. We have implemented a type inference and live- or channel safety (i.e. channels may be closed in Go, and sending ness and safety checks in a tool-chain and tested it against publicly on a closed channel raises a runtime error) of communication in available Go programs. well-typed code. Our work provides a framework for the static Categories and Subject Descriptors D.1.3 [Programming Tech- verification of liveness and absence of communication errors (i.e. niques]: Concurrent Programming; D.2.4 [Software Engineering]: safety) of Go programs by extracting concurrent behavioural types Software/Program Verification; D.3.1 [Programming Languages]: from Go source code (related type systems have been pioneered Formal Definitions and Theory; F.3.2 [Semantics of Programming by [48] and [28], among others). Our types can be seen as an Languages]: Program analysis abstract representation of communication behaviours in the given program. We then perform an analysis on behavioural types, which Keywords Channel-based programming, Message-passing pro- checks for a bounded form of liveness and communication safety gramming, Process calculus, Types, Safety and Liveness, Compile- on types and study the conditions under which our verification time (static) deadlock detection entails liveness and communication safety of programs. 1. Introduction 1.1 Overview Do not communicate by sharing memory; We present a general overview of the steps needed to perform our instead, share memory by communicating analysis of concurrent Go programs. Go language proverb [5, 49] Prime Sieve in Go To illustrate the challenges in analysing Go Go is a statically typed programming language designed with ex- programs, we begin by considering a rather concise implementa- plicit concurrency primitives at the forefront, namely channels and tion of a concurrent prime sieve (Listing 1, adapting the example from [43] to output infinite primes). This seemingly simple Go pro- gram includes intricate communication patterns and concurrent be- haviours that are hard to reason about in general due to the combi- nation of (1) unbounded iterative behaviour, (2) dynamic channel creation and (3) spawning of concurrent threads. The program is made up of three functions: Generate, that given a channel ch continuously sends along the channel an in- creasing sequence of integers starting from 2 (encoded as a for loop without an exit condition); Filter, that given a channel that a program consists of finitely many different communication 1 package main 2 func Generate(ch chan<- int){ patterns (that may themselves be repeated infinitely many times). 3 for i := 2; ;i++ { ch <-i} // Send sequence 2,3... For instance, the recursive call to rhbi in the equation r(x) is 4 } fenced, since the recursive instances of r will not know the channel 5 func Filter(in <-chan int, out chan<- int, prime int){ parameter x hence they cannot spawn threads which share x. This 6 for {i := <-in // Receive value from’in’. 7 if i%prime != 0 { out <-i} // Fwd’i’ if factor. restriction enforces a “finite memory” property wrt. channel names, 8 } insofar as given enough recursive unfoldings all the parameters of 9 } the recursive call will be names local to the recursive definition. 10 func main() { 11 ch := make(chan int) // Create new channel. Symbolic Semantics and Bounded Verification Fenced types can 12 go Generate(ch) // Spawn generator. 13 for i := 0; ;i++ { be symbolically executed as CCS processes in a representative 14 prime := <-ch finite-state labelled transition system that consists of a bounded 15 ch1 := make(chan int) version of the generally unrestricted type semantics. This enables 16 go Filter(ch, ch1, prime) // Chain filter. us to produce decision procedures for bounded liveness and safety 17 ch= ch1 18 } of types, which deems the type t0 as bounded live and safe. In the 19 } finite control case, the bounded correctness properties and their Listing 1. Concurrent Prime Sieve. unbounded ones coincide. The approach is similar to bounded model checking, using a bound on the number of channels in a type to limit its execution. for inputs in, one for outputs out, and a prime, continuously From Type Liveness to Program Liveness The final step is to forwards a number from in to out unless it is divisible by formally relate liveness and safety of types to their analogues prime; and main, which assembles the sieve by creating a new in Go programs. For the case of safety, safety of types implies synchronous channel ch, spawning the Generator thread (i.e. safety for programs. For the case of liveness, programs typically go f(x) spawns a parallel instance of f) with the channel ch, rely on data values to guide their control flow (e.g. in conditional and then iteratively setting up a chain of Filter threads, where branches) which are abstracted away at the type level. For instance, the first filter is connected to the generator and the next Filter, the Filter function only outputs if the received value i is not and so on. We note that with each iteration, a new synchronous divisible by the given prime, but the type for the corresponding channel ch1 is created that is then used to link each filter instance. process is given by f (x; y) which just indicates an internal choice The program spawns an infinite parallel composition of Filter between two behaviours. The justification for the liveness of t0 is threads, each pair connected by a dynamically created channel; and that the internal choice always has the potential to enable the output the execution of Generator and the Filter processes in the on y (assuming a fairness condition on scheduling). However, it is sieve is non-terminating. not necessarily the case that a conditional branch is “equally likely” Types of Prime Sieve Our framework infers from the prime sieve to proceed to the then branch or the else branch. In x 5, we de- fine the three classes of programs in which liveness of programs program the type t0 given by: and their types coincide. g(x) , x; ghxi f (x; y) , x;(y; f hx; yi ⊕ f hx; yi) 1.2 Contributions r(x) , x;(new b)(f hx; bi j rhbi) We list the main contributions of our work: To define and then t0() , (new a)(ghai j rhai) show the bounded liveness and safety properties entailed by our The type is described as a system of mutually recursive equations, analysis, we formalise the message-passing concurrent fragment of the Go language as a process calculus (dubbed MiGo). The MiGo where the distinguished name t0 types the program entry point. This language is equivalent to a subset of CCS [36] with recursion, calculus mirrors very closely the intended semantics of the channel- parallel and name creation, for which deciding liveness or safety based Go constructs and allows us to express highly dynamical and potentially complex behaviours (x 2); We introduce a typing is in general undecidable [7, 8].