Experiences from a Decade of TinyOS Development http://www.tinyos.net Philip Levis Stanford University [email protected] Abstract when sensor network research was beginning, starting When first written in 2000, TinyOS’s users were a hand- as a set of Perl scripts that auto-generated #define state- ful of academic computer science researchers. A decade ments [23]. Since then, it has evolved to use a C di- later, TinyOS averages 25,000 downloads a year, is in alect called nesC, has gone through four major revisions, many commercial products, and remains a platform used supports tens of sensor network platforms, and has ap- for a great deal of sensor network, low-power systems, proximately 25,000 downloads per year. TinyOS is the and wireless research. dominant software platform used for sensor network re- We focus on how technical and social decisions influ- search, enabling hundreds of research results. It is used enced this success, sometimes in surprising ways. As in numerous commercial products, such as Zolertia [3], TinyOS matured, it evolved language extensions to help Cisco’s smart grid systems (formerly Arch Rock), and experts write efficient, robust systems. These extensions People Power Company [2]. revealed insights and novel programming abstractions This paper examines how TinyOS evolved over the for embedded software. Using these abstractions, ex- past decade. TinyOS is interesting for two reasons. First, perts could build increasingly complex systems more like projects such as Xen [11, 44] and OpenFlow [16], easily than with other operating systems, making TinyOS TinyOS started as an academic research project that tran- the dominant choice. sitioned to significant success and impact outside academia. This success, however, came at a long-term cost. Sys- It managed to make this transition while simultaneously tem design decisions that seem good at first can have remaining a linchpin of the research community. Sec- unforeseen and undesirable implications that play out ond, TinyOS differs from these other examples in that it over the span of years. Today, TinyOS is a stable, self- is a successful, principled, and novel operating system contained ecosystem that is discouraging to new users. for a new class of computing devices. Other systems, such as Arduino and Contiki, by remain- This paper examines how technical and social deci- ing more accessible, have emerged as better solutions sions encouraged or restricted the growth of TinyOS and for simpler embedded sensing applications. therefore its impact on practice, sometimes in unfore- seen ways. For example, fine-grained software compo- 1. INTRODUCTION nents allow users to easily customize the OS with small, local changes. As TinyOS was still forming and being Wireless sensor network research is just over a decade used speculatively in a large number of domains, this old. Starting as a handful of academic institutions study- easy customization was beneficial. But once core OS ing networks of tiny, low-power wireless sensing de- services solidified, fine grained components became ul- vices, it now has numerous academic conferences and timately harmful, as reading a core system requires leaf- journals that serve a large, worldwide research commu- ing through many tiny components. nity. Sensor networks have also grown from research The paper is divided into four parts. Section 2 de- projects to commercial systems. Commercial systems scribes the two basic principles that have driven TinyOS. today include ad-hoc wireless smart meter networks, home The first principle is resource use minimization. The area networks, and industrial monitoring systems. When costs of scale and low power operation say that TinyOS Cisco talks about an “Internet of Things,” it means the code should trade off runtime flexibility or generality coming Internet with millions or billions of tiny net- for smaller code and data, in contrast to many modern worked devices that interact with and sense the physical “large” software systems. The bug prevention princi- environment: sensor networks. ple, motivated by the tremendous difficulty of debug- TinyOS is an operating system designed for such em- ging embedded systems, says that TinyOS should be bedded devices. It emerged from UC Berkeley in 2000 USENIX Association 10th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’12) 207 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 bidirectional interfaces, atomic, generics, safe, parameterized interfaces uniqueCount nx_types threads Figure 1: Timeline of major events in TinyOS development from 1999-2010. structured to make it hard to write bugs, sometimes at Model ROM RAM Sleep Price the cost of making it generally harder to write code. To F2002 1kB 128B 1.3μA $0.94 help support these principles, TinyOS developers chose F1232 8kB 256B 1.6μA $2.73 to design and use nesC [20], a new C dialect. The lan- F155 16kB 512B 2.0μA $6.54 F168 48kB 2048B 2.0μA $9.11 guage and OS co-evolved, such that it does not make F1611 48kB 10240B 2.0μA $12.86 sense to talk about one without the other: when we talk about the evolution of TinyOS, we mean the evolution (a) TI MSP430 Microcontrollers of both the OS and its language. Sections 3-6 walk through how four approaches TinyOS Model ROM RAM Sleep Price took had unforeseen long-term implications. The first LM2S600 32kB 8kB 950μA $2.73 two, memory allocation and isolation, relate to the unique LM3S1608 128kB 32kB 950μA $4.59 properties of embedded software. The second two, com- LM3S1968 256kB 64kB 950μA $6.27 ponents and systems language design, relate to systems software more generally. Section 3 discusses how new (b) TI ARM CortexM3 Processors language features allowed TinyOS to optimally allocate RAM while simultaneously removing the need for some Table 1: A representative sampling of popular pro- run-time memory access checks. Section 4 describes cessors used in low-power wireless sensors. The price how a novel software pattern based on this memory allo- values are from DigiKey’s catalog on March 3rd, cation, static virtualization, improves software isolation 2010, when purchased in quantities of 1,000 - 10,000. by making the finite state machine of each virtualized in- stance completely independent. Section 5 examines how TinyOS that can apply to embedded software, systems using nesC was critically important to TinyOS’s early more generally, and systems projects. One conclusion is success, but also how its evolution limited TinyOS from that fine-grained components are good for experimenta- even broader, long-term use. Section 6 looks at the ben- tion but add unnecessary and painful complexity to sta- efits and drawbacks of fine-grained, reusable software ble software that expects reuse (e.g., a kernel). A sec- components, concluding they are a poor fit for operating ond conclusion is the natural tendency to support long- systems. standing, dedicated users and evolve a system to bet- Section 7 examines the TinyOS project from a so- ter meet their needs undermines system adoption. Re- cial perspective: how did the project grow such a large search wants to push a frontier, but doing so can alienate developer community? Open source projects live and a broader audience and stifle long-term success. We dis- die based on their contributors. TinyOS today has a cuss some ways in which future projects seeking large- large community of developers and users from all across scale adoption might avoid these and other pitfalls. the world. It examines how this community is struc- tured and how that structure evolved. It presents sev- eral pitfalls the project encountered, relating to hiring 2. MINIMIZATION AND PREVENTION staff, managing code contributions, and the interactions TinyOS’s design has two major goals: minimizing between academia and industry. It also discusses the resource use and preventing bugs. Both are driven by role of documentation and target audiences and how the the unique intersection of requirements that sensor net- project was able to reduce the barrier to entry caused by works pose. its increasing technical complexity. The minimization principle states that TinyOS soft- Section 8 takes a step back to examine lessons from ware should use as few hardware resources as possible. 208 10th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’12) USENIX Association This means being computationally efficient (minimizing swap, where a slightly-too-big program will run slowly, cycle counts and wake time), requiring little state (min- there is no margin for error on a microcontroller. A too- imizing RAM) and having very tight code (minimizing big program either has a compile error or crashes almost code ROM). Traditional computing systems want to be immediately when the stack overruns data memory. efficient, but they typically trade off some efficiency for The prevention principle means preventing bugs through flexibility and efficiency in the form of kernel modules, software structure. All software wants to prevent bugs, plugins, or other mechanisms. In contrast, TinyOS fo- but TinyOS took a very extreme position due to how cuses on producing an ultra-optimized binary that can astonishingly difficult in-the-field debugging of sensor run unattended for months to years. networks is. Debugging is so difficult that it has prompted Two properties of embedded sensors motivate the min- a wide range of research [13, 37, 42]. A sensor net- imization principle. The first is energy. Within a device work is a highly distributed system, where nodes dy- class, parts with more hardware resources draw more namically react to the environment and each other. The power both when awake and when asleep. Since nodes limited resources, as well as possible energy constraints, sleep almost all of the time, even small sleep power on each device preclude extensive logging or other tra- draws are significant. Table 1 shows a selection of re- ditional debugging techniques.