A Characterization of State Spill in Modern Operating Systems Kevin Boos, Emilio Del Vecchio, and Lin Zhong Rice University fkevinaboos, edd5, [email protected] Abstract states change and propagate throughout the system, showing that modularity is necessary but insufficient. For example, Understanding and managing the propagation of states in several process migration works have cited “residual depen- operating systems has become an intractable problem due dencies” that remain on the source system as an obstacle to to their sheer size and complexity. Despite modularization post-migration correctness on the target system [38, 55, 34]. efforts, it remains a significant barrier to many contemporary Fault isolation and fault tolerance literature has long realized computing goals: process migration, fault isolation and the undesirable effects of fate sharing among software mod- tolerance, live update, software virtualization, and more. ules as well as the difficulty of restoring a process or whole Though many previous OS research endeavors have achieved machine statefully after a failure, due to the sprawl of states these goals through ad-hoc, tedious methods, we argue that spread among different system components [28, 26, 52, 51]. they have missed the underlying reason why these goals are Live update and hot-swapping research has long sought to overcome the state maintenance issues and inter-module de- so challenging: state spill. pendencies that plague their implementations when transi- State spill occurs when a software entity’s state undergoes tioning from an old to a new version, forcing developers to lasting changes as a result of a transaction from another entity. write complex state transfer functions [7, 25, 27, 48, 5]. Like- In order to increase awareness of state spill and its harmful wise, state management poses a problem to the virtualization effects, we conduct a thorough study of modern OSes and of arbitrary software components, significantly complicat- contribute a classification of design patterns that cause state ing the multiplexing and isolation logic of the underlying spill. We present STATESPY, an automated tool that leverages virtualization layer (e.g., kernel or runtime) [11, 6]. cooperative static and runtime analysis to detect state spill in In this paper, our primary contribution is to identify the real software entities. Guided by STATESPY, we demonstrate problem of state spill as a root cause of these problems and the presence of state spill in 94% of Android system services. show that it is the underlying reason why many computing Finally, we analyze the harmful impacts of state spill and goals are so difficult to realize, even in the face of proper mod- ularization. State spill is the act of one software entity’s state suggest alternative designs and strategies to mitigate them. undergoing lasting changes as a result of its interaction with 1. Introduction another entity. For example, if an application interacts with a system service, causing that service to store application- Over the past few decades, operating systems research has relevant states in its memory, then state spill has occurred gone to great lengths to achieve a spectrum of advanced com- from the application to the service. We formally define state puting goals: process migration, fault isolation and tolerance, spill and the conditions under which it occurs in §3. live update, hot-swapping, virtualization, security, general From this cursory explanation, it is apparent that state spill maintainability, and more. Better modularization, in which is a phenomenon that permeates all levels of a software sys- related functionality is grouped into bounded entities, is often tem. However, it is relative to the choice of entity granularity: touted as the most apt solution for realizing such goals, and it interactions that cross the boundary of an entity are consid- seems promising from an initial glance. However, we argue ered transactions of interest, but those within a single entity that modularization alone is not enough, and that the effects are not. As an example, if the entity granularity is defined to of interactions between modules have a more pronounced be a process, then an IPC call between two processes could impact on these goals, as shown below. constitute state spill, but a local function call within a process Many efforts towards these goals have been ad-hoc and could not. The definition of state spill is not confined to any platform-specific due to the complex nature of how software particular entity granularity; entity granularity is a platform- and goal-specific decision (§3.1). However, in this work, our analysis centers around entity bounds akin to those of a mod- ule: an entity consists of a group of related functions and the Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice data they modify. When analyzing Android system services, and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to for example, the best entity granularity choice is a Java class. lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. The second contribution of this work is a manual analysis EuroSys ’17, April 23 - 26, 2017, Belgrade, Serbia of state spill and its detrimental impact on the aforementioned © 2017 Copyright held by the owner/author(s). Publication rights licensed to ACM. ISBN 978-1-4503-4938-3/17/04. $15.00 computing goals across various real-world OSes. We find that DOI: http://dx.doi.org/10.1145/3064176.3064205 state spill is often a byproduct of applying the locality prin- Definition 1. A transaction is the flow of control (and ciple to OS design, in that it often stems from design choices optionally data) from one software entity to another. The that favor programming convenience or performance over ad- source entity, or client, initiates the transaction and the herence to strict architectural or modularity principles. Based destination entity, or server, receives and handles it, returning on these case studies, we establish a classification of state spill control to the source upon completion. (§4) according to common design patterns: indirection layers, Transactions can represent procedure calls, system calls, multiplexers, dispatchers, and inter-entity collaboration. interrupts, signals, other IPC, and more. The exact incarnation Our third contribution is the STATESPY tool that auto- of a transaction is defined by which entities are involved; for mates the detection of state spill in real systems, and an in- example, a function call is a transaction from one function depth, tool-guided analysis of state spill in Android’s system to another function in the same thread; system calls are service entities. As described in §5, STATESPY employs both transactions from a userspace entity to a kernelspace one; runtime and static analysis techniques that automate the cap- IPC is a transaction from one process to another. Transactions ture, inspection, and differencing of a software entity’s state, have the following explicitly-defined characteristics: with current support for Java environments. The runtime anal- ysis component of STATESPY captures a running entity’s state • If a transaction is interrupted (e.g., preempted), it is simply by interposing on an entity’s execution paths when handling treated as incomplete until it resumes and returns control. transactions invoked by real-world applications. Our key in- • If a transaction never completes, such as a call to an debugging sight for runtime analysis is to leverage existing endlessly-looping function that listens for messages, no frameworks to non-intrusively capture and compare entity future actions from different source entities can be affected states, a technique that forgoes environment-specific features by its changes; thus, it is irrelevant to state spill. and generalizes to any execution environment. The static anal- ysis component of STATESPY provides relevancy filters to the • If a transaction handler fails, e.g., by returning an error, it runtime component as part of a cooperative feedback loop that is treated the same as a completed successful transaction. iteratively improves the output of each component. Guided • If a transaction is asynchronous/non-blocking, it is treated by STATESPY, our experimental analysis of Android system as two separate ones: an initial transaction from the source services (§6) finds that harmful state spill is both prevalent to the destination, and a second transaction from the and deep: nearly all Android system services have state spill, destination back to the source as a completion event. and it often occurs in a chain across multiple services. We discuss how to mitigate the effects of state spill in §7 We are primarily interested in short-lived transactions that by rethinking existing software designs patterns and commu- leave a lasting change on the destination entity, such that the nication schemes. Modularization is not enough: reducing latter’s future behavior will be affected. the impact of state spill is key to simplifying software com- 2.2 Quiescence in Software Entities ponents and making them amenable to migration, live update, fault recovery, virtualization, and other computing goals. In In assessing state spill, we are interested in the effect of a fact, we have shown Android system services can be made single transaction on an entity; thus an entity’s state is only more fault tolerant with state spill mitigation techniques [17]. matters at certain points: before and after a transaction, not In summary, we identify and formally define the problem during. We use the concept of quiescence to identify this of state spill in modern OSes, classify its various incarnations, condition.