Region-based Memory Management

Advanced Operating Systems Lecture 10 Lecture Outline

• Rationale • Stack-based memory management • Region-based memory management • Ownership • Borrowing • Benefits and limitations

2 Rationale

• Garbage collection tends to have unpredictable timing and high memory overhead • Real-time collectors exist, but are uncommon and have significant design implications for applications using them • Manual memory management is too error prone

• Region-based memory management aims for a middle ground between the two approaches • Safe, predictable timing • Limited impact on application design

3 Stack-based Memory Management

• Automatic allocation/deallocation of variables on the stack is common and efficient:

#include #include #include

static double pi = 3.14159; Global variables

static double conic_area(double w, double h) { double pi = 3.14159 double r = w / 2.0; double a = pi * r * (r + sqrt(h*h + r*r)); Stack frame for main()

return a; double width = … } double height = … double area = … int main() { double width = 3; Stack frame for conic_area() double height = 2; double area = conic_area(width, height); double w = … double h = … double r = … printf("area of cone = %f\n", area); double a = … return 0; }

4 Stack-based Memory Management

• Hierarchy of regions corresponding to call stack: • Global variables • Local variables in each function • Lexically scoped variables within functions

double vector_avg(double *vec, int len) { double sum = 0; Lifetime of sum – local variable in function for (int i = 0; i < len; i++) { Lifetime of i – lexically scoped sum += vec[i]; }

return sum / len; }

• Variables live within regions, and are deallocated at end of region scope

5 Stack-based Memory Management

• Limitation: requires data to be allocated on stack • Example: int hostname_matches(char *requested, char *host, char *domain) { char *tmp = malloc(strlen(host) + strlen(domain) + 2);

sprintf(tmp, “%s.%s”, host, domain);

if (strcmp(requested, host) == 0) { return 1; } if (strcmp(requested, tmp) == 0) { return 1; } return 0; }

The local variable tmp (pointer of type char *) is freed when the function returns; the allocated memory is not freed • Heap storage has to be managed manually

6 Region-based Memory Allocation

• Stack allocation effective within a narrow domain – can we extend the ideas to manage the heap? • Create pointers to heap allocated memory • Pointers are stored on the stack and have lifetime matching the stack frame – pointers have type Box for a pointer to heap allocated T • The heap allocation has lifetime matching that of the Box – when the Box goes out of scope, the heap memory it references is freed • i.e., the destructor of the Box frees the heap allocated T

• Efficient, but loses generality of heap allocation since ties heap allocation to stack frames

7 Region-based Memory Management

• For effective region-based memory management: • Allocate objects with lifetimes corresponding to regions • Track object ownership, and changes of ownership: • What region owns each object at any time • Ownership of objects can move between regions • Deallocate objects at the end of the lifetime of their owning region • Use scoping rules to ensure objects are not referenced after deallocation

• Example: the Rust programming language • Builds on previous research with Cyclone language (AT&T/Cornell) • Somewhat similar ideas in Microsoft’s Singularity operating system

8 Returning Ownership of Data

• Returning data from a function causes it to outlive the region in which it was created:

Lifetime of local variables const PI: f64 = 3.14159; in area_of_cone fn area_of_cone(w : f64, h : f64) -> f64 { let r = w / 2.0; let a = PI * r * (r + (h*h + r*r).sqrt());

return a; }

fn main() { let width = 3.0; Lifetime of a let height = 2.0;

let area = area_of_cone(width, height);

println!("area = {}", area); }

9 Returning Ownership of Data

• Runtime must track changes in ownership as data is returned • Copies made of stack-allocated local variables; original deallocated, copy has lifetime of stack-allocated local variable in calling function • Allows us to return a copy of a Box that references a heap allocated value of type T • Effective with a reference counting implementation • Creating the new Box temporarily increases the reference count on the heap-allocated T • The original box is then immediately deallocated, reducing the reference count again • (An optimised runtime can eliminate the changes to the reference count) • The heap-allocated T is deallocated when the box goes out of scope of the outer region • Allows data to be passed around, if it always has a single owner

10 Giving Ownership of Data

% cat consume.rs • Ownership of parameters passed fn consume(mut x : Vec) { to a function is transferred to that x.push(1); function } Lifetime of a • Deallocated when function ends, unless it fn main() { returns the data let mut a = Vec::new(); • Data cannot be later used by the calling function – enforced at compile time a.push(1); a.push(2); Ownership of a transferred to consume() consume(a);

println!("a.len() = {}", a.len()); }

% rustc consume.rs consume.rs:15:28: 15:29 error: use of moved value: `a` [E0382] consume.rs:15 println!("a.len() = {}", a.len()); ^

11 Borrowing Data

% cat borrow.rs • Functions can borrow references to fn borrow(mut x : &mut Vec) { data owned by an enclosing scope x.push(1); } • Does not transfer ownership of the data A mutable reference • Naïvely safe to use, since live longer than fn main() { the function let mut a = &mut Vec::new();

a.push(1); a.push(2); • Can also return references to input parameters passed as references borrow(a); • Safe, since these references must live longer println!("a.len() = {}", a.len()); than the function }

% rustc borrow.rs % ./borrow a.len() = 3 %

12 Problems with Naïve Borrowing

% cat borrow.rs • The borrow() function changes fn borrow(mut x : &mut Vec) { the contents of the vector x.push(1); } • But – it cannot know whether it is fn main() { safe to do so let mut a = &mut Vec::new(); • In this example, it is safe If was iterating over the contents of a.push(1); • main() the vector, changing the contents might lead a.push(2); to elements being skipped or duplicated, or to a result to be calculated with inconsistent borrow(a); data println!("a.len() = {}", a.len()); • Known as iterator invalidation }

% rustc borrow.rs % ./borrow a.len() = 3 %

13 Safe Borrowing

• Rust has two kinds of pointer: • To change an object: • &T – a shared reference to an immutable • You either own the object, and it is not object of type T marked as immutable; or • &mut T – a unique reference to a mutable • You have the only &mut reference to it object of type T • Runtime system controls pointer • Prevents iterator invalidation ownership and use • The iterator requires an &T reference, so • An object of type T can be referenced by other code can’t get a mutable reference one or more references of type &T, or by to the contents to change them: exactly 1 reference of type &mut T, but fn main() { not both let mut data = vec![1, 2, 3, 4, 5, 6]; for x in &data { fails, since push takes Cannot get an reference to data data.push(2 * x); • &mut T an &mut reference of type T that is marked as immutable } } • Allows functions to safely borrow enforced at compile time objects – without needing to give away ownership

14 Benefits

• Type system tracks ownership, turning run-time bugs into compile-time errors: • Prevents memory leaks and use-after-free bugs • Prevents iterator invalidation • Prevents race conditions with multiple threads – borrowing rules prevent two threads from getting references to a mutable object • Efficient run-time behaviour – timing and memory usage are predictable

15 Limitations of Region-based Systems

• Can’t express cyclic data structures • E.g., can’t build a doubly linked list: Can’t get mutable reference a b c d to c to add the link to d, since already referenced by b • Many languages offer an escape hatch from the ownership rules to allow these data structures (e.g., raw pointers and unsafe in Rust) • Can’t express shared ownership of mutable data • Usually a good thing, since avoids race conditions • Rust has RefCell that dynamically enforces the borrowing rules (i.e., allows upgrading a shared reference to an immutable object into a unique reference to a mutable object, if it was the only such shared reference) • Raises a run-time exception if there could be a race condition, rather than preventing it at compile time

16 Limitations of Region-based Systems

• Forces programmer to consider object ownership early and explicitly • Generally good practice, but increases conceptual load early in design process – may hinder exploratory programming

17 Summary

• Region-based memory management with strong Region-Based Memory Management in Cyclone ∗

Dan Grossman Greg Morrisett Trevor Jim† ownership and borrowing rules Michael Hicks Yanling Wang James Cheney

Computer Science Department †AT&T Labs Research Cornell University 180 Park Avenue Ithaca, NY 14853 Florham Park, NJ 07932 danieljg,jgm,mhicks,wangyl,jcheney @cs.cornell.edu [email protected] • Efficient and predictable behaviour { } ABSTRACT control over data representation (e.g., field layout) and re- Cyclone is a type-safe programming language derived from source management (e.g., memory management). The de C. The primary design goal of Cyclone is to let program- facto language for coding such systems is C. However, in Strong correctness guarantees prevent many common bugs mers control data representation and memory management providing low-level control, C admits a wide class of danger- • without sacrificing type-safety. In this paper, we focus on ous — and extremely common — safety violations, such as the region-based memory management of Cyclone and its incorrect type casts, buffer overruns, dangling-pointer deref- static typing discipline. The design incorporates several ad- erences, and space leaks. As a result, building large systems vancements, including support for region subtyping and a in C, especially ones including third-party extensions, is per- coherent integration with stack allocation and a garbage col- ilous. Higher-level, type-safe languages avoid these draw- Constrains the type of programs that can be written backs, but in so doing, they often fail to give programmers lector. To support separate compilation, Cyclone requires • the control needed in low-level systems. Moreover, porting programmers to write some explicit region annotations, but acombinationofdefaultannotations,localtypeinference, or extending legacy code is often prohibitively expensive. and a novel treatment of region effects reduces this burden. Therefore, a safe language at the C level of abstraction, with As a result, we integrate C idioms in a region-based frame- an easy porting path, would be an attractive option. work. In our experience, porting legacy C to Cyclone has Toward this end, we have developed Cyclone [6, 19], a required altering about 8% of the code; of the changes, only language designed to be very close to C, but also safe. We 6% (of the 8%) were region annotations. have written or ported over 110,000 lines of Cyclone code, including the Cyclone compiler, an extensive library, lexer and parser generators, compression utilities, device drivers, Categories and Subject Descriptors amultimediadistributionoverlaynetwork,awebserver, D.3.3 [Programming Languages]: Language Constructs and many smaller benchmarks. In the process, we identified Further reading: and Features—dynamic storage management many common C idioms that are usually safe, but which the Ctypesystemistooweaktoverify.Wethenaugmentedthe • language with modern features and types so that program- General Terms mers can still use the idioms, but have safety guarantees. Languages For example, to reduce the need for type casts, Cyclone has features like parametric polymorphism, subtyping, and D. Grossman, G. Morrisett, T. Jim, M. Hicks, Y. Wang, and J. 1. INTRODUCTION tagged unions. To prevent bounds violations without mak- ing hidden data-representation changes, Cyclone has a va- • Many software systems, including operating systems, de- riety of pointer types with different compile-time invariants vice drivers, file servers, and databases require fine-grained and associated run-time checks. Other projects aimed at making legacy C code safe have addressed these issues with Cheney, “Region-based memory management in Cyclone”, ∗This research was supported in part by Sloan grant BR- 3734; NSF grant 9875536; AFOSR grants F49620-00-1- somewhat different approaches, as discussed in Section 7. 0198, F49620-01-1-0298, F49620-00-1-0209, and F49620-01- In this paper, we focus on the most novel aspect of Cy- 1-0312; ONR grant N00014-01-1-0968; and NSF Graduate clone: its system for preventing dangling-pointer derefer- Proc. ACM Conference on Programming Language Design Fellowships. Any opinions, findings, and conclusions or rec- ences and space leaks. The design addresses several seem- ommendations expressed in this publication are those of the ingly conflicting goals. Specifically, the system is: authors and do not reflect the views of these agencies. Sound: Programs never dereference dangling pointers. and Implementation, Berlin, Germany, June 2002. • Static: Dereferencing a dangling pointer is a compile- • Permission to make digital or hard copies of all or part of this work for time error. No run-time checks are needed to deter- personal or classroom use is granted without fee provided that copies are mine if memory has been deallocated. DOI: 10.1145/512529.512563 not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to Convenient: We minimize the need for explicit pro- republish, to post on servers or to redistribute to lists, requires prior specific • grammer annotations while supporting many C id- permission and/or a fee. PLDI’02, June 17-19, 2002, Berlin, Germany. ioms. In particular, many uses of the addresses of local • You’re not expected to read/understand section 4 Copyright 2002 ACM 1-58113-463-0/02/0006 ...$5.00. variables require no modification. • Will be discussed in tutorial 5

18