Determining When Objects Die to Improve

Garbage Collection

A dissertation submitted by

Nathan P. Ricci

In partial fulfillment of the requirements

for the degree of Doctor of Philosophy in

Computer Science

TUFTS UNIVERSITY

May 2016

ADVISER: Samuel Z. Guyer ii 0.1 Abstract

Although garbage collection frees programmers from the burden of manually free- ing objects that will no longer be used, and eliminates many common programming errors, it can come with a significant performance penalty. This penalty can become particularly great when memory is constrained.

This thesis introduces two pieces of work to address this problem. The first is

Elephant Tracks, a tool that uses novel techniques to measure memory properties of Java programs Elephant Tracks is novel in that it places object death events more accurately than any existing tool, and is able to do so with out modification to the underlying VM.

The second is the Deferred Garbage collector. Built based on observations from the Elephant Tracks traces, the Deferred Garbage Collector reduces redundant work performed by the garbage collector In this thesis, we show that the techniques used by the Deferred Collector can reduce garbage collector tracing workload in some programs.

iii 0.2 Acknowledgements

I would like to thank the Tufts University Department of Computer Science for building the environment that makes this work possible. I am especially grateful for the patient mentoring of my advisor, Sam Guyer. I also wish to express gratitude to my comitee members; Norman Ramsey, Alva Couch, Mark Hempstead, and Tony

Printezis.

Additionally, I would like to thank my wife Gizem, and my parents Susan, and

David, without whose support I never would have made it here.

I would also be remiss if I did not thank the Tufts Department of Athletics, for providing barbells, without which I would have long ago gone mad.

Lastly, I must also thank Tufts University and the National Science Foundation for their financial support.

iv Contents

0.1 Abstract ...... iii

0.2 Acknowledgements ...... iv

1 Introduction 1

2 Garbage Collection Background 3

2.1 On Object Lifetime ...... 3

2.2 Garbage Collector Overhead ...... 4

2.2.1 Mark Sweep ...... 6

2.2.2 Generational Mark Sweep ...... 7

2.3 Weak References ...... 8

2.4 The Java Virtual Machine Tool Interface ...... 9

3 Elephant Tracks 11

3.1 Elephant Tracks Introduction ...... 11

3.2 Trace time ...... 13

3.3 Background and related work ...... 15

3.3.1 Garbage collection tracing ...... 16

3.3.2 Why a new trace generator? ...... 19

3.3.3 Related work ...... 21

3.4 Elephant Tracks Design ...... 23

3.4.1 Kinds of trace records ...... 23

v 3.4.2 Execution model ...... 25

3.5 Implementation ...... 29

3.5.1 Timestamping strategy ...... 30

3.5.2 The instrumenter ...... 31

3.5.3 The agent ...... 36

3.5.4 Properties of our implementation approach ...... 39

3.6 Results ...... 40

3.6.1 Performance ...... 40

3.6.2 Trace analysis ...... 42

3.7 Conclusions ...... 43

4 Deferred Collector 49

4.1 Finding Key Objects ...... 50

4.2 Deferred Collector Design ...... 54

4.2.1 Defer all Object Reachable from the Key ...... 54

4.2.2 Large Object Space ...... 56

4.3 Mitigating Bad Hints ...... 57

4.3.1 Application Programming Interface ...... 58

4.4 JikesRVM ...... 58

4.4.1 Sanity Checker ...... 60

4.5 Experimental Results for The Deferred Collector ...... 61

4.5.1 Methodology ...... 61

4.5.2 Doubly Linked List ...... 62

4.5.3 sunflow ...... 63

4.6 Related Work ...... 66

5 Conclusion 68

vi Bibliography 68

vii List of Figures

2.1 The events in the life of an object. First it is allocated, then it is

used. Eventually, it is used for the last time. Sometime after this, it

may become unreachable. Finally its memory will be reclaimed. . .4

2.3 A weak reference object has a field called a ”referent” that refers

to some object (A). When the object referred to by the referent ob-

ject is reachable only through weak reference objects, it may be

collected. If this happens, the referent field is nulled (B)...... 9

3.1 Pseudo code for the merlin algorithm...... 18

4.1 The live size of each of the dacapo bench benchmaks with respect

to time. The blue line shows the number of objects live over time.

The lower green dashed line shows the number of live objects that

fall into clusters as described in 4.1 ...... 51

4.1 The live size of each of the dacapo bench benchmaks with respect

to time. The blue line shows the number of objects live over time.

The lower green dashed line shows the number of live objects that

fall into clusters and described in 4.1 ...... 52

viii 4.1 The live size of each of the dacapo bench benchmaks with respect

to time. The blue line shows the number of objects live over time.

The lower green dashed line shows the number of live objects that

fall into clusters and described in 4.1 ...... 53

4.2 mark/cons ratio for the doubly linked list program, run with a 16

MB heap, and differing values of immediate collection period. . . . 63

4.3 mark/cons ratio for sunflow, run with a 27 MB heap, and differing

values of immediate collection period...... 65

4.4 mark/cons ratio for sunflow, run with a 32 MB heap, and differing

values of immediate collection period...... 66

ix Chapter 1

Introduction

Since the task of allocating and freeing memory in computer programs is tedious and error prone, automated memory management (known as garbage collection) has been a boon to computer programmers. However, automated memory manage- ment comes with costs; extra memory overhead or time.

The hypothesis of this thesis is that a greater understanding of the structure of a program’s heap can improve the performance of garbage collectors. Intuitively, this seems obvious; the data structures used in a program govern in large part how the memory of a program is used, and understanding those data structures should give us more information about the lifetime of objects.

However, in order to examine this intuitive idea, we will need to develop new techniques to observe how programs use memory. In particular, we want to know precisely when objects become unreachable and may be collected. Thus, this thesis is two parts. In the first half, we present Elephant Tracks, a tool which uses novel techniques to trace the execution of Java programs. The traces produced by Ele- phant Tracks contain a record of all object allocation and death events, and enough information to place these events in a specific calling context (a novel feature of Ele- phant Tracks). These traces can be used to analyze runtime properties of programs,

1 including prototyping of new garbage collection algorithms. Since they place death so precisely, it is easy test schemes that require less precision by ignoring some of the information in the trace. A central claim of this thesis is that that this can be done with out relying on any modification to the underlying VM.

In the second half of this thesis, we present the deferred collector, built based on data gather with Elephant Tracks. The deferred collector exploits the observation that much of the heap does not change between two successive runs of a tracing- style garbage collector, and as a result there is much work repeated between such successive runs. The deferred collector reduces this repeated work based on pro- grammer hints. We claim that this can reduce garbage collector tracing workload in some programs.

2 Chapter 2

Garbage Collection Background

In order to understand the performance characteristics of garbage collectors, it is

necessary to understand how they work. Thus, we will describe their operation in

this section. As there is copious work on garbage collection, this chapter will only

focus on details relevant to this thesis.

2.1 On Object Lifetime

First, we must make a brief aside to explain the events of interest in an objects life-

time. We will borrow terminology first used by Rojemo¨ and Runciman (1996). A program allocates an object, and sometime later it is first used; the period between these events is called the lag. Eventually, the program uses an object for the last time, and this point we can say the object is dead. In principle, it would be safe for the garbage collector to reclaim an objects memory the very instant of its death.

However, in general it is not possible to determine whether a use is the last use.

So, instead the first point the collector could collect an object is later, when it be-

comes unreachable. The period between last use and the time an object becomes

unreachable is called use drag.

3 Lifetime of An Object

Figure 2.1: The events in the life of an object. First it is allocated, then it is used. Eventually, it is used for the last time. Sometime after this, it may become unreach- able. Finally its memory will be reclaimed.

Finally, although the garbage collector could collect an object as soon as the object becomes unreachable, in practice the collector runs only intermittently; the collector will not collect the object until its next run. This engenders another period of drag, the reachability drag.

2.2 Garbage Collector Overhead

How much overhead does garbage collection entail? While there have been many significant advances in garbage collection performance, overhead can still be very high when memory is constrained. To understand why, first consider how most typical garbage collection algorithms work: As the program requests new objects,

4 (a) Heap, memory usage below threshold (b) Heap full (c) Heap after collection

(d) Key: Blue objects are allocated and reach- able. Grey Objects are Al- located, but dead. White space is unallocated. memory for them is allocated on demand, until some threshold of allocated mem- ory, (H for ”heap size”), is reached. Then, the program is paused, and the garbage collector begins executing to reclaim memory. It begins by tracing all objects re- ferred to by the program roots, (the roots are just pointers into the heap from other memory locations, such as the stack). The garbage collector marks these initial ob- jects, and then finds all objects reachable from this set, and marks them, eventually marking all objects that are transitively reachable from the roots.

If the threshold H is larger than the total amount of memory the program allo- cates during its execution, H is never be reached, the program would never have to perform a garbage collection, and the cost of GC is zero. If instead H is a large fraction of the total amount of allocation, but less than the total amount of alloca- tion, the collector runs only infrequently. Imagine the collector has not run in some time , and then performs a new collection. Since most objects eventually die (that is, they are not immortal), and the cost of collection is proportional to the number of live objects, the collection is inexpensive. Furthermore, the collection will re- cover a large amount of memory, which means that it will be a long time before the

5 amount of allocated memory reaches H again. So, if H is large enough collections are infrequent and inexpensive.

In contrast, if H is small, then collections will occur frequently: Each collection will recover only a small amount of memory, leaving only a small amount to be allocated before H is reached again and a new collection must occur. In this case, we are in the unhappy world of frequent and expensive collections.

A direct comparison to manual memory management is not straight forward.

One could do as Hertz and Berger (2004) did and determine the exact unreachable time of every object in a Java program, and replay the program exactly, inserting manual frees at the exact moment of death, comparing that to the standard garbage collection scheme. However, the presence or absence of manual memory manage- ment changes how programs are written, and this experiment does not necessarily reflect the way that manual memory management is actually done. In practice, aux- iliary data structures may be used to track memory, defense copies may be made, or other techniques employed. If the data structures used to track memory are elab- orate enough, they could be seen as a partial implementation of a garbage collector.

However imperfect the analysis, (Hertz and Berger, 2004) concluded that when H is approximately four times the typical live-size of the program, the costs garbage collection are similar to the costs of manual memory management. When H is greater than this, garbage collection is more efficient than manual memory man- agement. However, if we move H in the opposite direction, the costs of garbage collection increases dramatically. With H only twice the live size, the execution of a program is slowed dramatically (Hertz and Berger, 2004).

2.2.1 Mark Sweep

One of the most basic algorithms for garbage collection is Mark Sweep. It proceeds as follows:

6 1. Once the threshold is reached, the mutator is paused

2. The collector enumerates all roots (references on the stack, or references to

global variables, that refer to objects in the heap)

3. The Mark Phase Begins. All objects reachable via the roots are marked, and

all objects transitively reachable from those objects are marked.

4. The Sweep Phase begins. The collector linearly scans through memory,

checking to see if objects are marked. All objects not marked are reclaimed

(”swept”).

5. The mutator resumes.

The marking can be done by modifying a bit in a side table, or the mark bit can be stored in memory adjacent to the object in a special header area. Since the sweeper is read-only with respect to live objects, and the mutator will never interact with unreachable objects, it is possible for the mutator to resume before the sweeper is complete; the mutator and sweeper then run concurrently.

2.2.2 Generational Mark Sweep

One important advance in garbage collector technology exploits an empirical ob- servation of object lifetimes known as the Generational Hypothesis: most objects die young.

In order to take advantage of this, new objects are allocated into a small space called the nursery. When the nursery is filled, survivors are copied to a larger space, called the mature space. This larger space is typically managed with the previously described mark sweep algorithm.

In order that the collector can collect the nursery without having to examine the mature space at all, it needs to keep track of any pointers located in the mature

7 space into that refer to objects in the nursery; such pointers are treated as roots during a nursery collection. In a JVM that relies on a just-in-time (JIT) compiler, this tracking is accomplished by the JIT compiler injecting code into the mutator surrounding any write of a pointer; this code is called the write barrier (a write barrier could also be implemented in an interpreter, or ahead of time compiler).

The write barrier code checks that this condition is met, and if so the address of that pointer is stored in a list called the remembered set.

Some generation collectors use an alternative called card marking instead of a simple list. In this scheme, one bit is associated with each ”card” of memory

(the size of a card is variable, but typically cards fit evenly onto pages, several per page). These bits are stored together in a single bit map called the card table. In the card marking scheme, instead of appending to a list, the write barrier sets the bit associated with the card in which a pointer is being mutated. Then, when a nursery garbage collection occurs, the garbage collector scans those cards that have been marked in the card table. After a collection, the card table is zeroed. Card marking has the advantage of bounded space, since the card table is of fixed size (and the list used in a remembered set is not). However, it has the disadvantage that the collector will have to scan an entire card even if only one pointer in it has been mutated.

2.3 Weak References

As with many other garbage collected languages, Java provides weak references. In

Java, these are implemented using a special container type, WeakReference. When a WeakReference contains an object, it can be accessed via the weak objects get method. Suppose an object is reachable only through WeakReference objects; such an object is said to be weakly reachable. When an object is determined to be weakly reachable, it will be collected. This is illustrated in fig. 2.3 Weak references are

8 Figure 2.3: A weak reference object has a field called a ”referent” that refers to some object (A). When the object referred to by the referent object is reachable only through weak reference objects, it may be collected. If this happens, the referent field is nulled (B). typically used to implement software caches, since it is not desired for the cache itself to keep objects resident in memory.

2.4 The Java Virtual Machine Tool Interface

Java Virtual Machines provide an standard interface for debugging tools called the

Java Virtual Machine Tool Interface (Sun Microsystems, 2004). This interface allows the JVM’s user to specify a shared library, called a JVMTI agent, to be loaded along with the VM. As classes are loaded by the JVM, their byte codes can be intercepted by the agent. The agent can then modify them before they are executed by the JVM.

JVMTI is a relatively sparse interface, and it is intended that agents primarily rely on byte code instrumentation to do most of the agents work. It does, however, provide some additional features beyond intercepting loaded classes. A JVMTI agent can traverse all the live objects in the heap, examine local variables on the stack, and intercept Java Native Interface calls.

9 In order to carry out the bytecode instrumentation used in the work presented in this thesis, we use version 3.3 of the ASM bytecode rewriting framework. ASM is widely used and actively maintained, and is part of the infrastructure of at least one major commercial JVM (Oracle’s Hotspot).

10 Chapter 3

Elephant Tracks

3.1 Elephant Tracks Introduction

Garbage collection tracing tools have been instrumental in the development of new garbage collection algorithms. A GC tracing tool produces an accurate trace of all the dynamic program events that are relevant to memory management, including allocations, pointer updates, and object deaths. We can quickly test a new GC algo- rithm by building a simulator that reads the GC trace, instead of developing a full

GC implementation in a real virtual machine, which is a considerable undertaking.

One of the widely used GC tracing tools for Java, GCTrace, is available as a component of the JikesRVM Java virtual machine (Alpern et al., 2005b). That tool, like ours, is based on the Merlin algorithm (Hertz et al., 2002, 2006), but suffers from several limitations. First, the implementation is integrated directly into the garbage collector. Due to the ongoing evolution of the JikesRVM Memory Man- agement Toolkit, it no longer functions with recent versions of JikesRVM, and older versions will not run modern Java software. Second, GCTrace measures time only in terms of bytes allocated, a fine metric for GC simulation, but not useful for pro- gram analysis since it cannot readily be tied back to points in the program. Third,

11 allocation time is not very precise for events other than allocation: many pointer

updates and object deaths can occur at various points between two allocations. Fi-

nally, the existing tool does not support a number of features found in real programs,

including weak references and multithreading.

In this chapter we present Elephant Tracks, a new GC tracing tool that is pre- cise, informative, and can run on top of any standard JVM. Our goal is not simply to address the limitations of prior work, but to provide new capabilities that allow our tool to support a wider variety of program analysis and runtime systems re- search. The implementation of Elephant Tracks uses a combination of bytecode instrumentation and JVMTI (JVM Tool Interface) callbacks to construct a graph representing the connectivity of the heap, on which it runs the Merlin algorithm to compute idealized object death times. Its attributes include:

Precise: Elephant Tracks measures time in terms of method calls (i.e., the clock

ticks at every method entry and exit), which are much more frequent than

allocations.1

Complete: The implementation properly handles all relevant events, including dif-

ficult cases, such as weak references, the Java Native Interface, sun.misc.Unsafe,

and JVM start up objects. Previous tools did not handle all of this features,

which could result in inaccurate traces if a program makes use of them.

Informative: Traces include much more than just GC-related events. We emit a

record for every method call and return, allowing us to tie memory behavior

back to the program structure. In fact, we can reconstruct the complete dy-

namic calling context for any time step. We also record information about

threads and exceptions, and, optionally, counts of heap reads and writes and

number of bytecodes executed. This ability is unique to Elephant Tracks.

1This includes constructor calls, thus tightly bounding most allocations as well.

12 Well-defined: We carefully define which aspects of the JVM execution model that

the traces capture , which embodies a number of subtle design issues that

affect the meaning of the traces. These include trace time, the definition of

object lifetime, and the ordering of events in multi-threaded programs. In this

thesis we explore these issues in detail and justify our choices.

Portable: Elephant Tracks is implemented as a JVMTI agent that runs on any com-

pliant Java virtual machine. Running with out modification to the underlying

Java Virtual Machine represents a research contribution of Elephant Tracks.

Fast: Elephant Tracks is as fast as existing tools, while providing much more in-

formation. Elephant Tracks includes performance tuning and optimizations

to reduce overhead, which is critical for larger, long-running programs. Pro-

grams instrumented with Elephant Tracks run hundreds of times slower than

uninstrumented programs, but that is comparable to existing tools, while pro-

viding much more information.

In the following sections we explore the design space of GC tracing tools, and explain the choices made for Elephant Tracks. We discuss the technical challenges of building such a tool using the JVMTI interface, which does not provide direct access to the JVM’s representation of Java Objects, or to the garbage collector. We also discuss the handling of weak references. This proved difficult because the JVM is able to side-step some of our instrumentation techniques in this case. Finally, we present some results, including overhead measurements, as well as new insights about the benchmarks gleaned from our precise traces.

3.2 Trace time

For Merlin-based tracing we need a notion of trace time, so that object death records, which are generated only at GC time, can be inserted in their proper place in the

13 trace. The choice of trace time has a profound affect on the implementation of the tool and on the resulting traces.

Real time is a poor choice, since it is dependent on many factors, including the virtual machine, the operating system, and the hardware. In addition, tracing tends to slow programs down significantly, so the real times are likely to be significantly different from uninstrumented runs. Real time is also, in some sense, too precise: we do not want the trace to reflect the time it takes to actually perform a timestamp or record a trace record.

The solution is to express time in terms of some program-level event: each time the event is encountered we tick the trace clock. In this way, time depends only on a property of the program, not on the VM or underlying machine. This model breaks time into discrete steps, each of which represents a small region of program execution.

The choice of which event(s) to use for the clock affects the granularity of time, which ultimately determines the precision of the trace, since trace records labeled with the same time cannot be ordered or localized within the region covered by that time step. The trade-off is that more fine-grained notions of time are more difficult to implement correctly, since we need to place the instrumentation more precisely to make sure that every event is labeled with the correct time. They may also incur more overhead. In the following subsections, we will contrast two approaches: allocation time, and method time.

Allocation Time

GCTrace measures time in terms of the number of bytes allocated since the program started (called allocation time). At each allocation, time advances by the number of bytes allocated. Allocation time is good for basic GC research, since the traces are precise enough to drive simulations of experimental GC algorithms. Allocation

14 time is fairly coarse, however, and a single time step can cover a large region of the code spanning multiple method calls. This notion of time was adequate for the GCTrace, since they were primarily interest in simulating garbage collection, and none of the collectors they were interested in simulating had any reliance on particular methods.

Method Time

Elephant Tracks measures time in terms of the number of method entries and exits executed (which we call method time). To get a sense of the difference in preci- sion, consider that across the DaCapo benchmarks there are, on average, 70 method entry/exit events between any two allocations (we present more measurements in

Section 3.6). Method time is almost a strict superset of allocation time, since every allocation of a scalar object also calls at least one constructor. The exception is array allocations (in Java, arrays do not have constructors), but in our experience these are not frequent enough to change the results significantly. Also, if a con- structor receives as an argument (not the receiver) a new object, there can be two allocations without an intervening constructor call. Again, this is not common. Ele- phant Tracks uses method time as it allows us to place events in a particular calling context.

3.3 Background and related work

In this section we describe the general garbage collection tracing problem and ex- isting solutions, and motivate the need for a new trace generator.

15 3.3.1 Garbage collection tracing

A GC trace is a record of the sequence of memory management events that oc- cur during a program’s execution. The events of interest may vary depending on the intended use of the trace, but typically include object allocation, object recla- mation, and mutations in the heap. Many of these events are straightforward to capture, such as object allocation, since they are explicitly invoked in the code.2

We can instrument those operations directly to emit a trace record with the relevant information.

The central challenge in GC tracing is determining object death times. An ob- vious solution is to emit an object death record when the garbage collector actually reclaims each object. This approach is easy to implement using JVMTI, but is un- appealing for at least two reasons. First, the particular timing of these events is collector-specific—we would measure a property of the GC algorithm used during trace generation, rather than a fundamental property of the program. Second, the resulting information is very imprecise. Most garbage collectors run infrequently, reclaiming large numbers of objects long after they are no longer needed by the program. As a consequence, object deaths appear far removed from the program events that actually cause them. This makes the traces poorly suited to evaluating new GC algorithms, as the Merlin work showed (Hertz et al., 2006).

Idealized death times

Our goal is to generate traces with idealized death records. That is, each object death appears in the trace at the earliest time at which the object could be reclaimed.

An idealized trace captures the behavior of a program independent of any particu- lar GC algorithm, with object death events appearing close to the program actions

2This is more subtle than you might think. In Java, the virtual machine may allocate application- visible objects as side-effects of other actions, such as class loading, and native libraries can also do so. Similar remarks apply to pointer updates.

16 that cause them. The exact nature of this problem depends on how we define “the

earliest time”. For example, we could compute death times based on object use

(see fig. 2.1): an object is dead immediately after its last use. While interesting as a

lower bound, this level of precision is potentially expensive to compute and cannot

be exploited by any practical memory manager. Therefore, we adopt the definition

used in garbage collection and in prior work on tracing: an object is dead when it

is no longer reachable from the roots (local and global variables) directly or indi- rectly through any sequence of pointers. This choice still leaves many subtle issues, however, including the granularity of trace time and the liveness of root variables, which are discussed in more detail in Section 3.4.

A naive algorithm for computing idealized death times is to run the garbage collector much more frequently. For example, we could produce a very precise trace by invoking the collector at every program point where an object could become dead. Not surprisingly, this approach is impractical.

The Merlin algorithm

The Merlin algorithm, introduced by: (Hertz et al., 2002) (reproduced here in

Figure 3.1 ), solves this problem by using timestamps to infer the idealized death

times of objects when they are reclaimed at regularly scheduled garbage collec-

tions. During normal execution the algorithm timestamps objects whenever they

lose an incoming pointer. At any point in time an object’s timestamp represents

the last time it was directly observed to be reachable. When the collector reclaims

an object, however, its timestamp is not necessarily its death time. In many cases

an object becomes unreachable indirectly, when an object that points to it becomes unreachable. In this case we need to determine which event occurred later: the di- rect loss of an incoming pointer (the timestamp), or the indirect loss of reachability

(the death times of the referring objects). So, the idealized death time of an object

17 void ComputeObjectDeathTimes() Time lastTime = sort unreachable objects from the earliest timestamp to the latest; push each unreachable object onto a stack from the earliest timestamp to the latest;

while (!stack.empty()) Object obj = stack.pop(); // pop obj with next earlier timestamp Time objTime = obj.timeStamp; if (objTime <= lastTime) // dont reprocess relabeled objects lastTime = objTime; for each (field in obj) if (isPointer(field) && obj.field != null) Object target = getMemoryWord(obj.field); Time targetTime = target.timeStamp; if (isUnreachable(target) && targetTime < lastTime) target.timeStamp = lastTime; stack.push(target);

Figure 3.1: Pseudo code for the merlin algorithm.

(Td(o)) is computed from its timestamp (Ts(o)) and the death times of any objects that point to it:

Td(o) = Max(Ts(o),{Td(p),∀p : p references o})

This insight leads to a practical approach for GC tracing that is also at central to the system we present in this chapter:

• During normal execution:

– Record ordinary events in the trace as they occur (e.g., object allocations

and pointer updates).

– Timestamp objects whenever they might become directly unreachable

(i.e., when they lose an incoming pointer).

• At GC time:

– Compute idealized death times using the formula above (implemented

18 roughly as a depth-first search on the graph of dead objects, pushing

computed death times across the pointers).

– Generate a death event record for each reclaimed object and insert it in

the proper place in the trace.

– Flush records to disk, and continue ...

An important implication of the Merlin algorithm is that it requires a notion of trace time for use in the timestamps. In fact all events in the trace must be associated with timestamps because the we cannot learn about death at the time it occurs— we discover the true death times of objects only at GC time, which is typically much later. The model of trace time (in particular, its granularity) has a profound impact on the implementation of the trace generator and the precision of the traces it generates.

3.3.2 Why a new trace generator?

The first realization of Merlin took the form of a customized garbage collector called GCTrace, implemented in JikesRVM. The main advantage of this approach is that the implementation can be integrated directly into the virtual machine code.

The compiler can be modified to add instrumentation to object allocations and pointer updates, and the garbage collector can be modified to perform the extra death time computation. GCTrace is a valuable tool. Its primary goal was to pro- vide a trace that allowed simulated garbage collections with great accuracy. We elaborate on some of its properties, and discuss where and why we want to improve on them, below.

Precise: GCTrace uses allocation time for its traces: the trace time clock “ticks”

at each object allocation. This is a good choice for its goals, as allocation is

what causes the garbage collector to eventually do work. However, it does

19 mean that object deaths and other events that occur in between allocations

cannot be ordered or precisely localized at any finer granularity.

A related problem is that allocation time does not correspond to anything

static in the program itself, so figuring out where events occur relative to the

code is very difficult (e.g., “In which method did the death of object 739229

occur?”). If a collector wants to take advantage of this information, for exam-

ple by doing something different in different phases of program execution (as

demarcated by methods), it would be difficult to prototype using GCTrace.

Completeness: GCTrace works with many Java programs, but since since it was

created Java programs have grown more sophisticated. Multithreading, use

of the Java Native Interface, and weak references are common; these were

not handled by GCTrace.

Integrated into VM GCTrace is implemented as a garbage collector within the

VM. This allows it to exploit internal structures of the VM for performance.

However, it does leave it exposed to changes in those structures. MMTk (the

memory management toolkit used in JikesRVM) has undergone a number of

radical refactorings, often leaving the GCTrace implementation out of date.

A major goal of our system therefore is to create a useful memory tracing tool

without modifying the JVM internally.

The calling context is potentially useful for a variety of optimizations. For

example Xu (2013) presents an optimization to re-use the memory of objects

if they die before the next time their allocation site is invoked. Although

Elephant Tracks is not fast enough to use on-line, it would allow researchers

to prototype such ideas by analyzing traces before investing time in complex

modifications to a JVM.

20 3.3.3 Related work

The work most closely related to ours is the original GCTrace implementation of the

Merlin algorithm (Hertz et al., 2006), which is discussed in detail throughout this chapter. Foucar reimplemented GCTrace using a shadow heap (see section 3.5.3) implemented in C++, like Elephant Tracks, but otherwise preserving the execution model and dependence on JikesRVM (Foucar, 2006).

Two prior papers explore the relationship between liveness and reachability for garbage collection. Agesen et al. (Agesen et al., 1998) examine the effects of apply- ing different levels of liveness analysis to the GC root set (variables on the stack).

They found that on average the differences were small, but on occasion static live- ness analysis would improve collection efficiency noticeably. This result suggests that our dynamic liveness model is reasonable for most purposes, but could be im- proved (see later discussion). Hirzel et al. (Hirzel et al., 2002) additionally consider the difference between reachability and last-use liveness of objects (see object life- time 2.1). They also find that schemes based on liveness of variables (i.e. employ- ing a compiler analysis to determine whether variables on the stack are no longer live, and therefore are not part of the GCs root set) have little impact on when a reachability-based garbage collector could collect objects. However, they do find that an objects last use time and the time it becomes unreachable are often signifi- cantly different. Elephant Tracks currently cannot compute equivalent information.

GC traces have been used to drive empirical studies of heap behavior, especially those examining the distribution and predictability of the lifetimes of objects (In- oue et al., 2006; Jones and Ryder, 2008). At a coarse level, measuring time in bytes allocated and measuring time with method events do not produce dramatically dif- ferent lifetime distributions. For analyses that are sensitive to program structure, however, this may be inadequate: many methods may be executed between any two allocations, and so the trace does not record accurately what event occurred in what

21 method. In addition, allocation time is not stable across runs of a program under different inputs.

Jones and Ryder (Jones and Ryder, 2008) show that the calling context of ob- ject allocation correlates well with lifetime i.e objects allocated in the same calling context tend to live the same amount of time. They could not determine, however, whether the calling context of object death correlates with lifetime, which might be a more useful fact for further improving garbage collection.

Inoue et al. (Inoue et al., 2006) look at what information is needed to precisely predict the lifetime of an object at the time of its allocation. They define a fully precise predictor as one that predicts lifetime to within a single quantum of time. By using bytes-of-allocation as their unit of time, they significantly reduce the coverage and accuracy of their predictors. The lifetime of an object in bytes-of-allocation time is much less stable than the calling context of its death, since the latter is directly related to its cause of death, while the former includes many irrelevant events (i.e., unrelated allocations).

Compile-time GC (Guyer et al., 2006) and connectivity-based garbage collec- tion (Hirzel et al., 2003) are two examples of techniques where knowing the pro- gram location at which an object dies is crucial. Such techniques are often evaluated using trace-driven simulation before embarking on a full implementation. Using

Elephant Tracks traces would yield a more accurate assessment of their potential.

Lambert et al. present a system for performing platform-independent JVM tim- ing (Lambert and Power, 2008). Although similar in spirit to our JVM-independent execution model, the focus of this work is on developing a model of code execution, rather than heap memory behavior.

Uhlig and Mudge (Uhlig and Mudge, 1997) present a survey of memory tracing techniques. While their focus is on tracing memory accesses for architecture and system research, they enumerate a set of features they consider desirable for tracing

22 systems in general: completeness (all relevant events are recorded), detail (events are associated with program-level information), low distortion (tracing does not change the program’s behavior), and portability. Elephant Tracks achieves many of these goals, although it significantly distorts actual running time, however, which is why we use a separate notion of time.

3.4 Elephant Tracks Design

Our goals in designing a new trace generator are to address the limitations of prior systems and to add new functionality to support new kinds of program analysis and memory management research. The central features of this design are (1) the kinds of program events recorded in a trace, and (2) the accuracy of this information with respect to some model of program execution. In this section we present the design of Elephant Tracks, and we discuss our choices in the context of the general GC tracing design space. In Section 3.5 we describe how this design is implemented.

3.4.1 Kinds of trace records

A minimal GC trace consists of just a sequence of object allocations and object deaths, labeled with the trace time and thread ID of each event. Without more information, though, such a trace has limited utility. In practice we add trace records for other kinds of relevant events to provide context for program analysis and to enable more kinds of trace-based simulations. For garbage collection research, for example, it is useful to add trace records for pointer updates in the heap, allowing a simulator to maintain an accurate heap model. Elephant Tracks can be configured to produce different kinds of trace records. We currently support the following kinds of records:

• Object allocations and object deaths (with idealized death times computed

23 using the Merlin algorithm).

• Pointer updates in the heap: These records include the source and target ob-

jects, as well as the object field or array index being updated. We also include

updates of static fields.

• Method entry and exit: These records allow trace times to be mapped to

specific methods, and even more precisely, to specific calling contexts.

• Exceptions: We augment method entry and exit to indicate when an exception

is thrown, the sequence of method calls (if any) that are terminated early

because of the exception, and the entry to a handler for the exception. The

main purpose of these events is to provide accurate information about method

execution context.

Separately from the trace, Elephant Tracks also emits information about each class loaded, each field declared in the class, each method declared in the class, and each allocation site in each method. This information is referred to by the trace, e.g., the trace will mention a unique allocation site number, which can be found in the side description file.

We currently do not generate trace records for object timestamps or for gen- eral memory accesses (including stack reads and writes). This information would enable an even wider range of applications, such as cache simulations. These events are extremely frequent, however, and would result in overwhelmingly large traces. In addition, instrumenting every single variable access would be technically challenging—bytecode rewriting might not be the best approach for this level of detail.

24 3.4.2 Execution model

Ideally, we would like to generate perfect traces, in which every event is recorded with a perfectly accurate and precise time. But this goal raises a critical question: accurate with respect to what? That is, what aspects of the execution model that we want the trace to represent? Elephant Tracks, like other trace generators, re- lies on a host virtual machine to execute the target program. It runs alongside the

VM, recording relevant events. The problem is that the timing of some events is highly VM dependent—directly recording these events as they occur produces a trace reflective of arbitrary VM implementation choices. Instead, we want to gener- ate traces that abstract away some details of the VM’s execution model, and record events in their own well-defined, less VM-specific, aspects. In particular, we would like a trace that places objects at the first time when any collector could collect them

(at the point where the object becomes unreachable, see figure 2.1). The possible models range widely, with some elements closer to the VM (essentially profiling the VM), and other elements more abstract, capturing an idealized execution of the program.

The main aspects of the execution model we wish to capture are (1) the defini- tion of object lifetime (in particular, when are objects considered dead), and (2) the definition of trace time (i.e., when does the trace time clock “tick” and with what frequency). The overall goal of the Elephant Tracks is to define these components in such a way that events can be localized precisely within the structure of the code.

The model is idealized for object lifetimes, but resorts to VM timing in cases where an idealized model is not possible, such as the interleaving of concurrent threads and the clearing of weak references.

One potential approach is to use non-deferred reference counting, which re- claims objects as soon as their reference counts becomes zero. Like reference count- ing collection, however, this approach cannot directly detect the death of cycles of

25 objects, and would require frequent tracing collections to achieve high precision.

Therefore, we do not use this approach.

Defining object lifetime

Object lifetimes are delineated by allocation and death events. Most object alloca-

tions are explicit in the program, so simply recording them as they occur produces

a VM-independent trace. We have found, however, that there are several other

sources of allocations, including VM internal allocations (e.g., String constants in

class files and Class objects themselves), objects allocated by the VM before it can

even turn instrumentation on, and objects allocated by JNI calls. We capture all of

these, but cannot associate them with a usual allocation site in the application, and

for those allocated very early in the run, we cannot provide relative time or context

of allocation.

For object deaths, however, an explicit goal of GC tracing is to compute ide-

alized death times. Both Elephant Tracks and GCTrace adopt the standard GC

definition: an object is dead when it is no longer reachable from the roots (local and

global variables). Even within this seemingly narrow definition, however, there are

a range of possible models. To see why, consider the program events that can cause

an object to become unreachable:

• The program overwrites a pointer in the heap (putfield, etc.)

• The program overwrites a static (global) reference (putstatic)

• A local reference variable goes out of scope

• The program changes the value of a local reference variable

• A weak reference is cleared by the garbage collector

While the first two (heap and global writes) are straightforward to instrument, local variables and weak references are more difficult to pin down. Furthermore, there

26 are roots inside the VM that we cannot observe and that the VM does not neces-

sarily inform us about when they change. Fortunately these are mostly “immortal”

references, such as to class objects, or relate to constants constructed from class

files (these may come and go).

Local variables

Tracing local variables presents many design choices and challenges. The key

question is: at what point is a local pointer variable dead, and therefore no longer

keeping the target object alive? At one end of the spectrum we could consider local

variables live throughout the method invocation with which they are associated. In

practice, however, most virtual machines apply some form of static liveness analysis

to compute more precise lifetimes. The virtual machine uses this information to

construct GC maps, which tell the garbage collector which variables to consider as

GC roots at a given point in the method.

GCTrace uses the GC maps in JikesRVM to determine which variables are live.

The advantage of this approach is that it is straightforward to implement. The down-

side is that the timing of the object death records depends on the specific liveness

analysis algorithm and choice of GC points made in JikesRVM.

Elephant Tracks currently uses a form of dynamic liveness to determine the life-

times of local variables. This choice reflects implementation decisions (described

in more detail below). A variable is considered dead after its last dynamic use. We

define a use as one of the following: (1) a direct dereference (access to an object or

array), (2) a type test, such as instanceof, (3) obtaining an array’s length, (4) use as a receiver of a dynamic method dispatch, or (5) a reference test, such as ifnull.

Dynamic liveness, however, is more precise than static liveness analysis, pri-

marily because it is not conservative about liveness on different execution paths.

The resulting traces show some object death times earlier than any real garbage

collector could achieve. For example, a reference that is held while a series of

27 methods are invoked, but never used or passed to any method, is considered dead in all the methods. We consider a variable live if it is passed to a method call as a parameter, or returned, even if it is never used with in those methods, or the return value is not used.

Weak references

Weak References are a special class in Java used to manage the collection of objects. An object reachable only via weak references can be collected at any time.

Listing 3.1: ”Weak reference usage”

WeakReference