CRAMM: Virtual Memory Support for Garbage-Collected Applications Ting Yang Emery D. Berger Scott F. Kaplan† J. Eliot B. Moss [email protected] [email protected] [email protected] [email protected] Dept. of Computer Science †Dept. of Mathematics and Computer Science University of Massachusetts Amherst Amherst College Amherst, MA 01003-9264 Amherst, MA 01002-5000 Abstract like Python and Ruby. Garbage collection’s popularity de- Existing virtual memory systems were designed to sup- rives from its many software engineering advantages over port applications written in C and C++, but do not pro- manual memory management, including eliminating dan- vide adequate support for garbage-collected applications. gling pointer errors and drastically reducing memory leaks. The performance of garbage-collected applications is ex- Garbage-collection application performance is highly tremely sensitive to heap size. Larger heaps reduce the sensitive to heap size. A smaller heap reduces the amount frequency of garbage collections, making them run several of memory referenced, but requires frequent garbage col- times faster. However, if the heap is too large to fit in avail- lections that hurt performance. A larger heap reduces the able RAM, garbage collection activity can trigger thrash- frequency of collections, thus improving performance by ing. Existing Java virtual machines attempt to adjust their up to 10x. However, if the heap cannot fit in available application heap sizes adaptively to fit in RAM, but suffer RAM, performance drops off suddenly and sharply. This performance degradations of up to 94% when subjected to is because garbage collection has a large working set (it bursts of memory pressure. touches the entire heap) and thus can trigger catastrophic We present CRAMM (Cooperative Robust Automatic page swapping that degrades performance and increases Memory Management), a system that solves these prob- collection pauses by orders of magnitude [18]. Hence, heap lems. CRAMM consists of two parts: (1) a new virtual size and main memory allocation need to be coordinated memory system that collects detailed reference informa- to achieve good performance. Unfortunately, current VM tion for (2) an analytical model tailored to the underly- systems do not provide sufficient support for this coordina- ing garbage collection algorithm. The CRAMM virtual tion, and thus do not support garbage-collected applications memory manager tracks recent reference behavior with low well. overhead. The CRAMM heap sizing model uses this infor- Choosing the appropriate heap size for a garbage- mation to compute a heap size that maximizes throughput while minimizing paging. We present extensive empirical results demonstrating CRAMM’s ability to maintain high JRockit -- pseudojbb performance in the face of changing application and sys- No Memory Pressure 1.2 tem load. Dynamic Memory Pressure 13.055 trans/ms 1 1 Introduction 0.8 The virtual memory (VM) systems in today’s operating systems were designed to support applications written in 0.6 the widely-used programming languages of the 80’s and 0.4 90’s, C and C++. To maximize the performance of these Normalized Throughput 0.2 applications, it is enough to fit their working sets in physi- 0.777 trans/ms 0.722 trans/ms 0 cal memory [16]. VM systems typically manage available 0 0.5 1 1.5 2 2.5 memory with an approximation of LRU [12, 13, 15, 16, 22], Normalized Elapsed Time which works reasonably well for legacy applications. However, garbage-collected languages are now increas- Figure 1: Impact of bursts of memory pressure on the per- ingly prevalent. These languages range from general- formance on the JRockit Java virtual machine, which de- purpose languages like Java and C# to scripting languages grades throughput by as much as 94%. collected application—one that is large enough to maxi- Java VirtualM achine (JVM ) mize throughput but small enough to avoid paging—is a G arbage Collector key performance challenge. The ideal heap size is one that makes the working set of garbage collection just fit Heap Size W orking Set within the process’s main memory allocation. However, an M anager Size M odel a priori best choice is impossible in multiprogrammed en- e y l e r b p g S o a a vironments where the amount of main memory allocated n l S M inorfault i Allowable M ajor m e a a e W h H overhead target v Faultoverhead c to each process constantly changes. Existing garbage- M A collected languages either ignore this problem, allowing M ajorFault W SS Estim ator e only static heap sizes, or adapt the heap size dynamically z i CostM onitor S l t using mechanisms that are only moderately effective. For o s i r M inorFault t L n e example, Figure 1 shows the effect of dynamic memory o CostM onitor v i C t pressure on an industrial-strength Java virtual machine, c a Page Fault n BEA’s JRockit [7], running a variant of the SPECjbb2000 I Handler Histogram benchmark. The solid lines depict program execution when the process fits in available RAM, while the dashed lines VirtualM em ory M anager(VM ) show execution under periodic bursts of memory pressure. This memory pressure dilates overall execution time by a Figure 2: The CRAMM system. The CRAMM VM system factor of 220%, and degrades performance by up to 94%. efficiently gathers detailed per-process reference informa- The problem with these adaptive approaches is not that tion, allowing the CRAMM heap size model to choose an their adaptivity mechanism is broken, but rather that they optimal heap size dynamically. are reactive. The only way these systems can detect whether the heap size is too large is to grow the heap un- til paging occurs, which leads to unacceptable performance CRAMM, including experimental measurements across 20 degradation. benchmarks and 5 garbage collectors, as well as compari- Contributions: This paper makes the following con- son to two industrial Java implementations. These results tributions. It presents CRAMM (Cooperative Robust Au- demonstrate CRAMM’s effectiveness in maintaining high tomatic Memory Management), a system that enables performance in the face of changes in application behavior garbage-collected applications to predict an appropriate and system load. heap size, allowing the system to maintain high perfor- In addition to serving the needs of garbage-collected mance while adjusting dynamically to changing memory applications, the CRAMM VM system is the first sys- pressure. tem to our knowledge to provide per-process and per-file CRAMM consists of two parts; Figure 2 presents an page management while efficiently gathering detailed ref- overview. The first part is the CRAMM VM system that erence histograms. This information can be used to im- dynamically gathers the working set size (WSS) of each pro- plement a wide range of recently-proposed memory man- cess, where we define the WSS as the main memory allo- agement systems, including compressed page caches [27], cation that yields a trivial amount of page swapping. To adaptive LRU mechanisms like EELRU [25], and informed accomplish this, the VM system maintains separate page prefetchers [20, 24]. lists for each process and computes an LRU reference his- The remainder of this paper is organized as follows. Sec- togram [25, 27] that captures detailed reference informa- tion 2 presents an overview of garbage collection algo- tion while incurring little overhead (around 1%). The sec- rithms and terminology used in this paper. Section 3 de- ond part of CRAMM is its heap sizing model, which con- rives the CRAMM heap sizing model, which relates appli- trols application heap size and is independent of any par- cation working set size to heap size. Section 4 describes ticular garbage collection algorithm. The CRAMM model the CRAMM VM system, which gathers detailed statistics correlates the WSS measured by the CRAMM VM to the allowing it to compute the precise current process work- current heap size. It then uses this correlation to select a ing set size. Section 5 presents empirical results, compar- new heap size that is as large as possible (thus maximizing ing static and previous adaptive approaches to CRAMM. throughput) while yielding little or no page faulting behav- Section 6 presents work most closely related to ours, and ior. We apply the CRAMM model to five different garbage Section 7 concludes. collection algorithms, demonstrating its generality. We have implemented the CRAMM VM system in 2 GC Behavior and Terminology the Linux kernel and the CRAMM heap sizing model in A garbage collector (GC) periodically and automatically the Jikes RVM research Java virtual machine [3]. We finds and reclaims heap-allocated objects that a program present the results of an extensive empirical evaluation of can no longer possibly use. We now sketch how, and 2 when, a GC may do this work, and along the way intro- MarkSweep -- SPEC _213_javac duce GC terminology and concepts critical to understand- 50 300 ing CRAMM. 45 250 Garbage collectors operate on the principle that if an 40 35 object is unreachable via any chain of pointers starting 200 from roots—pointers found in global/static variables and 30 on thread stacks—then the program cannot possibly use 25 150 the object in the future, and the collector can reclaim and 20 100 reuse the object’s space. Through a slight abuse of termi- 15 Working Set Size (MB) Execution Time (second) 10 nology, reachable objects are often called live and unreach- 50 5 Execution Time able ones dead. Reference counting collectors determine Working Set Size (95%) 0 0 (conservatively) that an object is unreachable when there 0 50 100 150 200 250 are no longer any pointers to it. Here, we focus primarily Heap Size (MB) on tracing collectors, which actually trace through pointer chains from roots, visiting reachable objects.