Detecting Cacheable Data to Remove Bloat
Total Page:16
File Type:pdf, Size:1020Kb
Cachetor: Detecting Cacheable Data to Remove Bloat Khanh Nguyen and Guoqing Xu University of California, Irvine, CA, USA {khanhtn1, guoqingx}@ics.uci.edu ABSTRACT 1. INTRODUCTION Modern object-oriented software commonly suffers from runtime Many applications suffer from chronic runtime bloat—excessive bloat that significantly affects its performance and scalability. Stud- memory usage and run-time work to accomplish simple tasks— ies have shown that one important pattern of bloat is the work re- that significantly affects scalability and performance. Our experi- peatedly done to compute the same data values. Very often the ence with dozens of large-scale, real-world applications [33, 35, cost of computation is very high and it is thus beneficial to mem- 36, 37] shows that a very important source of runtime bloat is the oize the invariant data values for later use. While this is a com- work repeatedly done to compute identical data values—if the com- mon practice in real-world development, manually finding invariant putation is expensive, significant performance improvement can data values is a daunting task during development and tuning. To be achieved by memoizing1 these values and avoiding computing help the developers quickly find such optimization opportunities for them many times. In fact, caching important data (instead of re- performance improvement, we propose a novel run-time profiling computing them) is already a well-known programming practice. tool, called Cachetor, which uses a combination of dynamic depen- For example, in the white paper “WebSphere Application Server dence profiling and value profiling to identify and report operations Development Best Practices for Performance and Scalability” [3], that keep generating identical data values. The major challenge in four of the eighteen best practices are instructions to avoid repeated the design of Cachetor is that both dependence and value profil- creation of identical objects. While finding and caching identical ing are extremely expensive techniques that cannot scale to large, data values is critical to the performance of many large-scale soft- real-world applications for which optimizations are important. To ware systems, the task is notoriously challenging for programmers overcome this challenge, we propose a series of novel abstractions to achieve during development and tuning. A large, long-running that are applied to run-time instruction instances during profiling, program may contain millions of instructions, and each instruction yielding significantly improved analysis time and scalability. We may be executed for an extremely large number of times and pro- have implemented Cachetor in Jikes Research Virtual Machine and duce a sea of data values. It would be extremely difficult, if not evaluated it on a set of 14 large Java applications. Our experimen- impossible, to find identical run-time data values and understand tal results suggest that Cachetor is effective in exposing caching how to cache them without appropriate tool support. opportunities and substantial performance gains can be achieved Motivation To illustrate, consider the following code example, by modifying a program to cache the reported data. adapted from sunflow 2, an open-source image rendering system. float[] fValues = {0, 1.0, 2.3, 1.0, 1.0, 3.4, 1.0, 1.0, Categories and Subject Descriptors ..., 1.0}; D.2.4 [Software Engineering]: Metrics—Performance measures; int[] iValues = new int[fValues.length] ; D.2.5 [Software Engineering]: Testing and Debugging—Debug- for (int i = 0; i < fValues.length; i++){ ging aids iValues[i] = Float.floatToIntBits(fValues[i]); } General Terms This simple program encodes each float value in array fValues Performance, Reliability, Experimentation using a bit array (represented by an Integer), which can then be stored in an Integer array. In this example, most of the values in fValues are 1.0, and it is unnecessary to invoke method Float. Keywords floatToIntBits (which is quite expensive) to compute the bit Runtime bloat, performance optimization, cacheable data, dynamic array for each of them. The program would run more efficiently dependence analysis if Float.floatToIntBits can be invoked the first time 1.0 is seen , and the result can be cached and reused for its future occurrences. However, this information may not be available to Permission to make digital or hard copies of all or part of this work for personal or the programmer during development, as fValues may be a dynam- classroom use is granted without fee provided that copies are not made or distributed Permission to make digital or hard copies of all or part of this work for ically computed array whose content is unknown at compile time, forpersonal profit or or commercial classroom advantage use is granted and that without copies bear fee this provided notice and that the copies full citation are on the first page. Copyrights for components of this work owned by others than ACM or the fact that most of its elements are the same is specific to a mustnot made be honored. or distributed Abstracting for with profit credit or is commercial permitted. To advantage copy otherwise, and that or republish, copies tobear post this onnotice serversand or to the redistribute full citation to lists, on requires the first prior page. specific To cop permissiony otherwise, and/or to a certain kind of input image being processed. As a result, it is neces- fee.republish, Request to permissions post on servers from [email protected]. or to redistribute to lists, requires prior specific sary to develop techniques and tools that can help the programmer permission and/or a fee. ESEC/FSE’13, August 18–26, 2013, Saint Petersburg, Russia 1Terms “memoize” and “cache” are used interchangeably. CopyrightESEC/FSE 2013’13 August ACM 978-1-4503-2237-9/13/08...$15.00 18-26,2013 Saint Petersburg, Russia 2 http://dx.doi.org/10.1145/2491411.2491416Copyright 13 ACM 978-1-4503-2237-9/13/08 ...$5.00. http://sunflow.sourceforge.net/. 268 Technical challenges and our solution The biggest challenge that stands in the way of implementing Cachetor is how to find these identical data values in a scalable way so that Cachetor can be applied to large, real-world applications. In particular, the im- plementation of I-Cachetor requires the comparison of run-time values produced by different executions of the same instruction. To do this, a natural idea is to perform whole-program value pro- filing [10], which records run-time values for all instruction exe- cutions. These values are compared offline to identify cacheable instructions. In addition, in order to compute data-structure-level CMs, a dynamic dependence analysis may be needed to understand which data structure an instruction may write into at run time. How- ever, both whole-program value profiling and dependence analysis Figure 1: An overview of Cachetor. are extremely expensive techniques that cannot scale to real-world applications. To improve the practicality of our analysis, we pro- find such missed optimization opportunities (e.g., report method pose a novel approach that combines value profiling with dynamic floatToIntBits is frequently executed with the same input dependence analysis in a way so that they mutually improve the and produces the same output), especially in a situation where a scalability of each other. Specifically, we use distinct values pro- significant performance issue is observed and tuning must be un- duced at run time to abstract dependence graph nodes, yielding a dertaken to make the application reach its performance goal. value-abstracted dynamic dependence graph. This graph contains Our proposal In this paper, we propose a dynamic analysis the value information necessary for our analysis, and yet is much tool, called Cachetor, that profiles large-scale applications to pin- smaller and easier to compute than a regular dynamic dependence point cacheable data values and operations producing these val- graph. We then propose a series of further abstractions based on ues. Cachetor has three major components, which are illustrated this dependence analysis to scale Cachetor to the real world. in Figure 1. At the lowest level of the tool is an instruction-level We have implemented this combined dependence and value pro- cacheable value detector called I-Cachetor, which identifies (byte- filing technique in Jikes Research Virtual Machine [16], and then code) instructions whose executions produce identical (primitive- built the three detectors based on the abstract representation. An typed) values. Finding only instructions that always produce the evaluation of the tool on a set of 14 large-scale Java applications same values may significantly limit the amount of optimization op- shows that Cachetor incurs an overall 201.96× running time over- portunities that can be detected. To improve usefulness, we pro- head and 1.98× space overhead. While these overheads are very pose to compute a cacheability measurement for each instruction, large, they have not prevented us from collecting any run-time data that captures the percentage of the most frequently-occurring value for real-world applications. In fact, it could have been impossible among all values produced by the instruction. For example, al- to implement such a heavyweight analysis on real-world applica- though the call instruction that invokes floatToIntBits does tions (such as those in the DaCapo benchmark set [7]) without the not always return the same value, this instruction has a high CM proposed abstractions. We have carefully inspected