Appears in the Proceedings of the ACM Symposium on Operating System Principles (SOSP '97), Saint-Malo, France, October 1997 HAC: Hybrid Adaptive Caching for Distributed Storage Systems Miguel Castro Atul Adya Barbara Liskov Andrew C. Myers MIT Laboratory for Computer Science, 545 Technology Square, Cambridge, MA 02139 castro,adya,liskov,andru ¡ @lcs.mit.edu Abstract These systems cache recently used information at client ma- chines to provide low access latency and good scalability. This paper presents HAC, a novel technique for managing the This paper presents a new hybrid adaptive caching tech- client cache in a distributed, persistent object storage sys- nique, HAC, which combines page and object caching to tem. HAC is a hybrid between page and object caching that reduce the miss rate in client caches dramatically. The combines the virtues of both while avoiding their disadvan- approach provides improved performance over earlier ap- tages. It achieves the low miss penalties of a page-caching proaches Ð by more than an order of magnitude on com- system, but is able to perform well even when locality is poor, mon workloads. HAC was developed for use in a persistent since it can discard pages while retaining their hot objects. object store but could be applied more generally, if provided It realizes the potentially lower miss rates of object-caching information about object boundaries. For example, it could systems, yet avoids their problems of fragmentation and high be used in managing a cache of ®le system data, if an ap- overheads. Furthermore, HAC is adaptive: when locality is plication provided information about locations in a ®le that good it behaves like a page-caching system, while if locality correspond to object boundaries. HAC could also be used is poor it behaves like an object-caching system. It is able to within a server cache. adjust the amount of cache space devoted to pages dynami- In persistent object systems, objects are clustered in ®xed- cally so that space in the cache can be used in the way that size pages on disk, and pages are much larger than most best matches the needs of the application. objects [Sta84]. Most systems manage the client cache us- The paper also presents results of experiments that in- ing page caching [LLOW91, WD94, SKW92]: when a miss dicate that HAC outperforms other object storage systems occurs, the client evicts a page from its cache and fetches across a wide range of cache sizes and workloads; it per- the missing object's page from the server. Page caching forms substantially better on the expected workloads, which achieves low miss penalties because it is simple to fetch and have low to moderate locality. Thus we show that our hy- replace ®xed-size units; it can also achieve low miss rates brid, adaptive approach is the cache management technique provided clustering of objects into pages is good. However, of choice for distributed, persistent object systems. it is not possible to have good clustering for all applica- tion access patterns [TN92, CK89, Day95]. Furthermore, access patterns may evolve over time, and reclustering will 1 Introduction lag behind because effective clustering algorithms are very expensive [TN92] and are performed infrequently. There- In distributed persistent storage systems, servers provide per- fore, pages contain both hot objects, which are likely to be sistent storage for information accessed by applications run- used by an application in the near future, and cold objects, ¢ ning at clients [LAC ¢ 96, C 89, WD94, LLOW91, BOS91]. which are not likely to be used soon. Bad clustering, i.e., a low fraction of hot objects per page, causes page caching This research was supported in part by DARPA contract DABT63-95-C-005, moni- to waste client cache space on cold objects that happen to tored by Army Fort Huachuca, and in part by DARPA contract N00014-91-J-4136, reside in the same pages as hot objects. monitored by the Of®ce of Naval Research. M. Castro is supported by a PRAXIS XXI ¢ ¢ fellowship. Object-caching systems [Ont92, D ¢ 90, LAC 96, C 94b, KK90, WD92, KGBW90] allow clients to cache hot objects Copyright £ c 1997 by the Association for Computing Machinery, Inc. Permission to without caching their containing disk pages. Thus, object make digital or hard copies of part or all of this work for personal or classroom use caching can achieve lower miss rates than page caching is granted without fee provided that copies are not made or distributed for pro®t or commercial advantage and that copies bear this notice and the full citation on the ®rst when clustering is bad. However, object caching has two page. Copyrights for components of this work owned by others than ACM must be problems: objects are variable-sized units, which leads to honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post storage fragmentation, and there are many more objects than on servers, or to redistribute to lists, requires prior speci®c permission and/or a fee. pages, which leads to high overhead for bookkeeping and for Request permissions from Publications Dept, ACM Inc., fax +1 (212) 869-0481, or [email protected] maintaining per-object usage statistics. Dual buffering [KK94] partitions the client cache stati- cally into a page cache and an object cache. This technique HAC has been incorporated in Thor-1, a new implemen- has been shown to achieve lower miss rates than both page tation of the Thor object database [LAC ¢ 96]; it retains the caching and object caching when the fraction of space ded- server storage management and concurrency control tech- icated to the object cache is manually tuned to match the niques of our previous implementation, Thor-0, but uses characteristics of each application. Such tuning is not fea- HAC for client cache management. The high performance sible for real applications; furthermore, these systems do of Thor-1 is furthermore achieved by adherence to two de- not solve the problems of storage fragmentation and high sign principles: think small, and be lazy. We were careful overheads in object caching systems. about the sizes of all data structures, but particularly about HAC is the ®rst technique to provide an adequate solu- objects; we designed our object format parsimoniously, and tion to all these problems. It is a hybrid between page and the result is better performance in all parts of the system, object caching that combines the virtues of each Ð low over- from the server disk to the client cache. Furthermore, we do heads and low miss rates Ð while avoiding their problems. work only when absolutely necessary, since this allows us to Furthermore, HAC adaptively partitions the cache between avoid it entirely in many cases. We will explain how these objects and pages based on the current application behav- principles were applied in our implementation and how they ior: pages in which locality is high remain intact, while for contribute to our performance. pages in which locality is poor, only hot objects are retained. The paper presents results of performance experiments To our knowledge, HAC is the ®rst adaptive caching system. that show the bene®ts of these techniques. The experiments O'Toole and Shrira [OS95] present a simulation study of an evaluate their impact on performance across a wide range of adaptive caching system, but they focus on avoiding reads to cache sizes and workloads. Our results show that HAC con- install modi®cations at servers and ignore storage manage- sistently outperforms page-based approaches and achieves ment issues. speedups of more than an order of magnitude on memory- HAC partitions the client cache into page frames and bound workloads with typically achievable clustering. We fetches entire pages from the server. To make room for also show that Thor-1 outperforms the best object storage an incoming page, HAC does the following: systems. The rest of the paper is organized as follows. Section 2 selects some page frames for compaction, gives an overview of our implementation, and describes the object format at both clients and servers. Section 3 describes discards the cold objects in these frames, HAC. We present performance results in Section 4. Section 5 compacts the hot objects to free one of the frames. summarizes the main results in the paper. The approach is illustrated in Figure 1. The policy to select page frames for compaction strives both to achieve low over- 2 System Overview head and to discard objects that are unlikely to be reused. HAC maintains usage statistics on a per-object basis using a In Thor, applications running at client machines make use novel technique with low space and time overheads. The of persistent objects stored at servers. Applications interact technique combines both recency and frequency information with Thor by calling methods of objects; these calls occur to make good replacement decisions. within atomic transactions. The methods are implemented in a type-safe language called Theta [LCD ¢ 94]. Before compaction To speed up applications, client machines cache copies of persistent objects. Clients communicate with servers only 5 to fetch pages or commit transactions; we use optimistic 1 3 6 7 concurrency control [AGLM95, Gru97] to serialize transac- 4 2 tions. The fact that computations run as transactions does not preclude the use of HAC in non-transactional systems, Framet 14 Frame 2 Frame 53 since the main impact of transactions is de®ning when mod- Cold object After compaction i®cations are sent to servers, and this could be de®ned in a Hot object 4 different way, e.g., for ®le systems, updates could be sent 1 5 every few seconds.