Context-Aware Prefetching at the Storage Server Gokul Soundararajan, Madalin Mihailescu†, and Cristiana Amza Department of Electrical and Computer Engineering Department of Computer Science† University of Toronto Abstract repeatable non-sequential access patterns as well. How- ever, the storage system receives interleaved requests In many of today’s applications, access to storage con- originating from many concurrent application streams. stitutes the major cost of processing a user request. Thus, even if the logical I/O sequence of a particular ap- Data prefetching has been used to alleviate the stor- plication translates into physically sequential accesses, age access latency. Under current prefetching tech- and/or the application pattern is highly repeatable, this niques, the storage system prefetches a batch of blocks pattern may be hard to recognize at the storage system. upon detecting an access pattern. However, the high This is the case for concurrent execution of several ap- level of concurrency in today’s applications typically plications sharing a network attached storage, e.g., as leads to interleaved block accesses, which makes detect- shown in Figure 1, and also for a single application with ing an access pattern a very challenging problem. To- multiple threads exhibiting different access patterns, e.g., wards this, we propose and evaluate QuickMine, a novel, a database application running multiple queries, as also lightweight and minimally intrusive method for context- shown in the figure. aware prefetching. Under QuickMine, we capture ap- We investigate prefetching in storage systems and plication contexts, such as a transaction or query, and present a novel caching and prefetching technique that leverage them for context-aware prediction and improved exploits logical application contexts to improve prefetch- prefetching effectiveness in the storage cache. ing effectiveness. Our technique employs a context We implement a prototype of our context-aware tracking mechanism, as well as a lightweight frequent prefetching algorithm in a storage-area network (SAN) sequence mining [38] technique. The context tracking built using Network Block Device (NBD). Our proto- mechanism captures application contexts in an applica- type shows that context-aware prefetching clearly out- tion independent manner, with minimal instrumentation. performs existing context-oblivious prefetching algo- These contexts are leveraged by the sequence mining rithms, resulting in factors of up to 2 improvements in technique for detecting block access patterns. application latency for two e-commerce workloads with repeatable access patterns, TPC-W and RUBiS. In our context tracking mechanism, we simply tag each application I/O block request with a context iden- tifier corresponding to the higher level application con- 1 Introduction text, e.g., a web interaction, database transaction, appli- cation thread, or database query, where the I/O request In many of today’s applications, such as, e-commerce, to the storage manager occurs. Such contexts are readily on-line stores, file utilities, photo galleries, etc., access available in any application and can be easily captured. to storage constitutes the major cost of processing a We then pass this context identifier along with each read user request. Therefore, recent research has focused block request, through the operating system, to the stor- on techniques for alleviating the storage access latency age server. This allows the storage server to correlate the through storage caching [14, 23, 29] and prefetching block accesses that it sees into frequent block sequences techniques [24, 25, 35, 36, 37]. Many traditional storage according to their higher level context. Based on the de- prefetching algorithms implement sequential prefetch- rived block correlations, the storage cache manager then ing, where the storage server prefetches a batch of se- issues block prefetches per context rather than globally. quential blocks upon detecting a sequential access pat- At the storage server, correlating block accesses is per- tern. Recent algorithms, like C-Miner∗ [24, 25], capture formed by the frequent sequence mining component of USENIX Association USENIX ’08: 2008 USENIX Annual Technical Conference 377 our approach.approachh. In particular,particular, we design and implement a MySQL/InnoDB MySQL/InnoDB lightweight aand dynamic frequent sequence mining tech- 1 2 3 5 7 9 IV V VI II IX X nique, called QuicQuickMinekMine. Thread-1 Thread-2 Thread-3 Thread-4 Justlik likee state-of-the-arts prefetching algorithmsalggorithms [24, 25], QuicQuickMinekMMine detects sequential as wwell as non- Operating System Operating System sequential co correlationsorrelations using a history-based d mechani mechanism.sm. 1 2 5 7 3 9 IV II IX V X VI QuickMine’sQuickMine’s keykey noveltynovelty lies in detecting andannd leveragingleveraging VolumeVolume 1 VolumeVolume 2 block correla correlationsations within logical application n conte contexts.xts. In Consolidated Storage additionaddition, QuQuickMineuickMine generates and adapts block corre-corre 1 2 IV 5 II 7 IX 3 V X 9 VI lations incrincrementally,rementallyemmentallyy,, on-the-fly , through a lightweight mining algor algorithm.rithm. As we will sho showw in our e experi-xperi- mental e evaluation,valuuation, these no novelvel features mmak makee QuicQuick-k- Figure 1: InterleavedInterleaved Accesses. Accessess from con- Mine uniqueuniquelyely suitable for on-line pattern n mining and current processes/threadsproceesses/threads are interleavedinterleaved at the storagestorage prefetching bby i) substantially reducing the e footprint of server.server. the block cor correlationsrrelations it generates, ii) impro improvingvving the lik like-e- lihood that th thehe block correlations maintained d will lead to accurate pref prefetchesfetches and iii) pro providingviding fle flexibilityxiibility to dy- prefetches or ii)ii) non-repeatable (false)(ffalse)alse) block correlationscorrelations namic chang changesges in the application pattern, and concur concur-- at conte contextxt bou boundaries,undaries, hence useless prefetch h rules in the rencrencyy de degree.gree. contecontext-obliviousxt-obliviious approaches. Our e evaluationvaluationn sho showsws that WWee implem implementment QuicQuickMinekMine in a lightwe lightweighteight storage QuicQuickMinekMine gegeneratesenerates substantially more ef effectiveffectifefective block cache prototypeprototyype embedded into the NetworkNetworrk Block De- correlation ru rulesules o overall,verall, in terms of both the e number of vice (NBD) code. WWe e also implement se severalveral al alterna-terna- prefetches tri triggeredggered and the prefetching ac accuracy.curacy. WWe e titiveve approach approacheshes for comparison with our sch scheme,heme, includ- also sho showw tha thatat QuicQuickMinekMine is capable of adjus adjustingsting its cor cor-- ing a baseline e LR LRURUU cache replacement algor algorithmrithm with no relation rules dynamically dynamically,, without incurring undue o over-ver- prefetching, and the follo followingwing state-of-the state-of-the-arte-art conte context-xt- head for rapid rapidlydly changing patterns. obliobliviousvious pre prefetchingefetching schemes: tw twoo adapti adaptiveve se sequentialquential The rest of f this paper is or organizedganized as fo follows.llows. Sec- prefetching sschemes [10, 16] and the recen recentlyntly proposed tions 2 pro providesviddes the necessary background an andnd moti motivatesvates C-Miner∗ storagestoorage prefetching algorithm [24,[244, 25]. our dynamic, , conte context-awarext-aware algorithm. Se Sectionection 3 in- troduces our QQuicQuickMinekMine contecontext-awarext-aware prefe prefetchingetching solu- In our e experimentalxpperimental e evaluation,valuation, we use th threehree standard tition. SSection ti 4 pro providesvidesid ddetails t il of f our iimpl implementation.llementation.t ti database app applications:plications: the TPC-W e-comm e-commercemerce bench- Section 5 des describesscribes our e experimentalxperimental platfor platform,rm, method- mark [1], th thehe RRUBiS UBiS auctions benchmark k and DBT DBT-T-- ologyology,, other approaches in storage cache m managementanagement 2 [40], a TPC TPC-C-likeC-C-like benchmark [30]. The appli applicationscations that we e evaluatevaluate in our e experiments.xperiments. Section n 6 pres presentsents hahaveve a wide range of access patterns. TPC TPC-WC-W and RRU- U- our e experimentalxperimeental results. Section 7 discu discussesusses r relatedelated BiS are read- read-intensive-intensive w workloadsorkloads with high highlyly repe repeatableatable wworkork and Sec Sectionction 8 concludes the paper paper.. access patter patterns;rns; the theyy contain 80% and 85 85%% read-only transactions, respecti respectively,vely, in their w workloadorkload mix. In con- trast, DBT DBT-2T-2-2 is a write-intensi write-intensiveve applicat applicationionn with rapidly 2 BackgroundBackgrround andMoti Motivationvation changing acc accesscess patterns; it contains only 44% read-only transactions in its w workloadorkload mix. WWe e in instrumentstrument the WeWe focus on improvingimproving the cache hit rate att the storage MMySQL/InnoMySQL/InnoDBSQL/I oDBDB ddatabase t b engine i tto ttrack k th thehhe conte contextst xtst of f cache in a SA SANAN en environmentvironmentv through prefet prefetching.tching. Our interest. WWe e found that changing the DBMS S to incorpo- techniques arearre applicable to situations wherewherre the work-work-