Fast and Energy-Efficient OLAP Data Management on Hybrid Main Memory Systems Hassan, A., Nikolopoulos, D., & Vandierendonck, H. (2019). Fast and Energy-Efficient OLAP Data Management on Hybrid Main Memory Systems. IEEE Transactions on Computers. https://doi.org/10.1109/TC.2019.2919287 Published in: IEEE Transactions on Computers Document Version: Peer reviewed version Queen's University Belfast - Research Portal: Link to publication record in Queen's University Belfast Research Portal Publisher rights © 2019 IEEE. This work is made available online in accordance with the publisher’s policies. Please refer to any applicable terms of use of the publisher. General rights Copyright for the publications made accessible via the Queen's University Belfast Research Portal is retained by the author(s) and / or other copyright owners and it is a condition of accessing these publications that users recognise and abide by the legal requirements associated with these rights. Take down policy The Research Portal is Queen's institutional repository that provides access to Queen's research output. Every effort has been made to ensure that content in the Research Portal does not infringe any person's rights, or applicable UK laws. If you discover content in the Research Portal that you believe breaches copyright or violates any law, please contact [email protected]. Download date:01. Oct. 2021 SUBMITTED TO IEEE TRANSACTIONS ON COMPUTERS, JUNE 2018 1 Fast and Energy-Efficient OLAP Data Management on Hybrid Main Memory Systems Ahmad Hassan, Dimitrios S. Nikolopoulos, Senior Member, IEEE, and Hans Vandierendonck, Senior Member, IEEE, Abstract—This paper studies the problem of efficiently utilizing hybrid memory systems, consisting of both Dynamic Random Access Memory (DRAM) and novel Non-Volatile Memory (NVM) in database management systems (DBMS) for online analytical processing (OLAP) workloads. We present a methodology to determine the database operators that are responsible for most main memory accesses. Our analysis uses both cost models and empirical measurements. We develop heuristic decision procedures to allocate data in hybrid memory at the time that the data buffers are allocated, depending on the expected memory access frequency. We implement these heuristics in the MonetDB column-oriented database and demonstrate performance improvement and energy-efficiency as compared to state-of-the-art application-agnostic hybrid memory management techniques. Index Terms—Non-volatile memory, hybrid main memory, database management system F 1 INTRODUCTION RAM) and Resistive RAM (RRAM) [19], are compelling alternatives due to their high density and near-zero leak- Online analytical processing (OLAP) is a key component age power [8], [13], [37]. NVM technologies however have of modern data processing pipelines and aims at analyzing several drawbacks, broadly involving increased latency and large data sets to inform business decisions. OLAP con- increased dynamic energy for NVM accesses, reduced mem- sists of long-running, complex transactions that are read- ory bandwidth and a fast wear-out of memory cells [8], [28], intensive. To maintain high processing throughput, data [46], [65]. On the plus side, NVM is predicted to scale to sets are kept in main memory during OLAP processing to smaller feature sizes [8] and has around 100× lower static avoid I/O bottlenecks. This approach, known as in-memory energy due to the absence of refresh operations [13]. More- processing, puts severe pressure on the required main mem- over, NVM is persistent, which is important to the design ory capacity as data sets are continuously growing. It is of DBMSs. This work, however, focuses on the orthogonal estimated that data sets world-wide double in size every aspects of energy-efficiency and performance. year [20]. As such, multi-terabyte main memory setups are Hybrid memory systems typically have a fast and small desired [48]. component and a large and slow component, each with DRAM, the dominant technology for main memory distinct energy consumption. In the case of a DRAM+NVM chips, has hit power and scaling limitations [22], [64]. hybrid memory, DRAM provides high performance but it is DRAM-based main memory consumes 30-40 % of the total not feasible to have large amounts of it. On the other hand, server power due to leakage and refresh power consump- NVM can be scaled up to provide large amounts of memory, tion [3], [30], [34]. The background power consumption of but it cannot be accessed as efficiently as DRAM. Hybrid DRAM moreover scales proportionally with the DRAM size, memory systems raise the key question of how to distribute adding to the total cost of ownership (TCO). Moreover, it is the data over the memory types? uncertain whether DRAM technology can be scaled below State-of-the-art techniques propose data migration poli- 40 nm feature sizes [22], [64]. It is thus unlikely that DRAM cies for hybrid memory where data migration is decided will continue to be the dominant memory technology. The on by the hardware [28], [46], [53], [62], the operating sys- techniques like High-Bandwidth Memory (HBM) and Intel’s tem [12], [50] or a mixture of both [47]. These techniques try Knight’s Landing processor provide ways of mitigating to second-guess the properties of the workload and migrate DRAM limitations [25][53]. large chunks of data, typically at the page granularity of the An alternative setup uses a combination of conventional virtual memory system. This reactive approach introduces DRAM and novel byte-addressable non-volatile memory runtime overhead and energy consumption due to monitor- (NVM) chips that are interface-compatible with DRAM [27], ing and migration. [28], [46], [47]. Emerging NVM technologies such as Phase- In this paper, we propose a radically different approach: Change Memory (PCM), Spin Transfer Torque RAM (STT- Building on intricate knowledge of the structure and mem- ory access patterns of column-oriented database, we pro- • Ahmad Hassan is with SAP. He was with Queen’s University Belfast, pose that the DBMS manages data placement in the hybrid United Kingdom at the time this research was conducted. E-mail: [email protected] main memory. We present a methodology to analyze a DBMS • Dimitrios S. Nikolopoulos and Hans Vandierendonck are with Queen’s in order to identify a good strategy for data placement. University Belfast. Our analysis uses a tool [18] that instruments all program Manuscript received June 6, 2018; revised December 2, 2018. variables and memory allocations and collects statistics on SUBMITTED TO IEEE TRANSACTIONS ON COMPUTERS, JUNE 2018 2 their size, their lifetime and on the number of memory e.g., to apply filter operations as early in the query plan as accesses made for each. Using these statistics, we formulate possible in order to reduce data volumes quickly. a strategy for data placement in a hybrid main memory Columnar databases require materialization, which is system. In our solution the DBMS analyzes the query plan to the act of “gluing” together the columns that make up a predict which data sets will incur frequent memory accesses relation [1]. Early materialization glues together all columns and should thus be placed on DRAM. It then allocates these that are needed in the evaluation of the query. This then is data sets on DRAM as they are generated. By predicting the similar to executing the query in the same style as a record- best placement with high accuracy, our solution does not oriented database where records are initially shortened to require migration of data between DRAM and NVM to fix contain only the columns relevant to the query. Late mate- wrong data placements. This contributes to its performance rialization postpones glueing together the columns as long and energy-efficiency. as possible. Algebraic operators are not applied to the data The key contributions of this paper are: columns themselves. Instead they operate on columns that hold the indices of the relevant records. • Derivation of runtime data placement heuristics for We focus our work on MonetDB [6], a highly effi- inputs and intermediate allocations of hash join us- cient columnar database. MonetDB uses late materializa- ing the access patterns of binary algebraic operators. tion, which results in high memory locality [6]. Moreover, The hash join is the single most important operation MonetDB evaluates each operator at a time, i.e., it processes in our OLAP workloads. Our solution, optimized the full input columns to produce the output(s) before for OLAP workloads, requires only two heuristics proceeding to evaluate the next operator. for MonetDB [6], a state-of-the-art column-oriented database. Our approach is, however, not limited to MonetDB. 2.2 The Memory Hierarchy • Demonstrating the performance and energy- The goal of cache memories (or caches for short) is to create efficiency of our data placement heuristics for hybrid the impression of a large and fast memory, where in reality memory in MonetDB. We furthermore compare our large memories are slow and fast memories must be small. heuristics to state-of-the-art application-agnostic Cache memories create this impression using the principle page placement and migration techniques. of reference locality, an empirical observation that programs • Demonstrating that query plan partitioning yields tend to repeatedly access only a limited amount of data intermediate data sets that are small enough to fit during any period of time. This limited amount of data is in DRAM without incurring significant overhead. called the working set of the program. Only the first access to The remainder of this paper is organized as follows. the working set is slow. Subsequent accesses are fast as the Section2 reviews the organization of memory systems and working set is then available in the cache. We call a cache their key performance and energy properties. Section3 access that finds the data in the cache a cache hit.