Everything You Always Wanted to Know About Compiled and Vectorized Queries But Were Afraid to Ask
Timo Kersten Viktor Leis Alfons Kemper Thomas Neumann Andrew Pavlo Peter Boncz? Technische Universitat¨ Munchen¨ Carnegie Mellon University Centrum Wiskunde & Informatica? {kersten,leis,kemper,neumann}@in.tum.de [email protected] [email protected]
ABSTRACT Like the Volcano-style iteration model, vectorization uses pull- The query engines of most modern database systems are either based iteration where each operator has a next method that produces based on vectorization or data-centric code generation. These two result tuples. However, each next call fetches a block of tuples state-of-the-art query processing paradigms are fundamentally dif- instead of just one tuple, which amortizes the iterator call over- ferent in terms of system structure and query execution code. Both head. The actual query processing work is performed by primitives paradigms were used to build fast systems. However, until today it that execute a simple operation on one or more type-specialized is not clear which paradigm yields faster query execution, as many columns (e.g., compute hashes for a vector of integers). Together, implementation-specific choices obstruct a direct comparison of ar- amortization and type specialization eliminate most of the overhead chitectures. In this paper, we experimentally compare the two mod- of traditional engines. els by implementing both within the same test system. This allows In data-centric code generation, each relational operator imple- us to use for both models the same query processing algorithms, the ments a push-based interface (produce and consume). However, in- same data structures, and the same parallelization framework to ul- stead of directly processing tuples, the produce/consume calls gen- timately create an apples-to-apples comparison. We find that both erate code for a given query. They can also be seen as operator are efficient, but have different strengths and weaknesses. Vector- methods that get called during a depth-first traversal of the query ization is better at hiding cache miss latency, whereas data-centric plan tree, where produce is called on first visit, and consume on compilation requires fewer CPU instructions, which benefits cache- last visit, after all children have been processed. The resulting code resident workloads. Besides raw, single-threaded performance, we is specialized for the data types of the query and fuses all operators also investigate SIMD as well as multi-core parallelization and dif- in a pipeline of non-blocking relational operators into a single (po- ferent hardware architectures. Finally, we analyze qualitative dif- tentially nested) loop. This generated code can then be compiled to ferences as a guide for system architects. efficient machine code (e.g., using the LLVM). Although both models eliminate the overhead of traditional en- PVLDB Reference Format: gines and are highly efficient, they are conceptually different from T. Kersten, V. Leis, A. Kemper, T. Neumann, A. Pavlo, P. Boncz. Ev- each other: Vectorization is based on the pull model (root-to-leaf erything You Always Wanted to Know About Compiled and Vectorized traversal), vector-at-a-time processing, and interpretation. Data- Queries But Were Afraid to Ask. PVLDB, 11 (13): 2209 - 2222, 2018. centric code generation uses the push model (leaf-to-root traver- DOI: https://doi.org/10.14778/3275366.3275370 sal), tuple-at-a-time processing, and up-front compilation. As we discuss in Section 9, other designs that mix or combine ideas from data-centric compilation and vectorization have been proposed. In 1. INTRODUCTION this paper, we focus on these two specific designs, as they have been In most query engines, each relational operator is implemented highly influential and are in use in multiple widespread systems. using Volcano-style iteration [14]. While this model worked well The differences of the two models are fundamental and deter- in the past when disk was the primary bottleneck, it is inefficient mine the organization of the DBMS’s execution engine source code on modern CPUs for in-memory database management systems and its performance characteristics. Because changing the model (DBMSs). Most modern query engines therefore either use vec- requires rewriting large parts of the source code, DBMS design- torization (pioneered by VectorWise [7, 52]) or data-centric code ers must decide early on which model to use. Looking at recent generation (pioneered by HyPer [28]). Systems that use vector- DBMS developments like Quickstep [33] and Peloton [26], we find ization include DB2 BLU [40], columnar SQL Server [21], and that both choices are popular and plausible: Quickstep is based on Quickstep [33], whereas systems based on data-centric code gener- vectorization, Peloton uses data-centric code generation. ation include Apache Spark [2] and Peloton [26]. Given the importance of this choice, it is surprising that there has not yet been a systematic study comparing the two state-of-the- This work is licensed under the Creative Commons Attribution- art query processing models. In this paper, we provide an in-depth NonCommercial-NoDerivatives 4.0 International License. To view a copy experimental comparison of the two models to understand when a of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0/. For database architect should prefer one model over the other. any use beyond those covered by this license, obtain permission by emailing To compare vectorization and compilation, one could compare [email protected]. Copyright is held by the owner/author(s). Publication the runtime performance of emblematic DBMSs, such as HyPer rights licensed to the VLDB Endowment. and VectorWise. The problem is, however, that such full-featured Proceedings of the VLDB Endowment, Vol. 11, No. 13 DBMSs differ in many design dimensions beyond the query execu- ISSN 2150-8097. tion model. For instance, HyPer does not employ sub-byte com- DOI: https://doi.org/10.14778/3275366.3275370
2209 vec
1 2 HyPer compiles to LLVM IR rather than C++, but this choice only affects compila- Technically, it would be possible to create primitives that work on multiple types. tion time (which we ignore in this paper anyway), not execution time. However, this is not practical, as the number of combinations grows exponentially.
2210 query(...) // build hash table tinues until the candidate vector is empty. Afterwards, the join uses for(i=0; i< S.size(); i++) buildGather to move data from the hash table into buffers for the ht.insert(
2211 r tl rvln npouto ytm n r sdb ohHPradVectorWise. and HyPer both by used are and systems production in prevalent still are in- as the “growing well (e.g., as dimensions 1“) other query across on Typer comparisons than teresting more tuple of per 41 sum instructions enables executes “Tectorwise the normalization (e.g., (i.e., This systems across query scanned). observations intuitive tables that all by of scanned cardinalities code tuples the our of by counters number compile CPU total and perf normalize the Linux’ we OS paper, using this obtained as Throughout were Linux API. counters events CPU use The We hardware 7.2. the GCC with in 4. found Table be experiments. in can our section CPU for this cores for 10 specifications with Detailed CPU X) (Skylake i9-7900X tel Setup Experimental 3.4 [6]. also suite be TPC-H likely full and will the them TPC-H on on efficient well of queries performs challenges that five engine performance these execution any Together, important most differently. and the Q1 quite both cover behave for they results being show as despite we Finally, Q6 queries, single-table Q18. ag- two to high-cardinality only similar a the in therefore smaller time are A the and Q9. of gregation and most by Q3 spend dominated queries to are of similar most number therefore queries, are 17 and remaining processing the join Of it. with early following: bottlenecks the performance op- in their listed and as subset are queries representative query selected a The individual chose TPC-H. we each of statistics, for summary be only statistics To to detailed posed experiments. show most to for able benchmark TPC-H well-known the Workload 3.3 2.1. vectorization Section in while algorithms describe straightforward, we these which quite Using adaptations, step is requires final code partition. a data-centric each Afterwards, in in groups partitions. the phases into two aggregates groups into spills split heavy and handles is pre-aggregation hitters operator traverse A to by parallelization. have friendly group cache not The for does usually list). miss collision probe the performance (a improves joins that encodes selective [22] dictionary for structure table filter-like hash Bloom the small pointer, a each of bits (unused) 16 table hash uses Join single expressions. a arithmetic does of checking implementation overflow our perform C not systems, corresponding production-grade the Unlike to expression code. the transforming by achieved is as expressed are statements Select relation. 3 lhuhrcn eerhage o attoe ahjis[,4] igetbejoins single-table 41], [5, joins hash partitioned for argues research recent Although
nesohrientd eueasse qipdwt nIn- an with equipped system a use we noted, otherwise Unless lin- grow and 1 (SF) factor scale for are cardinalities given The use therefore and performance, OLAP on focus we paper this In Runtime [ms] • • • • • 20 40 60 80 0 Typer TW Q18: Q9: Q3: Q6: Q1: iue3: Figure q1 on(ul:30Ketis rb:15Mentries) M 1.5 probe: entries, K entries) 320 M (build: 3.2 join probe: entries, K 147 (build: join filters selective aggregation groups) (4 arithmetic, fixed-point ihcriaiyageain(. groups) M (1.5 aggregation high-cardinality 10 15 0 5 Typer TW Performance 3 ihcann o olso eeto.Using detection. collision for chaining with q6 10 20 30 40 50 0 Typer TW – q3 P- F1 thread 1 SF=1, TPC-H 100 150 50 if 0 Typer TW rnhs Projection branches. q9 100 150 50 0 Typer TW q18 2212 h atta ohegnsueeatytesm ahtbelayout table hash same the exactly use given engines surprising both be might that This fact 32%). the is and Tectorwise 4% probing, (by Typer table than hash faster of efficiency the by determined is CPU in misses. results cache intermediate for few have efficient hold and more registers can is dif- Typer that key that queries is one computational models Thus, two the itself. between is computation ference which the materialized, as by be expensive dominated must similarly is results intermediate query aggregation. Tectorwise This in-cache In cheap why a Q1. and clear operations on arithmetic becomes fewer fixed-point faster it with significantly observations, operations is these same Typer in on the results Based perform intermediate instructions. thus keep often and ac- can registers cache contrast, CPU and in instructions Typer, additional in cesses. results which between steps, results intermediate these materialize must and 3.3 steps simple to into (up 2.4 misses to cache (up data signif- executes instructions Tectorwise more First, made. icantly be can observations of ber models. understand two the to of the differences weaknesses in and performance strengths Therefore, the the engine. vi- analyze both processing we makes a following which implement neither other to the options Nevertheless, by able dominated clearly systems. point is these the paradigm from implementing different appear someone models close— the two of quite the words, is that other fact paradigms In the processing despite [17]. query magnitude both of of orders performance two be- is and PostgresSQL one systems. and other tween HyPer to between difference gap es- the performance at differences, example large the For look not to we are compared these Before when that note pecially (Q9). we 32% this, 74% for by reasons by faster the faster being being Tectorwise Typer to from (Q1) efficient. ranges more is performance Tectorwise Q18), relative Q9) (Q1, (Q3, The queries others for some and For faster is queries. Typer TPC-H selected for models Performance Single-Threaded 4.1 6. Section to parallelization multi-core and 5 Section to (SIMD) on data-parallelism discussing focus defer initially and performance We sequential comparison. micro-architectural in-depth an ANALYSIS MICRO-ARCHITECTURAL 4. per misses cache additional 0.5 causes 10, of factor tuple“). a by size data 1: Table ubro ulspoesdi htquery that in processed tuples of number eosrefrhroe htfrQ n 9 hs performance whose Q9, and Q3 for that furthermore, observe We num- a which from statistics, CPU important some shows 1 Table two the of performance single-threaded the compares 3 Figure perform we paradigms, processing query two the understand To 1 W4 . 0 . .80.37 0.16 0.18 1.9 0.19 0.8 102 46 2.1 48 1.6 30 TW Q18 Typer Q18 9T 6137 . .70.39 0.34 0.47 0.08 2.1 0.46 0.27 0.16 76 1.7 0.01 0.9 0.16 42 1.3 0.06 0.29 42 0.5 56 0.03 0.2 0.6 0.35 21 1.8 0.01 74 15 0.3 0.57 24 TW 0.8 2.0 0.57 Q9 20 1.4 Typer 25 0.6 Q9 162 11 TW 1.8 Q3 68 2.8 Typer 11 Q3 59 TW 2.0 Q6 Typer 34 Q6 TW Q1 Typer Q1 P Counters CPU ylsICinstr. IPC cycles – P- F1 hed omlzdby normalized thread, 1 SF=1, TPC-H × .Tcows rasaloperations all breaks Tectorwise ). × n sal a oeL1 more has usually and ) miss L1 LLC miss branch miss 4 Memory Stall Cycles Other Cycles Query q1 q6 q3 q9 q18 3 Typer Tectorwise 60
40 q1 2 20 0 12 1 Time Relative to 1K
9 q6 6 1 16 256 1K 4K 64K 1M Max. 3 0 Vector Size (Tuples)
40 q3 Figure 5: Tectorwise Vector Sizes – Times are normalized by 1K 20 vector time. 0 Cycles / Tuple 100 q9 to executing almost twice the number of instructions. This means 50 0 that one has to be cautious when using IPC to compare database systems’ performance. It is a valid measure of the amount of free 40 q18 20 processing resources, but should not be used as the sole proxy for 0 overall query processing performance. 1 3 10 30 100 1 3 10 30 100 To summarize, looking at the micro-architectural footprint of the Data Size (TPC-H Scale Factor) two models we found that (1) both are efficient and fairly close in Figure 4: Memory Stalls – TPC-H, 1 thread performance, (2) Typer is more efficient for computational queries with few cache misses, and (3) Tectorwise is slightly better at hid- ing cache miss latency. and therefore also have an almost identical number of last level cache (LLC) misses. As Figure 4 shows, Tectorwise’s join advan- 4.2 Interpretation and Instruction Cache tage increases up to 40% for larger data (and hash table) sizes. The Systems based on Volcano-style iteration perform expensive vir- reason is that vectorization is better at hiding cache miss latency, tual function calls and type dispatch for each processed tuple. This as observed from the memory stall counter that measures the num- is a form of interpretation overhead as it does not contribute to ber of cycles during which the CPU is stalled waiting for memory. the actual query processing work. Generating machine code for a This counter explains the performance difference. On the one hand, given query, by definition, avoids interpretation overhead. Vector- Tectorwise’s hash table probing code is only a simple loop. It exe- ized systems like VectorWise are still fundamentally interpretation- cutes only hash table probes thus the CPU’s out-of-order engine can based engines and use Volcano-style iteration. In contrast to clas- speculate far ahead and generate many outstanding loads. These sical database systems, the interpretation overhead is not incurred can even be executed out of order. On the other hand, Typer’s code for each tuple but is amortized across the eponymous vector of tu- has more complex loops. Each loop can contain code for a scan, se- ples. Each primitive is specialized for a particular data type and is lection, hash-table probe, aggregation and more. The out-of-order called for (e.g., 1,000 values). This amortization is effective: Using window of each CPU fills up more quickly with complex loops a profiler, we determined that across our query set the interpreted thus they generate less outstanding loads. In addition every branch part is less than 1.5% of the query runtime (measured at scale factor miss is more expensive than in a complex loop as more work that is 10). Thus, the DBMS spends 98.5% of its time in primitives doing performed under speculative execution is discarded and must be re- query processing work. From Table 1 we observe that vectorized peated on a miss. Overall, Tectorwise’s simpler loops enable better code usually executed more instructions per tuple than compiled latency hiding. code. Since the vast majority of the query execution time is spent Another difference between the two executions models is their within primitives, also the time to execute these extra instructions sensitivity regarding the hash function. After trying different hash must be spent within primitives. As primitives know all involved functions, we settled on Murmur2 for Tectorwise, and a CRC-based types at compile time, we conclude that the extra instructions are hash function, which combines two 32-bit CRC results into a sin- not interpretation code that is concerned with interpretation deci- gle 64-bit hash, for Typer. Murmur2 requires twice as many in- sions and virtual function calls. It is rather due to the load/store structions as CRC hashing, but has higher throughput and is there- instructions for materializing primitive results into vectors. fore slightly faster in Tectorwise, which separates hash computa- Recent work has found that instruction cache misses can be a tion from probing. For Typer, in contrast, the CRC hash function problem for OLTP workloads [43]. To find out whether this is the improves the performance up to 40% on larger scale factors—even case for our two query engines, we measured L1 instruction cache though most time is spent waiting for cache misses. The lower misses for both systems and found that instruction cache misses are latency and smaller number of instructions for CRC significantly negligible, thus not a performance bottleneck for OLAP queries. improve the speculative, pipelined execution of consecutive loop For all queries measured, the L1 instruction cache (32 KB) was iterations, thereby enabling more concurrent outstanding loads.4 large enough to contain all hot code. As a note of caution, we remark that one may observe from Ta- ble 1 that Tectorwise generally executes more instructions per cycle 4.3 Vector Size (IPC) and deduce that Tectorwise performs better. However, this is The vector size is an important parameter for any vectorized en- not necessarily correct. While IPC is a measure of CPU utilization, gine. So far, our Tectorwise experiments used a value of 1,000 having a higher IPC is not always better: As can be observed in tuples, which is also the default in VectorWise. Figure 5 shows Q1, Tectorwise’s IPC is 40% higher, but it is still 74% slower due normalized query runtimes for vector sizes from 1 to the maximum 4 Despite using different hash functions, this is still a fair comparison of join perfor- (i.e., full materialization). We observe that small (<64) and large mance, as each system uses the more beneficial hash function. vector sizes (>64 K) decrease performance significantly. With a
2213 vector size of 1, Tectorwise is a Volcano-style interpreter with its 2.5 large CPU overhead. Large vectors do not fit into the CPU caches 2.0 1.4x and therefore cause cache misses. The other end of the spectrum is 2.0 1.5 10 1.5 2.7x to process the query one column at a time; this approach is used in 8.4x 1.0 MonetDB [9]. Generally, a vector size of 1,000 seems to be a good 1.0 5 setting for all queries. The only exception is Q3, which executes 0.5 0.5 Time [ms]
15% faster using a vector size of 64K. Cycles / Element 0.0 Cycles / Element 0.0 0 Scalar SIMD Scalar SIMD Scalar SIMD 4.4 Star Schema Benchmark (a) Dense Input (b) Sparse Input (c) TPC-H Q6 So far, we investigated a carefully selected subset of TPC-H. To Figure 6: Scalar vs. SIMD Selection in Tectorwise – (a) 40% show that our findings are more generally applicable, we also im- selectivity. (b) Secondary selection: Input selection vector selects plemented the Star Schema Benchmark (SSB), which consists of 4 40% and selection selects 40%. (c) Runtime of TPC-H Q6, SF=1 query templates (with different selections) and which is dominated by hash table probes. We use one thread and scale factor 30: code [34, 26], this is more challenging since the generated code is L1 LLC branch mem more complex. We will therefore use Tectorwise as the platform cycles IPC instr. miss miss miss stall for evaluating how large the impact of SIMD on in-memory OLAP workloads is. In contrast to most research on SIMD, we use TPC-H Q1.1 Typer 28 0.7 21 0.3 0.31 0.69 6.33 and not micro-benchmarks. Q1.1 TW 12 2.0 23 0.4 0.29 0.05 2.77 The Skylake X CPU we use for this paper supports the new AVX- Q2.1 Typer 39 0.8 30 1.3 0.12 0.17 18.35 512 instruction set and can execute two 512-bit SIMD operations Q2.1 TW 30 1.5 44 1.6 0.13 0.23 7.63 per cycle—doubling register widths and throughput in comparison Q3.1 Typer 55 0.7 40 1.1 0.20 0.24 27.95 with prior microarchitectures. In other words, using AVX-512 one Q3.1 TW 53 1.3 71 1.7 0.23 0.41 15.68 can process 32 values of 32-bit per cycle, while scalar code is lim- Q4.1 Typer 78 0.5 39 1.8 0.31 0.38 45.91 ited to 4 values per cycle. Furthermore, in comparison with prior Q4.1 TW 59 1.0 61 2.5 0.32 0.63 19.48 SIMD instruction sets like AVX2, AVX-512 is more powerful (al- most all operations support masking and there are new instructions These results are quite similar to TPC-H Q3 and Q9 and show like compress and expand) and orthogonal (almost all operations once more that Tectorwise requires more instructions but has an ad- are available in 8, 16, 32, and 64-bit variants). One would therefore vantage for join heavy queries due to better hidden memory stalls. expect significant benefits from using SIMD. In the following, we In general, we find that TPC-H subsumes SSB for our purposes and focus on selection and hash table probing, which are both common in the name of conciseness, we present our findings using TPC-H and important operations. in the rest of this paper. 5.1 Data-Parallel Selection 4.5 Tectorwise/Typer vs. VectorWise/HyPer A vectorized selection primitive produces a selection vector con- Let us close this section taining the indexes of all matching tuples. Using AVX-512 this can 6 Table 2: Production Systems by comparing Actian Vec- be implemented using SIMD quite easily . The comparison in- tor 5.0 (the current market- struction generates a mask that we then pass to a compress store HyPer VW Typer TW ing name for VectorWise) and (COMPRESSSTORE) instruction. This operation works across SIMD lanes and writes out the positions selected by the mask to memory. 44 the research (TUM) version Q1 53 71 85 We performed a micro-benchmark for selection, comparing a Q6 10 21 15 15 of HyPer (a related system is branch-free scalar x86 implementation with a SIMD variant. In the Q3 48 50 47 44 now with Tableau). The re- Q9 124 154 126 111 sults are shown in Table 2 use benchmark, we select all elements from an 8192 element integer Q18 224 159 90 154 one thread and TPC-H scale array which are smaller than a constant. Results for a best-case sce- factor 1. The first observation nario, in which all consumed data are 32-bit integers, are present in is that HyPer performs simi- the L1 cache, and the input is a contiguous vector, are shown in Fig- larly to Typer, and Tectorwise’s performance is similar to Vector- ure 6a. The observed performance gain for this micro-benchmark Wise. Second, except for Q65, either Typer or Tectorwise are is 8.4×. However, as Figure 6c shows, in a realistic query with slightly faster than both production-grade systems. It is unsur- multiple expensive selections like Q6, we only observe a speedup prising given that these must handle complex issues like overflow of 1.4×—even though almost 90% of the processing time is spent checking (that our prototype ignores). in SIMD primitives. Our experiments revealed two effects that ac- count for this discrepancy: sparse data loading due to selection vectors and cache misses due to varying stride. The remainder of 5. DATA-PARALLEL EXECUTION (SIMD) this section discusses these effects. Let us now turn our attention to data-parallel execution using Sparse data loading occurs in all selection primitives except for SIMD operations. There has been extensive research investigat- the first one. From the second selection primitive on, all primi- ing SIMD for database operations [51, 50, 36, 37, 38, 35, 46, 44]. tives receive a selection vector that determines the elements to con- It is not surprising that this research generally assumes a vector- sider for comparison. These elements must be gathered from non- ized execution model. The primitives of vectorized engines consist contiguous memory locations. A comparison of selection primi- of simple tight loops that can be translated to data-parallel code. tives with selection vectors (40% selectivity) is shown in Figure 6b. Though there has been research on utilizing SIMD in data-centric Performance gains in this case range only up to a 2.7× (again for
5 6 HyPer is faster on Q6 than the other systems because it evaluates selections using With AVX2, the selection primitive is non-trivial and either requires a lookup ta- SIMD instructions directly on compressed columns [19]. ble [35, 19, 26] or complex permutation logic based on BMI2 [1].
2214 4.0 Cycles Total SIMD Auto Manual Auto + Manual 3.5 L1-miss Cycles 3.0 80 20 2.5 Cycles 60 10 2.0 Scalar 40 SIMD 0 20% 40% 60% 80% 100% 20 Input Selectivity -10 0 Figure 7: Sparse Selection – Cost of selection operation with se- q1 q6 q3 q9 q18 q1 q6 q3 q9 q18 Reduction of Instr. [%] Reduction of Time [%] lection vector on input, depending on element size and implemen- Figure 10: Compiler Auto-Vectorization – (ICC 18) tation variant. Output selectivity 40%, data set size 4 GB
Cycles / Element Time [ms] arithmetic operations like integer shifts and multiplications that are 2 available in AVX-512. We can also apply SIMD to lookups into 1.1x q3 q9 10 1.4x hash tables by using gather, compress store, and masking. 4 40 1.1x 1.1x 2.3x 90 A performance breakdown of components necessary for hash 1 30 joins is shown in Figure 8. Figure 8(a) shows that for hashing 5 60 2 20 alone a gain of 2.3× is possible. For gather instructions, shown 10 30 in Figure 8(b), we observe an improvement of 1.1× (in the best 0 0 0 0 0 case). This is because the memory system of the test machine Scalar SIMD Scalar SIMD Scalar SIMD Scalar SIMD Scalar SIMD can perform at most two load operations per cycle—regardless of (a) Dense (b) Gather (c) TW join (d) TPC-H whether SIMD gather or scalar loads are used. Figure 8(c) shows hashing instruction primitive that when employing gather and other SIMD instructions to the Figure 8: Scalar vs. SIMD Join Probing – in microbenchmarks Tectorwise probe primitive, a best-case performance gain of 1.4× and full queries. can be achieved. With a SIMD speedup of 2.3× for hashing and 1.4× for prob- 32-bit types). Considering that the selections in Q6 consist of one ing, one may expect an overall speedup in between. However, as initial selection without input selection vector and four subsequent is shown in Figure 8(d) the performance gains almost vanish for selections that have to consider a selection vector, we can expect TPC-H join queries. This happens even though the majority of the the overall speedup to be closer to 3× than to 8×. time (55% and 65%) is spent in SIMD-optimized primitives. The The previous benchmarks only considered data sets which reside reason for this behavior can be found in Figure 9. With a growing in L1 cache. For larger data sets, accessing memory can become a working set, gains from SIMD diminish and the execution costs are limiting factor. Figure 7 shows the interplay of selection perfor- dominated by memory latency. SIMD is only beneficial when all mance and input sparsity on a 4 GB data set. Note that the per- data fits into the cache. We are not the first to observe this phe- formance drops for selectivities below 100%, while the scalar and nomenon: Polychroniou et al. [35] found this effect in their study SIMD variants are nearly equal when the is selectivity are below of application of SIMD to database operators. 50%. We also show an estimate of how many cycles on average 5.3 Compiler Auto-Vectorization are spent resolving cache misses. We observe that most of the time is spent waiting for data. Thus the memory subsystem becomes We manually rewrote Tectorwise primitives using SIMD intrin- the bottleneck of the selection operation and the positive effect of sics. Given that the code of most primitives is quite simple, one utilizing SIMD instructions disappears. In the selection cascade of may reasonably ask whether compilers can do this job automati- Q6, only the first selection primitive benefits from SIMD and se- cally. We tested the GCC 7.2, Clang 5.0, and ICC 18 compilers. Of lects 43% of the tuples. This leaves all subsequent selections to these, only ICC was able to auto-vectorize a fair amount of prim- operate in a selectivity area where the scalar variant is just as fast. itives (and only with AVX-512). Figure 10 shows how successful ICC was in relevant paths for query processing. Its vectorized vari- 5.2 Data-Parallel Hash Table Probing ant reduces the observed number of instructions executed per tuple by between 20% to 60%. By inspecting traces of the executed code, We next examine hash join probing, where most of the query we confirmed that automatic vectorization was applied to hashing, processing time is spent in TPC-H. There are two opportunities to selection, and projection primitives. Hash table probing and ag- apply SIMD: computing the hash values, and the actual lookups in gregation, however, were not transformed. We also show a variant the hash table. For hashing we use Murmur2, which consists of with automatic and manual SIMD application combined, which has a benefit for Q3 and Q9. 100 Unfortunately, these automatic SIMD optimizations do not yield any significant improvements in query runtime. Automatic vec- 30 torization alone hardly creates any gains but even introduces cases where the optimized code becomes slower. This means that even Scalar 10 SIMD though primitives can be auto-vectorized, this is not yet a fire-and- forget solution. 128KB 4MB 256MB Cyc. / Lookup [log] Working Set Size [log] 5.4 Summary Figure 9: Join Probe – Interaction of working set size and cost per We found with AVX-512 it is often straightforward to trans- tuple during Tectorwise hash table lookup late scalar code to data-parallel code, and observed performance gains of up to 8.4× in micro-benchmarks. However, for the more
2215 Table 3: Multi-Threaded Execution – TPC-H SF=100 on Skylake Table 4: Hardware Platforms – used in experiments. (10 cores, 20 hyper-threads) Intel AMD Intel Typer Typer TW TW Skylake T.ripper KNL Thr. Ratio ms speedup ms speedup model i9-7900X 1950X Phi 7210 cores (SMT) 10 (x2) 16 (x2) 64 (x4) Q1 1 4426 1.0 7871 1.0 0.56 Q1 10 496 8.9 867 9.1 0.57 issue width 4 4 2 Q1 20 466 9.5 708 11.1 0.66 SIMD [bit] 2×512 2×128 2×512 clock rate [GHz] 3.4-4.5 3.4-4.0 1.2-1.5 Q6 1 1511 1.0 1443 1.0 1.05 Q6 10 243 6.2 213 6.8 1.14 L1 cache 32 KB 32 KB 64 KB Q6 20 236 6.4 196 7.4 1.20 L2 cache 1 MB 1 MB 1 MB LLC 14 MB 32 MB (16 GB) Q3 1 9754 1.0 7627 1.0 1.28 Q3 10 1119 8.7 913 8.4 1.23 list price [$] 989 1000 1881 Q3 20 842 11.6 743 10.3 1.13 launch Q2’17 Q3’17 Q4’16 mem BW [GB/s] 58 56 68 Q9 1 28086 1.0 20371 1.0 1.38 Q9 10 3047 9.2 2394 8.5 1.27 Q9 20 2525 11.1 2083 9.8 1.21 Q18 1 13620 1.0 18072 1.0 0.75 and exclusive resources for every worker. To achieve that the work- Q18 10 2099 6.5 2432 7.4 0.86 ers can work together on one query, every operator can have shared Q18 20 1955 7.0 2026 8.9 0.97 state. For each operator, a single instance of shared state is created. All workers have access to it and use it to communicate. For ex- ample, the shared state for a hash join contains the hash-table for complicated TPC-H queries, the performance gains are quite small the build side and all workers insert tuples into it. In general, the (around 10% for join queries). Fundamentally, this is because most shared state of each operator is used to share results and coordinate OLAP queries are bound by data access, which does not (yet) bene- work distribution. Additionally, pipeline breaking operators use a fit much from SIMD, and not by computation, which is the strength barrier to enforce a global order of sub-tasks. The hash join oper- of SIMD. Coming back to the comparison between data-centric ator uses this barrier to enforce that first all workers consume the compilation and vectorization, we therefore argue that SIMD does build side and insert results into a shared hash table. Only after 7 not shift the balance in favor of vectorization much . that, the probe phase of the join can start. With shared state and a barrier, the Tectorwise implementation exhibits the same workload 6. INTRA-QUERY PARALLELIZATION balancing parallelization behavior as Typer. Given the decade-long trends of stagnating single-threaded per- 6.2 Multi-Threaded Execution formance and growing number of CPU cores—Intel is selling 28 cores (56 hyper-threads) on a single Xeon chip—any modern query We executed our TPC-H workload on scale factor 100 (ca. 100 GB engine must make good use of all available cores. In the following, of data). Table 3 shows runtimes and speedups in comparison with we discuss how to incorporate parallelism into the two query pro- single-threaded execution. Using the 10 physical cores of our Sky- cessing models. lake CPU, we see speedups between 8× and 9× for Q1, Q3, and Q9 in both systems. Given that modern CPUs reduce clock rates 6.1 Exchange vs. Morsel-Driven Parallelism significantly when multiple threads are used these results are close The original implementations of VectorWise and HyPer use dif- to perfect scalability. For the scan query Q6 the speedup is lim- ferent approaches. VectorWise uses exchange operators [3]. This ited by the available read memory bandwidth, and the large-scale classic approach [13] keeps its query processing operators like ag- aggregation of Q18 approaches the write bandwidth. gregation and join largely unaware of parallelism. HyPer, on the We also conducted these experiments on AWS EC2 machines other hand, uses morsel-driven parallelism, in which joins and ag- and found that both systems scale equally well. However, we ob- gregations use shared hash-tables and are explicitly aware of paral- serve that when we use a larger EC2 instance to speed up query ex- lelism. This allows HyPer to achieve better locality, load-balancing, ecution, the price per query rises. For example, the geometric mean and thus scalability, than VectorWise [22]. Using the 20 hyper- over our TPC-H queries for a m5.2xlarge instance with 8 vCPUs is threads on our 10-core CPU, we measured an average speedup 0.0002$ per query (2027 ms runtime). On an instance with 48 cores on the five TPC-H queries of 11.7× in HyPer, but only 7.2× in it is 0.00034$ per query (534 ms runtime). So in this case, running VectorWise. The parallelization framework is, however, orthogonal queries 4× faster costs 1.7× more. to the query processing model and we implemented morsel-driven Tectorwise and Typer have similar scaling behavior. Neverthe- parallelization in both Tectorwise and Typer, as it has been shown less, the “Ratio” column of Table 3, which is the quotient of the to scale better than exchange operators [22]. runtimes of both systems, reveals an interesting effect: For all but one query, the performance gap between the two systems becomes Morsel-driven parallelism was developed for HyPer [22] and can 8 therefore be implemented quite straightforwardly in Typer: The smaller when all 20 hyper-threads are used . For the join queries table scan loop is replaced with a parallel loop and shared data Q3 and Q9, the performance benefit of Tectorwise is cut in half, and structures like hash tables are appropriately synchronized similar Tectorwise comes closer to Typer for Q1 and Q18. This indicates to HyPer’s implementation [22, 23]. that hyper-threading is effective at hiding some of the downsides of For Tectorwise, it is less obvious how to use morsel-driven par- microarchitecturally sub-optimal code. allelism. The runtime system of Tectorwise creates an operator tree 8Q6 is memory bound and is the only exception. Typer’s branch- 7 We note that the benefit of SIMD can be larger when data is compressed [19] and on free selection implementation consumes more memory bandwidth vector-oriented CPUs like Xeon Phi (see Section 7). resulting in 20% lower performance at high thread counts.
2216 e.A ohpoesr aeadfeetcr onstegah are graphs the counts core different a have Threadrip- processors ex- on both and our Skylake As for on per. second) both / systems queries and queries (in performance. perimental performance absolute in the par shows on 11 roughly Figure are models CPU both that com- of quarter a only has but throughput. dollar, SIMD Intel. putational per de- and cores AMD the of more palettes represents offers product AMD rather CPU overall but the of coincidental choices sign not is processors these of h M hediprfaue 6cmuecrs lce tmax- However, at clocked cores, bandwidth. imally compute memory 16 features equal Threadripper almost AMD the an posses sys- Both also dollar. tems per same, performance the comparing almost allows cost directly Threadripper which AMD and Skylake Threadripper Intel AMD CPUs. vs. X Skylake Intel 7.1 its with AMD line. product at Phi look Intel’s also and microarchitecture now hold Zen we also recent platforms, results hardware our whether other out for find To Skylake. croarchitecture HARDWARE 7. analysis devices. in-memory I/O our modern with that ag- settings find and out-of-memory join we to the applies Overall, of performance affected queries. the more gregation than are bandwidth Q6 slower and the the Q1 expected, by as scan-dominated Furthermore, the of one). notice- performance still to but closer smaller moves slightly (“Ratio” are able engines differences query performance two the the that re- observe between can in-memory read we the is 3), Table with data (c.f. these when sults Comparing 100 shows factor 5 storage. scale Table secondary on from GB/s. threads 55 20 of with bandwidth runtimes a the has which memory, main Experiments Out-Of-Memory 6.3 al 5: Table 1 7729 0.98 1.16 2795 1.07 1.05 2827 2747 3268 1313 773 0.78 Q18 1405 808 Q9 1184 923 Q3 Q6 Q1 ntrso ur rcsigpromneoreprmnsshow experiments our performance processing query of terms In al hw h ehia pcfiain o u ne n AMD and Intel our for specifications technical the shows 4 Table mi- latest Intel’s on measured solely we experiments, previous In
4 Queries / Second . 0.0 0.5 1.0 1.5 2.0 2.5 GHz 5 Typer 4 5%100% 50% 0 ms . GHz 0 S Results SSD % CoresUsed u otisol 0crs h ifrne between differences The cores. 10 only contains but TW ms hl h ne klk lcsa ihrrate higher a at clocks Skylake Intel the while , q1 Ratio HT . Bsra adit nta of instead providing bandwidth SSDs 5 read SATA RAID GB/s 1.4 3 a in of data array we table this, the do stored To storage. from data secondary the fetching of measured impact also the we why is capac- which memory ity, main databases exceed main OLAP often in Large present was memory). data (i.e., table in-memory measure- all were far all so speed, ments maximum at 0 2 4 6 ocmaeTcows n Typer and Tectorwise compare To 5%100% 50% 0 Processor % CoresUsed q6 iue11: Figure klk Threadripper Skylake HT klk s Threadripper. vs. Skylake 0.0 0.5 1.0 5%100% 50% 0 % CoresUsed 2217 q3 efrac fTcows n ye ssmlro ohCPUs. both relative on The similar Skylake’s. is Typer than and higher Tectorwise 25% of performance execution to Landing’s 0 Knights from that observe is Q9 we and performance code Q3 the queries to join changes the any about Without 12. Figure of Skylake in against comparison shown Landing is A Knights Mode). of (Quadrant performance expose processing node query NUMA and one cache as L3 CPUs hardware-managed all a as memory bandwidth scenario. this in perform Typer and Tectorwise how machine. make OLAP available perfect are a like resources seem SIMD processor Landing many Knights that the of around fact number of memory, the available and bandwidth The cores, available. a is Additionally, second, with MB per core. 1 memory, GB neighboring 300 a high-bandwidth one and of with cache L1 shared GB 16 KB is 64 that a cache features L2 copied core to Each have not PCIe). does data through GPUs DDR4 to six contrast using (in accessed channels be memory directly can memory Main promising. pro- applications. vector That HPC 512-bit for TFLOPs. attractive multiple two it of with makes capacity aggregate equipped an is with On units core each cores. cessing each Xeon on to side, cores compared plus 72 slow relatively the to is 64 core inte- every are but an There chip, is It architecture: (HPC). many-core computing mi- grated high-performance This of for said line. processor a product be called Phi cannot also Xeon is This Intel’s croarchitecture of similar. generation quite second are the Threadripper and lake Phi) (Xeon Landing Knights 7.2 hyper-threads of use the degradation. queries performance SMT a some of in for benefit hyper- results and the from small, system, boost very AMD either performance the is On a queries. see all we for Skylake, threading much the be On to better. seems implementation hyper-threading Intel’s (SMT), Multi-Threading Simultaneous 2-way offer platforms both although same the almost is CPUs. platforms both per- hardware on relative two the the and ad- between similar an performance formance quite selective the has are Overall, Typer the platforms two Q1. and Tectorwise. characteristics query Q9 by computational and faster the on processed Q3 vantage are queries Q6 join in The scan similar. quite are ac fbt Psi eysmlrfrQ n 1 n h re- the and Q18 and Q6 (Q1 perfor- for similar < quite the similar still Notably, very are queries is avail- runtime. maining CPUs the the both achieve of of to percentage mance used which was show cores to able x-axis the on normalized o u xeiet nKihsLnigw ofiue h high- the configured we Landing Knights see on and experiments our qualities For machine’s this explore to want we Naturally, seems Landing Knights perspective, systems database a From Sky- manufacturers, hardware different two from being Despite that, is platforms two the between difference significant only The 40% Engine HT .As h eaiepromneo ye n Tectorwise and Typer of performance relative the Also ). – 0.0 0.1 0.2 0.3 0.4 0.5 SF=100 ye Tectorwise Typer 5%100% 50% 0 % CoresUsed q9 ngt Landing Knights HT 0.0 0.2 0.4 < 5%100% 50% 0 20% % CoresUsed n sdsge as designed is and Q3 , q18 < 25% HT Q9 , osbet opl l ure fasoe rcdr noasingle, a into procedure stored a of queries all is compile it contrast, to in possible single compilation, a With tra- over touch iteration. benefit Volcano-style only little ditional has might vectorization query workloads, OLTP in a For case where tuple. OLTP, the always in almost not is but which OLAP, processed, are values of vectors Support Multi-Language and OLTP importance. 8.1 greater of be may section, this effort, in implementation discuss mat- or we practical performance which OLTP a like as whether factors Therefore, recommendation other ter, compilation. general or a vectorization make use to differences to enough performance large The not are parallelized. properly par- when model—in ticular, two the between differences performance moderate FACTORS OTHER 8. the dollar, top. per on out speed come execution CPUs to commodity down when Thus broken is processors. AMD performance and each the Intel at our of comes the of price cost Landing at the the Knights twice almost looking Unfortunately, take to and account. need back into also 3 processor step we almost a picture, Q6 of taking performance of factor whole variant when scalar a However, the even for as Q6 applies). Sky- remark than query same faster the selection 50% (although to achieved the Knights up On query that join observe a lake. We execute to able account. is into Landing man- optimizations with measurements SIMD the take ual to need we Therefore, structions. per- Landings’s great. Knights As that Skylake, not faster seems than slightly two. formance being slower these queries slightly some between work- others with be mixed point, and this to a to difference up In the summary a expect Skylake. higher can on 10% one only performance load is respective performance In the Q6 cache. than L3 Landing’s an Knights have to case not of that configured measurement hardware our the use query with may one Q6 of reference query be much of evict frame would a likely As cache would data. the Q6’s which workload queries per- other the real boosts with a Q6, shared In query of GB, set unrealistically. 16 working of formance entire Each cache the The hold perspective: can repeatedly. in executed which is seen processor. measurements be Skylake our the must of of however, ahead measurement, Knights get to This of resources pro- channels required cache DDR4 the as memory vide 2 high-bandwidth extra the with the combined Landing Thus Knights Q6 bound. 2 on Finally, bandwidth to lower. up 30% about is is Landing it Q1 query On lower.
h etrzdeeuinmdlahee fcec hnmany when efficiency achieves model execution vectorized The only found and workloads OLAP on focused have we far, So in- SIMD of use heavy for designed is platform this fair, be To 20% about is performance Landing’s Knights Q18 query For Queries / Second 0.0 0.5 1.0 1.5 2.0 5%100% 50% 0 % CoresUsed q1 × atr ealta nSyaeqeyQ is Q6 query Skylake on that Recall faster. Processor HT 10 15 0 5 5%100% 50% 0 klk ngt adn KnightsLandingwithSIMD KnightsLanding Skylake % CoresUsed iue12: Figure q6 HT klk s ngt Landing Knights vs. Skylake 0.0 0.5 1.0 1.5 × 5%100% 50% 0 is % CoresUsed 2218 q3 re fcnuciefitrpeiae ree on satechnique a is joins even or predicates filter conjunctive of order Adaptivity 8.4 only can system basis. per-pipeline the a since on to currently performance operators, contributions is measure individual relational it the of reason know time this to For execution SQL Spark it. in for possible responsible not operator plan the relational query if the to the profiling, lines in code code generated sample-based back map using can done system be pipeline, could a it of in- time data- the though execution of For final the contribution to values. the operators separate thousand relational to dividual a hard on is works it function compilation, centric the overhead, to marginal call getting only each adds since primitive as queries, each for down counts slowing cycle clock without possible is profiling Debuggability and Profiling 8.3 byte Java KB 8 than more generates tuple-at- code. pipeline interpreted a to addi- if back of execution falls cost a-time Spark the the complexity. at answer obvi- system comes largely to compilation—but tional This enough of is downside [17]. this omitted that ates is if to compilation used data; LLVM is full of that query, morsels interpreter first al- IR LLVM allocation the register a execute scalable contains even more allocation own and its register gorithm, by as them such replaces passes and optimization switches LLVM HyPer certain compila- this. off against data-centric measures Real-world mitigating take systems generation. tion code some to leads o fcd ftable if as code such of but query lot code, SQL of small amounts large that a generate queries also LLVM will OLAP (optimizing) operators size. many code because of to large consist super-linear often is become is code also time compile generated can time the compilation if However, large time. be OLTP can of by code-generation in ahead countered case done issue which be in an can procedures, stored it be on workloads relying can transactional This in though queries, [48]. time execution dominating Time Compilation 8.2 execution same 42]. the 31, 32, into [11, languages environment multiple and functions defined [12]. compilation-based Hekaton the engine Server integrate additionally SQL to Microsoft compelled the felt 20]), team [21, mod- (Apollo a engine having compi- vectorized already of ern Despite benefit systems. HTAP major and a OLTP for is lation This fragment. code machine efficient dpieqeyeeuin o ntnet eodrteevaluation the re-order to instance for execution, query Adaptive detailed that is execution vectorized of advantage practical A iavnaeo oegnrto sters fcmiaintime compilation of risk the is generation code of disadvantage A user- integrating for beneficial highly be also can Compilation HT – 0.0 0.2 0.4 0.6 0.8 SF=100. 5%100% 50% 0 Engine % CoresUsed T a huad fclms sec column each as columns, of thousands has q9 ye Tectorwise Typer EET*FO T FROM * SELECT HT 0.0 0.2 0.4 5%100% 50% 0 % CoresUsed a rdc a produce can q18 HT for improving robustness that can compensate for (inevitable) esti- Compilation mation errors in query optimization. Integrating adaptivity in com- HyPer [28] Peloton [26] piled execution is very hard; the idea of adaptive execution works best in systems that interpret a query—in adaptive systems they can HyPer + Data Blocks [19] change the way they interpret it during runtime. Vectorized execu- tion is interpreted, and thus amenable for adaptivity. The combina- Sompolski [45] tion of fine-grained profiling and adaptivity allows VectorWise to make various micro-adaptive decisions [39]. Interpretation We saw that VectorWise was faster than Tectorwise on TPC-H System R [25] VectorWise [7] Q1 (see Table 1); this is due to an adaptive optimization in the for- Tuple-at-a-time Vectorization mer, similar to [15], that it not present in the latter. During aggre- gation, the system partitions the input tuples in multiple selection Figure 13: Design space between vectorization and compilation vectors; one for each group-by key. This task only succeeds if there – hybrid models integrate the advantages of the other approach. are few groups in the current vector; if it fails the system exponen- tially backs off from trying this optimization in further vectors. If it during runtime and profiling is easier. Finally, both systems have succeeds, by iterating over all elements in a selection vector, i.e. all their own implementation challenges: Implementing operators with tuples of one group in the vector, hash-based aggregation is turned code generation introduces an additional indirection, whereas vec- into ordered aggregation. Ordered aggregation then performs par- torization comes with a set of constraints on the code, which can tial aggregate calculation, keeping e.g. the sum in a register which be complicated to handle. strongly reduces memory traffic, since updating aggregate values in a hash table for each tuple is no longer required. Rather, the 9. BEYOND BASIC VECTORIZATIONAND aggregates are just updated once per vector. DATA-CENTRIC CODE GENERATION 8.5 Implementation Issues 9.1 Hybrid Models Both models are non-trivial to implement. In vectorized exe- Vectorization and data-centric code generation are fundamen- cution the challenge is to separate the functionality of relational tally different query processing models and the applied techniques operators in control logic and primitives such that the primitives are mostly orthogonal. That means that a design space, as visu- are responsible for the great majority of computation time (see alized in Figure 13, exists between them. Many systems combine Section 2.1). In compiled query execution, the database system ideas from both paradigms in order to achieve the “best of both is a compiler and consists of code that generates code, thus it is worlds”. We first describe how vectorization can be used to im- harder to comprehend and debug; especially if the generated code prove the performance of the compilation-based systems HyPer and is quite low-level, such as LLVM IR. To make code generation Peloton, before discussing how compilation can help vectorization. maintainable and extensible, modern compilation systems intro- In contrast to other operators in HyPer, scans of the compressed, duce abstraction layers that simplify code generation [42] and make columnar Data Block format [19] are implemented in template- it portable to multiple backends [32, 31]. For HyPer, some of these heavy C++ using vectorization and without generating any code abstractions have been discussed in the literature [29], while others at runtime. Each attribute chunk (e.g., 216 values) of a Data Block have yet to be published. may use a different compression format (based on the data in that It is worth mentioning that vectorized execution is at some disad- block). Using the basic data-centric compilation model, a scan vantage when sort-keys in order by or window functions are com- would therefore have to generate code for all combinations of ac- posite (consist of multiple columns). Such order-aware operations cessed attributes and compression formats—yielding exponential depend on comparison primitives, but primitives can only be spe- code size growth [19]. Besides compilation time, a second bene- cialized for a single type (in order to avoid code explosion). There- fit of using vectorization-style processing in scans is that it allows fore, such comparisons must be decomposed in multiple primitives, utilizing SIMD instructions where it is most beneficial (Section 5). which requires a (boolean) vector as interface to these multiple Since, Data Blocks is the default storage data format, HyPer (in primitives. This extra materialization costs performance. Com- contrast to Typer) may be considered a hybrid system that uses vec- piled query execution can generate a full sort algorithm specifically torization for base table selections and decompression, and data- specialized to the record format and sort keys at hand. centric code generation for all other operators. 8.6 Summary By default, data-centric code generation fuses all operators of the same pipeline into a single code fragment. This is often ben- As a consequence of their architecture and code structure compi- eficial for performance, as it avoids writing intermediate results lation and vectorization have distinct qualities that are not directly to cache/memory by keeping the attributes of the current row in related to OLAP performance: CPU registers as much as possible. However, there are also cases where it would be better to explicitly break a single pipeline into Lang. Comp. Pro- Adapt- Imple- multiple fragments—for example, in order to better utilize out-of- OLTP Support Time filing ivity mentation order execution and prefetching during a hash join. This is the Comp. XX (X) Indirection key insight behind Peloton’s relaxed operator fusion [26] model, which selectively introduces explicit materialization boundaries in Vect. XXX Constraints the generated code. By batching multiple tuples, Peloton can easily On the one hand, compiled queries allow for fast OLTP stored introduce SIMD and software prefetching instructions [26]. Con- procedures and seamlessly integrating different programming lan- sequently, Peloton’s pipelines are shorter and their structure resem- guages. Vectorization, on the other hand, offers very low query bles vectorized code (see Figure 13). If the query optimizer’s de- compile times, as primitives are precompiled: As a result of this cision about whether to break up a pipeline is correct (which is structure, parts of a vectorized query can be swapped adaptively non-trivial [24]), Peloton can be faster than both standard models.
2219 Table 6: Query Processing Models – and pioneering systems. of the reasons it has been adopted by Virtuoso [8]. One downside of the push model is that it is slightly less flexible in terms of con- System Pipelining Execution Year trol flow: A merge-sort, for example, has to fully materialize one System R [25] pull interpretation 1974 input relation. Some systems, mostly notably MonetDB [10], do PushPull [27] push interpretation 2001 not implement pipelining at all—and fully materialize intermediate MonetDB [9] n/a vectorization 1996 results. This simplifies the implementation, but comes at the price VectorWise [7] pull vectorization 2005 of increased main memory bandwidth consumption. Virtuoso [8] push vectorization 2013 In the last decade, compilation emerged as a viable alternative Hique [18] n/a compilation 2010 to interpretation and vectorization. As Table 6 shows, although HyPer [28] push compilation 2011 compilation can be combined with all 3 pipelining approaches, the Hekaton [12] pull compilation 2014 push model is most widespread as it tends to result in more efficient code [47]. One exception is Hekaton [12], which uses pull-based The two previous approaches use vectorization to improve an compilation. An advantage of pull-based compilation is that it au- engine that is principally based on compilation. Conversely, com- tomatically avoids exponential code size growth for operators that pilation can also improve the performance of vectorized systems. call consume more than once. With push-based compilation, an op- Sompolski et al. [45], for example, observed that it is sometimes erator like full outer join that produces result tuples from two dif- beneficial to fuse the loops of two (or more) VectorWise-style prim- ferent places in the source code, must avoid inlining the consumer itives into a single loop—saving materialization steps. This fusion code twice by moving it into a separate function that is called twice. step would require JIT compilation and result in a hybrid approach, thus moving it towards compilation-based systems in Figure 13. However, to the best of our knowledge, this idea has not (yet) been 10. SUMMARY integrated into any system. To our surprise, the performance of vectorized and data-centric Tupleware is a data management system focused on UDFs, specif- compiled query execution is quite similar in OLAP workloads. In ically a hybrid between data-centric and vectorized execution, and the following, we summarize some of our main findings: uses a cost model and UDF-analysis techniques to choose the exe- < Computation: Data-centric compiled code is better at compu- cution method best suited to the task [11]. tationally-intensive queries, as it is able to keep data in registers Apache Impala uses a form of compiled execution, which, in and thus needs to execute fewer instructions. a different way, is also a hybrid with vectorized execution [49]. Rather than fusing relational operators together, they are kept apart, > Parallel data access: Vectorized execution is slightly better in and interface with each other using vectors representing batches of generating parallel cache misses, and thus has some advantage tuples. The Impala query operators are C++ templates, parame- in memory-bound queries that access large hash-tables for ag- terized by tuple-specific functions (data movement, record access, gregation or join. comparison expressions) in, for example, a join. Impala has default = SIMD has lately been one of the prime mechanisms employed slow ADT implementations for these functions. During compila- by hardware architects to increase CPU performance. In theory, tion, the generic ADT function calls are replaced with generated vectorized query execution is in a better position to profit from LLVM IR. The advantages of this approach is that (unit) testing, that trend. In practice, we find that the benefits are small as most debugging and profiling can be integrated easily—whereas the dis- operations are dominated by memory access cost. advantage is that by lack of fusing operators into pipelines makes the Impala code less efficient. = Parallelization: With find that with morsel-driven parallelism both vectorized and compilation based-engines can scale very 9.2 Other Query Processing Models well on multi-core CPUs. Vectorization and data-centric compilation are the two state-of- = Hardware platforms: We performed all experiments on Intel the-art query processing paradigms used by most modern systems, Skylake, Intel Knights Landing, and AMD Ryzen. The effects and have largely superseded the traditional pull-based iterator model, listed above occur on all of the machines and neither vector- which effectively is an interpreter. Nevertheless, there are also ization nor data-centric compilation dominates on any hardware other (more or less common) models. Before discussing the strengths platform. and weaknesses of these alternative approaches, we taxonomize them in Table 6. We classify query processing paradigms regarding Besides OLAP performance, other factors also play an important (1) how/whether pipelining is implemented, and (2) how execution role. Compilation-based engines have advantages in is performed. Pipelining can be implemented either using the pull < OLTP (next) interface, the push (produce/consume) interface, or not at all as they can create fast stored procedures and (i.e., full materialization after each operator). Orthogonally to the < language support as they can seamlessly integrate code written pipelining dimension, we use the execution method (interpreted, in different languages. vectorized, or compilation-based) as the second classification cri- terion. Thus, in total there are 9 configurations, and, as Table 6 Vectorized engines have advantages in terms of shows, 8 of these have actually been used/proposed. > compile time as primitives are pre-compiled, Since System R, most database systems avoided materializing intermediate results using pull-based iteration. The push model be- > profiling as runtime can be attributed to primitives, and came prominent as a model for compilation, but has also been used > adaptivity as execution primitives can be swapped mid-flight. in vectorized and interpreted engines. One advantage of the push model is that it enables DAG-structured query plans (as opposed Acknowledgements. This work was partially funded by the DFG to trees), i.e., an operator may push its output to more than one grant KE401/22-1. We would like to thank Orri Erling for explain- consumer [27]. Push-based execution also has advantages in dis- ing Virtuoso’s vectorized push model and the anonymous reviewers tributed query processing with Exchange operators, which is one for their valuable feedback.
2220 11. REFERENCES [20] P. Larson, C. Clinciu, C. Fraser, E. N. Hanson, M. Mokhtar, M. Nowakiewicz, V. Papadimos, S. L. Price, S. Rangarajan, R. Rusanu, and M. Saubhasik. Enhancements to SQL Server [1] https://stackoverflow.com/questions/36932240/ column stores. In SIGMOD, pages 1159–1168, 2013. avx2-what-is-the-most-efficient-way-to-pack- [21] P. Larson, C. Clinciu, E. N. Hanson, A. Oks, S. L. Price, left-based-on-a-mask, 2016. S. Rangarajan, A. Surna, and Q. Zhou. SQL Server column [2] S. Agarwal, D. Liu, and R. Xin. Apache Spark as a compiler: store indexes. In SIGMOD, pages 1177–1184, 2011. Joining a billion rows per second on a laptop. [22] V. Leis, P. Boncz, A. Kemper, and T. Neumann. ”https://databricks.com/blog/2016/05/23/apache- Morsel-driven parallelism: A NUMA-aware query spark-as-a-compiler-joining-a-billion-rows-per- evaluation framework for the many-core age. In SIGMOD, second-on-a-laptop.html”, 2016. pages 743–754, 2014. [3] K. Anikiej. Multi-core parallelization of vectorized query [23] V. Leis, K. Kundhikanjana, A. Kemper, and T. Neumann. execution. Master’s thesis, University of Warsaw and VU Efficient processing of window functions in analytical SQL University Amsterdam, 2010. queries. PVLDB, 8(10):1058–1069, 2015. http://homepages.cwi.nl/∼boncz/msc/2010- [24] V. Leis, B. Radke, A. Gubichev, A. Mirchev, P. Boncz, KamilAnikijej.pdf. A. Kemper, and T. Neumann. Query optimization through [4] R. Appuswamy, A. Anadiotis, D. Porobic, M. Iman, and the looking glass, and what we found running the join order A. Ailamaki. Analyzing the impact of system architecture on benchmark. VLDB Journal, 2018. the scalability of OLTP engines for high-contention [25] R. A. Lorie. XRM - an extended (n-ary) relational memory. PVLDB workloads. , 11(2):121–134, 2017. IBM Research Report, G320-2096, 1974. ¨ [5] C. Balkesen, J. Teubner, G. Alonso, and M. T. Ozsu. [26] P. Menon, A. Pavlo, and T. Mowry. Relaxed operator fusion Main-memory hash joins on multi-core CPUs: Tuning to the for in-memory databases: Making compilation, underlying hardware. In ICDE, pages 362–373, 2013. vectorization, and prefetching work together at last. PVLDB, [6] P. Boncz, T. Neumann, and O. Erling. TPC-H analyzed: 11(1):1–13, 2017. Hidden messages and lessons learned from an influential [27] T. Neumann. Efficient generation and execution of TPCTC benchmark. In , 2013. DAG-structured query graphs. PhD thesis, University of [7] P. Boncz, M. Zukowski, and N. Nes. MonetDB/X100: Mannheim, 2005. CIDR Hyper-pipelining query execution. In , 2005. [28] T. Neumann. Efficiently compiling efficient query plans for [8] P. A. Boncz, O. Erling, and M. Pham. Advances in modern hardware. PVLDB, 4(9):539–550, 2011. Linked Open Data - large-scale RDF data management. In [29] T. Neumann and V. Leis. Compiling database queries into Creating Knowledge Out of Interlinked Data - Results of the machine code. IEEE Data Eng. Bull., 37(1):3–11, 2014. LOD2 Project, pages 21–44. Springer, 2014. [30] S. Padmanabhan, T. Malkemus, R. C. Agarwal, and [9] P. A. Boncz, M. L. Kersten, and S. Manegold. Breaking the A. Jhingran. Block oriented processing of relational database Commun. ACM memory wall in MonetDB. , 51(12):77–85, operations in modern computer architectures. In ICDE, 2008. pages 567–574, 2001. [10] P. A. Boncz, W. Quak, and M. L. Kersten. Monet and its [31] S. Palkar, J. J. Thomas, D. Narayanan, A. Shanbhag, geographic extensions: A novel approach to high R. Palamuttam, H. Pirk, M. Schwarzkopf, S. P. Amarasinghe, performance GIS processing. In EDBT, pages 147–166, S. Madden, and M. Zaharia. Weld: Rethinking the interface 1996. between data-intensive applications. CoRR, abs/1709.06416, [11] A. Crotty, A. Galakatos, K. Dursun, T. Kraska, C. Binnig, 2017. U. C¸etintemel, and S. Zdonik. An architecture for compiling [32] S. Palkar, J. J. Thomas, A. Shanbhag, M. Schwarzkopt, S. P. PVLDB UDF-centric workflows. , 8(12):1466–1477, 2015. Amarasinghe, and M. Zaharia. A common runtime for high [12] C. Freedman, E. Ismert, and P. Larson. Compilation in the performance data analysis. In CIDR, 2017. IEEE Data Eng. Microsoft SQL Server Hekaton engine. [33] J. M. Patel, H. Deshmukh, J. Zhu, H. Memisoglu, N. Potti, Bull., 37(1):22–30, 2014. S. Saurabh, M. Spehlmann, and Z. Zhang. Quickstep: A data [13] G. Graefe. Encapsulation of parallelism in the Volcano query platform based on the scaling-in approach. PVLDB, processing system. In SIGMOD, pages 102–111, 1990. 11(6):663–676, 2018. [14] G. Graefe and W. J. McKenna. The Volcano optimizer [34] H. Pirk, O. Moll, M. Zaharia, and S. Madden. Voodoo - A generator: Extensibility and efficient search. In ICDE, pages vector algebra for portable database performance on modern 209–218, 1993. hardware. PVLDB, 9(14):1707–1718, 2016. [15] T. Gubner and P. Boncz. Exploring query compilation [35] O. Polychroniou, A. Raghavan, and K. A. Ross. Rethinking strategies for JIT, vectorization and SIMD. In IMDM, 2017. SIMD vectorization for in-memory databases. In SIGMOD, [16] T. Kersten. https://github.com/TimoKersten/db- pages 1493–1508, 2005. engine-paradigms, 2018. [36] O. Polychroniou and K. A. Ross. High throughput heavy [17] A. Kohn, V. Leis, and T. Neumann. Adaptive execution of hitter aggregation for modern SIMD processors. In DaMoN, compiled queries. In ICDE, 2018. 2013. [18] K. Krikellas, S. Viglas, and M. Cintra. Generating code for [37] O. Polychroniou and K. A. Ross. Vectorized bloom filters for holistic query evaluation. In ICDE, pages 613–624, 2010. advanced SIMD processors. In DaMoN, 2014. [19] H. Lang, T. Muhlbauer,¨ F. Funke, P. Boncz, T. Neumann, and [38] O. Polychroniou and K. A. Ross. Efficient lightweight A. Kemper. Data Blocks: Hybrid OLTP and OLAP on compression alongside fast scans. In DaMoN, 2015. compressed storage using both vectorization and compilation. In SIGMOD, pages 311–326, 2016.
2221 [39] B. Raducanu, P. Boncz, and M. Zukowski. Micro adaptivity in Vectorwise. In SIGMOD, pages 1231–1242, 2013. [40] V. Raman, G. K. Attaluri, R. Barber, N. Chainani, D. Kalmuk, V. KulandaiSamy, J. Leenstra, S. Lightstone, S. Liu, G. M. Lohman, T. Malkemus, R. Muller,¨ I. Pandis, B. Schiefer, D. Sharpe, R. Sidle, A. J. Storm, and L. Zhang. DB2 with BLU acceleration: So much more than just a column store. PVLDB, 6(11):1080–1091, 2013. [41] S. Schuh, X. Chen, and J. Dittrich. An experimental comparison of thirteen relational equi-joins in main memory. In SIGMOD, pages 1961–1976, 2016. [42] A. Shaikhha, Y. Klonatos, L. Parreaux, L. Brown, M. Dashti, and C. Koch. How to architect a query compiler. In SIGMOD, pages 1907–1922, 2016. [43] U. Sirin, P. Toz¨ un,¨ D. Porobic, and A. Ailamaki. Micro-architectural analysis of in-memory OLTP. In SIGMOD, pages 387–402, 2016. [44] E. A. Sitaridi, O. Polychroniou, and K. A. Ross. SIMD-accelerated regular expression matching. In DaMoN, 2016. [45] J. Sompolski, M. Zukowski, and P. A. Boncz. Vectorization vs. compilation in query execution. In DaMoN, pages 33–40, 2011. [46] D. Song and S. Chen. Exploiting SIMD for complex numerical predicates. In ICDE, pages 143–149, 2016. [47] R. Y. Tahboub, G. M. Essertel, and T. Rompf. How to architect a query compiler, revisited. In SIGMOD, pages 307–322, 2018. [48] A. Vogelsgesang, M. Haubenschild, J. Finis, A. Kemper, V. Leis, T. Muhlbauer,¨ T. Neumann, and M. Then. Get real: How benchmarks fail to represent the real world. In DBTest, 2018. [49] S. Wanderman-Milne and N. Li. Runtime code generation in Cloudera Impala. IEEE Data Eng. Bull., 37(1):31–37, 2014. [50] T. Willhalm, N. Popovici, Y. Boshmaf, H. Plattner, A. Zeier, and J. Schaffner. SIMD-Scan: Ultra fast in-memory table scan using on-chip vector processing units. PVLDB, 2(1):385–394, 2009. [51] J. Zhou and K. A. Ross. Implementing database operations using SIMD instructions. In SIGMOD, pages 145–156, 2002. [52] M. Zukowski. Balancing Vectorized Query Execution with Bandwidth-Optimized Storage. PhD thesis, University of Amsterdam, 2009. [53] M. Zukowski, S. Heman,´ N. Nes, and P. A. Boncz. Super-scalar RAM-CPU cache compression. In ICDE, 2006.
2222