A Hardware-Oblivious Optimizer for Data Stream Processing Constantin Pohl supervised by Prof. Dr. Kai-Uwe Sattler Technische Universitat¨ Ilmenau Ilmenau, Germany [email protected] ABSTRACT properties come into play, like memory hierarchies or CPU High throughput and low latency are key requirements for core count. data stream processing. This is achieved typically through For this paper, we consider different aspects and para- different optimizations on software and hardware level, like digms of parallelization that are applicable on data stream multithreading and distributed computing. While any con- processing. In addition, first measurements on SIMD and cept can be applied to particular systems, their impact on multithreading realized on an Intel Core i7 and Intel Xeon performance and their configuration can differ greatly de- Phi Knights Landing (KNL) with our SPE PipeFabric are pending on underlying hardware. shown. Our final goal is a full optimizer on a SPE capable Our goal is an optimizer for a stream processing engine of dealing with unknown UDFs in queries as well as with ar- (SPE) that can improve performance based on given hard- bitrary hardware the system uses. Two consequential tasks ware and query operators, supporting UDFs. In this paper, arise from this. we consider different forms of parallelism and show mea- surements exemplarily with our SPE PipeFabric. We use a • Exploitation of opportunities given by modern hard- multicore and a manycore processor with Intel's AVX/ AVX- ware, like wider CPU registers on Intel's AVX-512 in- 512 instruction set, leading to performance improvements struction set, manycore processors with 60+ cores on through vectorization when some adaptations like micro- a chip or increased memory size. batching are taken into account. In addition, the increased • Analysis of performance impacts by possible UDFs and number of cores on a manycore CPU allows an intense ex- operations, like computational effort or possible paral- ploitation of multithreading effects. lelization degree in case of data dependencies. Keywords The rest of this paper is organized as follows: Next Sec- tion 2 is a short recapitulation about stream processing, Data Stream Processing, AVX, Xeon Phi, SIMD, Vectoriza- possible parallelism and opportunities given by hardware. tion, Parallelism Section 3 summarizes related work done on SIMD paral- lelism, hardware-oblivious operations and stream partition- ing in context of data stream processing. Our results are 1. INTRODUCTION presented in Section 4, followed by Section 5 with future Technological advancement leads to more and more op- work. Section 6 with conclusions tops off this work. portunities to increase application performance. For stream processing, data arrives continuously at different rates and from different sources. A stream processing engine (SPE) 2. PREREQUISITES has to execute queries fast enough that no data is lost and This section shortly summarizes requirements on stream results are gathered before the information is already out- processing and parallelization opportunities in addition to dated. A solution to achieve this is a combination of software information on used hardware, like supported instruction optimizations paired with modern hardware for maximizing sets. parallelism. It is difficult to find an optimal parametrization though, e.g. for the number of threads or load balancing 2.1 Data Stream Processing between them. It gets even worse when different hardware As already mentioned, high throughput and low latency are main requirements for stream processing. A data stream delivers continuously one tuple of data after another, possi- bly never ending. Queries on data streams have to consider that tuples can arrive at alternating rates and they get out- dated after a while, because storing all of them is impossible. Operating with windows of certain sizes are a common so- lution for this property. Proceedings of the VLDB 2017 PhD Workshop, August 28, 2017. Munich, As a consequence, a query has to be processed fast enough Germany. Copyright (c) 2017 for this paper by its authors. Copying permitted for to produce no outdated results as well as keeping up with private and academic purposes.. eventually fast tuple arrival rates. Handling multiple tuples at once through partitioning of data or operators exploiting parallelism possibilities is therefore a must. Data Stream T B Batch Op . 2.2 Parallelism Paradigms There are mainly three forms of parallelism on stream Tuple T Batch B processing that can be exploited. <A1...An> <T1.A1...Tm.A1> Partitioning. Partitioning can be used to increase speedup ... through splitting data on different instances of the same op- <T1.An...Tm.An> erator. Therefore every instance executed in parallel has to add its function on a fraction of data, increasing throughput Figure 1: Processing Model of a query. However, additional costs for assigning data to instances and merging results afterwards arise, influencing the optimal partitioning degree. Task-Parallelization. Operators of a query can execute gathered first on a batching operator with attribute group- in parallel if data dependencies allow such parallelism. A ing in memory realized by vectors until batch size is reached pipelining schema provides possibilities to achieve this, re- and then forwarded to the next operator. alized by scheduling mechanisms. Vectorization. A single instruction can be applied on mul- 2.3 Hardware Opportunities tiple data elements, called SIMD. For stream processing, There are mainly two different ways to increase compu- some preparatory work is necessary to use this form of par- tational speed on hardware, distributed and parallel com- allelism, like storing a number of tuples before processing puting. Distributed computing uses usually many high-end them at once with SIMD support. On the one hand, it in- processors connected to each other, sharing computational creases the throughput of a query while on the other hand work between them. The disadvantage comes with commu- batching up tuples worsens latency. nication costs. With requirements of low latency, we focus on parallel computing. Manycore processors like the Xeon We focus on partitioning and vectorization, because task- Phi series from Intel use simpler cores, but many of them in- parallelism is mainly a scheduling problem that is not of side their CPU. This eliminates most of the communication further interest at this point. costs, improving latency while providing wide parallelization opportunities compared to a single multicore processor. 2.2.1 Partitioning The latest Xeon Phi KNL uses up to 72 cores with 4 threads each, available through hyperthreading. In addi- A speedup through partitioning is achieved mainly with tion, the AVX-512 instruction set can be used for 512bit multithreading. Each partition that contains operators pro- wide operations (SIMD). This leads to great possibilities on cessing incoming tuples uses a thread, which leads to chal- partitioning and vectorization to reduce latency of a query. lenges on synchronization or load balancing, especially on There are more interesting improvements on KNL like clus- manycores, as shown by Yu et al. [6] for concurrency con- tering modes, huge page support or high-bandwidth memory trol mechanisms. Partitioning is a key for high performance on chip (MCDRAM) which we will address in future work. when using a manycore processor, which provides support for 200+ threads at the cost of low clock speed. To investi- gate the right partitioning degree between reduced load on 3. RELATED WORK each partition and increased overhead from threads as well For parallelism through vectorization and partitioning on as an appropriate function for forwarding tuples to parti- data streams, a lot of research has been done already, espe- tions additional observations have to be made. Statistics cially since manycore processors are getting more and more are a common solution, but can be far away from optimal common. To achieve performance benefits, those manycore performance in worst cases. CPUs rely massively on SIMD parallelism and multithread- ing for speeding up execution time. 2.2.2 Vectorization For data stream processing, the functionality of operations To use vectorization, certain requirements must be ful- like joins or aggregations to give an example, are basically filled. Without batching up tuples it is impossible to ap- the same. However, for realization the stream processing ply a vector function on a certain attribute. This leads to properties have to be taken into account. the next challenge, the cache-friendly formation of a batch. Without careful reordering, any vectorization speedup is lost Polychroniou et al. [4] take a shot on different database through expensive scattered memory accesses. A possible operators, implementing and testing vectorized approaches solution for this is provided by gather and scatter instruc- for each one. Results show a performance gain up to a mag- tions that index through masking certain memory addresses nitude higher than attempts without SIMD optimization. In for faster access. Additional requirements on vectorization addition to this, their summary of related work gives a good arise through the used operator function and data depen- review about research done with SIMD on general database dencies between tuples. The function must be supported by operations. used instruction set, while dependencies are solved through For stream partitioning, the degree in terms of numbers of rearrangements or