Rumble: Data Independence for Large Messy Data Sets Ingo Müller Ghislain Fourny Stefan Irimescu∗ ETH Zurich ETH Zurich Beekeeper AG Zurich, Switzerland Zurich, Switzerland Zurich, Switzerland
[email protected] [email protected] [email protected] Can Berker Cikis∗ Gustavo Alonso (unaffiliated) ETH Zurich Zurich, Switzerland Zurich, Switzerland
[email protected] [email protected] ABSTRACT 1 INTRODUCTION This paper introduces Rumble, a query execution engine for large, JSON is a wide-spread format for large data sets. Its popularity heterogeneous, and nested collections of JSON objects built on top can be explained by its concise syntax, its ability to be read and of Apache Spark. While data sets of this type are more and more written easily by both humans and machines, and its simple, yet wide-spread, most existing tools are built around a tabular data flexible data model. Thanks to its wide support by programming model, creating an impedance mismatch for both the engine and languages and tools, JSON is often used to share data sets between the query interface. In contrast, Rumble uses JSONiq, a standard- users or systems. Furthermore, data is often first produced in JSON, ized language specifically designed for querying JSON documents. for example, as messages between system components or as trace The key challenge in the design and implementation of Rumble is events, where its simplicity and flexibility require a very low initial mapping the recursive structure of JSON documents and JSONiq development effort. queries onto Spark’s execution primitives based on tabular data However, the simplicity of creating JSON data on the fly often frames.