Generating Flexible Workloads for Graph Databases ˚ Guillaume Bagan Angela Bonifati Radu Ciucanu CNRS LIRIS University of Lyon 1 & CNRS LIRIS University of Oxford [email protected] [email protected] [email protected] George H. L. Fletcher Aurélien Lemay Nicky Advokaat TU Eindhoven University of Lille 3 & INRIA TU Eindhoven g.h.l.fl[email protected] [email protected] [email protected] ABSTRACT management systems. Often in practice, graph database sce- Graph data management tools are nowadays evolving at narios are driven by sophisticated query workloads. Funda- a great pace. Key drivers of progress in the design and mental examples include multi-query optimization, mapping study of data intensive systems are solutions for synthetic discovery and query rewriting in data integration systems, generation of data and workloads, for use in empirical stud- and workload-driven graph database physical design [2]. ies. Current graph generators, however, provide limited Recently, there has been a flurry of works on graph data- or no support for workload generation or are limited to base benchmarking [1, 4, 5], some of them addressing syn- fixed use-cases. Towards addressing these limitations, we thetic workload generation [1]. In our work, we extend this demonstrate gMark, the first domain- and query language- line of research by pushing forward the diversity and flexi- independent framework for synthetic graph and query work- bility of graph and query workload generation. load generation. Its novel features are: (i) fine-grained con- In particular, we have designed and implemented gMark, trol of graph instance and query workload generation via a domain- and query language-independent framework tar- expressive user-defined schemas; (ii) the support of expres- geting highly tunable generation of both graph instances sive graph query languages, including recursion among other and graph query workloads based on user-defined schemas. features; and, (iii) selectivity estimation of the generated One of the most notable features of gMark is support for queries. During the demonstration, we will showcase the controlled schema-driven generation of fully diverse queries highly tunable generation of graphs and queries through var- not only in terms of syntactic features but also in terms of ious user-defined schemas and targeted selectivities, and the expressiveness (in particular, recursive queries) and query variety of supported practical graph query languages. We selectivity (size of the query result as a function of the will also show a performance comparison of four state-of- size of any input graph, reflecting the runtime behavior of the-art graph database engines, which helps us understand queries). These features, which are not supported by current their current strengths and desirable future extensions. tools, permit fine-grained targeted testing of current graph data management systems, to better understand their per- formance, limitations, and optimization opportunities. 1. INTRODUCTION Contributions. We demonstrate gMark for graph and que- Graph data management tools are rapidly evolving, as ry generation. In summary, our system is highly tunable, new features and capabilities are explored in response to featuring: (i) schema-driven data and query generation, (ii) practical demand and increasingly sophisticated user require- multiple practical graph query languages (SPARQL, open- ments. To better understand and drive forward this evo- Cypher, SQL, Datalog) including recursion among the fea- lution, solutions for the generation of synthetic graph in- tures, and (iii) selectivity estimation of generated queries. stances and query workloads are fundamental for the design We will showcase graph instance and query workload gener- of empirical studies of graph systems. Indeed, graph and ation through a set of highly tunable parameters and user- query generation solutions are important for understanding defined schemas. We will show that our system is able to the current limitations and potential improvements to graph simulate realistic graph domains and to fruitfully leverage user interactions. We will also present our generated in- stances and query loads at work with four current (commer- ˚ Supported by EPSRC platform grant DBOnto. cial and open-source) graph database management systems. The goal of the demonstration is to highlight the novel gMark capabilities and features. We will call attention to the importance of the continued study and development of domain- and query language-independent synthetic gen- This work is licensed under the Creative Commons Attribution- eration solutions, which are crucial in understanding the NonCommercial-NoDerivatives 4.0 International License. To view a copy strengths and desirable directions for future research of graph of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0/. For data management systems. We present a system overview in any use beyond those covered by this license, obtain permission by emailing Section 2 and our demonstration scenarios in Section 3. In [email protected]. Proceedings of the VLDB Endowment, Vol. 9, No. 13 our technical report [3] we focus more deeply on the gMark Copyright 2016 VLDB Endowment 2150-8097/16/09. application to system performance comparison. 1457 Graph configuration ‚ Size ‚ Node types Graph instance file ‚ Edge predicates Graph&query generator (CSV) ‚ Schema constraints ‚ Degree distributions SPARQL Query workload configuration openCypher ‚ Size ‚ Selectivity Query workload file Query translator ‚ Recursion (UCRPQs as XML) PostgreSQL ‚ Shape ‚ Arity Datalog Figure 1: Overview of the gMark workflow. Node type Constr. 2. SYSTEM OVERVIEW Edge predicate Constr. researcher 50% authors 50% Given a graph configuration and a query workload config- paper 30% publishedIn 30% uration, our system generates directed edge-labeled graphs journal 10% heldIn 10% conference 10% and query workloads coupled to these graphs. Moreover, extendedTo 10% city 100 (fixed) our query generator supports four concrete syntaxes, and (b) Edge predicates. can be easily extended to support additional practical query (a) Node types. language syntaxes. The goal of this section is to highlight source type predicate target type In-distr. Out-distr. the main features and workflow of the system, as illustrated ÝÝÝÝÝÝÑ 1 researcher authors paper Gaussian Zipfian in Figure 1. We provide gMark as open-source software ÝÝÝÝÝÑ paper publishedIn conference Gaussian Unif. [1,1] for use in the graph processing community. Our system is ÝÝÝÝÝÝÝÝÑ paper extendedTo journal Gaussian Unif. [0,1] implemented in C++ using the Standard Library. ÝÝÝÝÝÝÝÑ conference heldIn city Zipfian Unif. [1,1] ÝÝÝÝÑ 2.1 Graph generation (c) In- and out-degree distributions. Graph generation is driven by a graph configuration, which specifies constraints that the generated graphs should sat- Figure 2: The bibliographical motivating example. isfy. The first parameter is the graph size, defined as the number of nodes. The next parameters (cf. Figure 1) en- code a graph schema which allows the user to specify the instance, the first line encodes that the number of authors on sets of node types and edge labels; schema constraints (as papers follows a Gaussian distribution (the in-distribution proportions of a node type’s or an edge label’s occurrences of the schema constraint), whereas the number of papers au- or a fixed constant value); and, the in- and out-degree dis- thored by a researcher follows a Zipfian (power-law) distri- tributions of edge labels, with support for the following dis- bution (the out-distribution of the schema constraint). The tributions: uniform, Gaussian (also known as normal), and following lines in Figure 2(c) encode constraints such as: Zipfian. For each distribution, the user can specify the rel- a paper is published in exactly one conference, a paper can evant parameters (i.e., min and max for uniform, µ and σ be extended or not to a journal, a conference is held in ex- for Gaussian, and s for Zipfian). If the user wants to specify actly one city, the number of conferences per city follows a only the in- or the out-distribution of an edge label, she can Zipfian distribution, etc. We can specify all aforementioned mark the other one as nonspecified. constraints in gMark via a few lines of XML. Example 2.1 Assume that we want to generate graphs sim- 2.2 Query workload generation ulating a bibliographical database that uses a simple schema Before introducing the parameters of the query workload consisting of 5 node types and 4 edge predicates. Intuitively, configuration (cf. Figure 1), it is important to first present the database consists of researchers who author papers that the abstract query language supported by gMark. are published in conferences (held in cities) and that can be Query language. We focus on unions of conjunctions extended to journals. We also want to specify constraints on of regular path queries (UCRPQ), a fundamental language the number of occurrences for both the node types and edge which covers many queries that appear in practice [7], e.g., predicates, either as proportions of the total size of the graph the core constructs of SPARQL 1.12 (SPARQL with recur- or as fixed numbers (cf. Figure 2(a) and 2(b)). For instance, sion) and openCypher3. Apart from recursive and non re- for graphs of arbitrary size, half of the nodes should be re- cursive graph queries, our language is expressive enough to searchers, but a fixed number of nodes should be cities where cover typical analytic workloads including listing triangles, conferences are held (in a realistic scenario the number of vertex neighborhood, and clique detection. authors increases over time, whereas the number of cities Given an input graph schema, let Σ` “ ta; a´ | a P Σu, remains constant). Then, we want to specify real-world rela- where Σ is the set of edge labels and a´ denotes the inverse tionships between types and predicates as in Figure 2(c). For 2http://www.w3.org/TR/sparql11-overview/ 1https://github.com/graphMark/gmark 3http://www.opencypher.org/ 1458 of edge label a.