Analyse und Vergleichvon gängigen Graph-Anfragesprachen

DIPLOMARBEIT

zur Erlangung des akademischen Grades

Diplom-Ingenieur

im Rahmen des Studiums

Software Engineering &Internet Computing

eingereicht von

Martin Klampfer,BSc Matrikelnummer 01526110

an der Fakultät für Informatik der Technischen Universität Wien Betreuung: Univ.Prof.Mag.rer.nat. Dr.techn. Reinhard Pichler Mitwirkung: Univ.Ass.Dipl.-Ing. Matthias Lanzinger,BSc

Wien, 9. Dezember 2020 Martin Klampfer Reinhard Pichler

Technische UniversitätWien A-1040 Wien Karlsplatz 13 Tel. +43-1-58801-0 www.tuwien.at

Analysis and Comparisonof CommonGraph Query Languages

DIPLOMA THESIS

submitted in partial fulfillmentofthe requirements forthe degree of

Diplom-Ingenieur

in

Software Engineering &Internet Computing

by

Martin Klampfer,BSc Registration Number 01526110

to the Faculty of Informatics at the TU Wien Advisor: Univ.Prof.Mag.rer.nat. Dr.techn. Reinhard Pichler Assistance: Univ.Ass.Dipl.-Ing. Matthias Lanzinger,BSc

Vienna, 9th December,2020 Martin Klampfer Reinhard Pichler

Technische UniversitätWien A-1040 Wien Karlsplatz 13 Tel. +43-1-58801-0 www.tuwien.at

Erklärung zur Verfassungder Arbeit

Martin Klampfer,BSc

Hiermit erkläre ich, dass ichdieseArbeit selbständig verfasst habe,dass ichdie verwen- detenQuellenund Hilfsmittel vollständig angegeben habeund dass ichdie Stellen der Arbeit–einschließlichTabellen,Karten undAbbildungen –, dieanderen Werken oder dem Internet im Wortlaut oder demSinn nach entnommensind, aufjeden Fall unter Angabeder Quelleals Entlehnung kenntlich gemacht habe.

Wien, 9. Dezember 2020 Martin Klampfer

v

Danksagung

Ichmöchtemichganz herzlichbei meinemBetreuer, Prof. Dr.Reinhard Pichler, für seine Unterstützung durchden ganzen Diplomarbeits-Prozess bedanken. Er unterstützte mich beider Auswahl einesspannendenThemas, ließ mir nützliche Unterlagen zukommen und warimmer füreinen Ratschlagerreichbar. Außerdemgilt mein herzlicher Dank meinen Elternund meinem Bruder,die michwährend meiner Studienzeit immer unterstützten, ermutigten und anspornten.

vii

Acknowledgements

First and foremostIwanttothank my supervisorProf. Dr. Reinhard Pichlerfor his support and guidancethroughoutthe whole process. He not onlyhelpedmewhen first choosing atopicbut providedvaluable materialsand wasalwaysreachable whenIneeded some advice.Iwould also like to thank my parentsand my brother, they notonly supported me withmywish to studybut also encouraged me throughout those years.

ix

Kurzfassung

Graphdatenbanken werdenimmeröfter verwendet um vernetzte Daten abzuspeichern. Viele dieser Systeme bietenein flexiblesDatenmodellbasierend auf property graphs, dassind GraphenwoKnoten und Kanten mit Tags gekennzeichnet und weitere Eigen- schafteninSchlüssel-WertPaarengespeichertwerdenkönnen. Um auf Daten in solch einerDatenbankzuzugreifenverwendet manAnfragesprachen. Im Unterschied zu re- lationalen Datenbanken, wo SQL die standardisierte Anfragesprache ist,gibt es noch keine standardisierte Anfragesprache für Graphdatenbanken. Nutzer könnendahervon einerbreiten Palette an Sprachenwählendie sichindem verwendeten Datenmodell, der Ausdrucksstärke,der Einfachheitder Verwendung und so weiter unterscheiden. Das Ziel dieser Arbeitist einsystematischerVergleich vonsolchen Sprachen. Daraus wollen wir Hilfestellungenzur Sprachauswahl beibestimmten Anwendungsfällen ableiten. In einemerstenSchritt identifizieren und analysieren wirKernfunktionen dieinallen Anfragesprachenfür Graphdatenbanken vorhandensind. Dazu zählt dasFindenvon Teilgraphen anhand vonGraphmustern (patterns), vonPfaden mittels Pfadanfragen (path queries) und die Kombinationdieser beidenzuNavigationsanfragen(navigatio- nalqueries). Wir erweitern diese Funktionen um strukturunabhängige Anfragen sowie Datenmanipulations- undDatendefinitions-Operationen. Anhanddieser Funktionen analysieren wir fünf moderne Anfragesprachenfür Graphda- tenbanken: Cypher, Gremlin, PGQL,GSQL und G-CORE. Wir analysieren dieSprachen nicht nuranhand vongenerellen Charakteristika, sondern implementieren auch Teile des “Social Network Benchmarks” und analysieren diese Anfragen.Esstelltsichheraus, dass Cypher, PGQL und G-COREeine ähnlicheSyntax verwendensowie ähnlich aus- drucksstarkund einfachzuverwenden sind. Aufder anderenSeite ähneln sichGSQL und Gremlin: beide sindTuring-vollständig undunterstützen nichtnur deklarative, sondern auchimperativeKonstruktionen.

xi

Abstract

Graph databasesare increasingly adoptedinindustry as they offer nativesupport for graph-like data. Manysuchsystems use aflexible datamodel based on property graphs, theseare graphswherevertices and edges can be labeledand augmented with key-value pairs,the properties. Data stored in suchadatabaseisaccessed viagraph query languages. UnlikeSQL,thatisthe standardized query language for relational databases,there is no standard foragraphquery languageyet. Userscan thereforechoose from arange of languages that vary in theirspecific datamodel, expressiveness, ease of use and so on. The goal of this thesisisasystematic comparisonofgraphquery languages and to give guidelines for choosingalanguage for specific application scenarios.Towardsachieving this goal, we identify and analyze corefeatures inherent to suchlanguages. The most common onesare subgraph discovery by matching graph patterns, pathdiscoveryusing path queries and thecombination of both in navigational queries. We extend these featurestoalso include structure independent queries as well as datamanipulation and data definition operations. Basedonthese features,wethen analyze fivecontemporarygraphquery languages: Cypher, Gremlin, PGQL, GSQL and G-CORE. Theanalysis is notonly basedon general characteristicsofthe languages as we alsoimplementpartsofthe SocialNetwork Benchmarkand analyze thesequeries. It turns out that Cypher, PGQL and G-CORE are quitesimilarintheir syntax, expressiveness and ease of use. On theotherhand, GSQL and Gremlin sharesome similarities as they are both Turing-completelanguages thatare notsolely declarativebut alsoofferimperativeconstructs.

xiii

Contents

Kurzfassung xi

Abstract xiii

Contents xv

1Introduction1

2Graph Databases 3 2.1 BriefHistory and Scope...... 3 2.2 BasicConcepts ...... 4 2.3 Graph Data Models ...... 8 2.4 Query Languages ...... 10 2.5 LDBCSocialNetwork Benchmark ...... 17

3QueryLanguage Features 21 3.1 Structure Independent Queries ...... 21 3.2 Pattern Matching Queries...... 22 3.3 Path and Navigational Queries ...... 28 3.4 DataManipulation ...... 38 3.5 DataDefinition ...... 40 3.6 Further Features ...... 41

4QueryLanguage Analysis45 4.1 Cypher...... 46 4.2 Gremlin...... 59 4.3 PGQL ...... 74 4.4 GSQL ...... 83 4.5 G-CORE ...... 95 4.6 Further Languages ...... 103 4.7 LessonsLearned ...... 106

5Conclusion 113

xv ListofFigures 115

List of Tables 117

Listings117

Acronyms 121

Bibliography 123 CHAPTER 1

Introduction

Relational databasesare well studied and optimized for datathatcan be modeledvia tabular relations. However, using arelational database comeswith serious restrictions, especiallywhen the informationishighly interconnected. In this case, querying data often meanstomovealong some relationships between theentities. This is typicallydone by joiningmultiple relations,whichadds computationaloverhead compared to systems tailored forsuchqueries. Furthermore, relational databases are not well suitedtohandle adynamic model where the structure changesovertime. Graph databasesare atypeofaNoSQLdatabase and specifically developedtohandle graph-like data, where information coming from relationships is at least as importantas informationstoredinentities [Ang12]. Theyare increasinglyadopted in industry as for examplechemistry-, biology-, or social network- related data canoften be modeledas agraph withinformation storedinnodes andrelationships connecting them. In these cases graph databases offer amorenatural andflexible model as well as nativesupport forqueriesonagraph structure, likenavigating alongthe relationships [AG08, RWE15]. Graph databases have been aroundsince the 1980s and enjoyedariseinpopularity over the last decade. Thiscomes from technicaladvancements that allow large graphslike social networks with millionsorevenbillions of nodestobestoredand processed [AG18]. Data stored in agraph database is accessedvia aquerylanguage. Unlike for relational database managementsystems1 ,whereSQL is the standardized query language,there is no standardfor agraph query language yet, as work on suchastandard started in 2019. Therefore, manyvendorsofgraphdatabasesystems define and use their ownquery language that differs from others in their expressiveness, their supported datatypes, theirease of use andsoon. The rise of interest in graph databasesisaccompanied by

1 We use the terms database management system, database system or simply database interchangeably for the system that “enables users to define, create, maintainand control access” [CB05]toacollection of data.

1 1. Introduction

academiaputting more focus on this area as well. This resulted in abetter understanding of somesystems and theirunderlying structures, andalso in the invention of morequery languages. Choosing asystemtherefore involves notonly the propertiesofthe system but alsothe supportedquery language. With many languages to choose from, it is important to get an overviewabout theirexpressiveness, their characteristicsand theirease whenformulating queries.But to our knowledgethere is no comprehensive comparisonofcurrentmajor and most promising languages regardingthese factors. Thisthesis providesacomparison of common query languages, especially regardingtheir expressiveness and ease of use. To get acomprehensible comparison, core features of query languages are identifiedand analyzed, andthe comparisonisdone based on these features. We analyze four languages from industry (Cypher, Gremlin, PGQL,GSQL) and one promising language from aresearchproject(G-CORE). Theanalysis is not onlybasedongeneral information as we alsoimplementparts of theSocialNetwork Benchmark. Whereas afull implementation of all chosen queriescan be found on our GitHub repository2,weanalyzeonly some of them in detail in this thesis. It turnsout that Cypherisagood choice, both when first starting with graph databases and for theuse in applications, as longasmore expressive path queriesand aschema are not needed.PGQL is very similartoCypher, and what it lacks in functionality it makes up for with sophisticated path querying capabilities. AlthoughGremlin allows for expressive queries, its low-level approachwith afocus on imperative traversals makes it harder to get into for someone familiar with high-level declarative querylanguages. GSQL is designed forhuge graphs with support fornativeparallel query execution. That results in aslightly different syntax comparedtoCypher and PGQL, but themajor difference is the support of astrong type systemand schema.Weconcludewith G-CORE, an expressive researchlanguage that combinesusefulfeatures from existinglanguages, achieves fullcomposabilityand elevates pathstofirst-class citizens in the graph. The remainder of this thesisisstructured as follows: Chapter2introducesgraph databases in general,including theirunderlying data modelsand the query languages we focus on in our comparison. We then identify commonfeatures of query languages and analyzethese languages regardingthe expressiveness they enable in Chapter 3. The fivequerylanguages are introducedand analyzed in Chapter 4. We endthis chapter with ageneral comparison and provide guidelines for selecting alanguage for specific application scenarios.Finally, Chapter 5concludes thethesisand includes notes on future work.

2https://github.com/martin-kl/Diploma_Thesis_01526110_code

2 CHAPTER 2

GraphDatabases

Since the 1980s, amultitude of database systemssupporting differentgraphmodels and query languages targetingdifferent applications have been introduced. We start with adescription of graph databasesingeneral and givesome historical background in Section 2.1.The basicconceptsofgraph databasesand underlying data modelsare introduced in Section2.2 and Section 2.3, respectively.Section 2.4 gives an introduction to fiveofthe most commongraph query languages. These are thelanguages we analyze and compare in Chapter 4.

2.1BriefHistory and Scope

Graph databasesand query languages have been aroundfor multiple decades.Initially introduced in the 1980s, with theinventionofgraphdatabaseslikeG-BASEin1987 [Kun87]and corresponding data modelsand query languageslikeG[CMW87], interest inthe areadeclinedinthe later1990s as databasestargetingfor examplespatial and geographical dataemerged [AG08]. In the late 2000s and especiallyduring the last decade we sawabreakthroughofgraphdatabases, as theneed for frequentschema changesand real-time query responsesondatabases containing vast amounts of interconnected data became morerelevant[FB18]. They are nowused to handledatafrom awide range of fields: from transport networks overbiological information like genes and protein interactions to socialnetworks [RWE15]. Using agraph databasebecamearequirementtohandle vast amounts of interconnected data in fields likethese, as they are designed to allow amore natural and faster access to graph-like data [Ang12]. As an examplefor this, assume that we modelsuchdata in arelational database.This could be done by storingthe edges in atablewhereeach rowrepresents an edgethatcontains twoforeign keyreferences, one for eachadjacent node. We endupwith lots of many-to-manyrelationshipsfor all theconnections carrying data.Althoughweare abletomodel graphsinthis way, queryingthe structure can

3 2. Graph Databases

become quitecumbersome. In contrasttoconventionaldatamanagementweare not only interested in the valuesstoredinthe tablesbut alsointhe connection patterns [EAL+15]. If we search for suchpatterns or query not onlyinformation stored in asingle relation,we use join operations tomovealongapath 1 in the graph. These joinscan easilybecome complex,therefore computationally demandingand impact theperformance,evenifonly asmallpart of thegraphisqueried. Graph databases on the otherhand are designedto store and process such interconnected data.They canhandle large graphswhile providing ahighperformance, especially for queries traversingthem [HP16]. One cansplit the numerous available systemshandling graph dataintotwo maincategories basedontheir focus on either transactional or analytical processing: [RWE15, RH19]

Transactional Systems thatare focused on transactional processingare usuallyonline, meaningthat they are built for real-time access to thecurrentdata. Similar to the world of relational databasestheyare designed for OnlineTransactional Processing (OLTP) and called graph database managementsystems or simply graph databases. We focus on these tasks and systems in ourthesis.

Analytical Systems focused on analyzinggraph-likedata can either be onlineoroffline, i.e. they canwork on thecurrentdata or on an offline copyofit. Regardless of this design choice, thesetasksare often called Online AnalyticalProcessing(OLAP)2 andare typically done in batch-processing fashion.Graph databases support these tasks to some degree but are not optimized for them and sometimes even lackthe ability to perform them at all.Therefore, specialtechnologiesexist thatare often called graph compute engines or graphanalytics systems.

2.2 Basic Concepts

Unlikerelational databasesthat are builtontabular relationsasunderlying data structure, graph databasesare builtonthe mathematicalstructures of graphs. Although thereare multiple differentdata modelsusedinvarious graph databases (see Section 2.3), they all representtheir domain using itsstructure via nodes (vertices), edges (relationships,links) andinmost casesproperties. Unlessmentionedotherwise,weassume directed graphsas most systems do not support undirected edges. Let’s look at an example for suchadata structure. Figure2.1 depicts asmallsocialnetwork containing three nodes representing its users thatare connected via multiple edges. Edges and nodesare either labeledas User or FOLLOWS and eachnodehas apropertycalled name.The structure tells us thatAlice follows Bob, Carol follows Alice and Carol and Bob follow eachother. This is asmallmotivating exampleofasocial network thatcouldeasily be extended to also containfurtherinformationabout the users or relationships by addingmore properties.

1Moving along apath,that is aconcatenationofadjacentedges,means to traverse the pathfrom its start to its end node. 2They are sometimes also denoted as Offline Analytical Processing [BPG+19].

4 2.2. BasicConcepts

Figure2.1: Asmallsocialgraph.

Furthermore, the graph couldalso be extended to includeother entities likepostsor messages by adding appropriate nodes and edges. Depicting information in such aconnected data model allowsagraph database to handle highly interconnecteddatawell, but one has to keep in mind that building on graphs comes with its owndrawbacks.While queryingfor related dataand traversingthe graph can be done efficiently, queryingthe global structure is often challenging and shows notably worse performance compared to relationaldatabasesinsome cases. An example forsuchaglobal querywould be to searchfor all usersataspecific age or for persons thathaveatleastone friendfollowing them (i.e. one incoming edge labeled FOLLOWS in our example). As graph databasesdonot provideparticularly efficientsolutions for all problems, they are not seen as areplacementfor theirrelational counterparts but more as acomplementary system thatisused especiallytoanalyze stronglyinterrelated data. [FB18]

2.2.1 NativevsNon-NativeGraph Representation Databases with built-in support forgraphscan be split into two categories that differ in theirinternal datarepresentation:native(alsopure)graphdatabasesand multi-model (also non-native) ones [EAL+15, AG18, BPG+19].

• Native graph databases like Neo4j3 and TigerGraph4 focussolely on datastructured in graphs. As theirdatamodel and querylanguages are specificallydesignedwith graphsinmind, graphsare first class citizens in thesesystems. Most of these databases do not only presentthe data as agraph to the userbut alsouse agraph data structure internally.Thisallows them to be optimized for certaingraph queries. An examplewouldbethe provisionofconstanttime adjacency retrievals regardlessofthe sizeofthe graph thatispossibleinsuchsystems. Furthermore,

3https://neo4j.com/ 4https://www.tigergraph.com/

5 2. Graph Databases

manysuchsystems provide graph specific functions like the shortestpathalgorithm or breadth-first search out of thebox.

• Multi-modeldatabases support other models apart from graphs, like the relational or documentoriented one, as well.Manyofthese systemstransformthe data into adifferentmodel to store it.Consequently, aglobal indexiscommonlyused to store and retrieve adjacencyinformation. Although some of thesesystemsalso provide graph specificfunctions out of thebox,they are in manycases not that performant caused by the use of differentstorage and processingmechanisms. On the other hand, multi-model databasescan integratedatacoming in various different structuresand possibly frommultiple sources into asingle system. Having all data in asingle system allowsfor auniform access via asinglequery language and the data can be analyzed overdifferent structures. Examplesfor such systems are ArrangoDB5 and AzureCosmos DB6.

Regardlessofagraph databasebeing categorized as nativeormulti-model we can identify two major components in suchasystem: thestorage and theprocessing component [RWE15]. Although thesecomponents are commoninbothdatabase categories, they differsubstantially and their designand mechanismsinfluence thecategory of adatabase.

Storage The storage componentisresponsible for reliablestorage and access to thedata. One possibility to achievethis is to serialize the graph andstore it in arelational table. This allowsfor theuse of asingle copy of the data thatcan be accessed by either SQLqueriesorquerieswritten in agraph query language,depending for exampleonthe knowledge of the engineers or theease-of-use for thegiven problem. However, serializingthe graph andkeeping it in adatamodel that is not specifically designed and optimized for suchinterconnected data comes with all performance drawbacks mentionedearlier. Thisisone of thereasons whydatabaseswith sucha non-native storage component are not categorizedasnativegraph databases, but as multi-model ones. Anotherpossibility is to storedatadirectly in aconnected structure thatallows for faster, in manycases constanttime, retrievals of adjacententities.Byusing such a storage component, thesedatabasesare usuallyabletoscale notablybettertolarge graphscontaining millionsofnodes while stillprovidingahighperformance for most graph related queries. Astorage component built on suchadesign is referred to as native graph storage.Asmentioned earlier,most nativegraph databases representthe data internallyasagraph and use such anativegraph storage.

Processing The processing componentconnects to thestorage componentand is re- sponsible for all operations on the data.Asthe choice of the storage mechanism heavily influences the innerworkings of theprocessing component, likethe use of a

5https://www.arangodb.com/ 6https://azure.microsoft.com/services/cosmos-db/

6 2.2. BasicConcepts

nativegraph storage that providesdifferentperformancecharacteristics and query possibilities to theprocessing component, the borderbetween thesecomponents is quite blurry.Therefore, the most interesting part of theprocessing component concernsalsothe storage technique andisabout the questionwhether the system reliesonindex-free adjacency or not. Whenusing index-free adjacency,adjacentnodes and edges are physically stored next to eachother with direct referencestoeachother. Thesecomponents are also referred to as native graph processing and are used in most nativegraph databases. As an example, assume that we have the nodes x, y and therelationship e =(x, y) in asystem. With index-free adjacency,the processing componentcan “move”from x to e and further to y by directly retrieving theirinformation without the needof aglobalindexcontaining therelationships.Having thesedirectpointersallows for constanttime retrieval of an adjacentnodeorrelationship,regardless of thesizeof the whole graph.Onthe downside,graphdatabasesingeneral and especiallynative graph processingsystems thatrelyontheir localadjacency information instead of global indexes have the problemthatsharding agraph is very hard.Sharding is the process of partitioning agraph into multiple subgraphs that can be stored on differentmachinestoachievehorizontal scalability. [RN11] Anotheroption for thedesign of aprocessing component is to use aglobalindex thatcontains references to relationships and thenodes in theserelationships.To navigatethe graph,data is joined with the global index. This approachisused in multi-model graph databases.

2.2.2 Indexes Indexes are used to speed up access to relevantdata. Relational databases relyheavily on theseindexes, as they enable tablestobeefficientlyjoined and data entries to be efficiently found. Graphdatabasesalso rely on indexesinvarious ways. Systems not built on theprinciple of index-freeadjacency use an index containing adjacency information. Thisindexcan then be usedtomovebetween connected entities in thegraph by joining the data with the index. Depending on thesystem,this canwork similar to relational databases. Graph databases relying on index-freeadjacencyhave localindexes integrated into the graph structure. As briefly explained in the processing componentabove,every node and edge in suchasystem keepsdirectpointerstoall adjacentnodes and edges. These are stored together with theentityand form alocal indexfor every node andedge. Once the start node of atraversalisfound, they can traverse the neighborhood without accessingadedicated or global indexbyusing thepointers in their local index [BPG+19]. Thisisasignificantadvantage compared to systems without index-free adjacency where performance usually deteriorates as thegraphbecomes bigger, caused by an increasein the sizeofthe index.However, thesedatabasesalso use additional indexestofind specific nodes in the graph,likethe start node in aquery,and to accelerateother operations, especiallyglobal ones. As an example, assume that our social network fromFigure 2.1 alsocontains theage of every user. If we want to getthe names of all usersolder than20

7 2. Graph Databases

years for example, an indexonthe age allowsthe system to checkfor thecondition on the index, and limitsthe access to nodes that satisfyit. [RN11]

2.3GraphData Models

Depending on thedefinition,the term data model (also database model,db-model or model) comprises not onlythe underlying datastructure usedbyadatabase system but alsothe data description, integrityconstraints,maintenanceand even access and query mechanisms [AG08, Ang18]. We focus on theunderlying data structure used by a database systeminthis section. As brieflymentionedearlier, it is defined aroundthe mathematical structures of graphs, nodesand edges [Ang12]. Unlikeinthe world of relational databasesthere is no standard data model usedingraph databases. Figure 2.1 is an example of a labeledproperty graph that is widely used, especially in native graph databases. Other notablemodelsinclude the Resource Description Framework (RDF) modelused inthe domainofSemanticWeb and hypergraphs thatgeneralize thenotion of agraph to allow edges between morethantwo nodes [AG18]. We nowtakeacloser lookatthe RDF and various propertygraph modelsasthese are the data modelsused in most graphdatabases of our time.

2.3.1 RDF The RDF model encodesdata in subject-predicate-object expressions and databasesusing this model aretherefore alsodenoted as Triplestore.Incontrast to other models,ituses aschema that is integrated together with thedataintoasingle graph.The schemaisa smallpart of thegraphand contains information about possibleconnections as well as node and edge types. SinceitisaWorld Wide WebConsortium(W3C)recommendation, it enjoyswidespreadrecognition,gooddocumentationand the available systemsare unified by acommon and standardizedquery language:SPARQL. It wasoriginally developedtorepresentweb metadata, butcan alsobeused in amoregeneral wayto modelinterconnected resources using theaforementioned triples. On the other hand, thesesystemsare usually notnativegraph databases as manytransform the triplesinto other formatsand store them forexample in relational tables. Nonetheless, they fall under the category of graph databasesaslarger amounts of thesetriples canform abody of interconnected nodesand therefore agraphstructure. [RWE15, AG08, AG18] Figure 2.2 gives an exampleofanRDF graph containing thedatafrom our social network example. An exampleofatripleinthis case is “Alice followsBob”, where Alice is the subject(theresource thatisdescribed), follows the predicate (property) and Bob the object (or property value) [AG08]. The schemaisgiven in aseparatepart of the graph thatisconnected to theinstancesvia type-edges. These connections representthe information given by the labels in Figure 2.1. Apartfrom the missing labels there arealso no inherentpropertiesonnodes.Suchapropertyisstoredinatriplethatintroduces a node containing thevalue and an edge labeled by the nameofthe property. An example for thisisthe node "Alice" connectedbythe name-edge to thenode user1.

8 2.3. Graph Data Models

Figure 2.2: Asmallsocialgraphinthe RDF data model.

Despite being endorsed by W3C and having,incontrast to other graph databases,a standardizedquery language, theseRDF stores are mainlyused in the domain of Semantic Web. Areason for thelackofutilization in other domains couldbethe comparably bloated datastructure that uses an additional node and edge for every label and property. Anotherreason couldbethe use of aschema that limitsthe flexibilityofthe data model [SS19].

2.3.2Labeled (Property) Graphs Edge-labeledgraphs are one of thesimplest form of graphswherenodes and edges can be eachlabeled with one label.Labelsassignedtoedges indicate the type of therelationship between thenodes. Thisallows us to model different relationships as well as multiple edges between anytwo nodes in asingle graph.RDF-graphsare builtonthe basis of edge-labeled graphsbut differintheir use of aschema andtherefore the use of node and edge types. Edge-labeled graphsdonot provide aschema and typescan onlybegiven implicitly by setting an appropriate label. Figure 2.3 shows thedata fromour small social network example encoded in an edge-labeled graph.Weadded the information thatCarol is thesister of Alice to showthe powerofedgelabels. Without theselabels, we would simply have twoedges fromCarol to Alice withoutbeing abletodistinguish them.[AAB+17] (Labeled)Property graphs,also LPGsorsimply propertygraphs,extendthis model by addinglabelstonodes and including propertiesdirectly in the nodes and edges. Properties, alsocalled attributes, are key-valuepairs containingnon-graphical information like name: Carol.The inclusionofpropertiesallows foramore natural datamodelingand faster retrievalofthe data related to anodeoredge[Ang12]. Therefore, propertygraphs can be described as a"directed, labeled, attributed multigraph"[AG18], where multigraph denotes that multiple edges betweenany two nodes are possible.Although this model

9 2. Graph Databases

Figure 2.3: Asmallsocialgraphinthe edge-labeleddata model.

is more popularthanothers [FGG+18b, BPG+19]and usedinmanygraphdatabases, it is notstandardized and manyvariants of it exist.Angles [Ang18]presents aformal definitionofthe propertygraph data model thatallows nodes and edges to be labeled with multiple labels and everypropertytobeassigned aset of values. Other definitions for examplerestrictedges to be labeledbyonlyasinglelabel [FGG+18b]. Somepapers also requirethe useofaunique identifier for every node and edge,orrestrict properties to onlystore single values insteadofsets[AAB+17]. Let’s look at an enriched examplebasedonour small socialnetwork. Thepropertygraph in Figure 2.4 containsauniqueidentifierfor everynodeand edge. This allowsfor fast and clear identification of entities and is alsoused in manygraphdatabases. As property graphsdonot have an explicit schema, thetypes of nodes and edges are given through labels attachedtothem.Our example allows anodetohave multiple labels,like User and Admin on node n1.Incontrast to theRDF model fromFigure 2.2, theproperties of anodeare stored inside the node itself. This results in amorecompact modelaswe do not need an additionalnodefor everyproperty.Asmentionedbefore, edges can also have propertieslikethe since attributes on the follows relationships. Propertygraphs are very flexible as they do not use an explicit schema. Onecan simply omitpropertiesthatare notavailable forsome parts in thegraph, likethe missingage attribute in the node n3.Likewise,other propertiescan be added to nodes, like the mail propertyinn3.Furthermore,ifwewanttoadd new informationlikeapicture thatisposted by one of our users, we can add anodelabeled Picture and connect it via an edge labeled posted to the user that posted it. Sincemostnative graph databasesuse the propertygraph data model,they do not have adefinedschema and are thereforemore flexiblecompared to relational or other schema-dependent databases. We will focusonthe propertygraph model and some variations of it in therestofthis thesis as manypopular graph query languagesare designed to operate on some variationofthatmodel.

2.4Query Languages

Aquery language is ahigh-level language used to accessdatastoredinadatabase [ARV19]. In the strictestdefinition, aquerylanguage onlysupportsoperations to retrieve

10 2.4. Query Languages

Figure 2.4: Asmallsocialgraph in the propertygraph data model.

data.Codddescribes suchalanguage moreformally as “a collectionofoperators or inferencingrules, whichcan be applied to any validinstances of the data types (ofthe model), to retrieve or derive data from any partsofthosestructuresinany combinations desired” [Cod80]. This descriptionlimitsqueriestoonly retrievedata, andsuchoperators form theclass of Data Query Language (DQL).Most definitionsextend this strictform to alsoinclude manipulationoperations thatallowtoadd,modify and remove data.These operationsform theclass of the Data Manipulation Language (DML).Databases that rely on, or support aschema furthermoreinclude operationsofthe Data Definition Language (DDL).Analogously to the relationalworld wherethe StructuredQuery Language (SQL) alsosupportsmore than just dataretrieval operations but is denoted as querylanguage, agraph query language supports DML and sometimes DDLoperationsaswell. Taken together,agraphquery languagedefines operations to access and manipulatedatathat is structured as agraph and provides support for graph specificoperations[ARV19]. [Cha12, Ang12] Figure 2.5 gives an overviewofsome graph query languages thatare specificallydesigned forgraph data models. As brieflymentionedearlier, thegraphical query language G [CMW87] wasintroduced in 1987 and is one of thefirst suchlanguages.Wewill briefly analyzethis languageasitintroduced importantconcepts that are stillinuse in modern query languages.Gis built on thedatamodel of edge-labeled graphs and introduced the concept of agraphicalquery.Aquery Q consistsofaset of labeled and directed graphs, called querygraphs. Nodesinaquerygraph are labeled by either constants or variables, and edges can be labeledbyregularexpressions over constants and variables. These querygraphs canbeseenasgraphpatterns that are matched to subgraphsofthe graph database G to evaluatethe query.

Figure 2.6 shows an example of aquery Q = {Q1, Q2} thatcontains twoquerygraphs.

11 2. Graph Databases

Figure 2.5: Evolution of graph query languages. Source:AnIntroductiontoGraph Data Management, Figure 1.7[AG18].

Q1 : Q2 : y

’AC’ C’ + ’A w ’Tor’ x ’Tor’ z ’AC’ ’AC’

Figure 2.6: An exampleofagraphicalquery in the query languageG.Source: AGraphical Query Language Supporting Recursion, page 2, example 2[CMW87].

Each node in thedatabase G represents acityand is labeledbyits namewhile the edges represent airline connections betweenthe cities and are labeled by the nameofthe airline. The query Q findsthe first and last citiesvisited by roundtrips from’Tor’ (Toronto), with therestriction that thefirst and lastconnections are via ’AC’ (Air Canada).Another restrictionisgiven by the regularexpression w+ thatlimits all connections between the cities y and z,the first and last citiesinaroundtrip, to be with thesame airline.Such aregular expression is depicted as adashed line to highlightthat this is apath and does not have to be asingle edge. Thesupport for regular expressions in query graphs allows foramore general and simplerexpressionofrecursivequeries compared to other query languageslikeStructured Query Language (SQL) for example [AG18].

Allmajor propertygraph query languages introduced in this millennium are built around some core languagefeatures. Thestructureofaquery graph as agraphical pattern, like the simplepattern depicted by Q1,isarguable the most popularfeature supported by most,ifnot all,graph query languages. Depending on thelanguage, thesepatterns can be either given directly as agraph patternlikeinG,orasatextual representation of the patternasinmanymodern query languages. We define this patternand other featuresof query languages and provide examplesfor them in Chapter3.Anothernotabledifference

12 2.4. Query Languages

between manygraph query languages and theirrelational counterpart SQLis, that their inputformat differsfromtheir output format. Unlikeinthe relationalworld, wherea query takes atable(relation) as input andoutputsatable, agraph query takes agraph as inputbut in many languages outputs atablecontaining values[Ang18]. Thisisthe mainreason whymanygraph query languages are not composable. We givemore details on thecomposability of queriesinSection 3.6.

The language Gevolved into G+ [CMW88]that added support for graph specificfunctions like the computationofthe sizeofthe shortestpath, and extended thecapabilities of querygraphs. Since then, manydifferentgraph query languages builtonvariousdata models were introduced. Examples are thelanguages THQL[WS90]and HQL [The02] forhypergraphs, Graphlog [CM90], GRE [Woo90], Gram [AS92]and PDL [ABR13] (as PDQL in Figure 2.5) for edge-labeled graphsand SPARQL version1.0 [PS08]and version1.1 [HS13]for RDFgraphs. Many of the query languagesinFigure 2.5 are moretheoreticaland notused inreal world database system. Therise in popularityof graph databases thatmainlyuse thepropertygraph data model wasaccompanied by the introduction of query languages builtonthis model.Examples of suchlanguages are PRPQ [LS06], Gremlin[Rod15], Cypher [FGG+18b]and PGQL [vRHK+16], thelatter three being actively used in currentgraph databases.

Apartfrom SPARQL, that is standardizedbythe W3C, there areavarietyoflanguages builtonotherdatamodelsthan RDF as no standardexists forthese models. Furthermore, at thetime of writing,noquery language that is currentlyused by anativegraph database has acomplete and formal specification. One reasonfor this is that these query languages have notbeen aroundfor that longand researchers are onlystarting to formalizeparts of them, like with the popular query language Cypher and itsopensource variant openCypher[MSV17, FGG+18a]. As manyofthesequery languagesare still being activelydeveloped,they evolveand change frequentlywhichmakes an effort for aformal specification somewhat impractical at thecurrenttime.Anotherreason for thelackofsuch aspecification could be thatqueryinggraph patterns is computationally hard [BLR11]. We will seethat querying suchpatternsisone of themainfeatures of everygraphquery languageinChapter3.Therefore, queryinggraphdatabasesisalso computationally hard [Bar13]. This is also one of the reasons forthe existenceofthe multitudeofgraphquery languages that provide differentpossibilities,functionalities and expressiveness.

The ISO/IEC JTC1/SC 327 is thecommittee responsiblefor datamanagementand interchangeinthe International Organization for Standardization (ISO). Members of theirworkinggroup 3, that is responsible for database languages,raised theidea of astandalonegraphquery language that complementsSQL in 2017 [MHM17]. This idea wasfurther publicizedinthe GQL manifesto [Gre18]in2018 that motivated the move towards the creation of astandardized query languagefor thepropertygraph data model. National standardization bodies aroundthe world voted in favorofaproposal for suchanewlanguage standard, calledGraph Query Language(GQL), in 2019 [Gre19b].

7https://www.iso.org/committee/45342.html

13 2. Graph Databases

Thiswas the first time in the last 35 years that theISO consideredthe creation and standardization of anew database language [Gre19a]. The project is planned to takefour years and even then it will most likely take some time until the vendors of graph databasesimplementand supportthe new standard, if they choose to do so in thefirst place.Therefore, choosing agraph databasesystem before the widespread support of GQLinvolves not onlythe propertiesofthe system but alsothe supported querylanguage. We will nowbriefly introduce the fivegraph query languages thatwefocus on in ourcomparison.

2.4.1 Cypher Cypher[FGG+18b]isahigh-level,declarative querylanguage that wasoriginally invented for the use in the graph database Neo4j.Asthatdatabase is one of themost used and well known nativegraph databases8,Cypherisregarded as perhaps the most well-known propertygraph query language [AAB+17]. Its syntax is inspiredbySQL to ease the transitionfor userscoming from relational databases and thelanguage is identifiedas “expressive”compared to other languages[RABM17]. Apartfrom being inspired by SQL, it alsoincorporates conceptsfrom other languages like Pythonand SPARQL as well as from functional programming in general. The developmentofCypher came handinhand with thedevelopment of Neo4j and waslargely an invention of Andrés Taylorin2011. Sincethen,the language has been continually improvedand extended. [FGG+18b] The companybehindNeo4j,Neo4j Inc., announced theopenCypherproject9 in 2015 that opened up thedevelopment of the language andthe currentversion of thelanguage is nowreferencedasversion9[ope18]. Theprojectprovidesanopenplatform that should enable Cyphertobecome afully-specifiedstandard [GJK+18]. Thiswouldenable other vendors of graph databasestoprovide implementationsofCypherintheir products. Even though the language is not yetfully-specified or standardized, it has been implementedin other products like SAPHANA Graph,Redis Graph or Memgraph.The move towards standardization came from thegoal of thecompanytoestablish Cypher“as the property graph querylanguage”[FGG+18b]. That companywas alsobehind, or at least involved in, some of themore influential pushes towards the creation of GQLasanew query language standardbyISO/IEC.Aninfluentialand driving figure in this areaisfor exampletheir productmanager for Cypher, AlastairGreen, whopublished the aforementioned GQL manifesto[Gre18] among otherthings. That also explains whysome information about the progress of GQLispublished on the website of Neo4j, like the notificationthatthe GQL project wasapprovedbythe nationalstandardization bodies [Gre19a].

2.4.2 Gremlin Gremlin [Rod15]isalow-level graph query languagethat supportsboth, thedeclarative patternmatching style but alsothe imperativegraph traversalstyle [HP16, RWE15].

8https://db-engines.com/en/ranking/graph+dbms 9https://www.opencypher.org/

14 2.4. Query Languages

Thisallows the language to be used in anygraph computingsystem,beitanOLTP graph databaseoranOLAP graph processing system[TPAV19]. Although Gremlin supports both styles,itismore imperative in nature with themainfocus on traversing the graph [AAB+17]. Together with thecompactsyntaxused in itsqueries, it is not that easytoread,especially compared to ahigher-level language like Cypher [HP16].

Gremlin wasintroduced together with theApacheTinkerPop open-source project10 in 2009 and is stilldesigned and developedbythis project[Rod15]. TinkerPop provides a graph computing framework that is not bound to aspecific vendorbut canbeintegrated into other database and processing systems. Gremlin is an integral partofthis framework and denotes not onlythe query language,but alsothe graph traversalmachine called Gremlin TraversalMachine (GTM).The GTM processes queriesand allowsthe user to define and use adomainspecific query language that is “compiled” to theGTM. Systems thatintegrate theTinkerPop framework are called TinkerPop enabled. Notable examples of suchsystems are theAmazon Neptune11 graph database and themulti-model databases OrientDB12 and AzureCosmos DB13.

As an interestingside node, parts of theTinkerPop framework including theGremlin APIs originated in thedevelopment of theNeo4j database from 2007 onwards [FGG+18b, Lin18]. While early versions of theNeo4j database supported thedirectuse of Gremlin, this has been removedbut is still possible viaaplugin likeNeo4j-Gremlin14.

2.4.3 PGQL

PGQL [vRHK+16]isahigh-level, declarativegraph querylanguage designedand devel- oped by Oracle as partoftheir Parallel Graph AnalytiX (PGX)15 framework.The first versionofthe language wasintroduced in 2016 and it is stillunder activedevelopment, with version1.3 [Ora20]being released in March 2020. In contrast to other languages like Gremlin, PGQL is closely aligned to SQLand supportsmanyconceptsand keywords fromSQL.These areextendedbygraphspecific conceptsand algorithms like powerful path expressions and graph pattern matching [AAB+18, AG18]. As in manyother graph query languages, the output of aqueryisstructured in tabular form. Thisallows PGQL queries to be nestedinside,and usedseamlesslytogether with SQL queries.

The language is published as an open sourcespecification together with aparsing software16.Implementationsofthe language exist in research projects as adistributed [RTH+ 17]and non-distributed version[SHvR+16]and it is usedinthe Oracle (Big Data)

10http://tinkerpop.apache.org/ 11https://aws.amazon.com/neptune/ 12http://orientdb.com/ 13https://azure.microsoft.com/services/cosmos-db/ 14https://github.com/neo4j-contrib/neo4j-tinkerpop-api-impl 15https://www.oracle.com/middleware/technologies/parallel-graph-analytix. html 16https://github.com/oracle/pgql-lang

15 2. Graph Databases

Spatialand Graph databases 17.Thesedatabases support the propertygraph as well as theRDF data model but one has to keep in mind that while in some configurations an in-memorynative graph storage via the PGX frameworkisused, the data is kept in arelational database in other configurations [BPG+19]. Overall, the SQL-like syntax of PGQL tries to lure theSQL communitythatisalready using Oracle products but lacks standardizationand support by the broader community and other database vendors [TPAV19].

2.4.4 GSQL GSQL [Tig]isahigh-level querylanguage developedbyTigerGraph Inc, the company behind the TigerGraph database. Awhite paper for thelanguage [WD18]togetherwith one for thedatabase [DXWL19]werepublished in 2018 and 2019, respectively. Similar to PGQL,GSQL hasanSQL-likesyntax and goes even further,asaGSQL query that does not containgraph-specific primitives is astandard SQL query. In that sense,GSQL extends therelational SQLtoincludeprimitives to query and analyze graphs. The goal whendeveloping theTigerGraph database wastodesign “aproperty graph databasefor tomorrow’s big data and analytics”[HLP+19]. To achievereasonableperfor- manceingraph analytics workloads on graphs spanning billions- or trillions of nodes and edges, thedatabase provides computation in massively parallel processing fashion.This objectivealsoinfluenced the design of GSQL thatprovidesnot only primitives forOLTP workloads but also allows forthe specification of iterativealgorithms and supports the MapReduceinterpretation [Wu18]. Anotherdesign decisiontowards better performance, thatalso came from the tightalignmentwith SQL,isthe use of aschema and type system. In contrast to all other systems and languages we looked at untilnow,one has to specify the schemaofagraph in GSQL’s data model.The schemadefinesthe typesof verticesand edges in thegraph and theirpossible relationshipstoeachother [Tig]. This allowsfor abetter queryoptimization, space savingswhen storing huge graphsaswell as moresophisticated securityand privacy features like limited access to parts of the graphs for some users [Wu18]. Although thelanguage and thesystem built aroundthe language isdesigned to handle and analyzehuge graphs, thesupport for OLTP workloads and unique usage of aschema made us include thelanguage in our comparison.

2.4.5 G-CORE G-CORE[AAB+18]isahigh-level query language designed in acollaboration between industry and academiaunderthe patronage of theLinked Data Benchmark Council (LDBC)18.Introduced in 2018, thequery languageintegratessome of themainfeatures frommultipleexisting languages like Cypher, Gremlinand PGQL in order to provide

17https://www.oracle.com/database/technologies/spatialandgraph.html, https:// www.oracle.com/database/technologies/bigdata-spatialandgraph.html 18http://ldbcouncil.org/

16 2.5. LDBCSocialNetwork Benchmark acore for future graph query languages [AG18, ARV19]. As it is acore andreference for futuredevelopments, thequery language does notsupport DML or DDLoperations. One of themajor differencescompared to other querylanguages is that G-CORE is a composablelanguage, meaningthataquerytakes agraph as input, processes it and outputsagraph as well. This allowsqueriestobechained or nested, as an output of a query canbedirectlyused as theinputofanother query.Asecond major noveltyisthe treatment of pathsasfirst-classcitizens. Theycan be stored next to nodesand edges in an extended property graph datamodel and can alsohavepropertiessimilartonodes and edges. We willanalyzeand explain this in more detailinSection 3.6. As G-CORE is aresearchlanguage, thereare onlypartial implementationsinopen-source researchprojects like aparserthatisavailable on GitHub19.

2.5 LDBC SocialNetwork Benchmark

We analyze theselanguages based on theircharacteristicsand commonlanguage features and compare them by implementing multiple queriesinChapter4.Topickqueries thatrepresentapotential use case in areal world scenario,weuse theschema and some queriesfrom the interactiveworkload of LDBCs Social Network Benchmark (SNB) [EAL+15]. The goal of this benchmark is “todefine aframework where different graph basedtechnologiescan be fairly testedand compared ”[AAA+20]and is thereforeaperfect fit to test and compare graph query languages.Itisdevelopedbyaninitiativefrom major actors in industry and academiaand currently contains two groups of workloads: the businessintelligence workload that focusesonOLAP queries, and theinteractive workload with afocus on transactional queries. We limitourselves mostlytothe interactive workload as this one fitsour focus on OLTP queries. The latest stable version at thetime of writing is v0.3.2[AAA+20]and we use mainly this versioninour experiments. On one occasion, namely for queriesthatremove data items, we use the currentsnapshotofversion 0.4.0[LDB20]asdata deletions are not presentinearlierversions. Theunderlying schemaaswell as all other queriesinuse did not change betweenthese versions, so we can safelyuse thenew additions together with the model fromthe stableversion. The interactive workload itself contains queriesthat are grouped into the followingquery classes:

• Short Reads (IS, interactive short): This class contains 7short read-only queries thattouchonly asmallpart of thegraph.

• ComplexReads(IC, interactive complex): Thisclass contains 14 complex read-only queriesthattoucharathersignificantamountofnodes and edges, most often in the neighborhood of astarting node.

19https://github.com/ldbc/ldbc_gcore_parser

17 2. Graph Databases

• Insertions (IU, interactive update;inthe snapshotofversion0.4.0:INS, insertbut with thesame queries): This classcontains insertions of either singleedges or nodes together with connections to other, existing nodes.Contrary to thename of these queriesinthe stable version, interactive update,the benchmark does not contain anyqueries thatfocus solely on the updateofexisting entities.

We furthermore use the class thatcontains deletion queries(DEL), that is nowcontained in the business intelligence workload.This class is only presentinthe currentsnapshot of version0.4.0 and contains queriesthat remove either asingle edgeoranodetogether withall incident edgesand, if needed,nodes that depend on theinitially deleted one. The queriesare enumerated inside eachclass and use the abbreviation of theclass as aprefix, i.e. the first query in the class of shortreads is IS1.Wewill state which querieswere taken from the benchmark in the next chapter whenweintroducequeries representing the language features.

2.5.1 Schema The queriesofthe SNBare defined on themodel of asocial network, hence the name. The schema in UML notation can be seeninFigure 2.7. It revolves mainly around persons interactingwith one or multipleforums. Tags associatedwith forumsormessages allow groupsfor specific topicsorinterests. Aperson can create or likeamessage, which in turn canbeeitheratextoranimage, but not both. Persons in aforumcan further reply to postsorprevious commentsbyreplying with anew comment. The schema also supports friendship relationships between multiple persons via the knows edge. Although this edgeisgiven as adirected edgeinFigure 2.7, thenewer versionofthe benchmark states that it shouldbetreated as undirected, thereforerepresentingmutualrelationships between two persons [LDB20]. The schemaalso allowsfor furtherinformationtobe stored,including the citythatapersonisliving in and the universityorcompanythey are studying respectively workingat. Apartfrom the queries, the benchmark comes with adata generator20 thatgenerates parts of thedatastatically and parts of it dynamically. Thiscan alsobeseeninthe schema butisofnofurtherconcerntousaswefocus on thequeriesworkingonthat model. The generated data using thedefaultparameters representsasnapshotofa potentialsocialnetwork duringaperiodofroughly 3yearswithmanycharacteristics taken from real world social networks. As we do notcomparethe database systems but onlythe query languages,itisnot necessarytogeneratedata and store it in a database. Nonetheless, we did generate and import thedataintoaNeo4j, aTigerGraph and aTitan21 database.Titanisascalable graph database with nativesupport of the TinkerPopstack, thereforesupporting Gremlin asagraph query language. Thisallows us to run the queriesfrom the benchmark as well as implementand test ourown queries on these databases and the supported query languages. 20https://github.com/ldbc/ldbc_snb_datagen 21http://titan.thinkaurelius.com/

18 2.5. LDBCSocialNetwork Benchmark

Figure 2.7: Thedata modelinUML notation of theSNB. Source:The LDBCSocial Network Benchmark, Figure 2.1[AAA+20].

19

CHAPTER 3

QueryLanguage Features

Although thereare many graph querylanguages based on different ideas thatvary in theirstyle, expressiveness or purpose, they all share aconceptual core[AAB+17]. This coreincludesprimitives thatare basedongraphfeatures like graph patterns, paths and neighborhoods(adjacency)[AG18]. As briefly mentioned whenweintroduced the languageGin Section 2.4, PatternMatching Queries formthe basisofevery graph query language. Apartfrom them,primitives thatallowustonavigate agraph on itspaths form asecondmajor feature that we denoteasPath Queries.Weadd the feature of Structure Independent Queries for queries and primitivesthat do not consider thegraph structure. Thisisanextensionofthe summarization queriesfromAngles [Ang12], that contains simple aggregate functions to summarize theresultsaswell as further operations thatdonot rely on thegraph structure. As thesefeatures formthe basisofevery graph query language, theirunderlying primitives are well studied [CM90, BLR11, LV12, Woo12, Bar13, LMV16, TKL19]. We willintro- duce these groups of queriesinmore detailand giveexamples forthem. The examples are then implementedinChapter4to compare thelanguages.Westart with an introduc- tionofstructureindependent andother summarization queriesinSection 3.1. Pattern matching and pathqueriesare explained in Section 3.2 and Section 3.3, respectively. We continuewith two further features that do not concern thequeryingpart of thelanguages but theirabilitytomanipulate and potentially define the structure of the data: Data Manipulation in Section 3.4 and Data Definition in Section 3.5.

3.1 Structure IndependentQueries

Thisfeature of agraph query language providesprimitives that allow for queriesthat mainlydonot consider thegraph structure but focus on theinformation stored in entities. In itssimplestform, astructureindependent query operates on theentities (either nodes or edges).Itprovidesprimitives thatallowthe usertoselectsuchentities and

21 3. QueryLanguageFeatures

theirpropertiesaswell as aggregate functions that summarize structure independent information. Asimple example of suchaqueryinthe setting of theLDBC SocialNetwork Benchmark(SNB)would be to select the userwith agiven first and last name. We do notset thename in thequery but expect theuser to pass it viathe $firstName and $lastName parameters:

Selectthe personwith firstName=$firstName and lastName=$lastName. (1)

Anotherexample,this time fromthe benchmark itself, would be the query IS4(interactive short4)that selects amessage with agiven id. Thequery expects the $messageId as input and returns thecontent andthe creation dateofthe message. It is moresophisticated compared to theprevious one as amessage can be either apost that contains an imageFile,apost that contains atext-only content or acomment with atextual content.Ifthereisanimage, theresultshould containthe image file, otherwise the content.

Given a$messageId,retrievethe contentofthe message and thecreationdate. (2)

These structure independent queries are not limitedtoselectasingle entity, they can forexample alsoreturn all entities of agiven type,likeall edges labeledwith knows,or containmore complicated selection criteriathansimple equality. As brieflymentionedbefore, the feature alsoincludes summarization queriesand therefore primitivesthatallow us to summarizeagraph,orparts of it.Theseprimitives are often applied on theintermediate result of aquerythat selects partsofagraph, and aggregate the intermediate valuestoreturn asingle one.Manygraph query languages support the same basic aggregate operationsasSQL, namelycount, sum, max, min and average, and possiblyfurtherdomain- or graph-specific ones [Ang12]. An exampleofsuchaquery would be to calculate the average number of spoken languagesoverall persons:

Retrievethe average number of spoken languagesper person. (3)

Most query languages alsocontainprimitives thatallowustocompute properties of a graph itself, not the data contained in thegraph. Examples forsuchpropertiesare the number of vertices (order),the diameter,the average degree of anodeand so on. In some definitions, likein[Ang12], queries basedonsuchprimitives alsobelong to the group of Summarizationqueries. However, we do not include theseprimitives in our definition of this feature as queriesusing them focusonthe structureofthe graph and thereforeare not structure independentanymore.

3.2 Pattern Matching Queries

Before we start with an explanation of pattern matching querieswewill introduce an extension of our smallsocialgraphfrom Figure 2.4 that allowsustogointomore detail

22 3.2. Pattern Matching Queries

Figure 3.1: Avariation of our smallsocialnetwork exampleincludingpostsinthe propertygraph data model. in this section. The variation of thenetwork is given in Figure3.1 where we removed Alice from thenetwork and instead added twoposts, i.e. nodeslabeled as Post,each containing itstitle. Furthermore, the graph nowcontainsrelationships between theusers and theposts, given by edges labeled likes and created. As mentionedbefore, matching graph query patterns on agraph form a, if not the, major operation on graphsand thereforeare at thecore of manygraph query languages.Most languages are designed startingwith graph patterns and other features are added to these patterns [ARV19]. Theidea behind suchagraph pattern is to specify theshape of the result via apropertygraph 1 itself. In difference to thedata graph,aquery graph is not limitedtoconstants but canalso contain variablesinstead of anyconstant[AAB+17]. Thisallows us to query forinformationand return the valuesmatched by the system. Suchaquerypattern defines aclass of subgraphs[BPG+19]. The system in turn matches the querypattern to the data graph by findingoccurrences of thesesubgraphs thatmaps possible variablestoconstantswhile preservingthe structure. In itssimplestform, as explained untilnow,suchapatternisdenoted as basic graph pattern(bgp).Asanexample of such abgp we can queryfor relationshipsbetween the post with thetitle “GQL” and usersinour small socialnetwork. Figure 3.2 depictssuch aqueryasapossible pattern in apropertygraph data model. Manymodernquery languages do not support queriesinagraphical representation but onlyinatextualone. ADatalog related syntax is often used whenstudying the theory behind query languages and consistsof(node, edge, node) triples.Assuming a simpler datamodel without node and edge IDs, thequery could be represented in sucha syntax as: (?W, ?X, GQL), (?Y, ?Z, GQL) where W and Y representnodevariables

1Assuming thatthe system uses apropertygraph datamodel. In other words, suchagraphpattern is given in the same shapeand style as the data model used by thesystem and query language.

23 3. QueryLanguageFeatures

Figure 3.2: An exampleofabasic graph query searching for relationships between a specific post and users in ahypothetical graphical querylanguage.

and X and Z edge variablesthat are matched against constants in thedatagraph. Most modern query languages used in industry relyonanASCIIart representation of thequery thatmimicsconnections between entities via “−”, “>”and “<”signs. The greater and smallersignsare usedtospecify thedirection of arelationship,whereas the lack of these signs describes an undirected relationshipinmanylanguages.Apossiblerepresentation of thequery in aCypher-related syntaxlooks likethis: (w:User) -[x]->(:Post {title:GQL}) <-[y]- (z:User)

In termsofrelational operations, basic graph patterns allowfor selectionbased on equality via the use of constants,like title:GQL,aswellasnatural joins viathe patternand the graph structure itself. These basicgraph patterns canbeextended by furtherrelational operations: union, difference,projection, left outerjoin(optional)and more sophisticated filtering (selection) rules. Theoptional operator is especially interesting in most graph databases as it allowsustoquery incomplete information that is caused by the lackofa schema in manysuchsystems [ARV19]. Aquerypattern using suchextendedprimitives is denoted as complexgraph pattern(cgp).The risinginterest in graph databases over the last decade broughtthe invention of further additions likeinexact and approximate matching to graph patterns [CL14, AG18, FPSW19]. However, we limitourselves to bgps and cgps as mentionedabove as theseform thecoreofmost modern querylanguages. Furthermore, additionslikeinexact matching are stillinaresearchstate and not yet supported by anyofthe majorgraph query languages. [AAB+17] We have already seenanexampleofsuchpatterns in the previous chapter:the query graph Q1 in Figure 2.6 is abasicgraphpattern with asingle variable.Furthermore, if we disregard theregular expression in Q2 for now, the query Q = {Q1, Q2} forms acgp using the unionoperator.

3.2.1 Evaluation Semantics We will nowlookatdifferentevaluationsemantics in use by the query languagesand thesystems using theselanguages as thesemantics can influence theoutcome of aquery. Sincecgps are an extension of bgps, theirevaluation semantics are alsobasedonthe ones frombgps.For this reason, we limitourselves to bgps in this part. As brieflymentioned before, abgp is evaluated by matching the patternontothe data graph such that the variables are replacedbyconstants while thestructure is preserved, at least in some way.This means, that the system searches for allsubgraphs in the data graphthat are isomorphic to the bgp [AG18]and thereforeclosely resemblesthe subgraph isomorphism

24 3.2. Pattern Matching Queries problemthatisNP-complete [Yan90]. One canidentify twomajor approachesfor the evaluation semantics of abgp: thehomomorphism- and theisomorphism-based approach [AAB+17].

• In the homomorphism-basedsemantics,the matches of apattern on the data graph are not restricted.This means that theevaluation of abgp Q on agraph G consists of all homomorphisms from Q to G and thereforeresembles thesemantics of queries in relational databases. As it is used in therelational world and heavily studied in theory,this is the most commonsemantics [ARV19]. However, when using this strategy thedatabase designer has to deal with thepotential problemofinfinite resultsetswhen enumerating the paths in apatternaswedonot restrict the matches [AAB+18].

• Under thetypeofisomorphism-basedsemantics,the matches of apatternare restricted in some sense. This restrictionmostlyconcerns thedistinctness of matchingsfrom variables to constants. We can further refinethis type of semantics based on therestriction it enforces:

– If no twovariables can be bound to the samedata item in amatch, we are limited to injectivemappings and this is denoted as no-repeated-anything semantics.Asaneffect,the evaluation of apattern underthis semantics preserves the structure of the query patternasneitheredges nornodes can “collapse” by binding to the same data item.

– If therestriction onlyappliestovariables on nodes, we call it no-repeated-node semantics.

– Consequently, if the restrictiononly applies to variables on edges we speak of no-repeated-edge semantics.

Note that thelimitations on theno-repeated-node and -edge semantics apply only to variablesthat map to IDs, i.e. variablesfor labelsorpropertiescan stillmap to the same dataitem.

If we evaluate thebasicgraphpattern from Figure3.2 on thenew variation of our social network given in Figure3.1,weget thefollowing non-restricted matches:

25 3. QueryLanguageFeatures

x1 x2 x3 x4 x5 x6 x7 x8 x9 n2 Carol e5 created n1 e4 likes n3 Bob n3 Bob e4 likes n1 e5 created n2 Carol n2 Carol e1 likes n1 e4 likes n3 Bob n3 Bob e4 likes n1 e1 likes n2 Carol n2 Carol e5 created n1 e1 likes n2 Carol n2 Carol e1 likes n1 e5 created n2 Carol n2 Carol e1 likes n1 e1 likes n2 Carol n2 Carol e5 created n1 e5 created n2 Carol n3 Bob e4 likes n1 e4 likes n3 Bob

As thehomomorphism-based semantics does not restrict the matches,all of them are valid. Only the first two matches are valid under the no-repeated-anything semantics as the latterones bindatleasttwo variables to thesame dataitem.Notethatwedonot onlyrestrictthe variablesmappingtoIDs but all variables, including the ones mapping to labels or properties, in this semantics.Anexample of this is the third resultthat is notvalidasthe variablesx4and x7 are both mapped to the label likes.The first four matches are valid under the no-repeated-node semantics sincethey donot bind variables that map to node IDs to the sameID. As we do notrestrictthe mapping of edge variablesthere canbemultiple suchvariables mapping to the same edgelabel,as is the case in the matches 3and 4with the label likes on x4 and x7. Therefore, an evaluation under this semantics still preserves the structure of thequery pattern as the nodes cannot“collapse”, while it does not further restrict therelationsbetween them. Under theno-repeated-edge semantics thefirst 6matchesare valid as thelast threebind the twovariablesmapping to edge IDs, x3 and x6,tothe same ID. The semantics explained above solelyfocus on asingle matchand possiblyrestrictsuch amatch. Thisisenoughinthe setting of bgpswherenotwo suchpatterns can be evaluated to theexactsame match.However, whendealing withthe evaluation of cgps and thereforewith further operators like union, we can have duplicate matches regardless of thechosensemantics.Inthat case we can further distinguish between a set semantics, meaningthat the evaluation of apattern is accumulated in aset of matches and therefore cannot containduplicates, and a bagsemantics thatallows duplicates of matches in the result. According to the choice of either set or bag semantics we can for example speak of ahomomorphism-based bag semantics or ano-repeated-node-based setsemantics. [AAB+17]

Each of thesemantics can be desired in some application scenarios and thereforethere is no single one used in all database systems and query languagesbut each systemis basedonone semantics. While some systems onlysupport asingle semantics, others like Neo4j with theCypher query language allowfor queryconstructs thatresult in a differentevaluationsemantics.Still otherlanguageslikePGQL even provide keywords thatallowthe user to switchtoadifferent semantics on aquery by querybasis.

26 3.2. Pattern Matching Queries

3.2.2 Queries Nowthat we understand the concept behind thesequerieswewill look at some examples from theLDBCSNB. The query IS1(interactiveshort 1) representsasimple example of aqueryusingabasic graph pattern.

Given a$personId, retrievethe first name, last name, birthday, IP address, browser, gender,the creationdate of the entryaswell as thecityofresidenceofthe person. (4) Thisquery canalsobegiven via the graph pattern in Figure3.3 thatshows the single directed relationship betweenthe twoentitiesinquestion.

Figure 3.3: Query 4(IS1) as agraphicalquery pattern. Source: The LDBCSocial Network Benchmark, page56[LDB20].2

Note that if thequery did not askfor thecityofresidence, it is astructureindependent queryasthe other information is stored directly with the person. Atypical query in the setting of asocial network is to searchfor friends of aperson.IS3 represents this in asimple manner where all friends of agiven personand the creation date of their relationship are returned.

Given a$personId, retrievethe id, first name and lastname of all theirfriends as well as thedateofthe friendship-creation. (5)

Suchpatternmatchingqueriescan alsogenerate boolean valuesthatindicatewhether there exists apossiblerelationshipbetween some entities.Asanexample,the query IS7 fetches the author of agiven message as well as all persons thatcreated adirectcomment to it. The optional relationship,denotedasknows,indicates whether thesetwo persons knoweachother.

Given a$messageId, get alldirectcomments replying to this message and abooleanvalue that indicates if the authors knoweachother. (6)

2The graphicalrepresentations of thisquery and allfollowing ones are taken from the currentsnapshot of version0.4.0 of the benchmark [LDB20]. The queries themselvesare identical between the stable version 0.3.2 [AAA+20]and the currentsnapshot,but some graphicalpatternsinversion0.3.2 do not depict the specified query.

27 3. QueryLanguageFeatures

Agraphicalvisualization of this pattern can be seeninFigure 3.4,wherethe dotted line indicatesthe flag, or optional relationship.

Figure 3.4: Query 6(IS7) as agraphical query pattern. Source: The LDBCSocial Network Benchmark, page 60 [LDB20].

3.3Pathand Navigational Queries

Graph patterns as explained untilnow allow us to queryfor boundedand fixed rela- tionships between entities in agraph.However, they are not powerful enoughfor many queriesthatnavigatemore complexrelationshipsinagraph.This in turn is one of the coremotivations of using agraph database,asrelationships either containinformation themselves or tell us something about theentities they connect. Path queriestacklethis problembyprovidingqueriesthatespeciallyfocus on one thing: paths. Navigational queriesthen integrate path queriesintopatterns to allow formore sophisticated data retrievals.

3.3.1 Path Queries In itssimplestform, apath querycan test for theexistence of an unrestricted path between twonodes. Thisisaninherently recursiveoperationasthe database engine traverses from one node to another, often without anybounds on thelengthofthe path. Compared to theworld of relational databases, where recursiveoperations are supported onlyinalimited way, theseoperations are heavily usedingraphdatabasesand form one of themajor advantages whenqueryingrelated information. Apartfrom this simplistic form thattests for theexistence of an unrestricted path, we can add some criteriathat the path has to satisfy.Asimple way to achievethis is to allow restrictions on the path by requiring the edges to have aspecific label.Ifwethink about asocial network, this allows us to searchfor apath between twousersthatislimitedtofriend-relationships(edges labeledwith follows or friend forexample). In general,suchqueries canbegrouped as Reachability Queries and we will explain further characteristics and ways to specify restrictions on paths in thefollowing partonRegular Path Queries. [AG08, vRHK+16] Apartfrom testingfor theexistence of apath,pathqueriescan alsobeused to retrieve nodes connected by apath.Asanexample, this allowsustoretrieveall nodes on paths

28 3.3. Path andNavigational Queries

of theform friends-of-friends in asocial network, either with arestricted lengthoreven with arbitrarylength. Friends-of-friends,orfriends-of-a-frienddenotes acommonlyused query in social networks. Startingfromagiven node v,the query navigates to alladjacent nodes with the restriction thatatraversed edgehas to denote afriend-relationship. After the friendsofvare explored, it continues to theirfriends. Note here that,causedbythe lackofadefinitionfor this query,itiseitherused as explained where the result contains all friends andthe friendsofthe friends, stoppingatpaths of length2.Onthe other hand, friends-of-friends can alsodenote thequery thatdoesnot stop at pathsoflength 2but either at auser-specifiedlengthorwhen the full graph isexplored, meaning an unrestricted length. Assuming that we stop after twohops,the query does notonly containthe existing friendsofaperson,but alsothe friendsoftheir friends, whichcan be usedtosuggest new friendsortoanalyze thenetwork around aperson [AAB+17]. More generally,this operation traverses theneighborhood,oradjacency, of agiven node. This is an important operation and manygraph databasesprovide dedicated operators for suchqueries,that are alsodenoted as Adjacency Queries [AG18]. An exampleofsuchan operators would be ak-neighborhood one that returns theneighbors up to distance k fromagiven node [Ang12]. Takentogether, Angles et al. summarize theconcept behind path queriesasfollows: “Theideaofapath queryistoselectpairs of nodesinagraph whoserelationship is given by apath (ofarbitrary length) satisfyingcertainproperties.” [ARV19]. This canbe rewritten to deriveacompactform of apathquery P given in [AAB+17]: P = x −→α y , where α specifies therestrictionsapath between x and y hastosatisfy suchthat the node pair (x, y)isselected.What we lack until nowisanotion that allows us to express the conditions on thepath,i.e. α.

Regular Path Queries When looking forpossible notations for this, we probably end up with some variant of regular expressions over theset of edge labels,asthis form is usedinmost languages until now[LMV16, AAB+17]. Thisnotation is formalized as a RegularPath Query (RPQ) and wasintroduced in [CMW87]togetherwith thegraphical query language G. At its core, an RPQ “selects nodes connectedbyapaththat belongs to aregular language over the labeling alphabet” [LV12]. That tellsustwo things. First, that suchaquery onlyever selects pairsofnodes that are connected by apath.And second,that anodepair is only selected if theconcatenation of thelabelsonapath connectingthem form aword in the languageofthe regularexpression[AAB+17]. As asimple example of this, we can write the friends-of-friendsquery with unrestricted length forCarol in thecompact formofa path query as: + P = Carol −−fo−−llows−−→ y.

Thisretrieves all node-pairs (Carol, y) thatare connected by apath of lengthat least 1, with therestriction that all edges on thepathare labeledasfollows.Note thatwecan alsouse theKleene Star“∗”inour query,and, depending on thechosen

29 3. QueryLanguageFeatures

evaluation semantics,the query:

∗ P = Carol −−fo−−−llows→ y can either givethe same resultasthe one from above, or can additionallycontainthe node pair (Carol, Carol).Furthermore,wecan alsoquery for the unionofpaths via the “|” symbol.Asanexample,wecan search for all poststhat are either created or likedbyauser thatCarol follows as:

follows·(likes|created) P = Carol −− −−−−−−−−−−−−→ y where the“·”denotes aconcatenation. We can summarize thenotation of an RPQ as aqueryofthe form P = x −−re→g y ,where reg is aregular expression overthe alphabet Σthatcontains all possibleedgelabels [LMV16]. The pathqueryingfunctionalityofmost modern query languages is basedonthis notation as it provedtobeagood fit to specifyrestrictions or conditionsonthe inherently recursive natureofapath [vRHK+16]. As RPQshavebeen aroundsince the late 1980s and are used in languages until now, the class is well studied,especiallyalso regardingits complexity [Woo12, Bar13, LMV16]. Over time,RPQs have been extended in manyways. An RPQ as explained untilnow can onlytraverse edges in aforward direction, but onecan also be interested in traversingthem backwards.This givesrise to anotable extension, namely the class of theTwo-way Regular Path Query (2RPQ) [CGLV00b, CGLV03]. 2RPQs add support for an inverse operator a− for a ∈ Σthat denotesthat theedgelabeled a is to be traversed backwards. Let’s look at an example of thisinoursmall social network fromFigure 3.1. We can queryfor thefriendsofall users thatliked apost p with the following two-way regularpathquery:

− P = p −−like−−−−−−s ·follo−→ws y. Starting fromagiven node p,the post in our case,the query follows all incoming likes- edges in abackward manner and from thereall outgoing follows-edges in aforward manner. There are further variantsofRPQs thatfor exampleadd variables[Woo90]ornode tests [BPR12]topath queries. We will introduce some of these as we progress with our explanation of navigational queries. As RPQsallowustoexpress path queriesand are usedinmost modern query languages,atleastinsome variant,the terms path query and RPQ are both usedtodenotethis class of queries. The difference is thatapath query denotessuchaquery in general, regardlessofthe underlying variantused by the query language, be it forexample asimple RPQ or a 2RPQ.Incontrast,the term RPQ is more relevant in theoretical literaturewhereitisimportanttostate and definethe variant thatisused. We will alsouse this approachfrom nowonand denote the generalclass of thesequeriesaspath queries and use thecorresponding theoreticaltermswhen we speak about theunderlying variants.

30 3.3. Path andNavigational Queries

3.3.2 NavigationalQueries Early graph query languages, likethe language Gthatwelookedatinthe previous chapter,allowustorestrictthe labels on apath,whichisenoughtoexpress the friends- of-friends queryfor example. However, one often wantstoexpress more sophisticated constraintsthat are not limitedtothe edgelabelsonthe path alone. With path queries as explained above, we can query for apath between two users in our social network over friends-relationships. If we want to query forsuchapath with thefurther restriction that aspecific user is visited on thepath, pathqueriesare not expressiveenoughaswehave no waytoexpress suchanintermediary node. Thisleadsustonavigational queries,also called navigational graph patterns (ngps),thatallowfor acombination of pathqueries and basic graph patterns. When looking at it from theperspectiveofapattern, we are nowable to replaceanedgeinapatternbyapath that is given by aregular expression [AAB+17]. Therefore, we can write thequery thatrequires aspecific user n to be on the path as follows:

friend+ friend+ x n y

We have already seenanother basic example of suchanavigational query whenwe + introduced the languageG.The expression w on thequery graph Q2 in Figure 2.6 is a regularexpression thatisembeddedinabasic graph pattern. As paths in ngpsare given as regular expressions over edge labels, and theseare nothing else then RPQs, we can alsolookatngps from theperspective of regularpathqueries. From there, ngpsare an extension of RPQs andform the class of Conjunctive Regular Path Query (CRPQ) [CM90, FLS98]. Queriesinthatclass candeal with intermediate nodes on paths [BLR11], which is exactly what we needed in ourmotivating example. The term “conjunctive” in CRPQ comes from the fact thatbyallowing intermediate nodes on apath,weessentially query forthe conjunctionofmultiple paths, eachgiven by an RPQ,thatshare the intermediary node(s)[LMV16]. Takentogether, anavigational graph pattern is agraph patternwhere nodes are either constants or variables, and edges can be labeledbyconstants,variables or RPQs[AAB+17]. Similar to basicgraphpatterns and the queryinFigure 3.2, thepatterns in asystem builtonthe propertygraph model canbefurtherenriched by elements from the property graph.Therefore, we can for example also restrictthe nodes to have aspecific label,as we did in the example with bgps in said figure.

FurtherVariants CRPQs, and navigational graph patterns in general, received lotsofattention throughout the last decades as they formthe base of manymoderngraph query languages [TKL19]. Thismanifested in thetheoreticalstudy of this class [CGLV03, BLR11, Bar13]aswell as the invention of newvariations of it.Asexplained until now, the addition of pathqueries is limited to basicgraph patterns. Therefore, anatural variation also allowspathqueries

31 3. QueryLanguageFeatures

on complexgraph patterns, forming complexngps (cngps).However,wewanttonote that this differentiation, and theclass of cngps, is notalwaysenforced, most likely since the term ngp often denotes thewhole class and all itsvariations. In themore theoretical terminologyofRPQs, we can alsoadd the inverse operator of 2RPQstoCRPQsand derivethe class of conjunctive 2PRQs (C2RPQs,also 2CRPQs)[CGLV00a]. U2CRPQs in turn denote afurtherextensionthatalso includesunions. However, all of thesevariations are stillnot expressive enough for several queriesin modern use cases and applications. Under thedefinitionsofRPQsand theirvariants until now, the evaluation of thequery P = x −→α y on agraph G, P (G), contains thenode pairs (x, y)thatare connected by apath that satisfies α.However, we cannot output the path or process it,for example by comparing it to othersinthis evaluation.The class ExtendedConjunctive RegularPath Query (ECRPQ) [BFL12, BLLW12]tackles this problembyallowingpathstobenamed, includedinthe outputand compared to others [LMV16]. Thisessentially raises paths to first-class citizens and as of nowseems to be one of themost promising variantsfor currentand future graphquerylanguages. [Bar13] There is amyriadoffurther variantsusing multiple concepts, but, as this is not thefocus of thethesis, we will onlyintroducetwo moreapproaches. In all variantsintroduced until now, we are limited to asingle recursiononapath viaaregular expression. Nested regularexpressions (NREs) [PAG10, BPR12], originallydeveloped to querydatainthe SemanticWeb,alsoallowfor anotherform of recursion.They are an extension of 2RPQs3 with the addition of abranching operator · that allowsRPQs to be nested. Theclass RegularExpressionswithMemory(REM) [LV12, LMV16]onthe other handcovers a differentuse case, namely that of the integrationofsocalleddata values on apath. A data value in this case is another word foraproperty of anodeinthe propertygraph data model.Anexample of this would be the age of aperson in asocial network. The basicidea is thatone canspecify whenadata value is stored and can than use these stored values. This allowsfor queriesthat arenot only limited to restrictionsofthe path via aregular expression, but can also deal withthe data valuesofthe nodes on thepath. Assume that we are interested in avariation of thefriends-of-friends query where we onlytraverse to friendsofthe same age. With REMs, we areabletostore the data value (age) of thefirst node on thepath, then traverse an outgoing friends-relationship, read the stored data value and compare it to thenew one.

3.3.3 Query Evaluation:Semantics, Outputand Complexity Nowthat we have introduced path and navigational querieswewill takeacloser look at theirevaluation in general as well as some semantics,possibleoutputs of suchqueries and finishwith anoteonthe evaluation complexity. We limit ourselves to pathqueries in this sectionasthe evaluation of navigational ones can be derived fromthis, together withthe evaluation of bgps (and possiblycgps). 3Note that they are aclass of pathqueries in theiroriginal form, therefore there are no patterns in these queries.

32 3.3. Path andNavigational Queries

Queryevaluation and semantics

As mentionedabove,the evaluation of an RPQ P = x −→α y on agraph G contains the node pairs (x, y)thatare connectedbyapath that satisfies α.ECRPQsonthe other hand alsoallow us to retrievepathsasaresult of aquery. From thepractical perspective of adatabase system however, the system has to find all paths that match agiven pattern before it canprojecttothe desired elements. As an example, let’slookatthe evaluation of theRPQ P fromabove.Inafirststep, the systemhas to find all paths in thegraph thatsatisfy α,and canthen projecttothe start and endnodes of thesepaths to get (x, y) as output. Therefore, when looking at theevaluation semantics of pathqueries from a practical perspective, it does notmakeadifferencewhether thequery language allows the output of paths or not.Toexplain the evaluation semantics in this section, we will look at the paths that are found during theevaluation of aquery in adatabasesystem.

Before we can explain theevaluation of pathqueries, we needaformal definitionofa path and itslabel in agraph G.Apath π is asequence n0e1n1e2 ...nk−1eknk | k ≥ 0 where each ei is an edge between thenodes ni−1 and ni.Asbrieflymentionedbefore, the evaluation of apath querycontains onlypaths whoseconcatenation of theedge-labels form aword in thelanguage of theregular expression. Lab(π)denotesthe label of π, ∗ 4 given as Lab(π)=a1a2...ak−1 ∈Σ ,whereeach ai denotes thelabel of ei and ai ∈ Σ . The evaluation of apath query P = x −→α y on agraph G,denotedasP(G), contains all paths in the databasewhose label Lab(π)satisfies α.Note that our definition of apath alsoallows forpaths of length0that containonly the node n0.Ifweuse theregular ∗ expression that acceptsall words overΣ,Σ∗,inanRPQ P = x −−→Σ y ,wedonot impose anyconstraintonthe path. Therefore, the evaluation of this query alsocontains the + emptypath with k =0and thesinglenode n0.[AAB 17, ARV19]

Thisdefinition allowsustoevaluate path queries on graphsbut alsoleadstoapotential problem: the evaluation can result in an infinitenumberofpaths. As an example, we can look at an extension of thefriends-of-friends query (with unlimited length) where we are not interested in the transitive closure of thefriends but in the poststhat they liked. Starting form afixednode n2(the userCarol), this querycan be written as:

+ P = n2 −−fo−−−llows−−−−·like→s y.

If we evaluate this query on thegraph G fromFigure 3.1, we get an unlimited number of paths caused by the cycle between Carol and Bob. The evaluation P (G)contains, among others, thefollowing paths that are given by theirnodes and edges as well as theirlabel:

4As areminder, Σcontainsall possible edge labels.

33 3. QueryLanguageFeatures

π Lab(π) n2 e3 n3 e4 n1 follows·likes =follows1·likes n2 e3 n3 e2 n2 e1 n1 follows2·likes n2 e3 n3 e2 n2 e3 n3 e4 n1 follows3·likes n2 e3 n3 e2 n2 e3 n3 e2 n2 e1 n1 follows4·likes ......

Similar to the differentsemantics when evaluating graphpatterns,there are multiple onesfor theevaluationofpathqueriesthat differmainly in their handling of duplicates in the evaluation P (G). We will nowdescribe fourcommonevaluationsemantics in use by graph querylanguages [AAB+17].

• Under arbitrary path semantics,the paths in P (G)are notrestricted in anyway. Therefore, all paths that satisfy therestrictionsofthe path query are presentin the evaluation. If we look at our example queryabove,the evaluation contains an infinite number of paths causedbythe loop in theoriginal graph thatsatisfiesthe condition in P .Asitisnot feasibletoenumerateaninfinite number of paths, one solution is to restrict them in some way. In some scenariosweare onlyinterested in whetherthere is suchapathornot,and the system canstop as soon as thefirst suchpathisfound. Anotherpossibilityistoreturnnot thepaths, but thepairs of nodes thatare connected by them,whichresultsinafinite number of node pairs. Thisresults in an outputasdefinedfor RPQs. There are further possibilities to dealwith this problembut the main takeawayisthat P (G)may containaninfinite number of paths under such an unrestricted semantics.

•The shortest path semantics restricts the paths in P (G)tobeshortestpaths only. That means, onlypaths of minimal lengthbetween anytwo nodes thatsatisfy the conditionsinP are contained in P (G). In the evaluation above, onlythe first path is valid under this semantics as all further ones are notpaths of minimal length between thenodes n2and n1.

• In case of the no-repeated-node semantics,apath thatsatisfiesthe conditions in P is onlyincluded in P (G)ifnonodeappears more thanonce on the path. These paths are alsoknown as simple paths and theevaluationofRPQsunderthis semantics hasbeenstudied extensively [MW89, ACP12, LM13], which cannot be saidfor manyofthe other semantics.Using this semantics,only the first path giveninthe table aboveisvalidasatleastone node appears morethanonce on theothers.

• The evaluation underthe no-repeated-edge semantics contains onlymatching paths where no edgeoccurs more thanonce. In our evaluation of P (G)above,only the first twopathssatisfy this as thelatter ones repeatedly takethe same route.

34 3.3. Path andNavigational Queries

Output Nowthatwehaveseensome evaluation semantics for pathqueries, we will brieflylook at differentforms of possible outputsfor such queries.Again, we look at thisfrom apracticalperspective, where auser mayfor exampleonly be interested in asimple true/false-answer and not in nodes, properties or paths. In suchacase, it is enough to output aboolean value.Asanexample, the query P = a −→α b with given nodes a and b asks whether the nodes are connected by apath satisfying α and can be answeredbya booleanvaluedepicting whether P (G)isempty or not.Inothercases,the start and end nodes (a and b in the previous example)are notgiven but we are searching forthem, i.e. we want to find the nodes connected by apath satisfying α.The outputofsucha query contains thesenodepairsand thereforecoincides with the definitionofthe output of RPQs. Apartfrom theseoutputs thathave afixed arity,wecouldalso be interested in the paths themselves whichare inherently of variable length. As we have seenbefore, including all pathsthatsatisfy aqueryunder thearbitrary path semanticscan result in alarge, and potentially infinite, number of paths. Furthermore, adatabase system offering such outputs also has to dealwith paths of potentially unlimited length. Oneapproachto get acompactrepresentation of theoutput is to return agraph consisting of thepaths thatare contained in theevaluation of thequery.This in turn would allow for query composition as aquerytakes agraph as input and would output agraph as well. We will look at this in more detail in Section 3.6. However, this is not supported by anymodern databasesystem or graph query languageexcept in the researchlanguage G-CORE as of now. Generally,the representation of paths in theoutput of aquery differsfromsystem to system as thereisnoconsensus for this until now. [AAB+17].

Complexity Evaluating apath queryonagraph is acomplex undertaking in general.Caused by theirrecursivenature,the system often needstoload, or at least go through sizable data space. Forexample, asearch for all node pairs connected by apath that satisfies a regularexpressionunder the simple path semantics (no-repeated-node) is NP-complete in the graph size[MW89, Bar13]. Thishas led to thedevelopment of manysemantics basedonthe arbitrary-pathone, thatfor instance stop as soon as one pathisfound. Barcelò showedthatsuchasemantics leads to tractablecomplexity when evaluating RPQs[Bar13]. However, this does notmean that all modern querylanguages and databasesystemsuse avariantofarbitrarypathsemantics as we will see in our analysis. [AG18,AAB+18]

3.3.4Queries: Nowthatweunderstand theconceptsand evaluation semantics of navigational queries, we will look at some examples from theLDBC SNB. In our explanation of pathqueries, we started with reachability queries that test for theexistence of apath between two

35 3. QueryLanguageFeatures

nodes.Wealso mentioned the possibilitytoconstrain suchapath,for examplebyonly allowing edges with aspecific label.The query IC13 (interactivecomplex 13) is an exampleofthat, with the addition that we do notonly test for theexistence of sucha path, butare also interestedinthe path itselfand the length of it.

Given two persons by theirid, findthe shortestpathbetween them in the subgraph induced by the knows relationships. Furthermore return the length of the path, 0ifboth ids are equaland -1 if there is no suchpath.(7)

Figure 3.5 depictsthe query as anavigational graph pattern that uses aslightly different notation forthe regularexpression knows∗,namely“knows∗0..”. The “∗”signalizesa path of some length where 0.. gives thelowerbound. Likewise, knows∗1..3woulddenote apath of lengthatleastone and maximumthree,whereall edges have to be labeled with knows.Note that we are not abletospecify that we are onlyinterestedinthe

Figure 3.5: Query 7(IC13) as agraphicalquery pattern. Source: The LDBCSocial Network Benchmark, page 54 [LDB20].

shortest pathinthis representation.The same limitationapplies to the representation as an RPQ: ∗ P = person1 −−kn−−ows→ person2.

In our introductionofpathqueries, we thenmovedontoadjacencyqueriesthat explore the surroundings of agiven node. Query IC1isanexample of suchaquerythat examines the adjacency of aperson.

Given aperson, find persons with agiven first name in theiradjacencyover knows relationshipsofatmost 3steps. Return thesepersons, their distance as well as summariesoftheir workplaces and place of studies. (8)

Figure 3.6 depictsthis query as anavigational pattern. As an advantage of sucha representation, we can get afeelingofthe difficultyofaquery in termsofaccesses to different elements, likethe ones on City, Company and Country here. Nowthatwehaveseenanexampleofboth, areachabilityand an adjacency query, we will look at general navigational query. Query IS2 gives us thelast messagescreated by auser,the originalpostsinthe conversationsaswell as thepersonsthatcreated these posts. An application could then use this informationtoanalyze relationships between thesepersonsfor example.

36 3.3. Path andNavigational Queries

Figure 3.6: Query 8(IC1) as agraphicalquery pattern. Source: The LDBCSocial Network Benchmark, page42[LDB20].

Retrievethe last 10 messages created by agiven person. Foreachofthose messages, return thatmessage, theoriginal post in its conversation as well as theauthor of that post.(9)

Thisisanavigational queryaswehaveapath of variable lengthbetween themessage and theoriginalpost in itsconversation. Whatmakes this query abit challenging is the factthat amessage can alsobeapost. In this case, theoriginalpost of theconversation will be the same message and should occurtwice in the output. Figure 3.7depicts this queryasanavigational pattern.

Figure 3.7: Query 9(IS2) as agraphicalquery pattern. Source: The LDBCSocial Network Benchmark, page57[LDB20].

Summary: These features,namelygraph patterns, paths and theircombination in navigational queriesform thecoreofmost,ifnot all,moderngraph query languages. In all example queriesthatwelookedatuntil now, we usedqueriesthatretrievedata fromagiven graph.Suchqueries form asub-language of ageneral querylanguage that is denoted as DQL. As briefly mentioned in the introduction of querylanguages in Section 2.4, a general querylanguage alsocontains operationstomanipulatethe data and sometimes even theschema. These sub-languages are in turn denoted as DMLand DDL and we will nowlookatthem,their underlying mechanicsand potentialproblems.

37 3. QueryLanguageFeatures

3.4 Data Manipulation

Queriesthatmanipulatethe data allow us to change thegraphbyadding nodes or edges, but also by removing or updatingexisting ones. These graph transformations are specified in aqueryusing primitivesthat are basedonthe previously introduced patterns and paths [AG18]. Therefore, queriesthatmanipulatethe data are expressedsimilarly to queriesthatretrievedataasbothrelyongraph-orientedoperations[AG08]. In order to updateorremove an entity, we first have to identify it. This canbeproblematic in some scenarios. Think about asocial network thatcan containmultiplepeople with the same first andlastname. To avoid an identification problem, manygraph databases use globally unique IDs that are associatedtoeachentity, similarlytothe world of relational databases. Therefore, if one wantstoremove aperson with thename of “John Doe”,wecouldfirst query forall suchpersons, select theIDofthe one that we want to delete and then send theDML query to remove the entity. Anotherthing thatwehave to keep in mindwhen removing nodes are theirrelationships to other nodes. When we remove suchanentity, allincoming and outgoingedges become invalid as theyare left with onlyone adjacentnode. It is up to thesystem to either removethese stale relationships together with the node or to manipulate them in order to achieveavalidgraphagain. There is no correctbehaviorfor all scenarios.While some systems remove suchedgeswiththe node, othersprovide explicit operators to either delete onlythe node or alsothe adjacentrelationshipsand there areevenproposals for algorithms that can deal with stale edges [SW14]. Insome scenarios, the removal of a node could alsoimplythat other, often adjacent, nodesshould be removed. Let’s look at the relationships between aforumand the postsinthis forum in the data model of the SNBinFigure 2.7. If we are to remove apost,all comments that reply to this post hang in the air.Similarly,removing aforumrendersall postsinthatforum to be without a container. Apartfrom the removal of entitieswecan alsoinsert and updateentities using such queries. This allowsfor adynamic change of thedataitself and,assuming that the database does notuse aschema as is thecase for some graph databases, alsothe data model. Let’s look at our smallsocialnetwork where anodeisidentifiedasapersononly by its User label. If we update suchanentityand change thelabel to Person,we are left with adatamodel containing both,nodes labeledasUser and Person.This gives us aglanceatthe extreme flexibilityofsuchsystems as oneisnot boundtoany schema.Onthe otherhand, querying acompletely schema-less graphbecomes nearly impossibleasfor example persons can be labeled as User, Person, Man, Woman ....This problemdoesnot only concern node labels but also edge labelsand properties in propertygraphs as we couldstore the name of aperson in asingle propertyorsplit it up into multiple ones for example. That means, that even if thedatabase system does notenforce aschema it is in theinterestofthe users to agreeonaset of labelsand propertiesand to use them throughout the database. Theflexibilitythat remains then is the adaptabilitytofuture changeswhereanew label can be seamlesslyadded to either

38 3.4. DataManipulation existingentities or newones and can then be usedinfuture queries. We do nothave theseproblems in systems relying on aschema as theschema defines theavailable labels and potentially alsothe properties. However, we have to adapt the schemabeforewecan introduceanew label, property or entity.

3.4.1Queries: Nowthatwehavehighlighted some of thepotential problems and typesofqueries that are possible with this sub-language,wewill look at afew examples. In thesetting of our social network, addinganew persontothe graph isanimportant operation.However, we do not only want to add anodecontainingthe information about the personbut alsorelateittoother entities in asingle query.The query INS15does exactlythatbyadding anodeaswellasedges depicting their location, interests and so on.

Addanode denoting aperson and connect it to existing nodes depicting their location, interests and place(s) of work and/or study.(10)

Figure 3.8 depictsthis query as agraphicalpattern and thevarious parameters (pre- ceded by adollar-sign)thatwill either be set as properties or are needed to buildthe relationships.

Figure 3.8: Query 10 (INS1, IU1 in [AAA+20]) as agraphicalquery pattern. Source: The LDBCSocial Network Benchmark, page61[LDB20].

As an exampleofaquerythatdeletes one or multiple entities we have to look at a snapshot of version0.4.0 as deletions are not presentinearlierversion. Thequery DEL7 removes acomment togetherwith the corresponding relationships as well as all further comments thatreply to theinitial one.

Delete acomment, itsincidentedges and recursivelyall comments replying to the intitialone. (11)

5Remember: the insertionqueries INS1-INS8 in the snapshot of the most recentversion0.4.0 [LDB20] equal the queries IU1-IU8 (interactiveupdate) in the stable version 0.3.2 [AAA+20].

39 3. QueryLanguageFeatures

Figure 3.9 depictsthis query as agraphicalpattern. We can see that theremovaltriggers the invocation of arecursivedeleteonthe replying comment(s). Thisinturndeletes also theiredges until there arenodangling commentsleft.Therefore, this query potentially touches many entitiesand, depending on the level of support forsuchtransitiveremovals fromthe systemand thequerylanguage,can be quite sophisticated.

Figure 3.9: Query 11 (DEL7) as agraphicalquery pattern. Source: The LDBC Social Network Benchmark, page 87 [LDB20].

The SNB does not containqueriesthat update an already existingentity, be it anodeor an edge.Tothis end, we include Query12that updates thepropertyofanedge, namely the year aperson graduated at auniversity.

Updatethe year (classYear)agiven persongraduated at agiven university. (12)

3.5 Data Definition

When we introduced thedifferentdata models in Section 2.3,wehighlighted that some of them support aschema while others do not. An exampleofsuchaschema can be seen in the RDF data model in Figure 2.2. We alsomentionedthat the widely usedproperty graph datamodel does not support aschema out of thebox.However, even if adatabase system and theunderlying data model do notsupport aschema, it is in theinterestof the users to agree on some sortofaschema as we have motivatedinthe previoussection. Therefore, manygraph databases allow, or even require, the specificationofsome sortof schema or restrictionsonthe graph,evenifthey are builtonthe propertygraph data model. Using astrong type system where labels, properties and theirdatatypes as well as possiblerelationships are specified in theschema limits the flexibility. This lossis traded forperformance as aschema can be usedtooptimizequeryexecution, improve access to entities or even reduce thespace needed to store them [DXWL19]. Apart from performanceimprovements, aschema also allowsfor securityand privacy features

40 3.6. Further Features

[WD18]. One canlimit for example the access to parts of thegraphfor certainusersand hide other parts from them. In applications dealing with datainaknown and structured form, using apre-definedschema representing this structure does notlimit the flexibility while providing the aforementioned performance and securitybenefits.Therefore, we cannot derive asingle best setting regarding theuse of aschema for all applications. The varying trade-offs between theflexibilityand havingatleast some sort of schema are one of thereasons for theexistence of thevarious graph databasesand often amajor differentiation between them. Even if we limit ourselves to databases and languages relying on aschema we endupwith multiple variants of suchschemata. Whereone language for exampleallows an edge to be related to its reverse edgeasinGSQL,we cannot express this relationshipinother languages likeCypher. While DML queriesallow us to manipulate the data graph,DataDefinitionLanguage (DDL)queriesare usedtomanipulatethe schema. Thisincludesthe initialcreationof the schemaaswellasadaptions lateron.

3.5.1Queries: The SNB does not containany queries thatmanipulatethe schemaasitassumes the pre-definedmodel from Figure2.7.Inour comparisonofthe query languageswewill describewhether alanguage supports or relies on aschema as well as thetypeofthe schema. As an exampleofsuchaDDL query we try to add astreettoour data model in Query 13, meaning thatweadapt it suchthatapersonisnot directly located in acity but in astreetwhichinturn is located in acity.

Addastreet entitytothe data model andchange therelationship Person[0..*]-isLocatedIn->[1]City to Person[0..*]-isLocatedIn->[1]Street[0..*]-isLocatedIn->[1]City. (13)

3.6 Further Features

The first three featuresthat we introduced, namely structure independent,pattern matching and navigational queries, are themajor building blocks used in many graph query languages. These featuresallowustoquerydataand are alsoused in DML queries to modify it. In adatabase that supports aschema, DDLqueriesare usedtomodify it. Depending on therepresentation of theschema, they can alsobebuilt on top of these three features as one couldfor examplespecify aschema using apattern. Apartfrom these featuresthatform thebasis and group possible queries, more general features of graph query languages have been identified and we will nowlookattwo of them. These two featuresare more acharacteristic of alanguage and tell us something about its

41 3. QueryLanguageFeatures

expressiveness in certain use cases.The LDBCTechnical User Communityidentified theseastwo of themajor issues in existinggraph query languages [AAB+18].

• Composability The first one, composability,denotes that theoutput or result of aquerycan be usedasinputtoanother query in the same language.This allowsfor nested subqueries as in SQL but alsofor alinear composition of querieswherethe first part of aqueryselects some datawhichisthen passed on as theinputtoasecondquery. Acomposablequery languageencourages amodularapproachand interoperability between queriesaswecan plug one query into another one.This in turn allows for an abstraction of problems as we can decomposebigger queries into smaller pieces where theinputs and outputs are compatible [AAB+18]. The inputisonly compatible to theoutput if they have the same,oratleasta compatible, datamodel [ARV19]. As brieflymentionedinthe introduction of query languages,agraphquery takes agraphasinput but in many languages outputs notagraph but atablecontainingvalues[Ang18]. Therefore, queriesare not composableinthese languages.The paperthatintroduced the researchlanguage G-COREgoesevenone stopfurtherand statesthat “current querylanguages do not provide full composabilitybecause they output tables of values, nodes or edges.” [AAB+18]. Following this argument,agraph query languageisfully composable if it outputs theresult of aqueryasagraph.Unfortunately,asofnow this is not supported by anyofthe currentgraph query languagesexcept by G-COREitself.

• Pathsasfirst-class citizens The second feature deals with the representationofpaths in thedata model. We already mentionedthis feature when we introduced the class of ECRPQin Section3.3.2. Paths are raised to first-classcitizens forECRPQs as they can be named, included in the output and compared to others [LMV16]. An entity in any programming language is saidtobeafirst-class citizenifitsupportsthe operations thatare generallyavailable alsotootherentities. If it cannotbereturned or stored however, it is reducedtoasecond-class citizen[Sco06]. Consequently,raisingpaths to first class citizens allowsustoadd labels andproperties not only to nodes and edges butalsotopaths [AAB+18]. These changesrequire amore sophisticated data model,asthe modelsexplained untilnow cannotdeal with stored paths. As of now, thefeatureisagain primarilysupported by the researchlanguage G- CORE.Toachievethis,G-CORE uses an extension of thepropertygraph data model, namely the Path Property Graph (PPG) data model,that allows paths to be stored and enriched with properties andlabels. Figure3.10depicts asmallsocial network in this model where node labels are depicted as shapes and edge labels as lineswith either an arrow or rhombus.The noveltyinthis data model lies in the lowerleftpart where we findtwo stored paths,one fromJohn to Alice and one from JohntoCeline. As theusersonthe start and endnodeonbothofthese

42 3.6. Further Features

Figure 3.10: Asmallsocialnetwork in the path propertygraph data model thatincludes stored paths. Source: G-CORE ACore for Future Graph Query Languages, Figure 5 [AAB+18].

paths liveinthe same city, namely Houston,wecouldlabel them for exampleby sameCityRelation.Furthermore,wecan store thelengthofsuchapath as a propertyofitsuchthat aquery can directly access this attribute without needing to compute it.

43

CHAPTER 4

Query LanguageAnalysis

Nowthatwehaveintroduced the featuresthatform theconceptualcore of all graph query languages, we analyze 5contemporarylanguages based on those features.The 5 languagesare:Cypher as it is perhaps the most well-known propertygraph language, Gremlin forits imperativeapproach, PGQL for itspowerful path patterns, GSQL forits use of aschema and nativelyparallelapproachand G-CORE, thatbuilds upon theother languages andoffers full composability.Wedid not only choose those languages fortheir differentfeatures and approaches,asnearly everymajor contemporary graph database supportsatleastone of those query languages. To achieveacomprehensiblecomparison, we analyzethe languages oneafter another. For eachlanguage, we start with ageneral introductionthat includes the core principlesofthe languageand giveexamples forthe structure of aquery. We then analyze thelanguages in depth basedonthe previously identifiedfeatures.Wegothrough the implementations of thequeriesfrom Chapter 3, highlightdifferencestoother languagesand look at the expressiveness of alanguage by the supportfor specificfunctions as well as potential shortcomings.Althoughweimplementedall queriesfrom theprevious chapter in the 5languages, we limit ourselvestothose thatare differenttoother implementations in this thesis.Furthermore, thequerydefinitionsinthe SNBexpect an implementation to rename eachvariableinthe result. We omitthis forbrevityonmost queriesinthis chapter as this does notfurther our cause of comparingthe languages.Nonetheless, a full implementationofall queries can be foundonour GitHub repository1.

1https://github.com/martin-kl/Diploma_Thesis_01526110_code

45 4. QueryLanguageAnalysis

4.1Cypher

Cypherisperhaps the best-knownquery languagefor propertygraphs and focuses heavily on patterns. The language is used in multiple databasesand projects, with the version in the Neo4j database [Neo20]and the one from theopenCypher project[ope18]being the most influentialones. As thelanguage is not yetstandardized, it is stillsubject to change and theversions and implementationsofitdifferinsome areas.Unless specifically mentioned, theconceptsand functions apply to all currentimplementations of thelanguage, especiallyalsoCypher 9, thecurrent versionofopenCypher, andthe version in theNeo4j database. [AAB+17] Whereas programminglanguages like Java and other graph query languages likeGremlin are imperativeinnature,meaningthatthey focus on thedescription of theprogram’s operation and in termsofquery languagesonhow to compute theresult, Cypherisa declarativelanguage. Therefore, it focuses on what aqueryshould retrieveinstead of howitachieves the result.This is one of theareaswhere CypherissimilartoSQL, that is declarativeaswell. Apartfromthat, many keywordsinCypherare inspired by SQL to ease the transitionfor userscoming from theworld of relational databases. Being adeclarativequerylanguage and using manyofthe same keywordsasSQL, it comes onlynatural that thegeneral structureofaCypher query resemblesthe one of an SQL query.Aquery in both languages consists of clauses that are chained together [ope18]. In contrast to SQLhowever, Cypher queriesare conceptually structured linearly. Therefore, we can thinkofthe executionofaqueryasstarting with the evaluation of thefirst clause by generating aresult for it, whichinturn is passed on as inputtothe second clause and so on. Thislinear combination leads to another difference as it does notallowthe projection(the RETURN statement,Cypher’s variant of SQL’s SELECT)tobeatthe beginningofthe queryasfirst clause,ithas to be at the end of it. [FGG+18b] Algorithm 4.1 depictsasimple Cypher query consisting of threeclauses, one in eachline. The MATCH clauseisthe entry pointofaquery and contains thebasicgraphpattern (bgp) thatthe query is searching for.Together with the WHERE clausethatfurtherrestricts the pattern, this query searchesfor relationships between theperson JohnDoe and other nodes.Parentheses “()”inthe patterndepictanode whereas edges are depicted by “-[ ]-”. As we can see in our example, thesebrackets cancontaineitherasingle word thatdenotes avariable(rel, dst)oravariable together with afilter on alabel that is given after thecolon. (pe:Person) forexample matches onlythese nodestothe variable pe thatare labeledasPerson.[AAB+17]

1 MATCH (pe:Person)-[rel]- (dst) 2 WHERE pe.firstName=’John’ AND pe.lastName=’Doe’ 3 RETURN p, rel, dst Listing 4.1: ACypherquery thatselects theneighborhood of thepersonJohn Doe.

We can alsosee theconceptuallylinear structure of aqueryand observe thatitcan be evaluated oneclause after theother, starting fromthe topmost that selects all such

46 4.1. Cypher

patterns, followedbythe restrictiononthe variable p to the personJohn Doe, to the projection in the RETURN clause. However, asystemimplementing thequery languagecan re-order clauses as longasthe semantics of thequeryisretained[FGG+18b]. Therefore, the linear structure holdsonly conceptually and an implementation is not bound to the order of clauses whenevaluating aquery. That allowsanimplementation to usethe restrictions given in the WHERE clausealready whensearching forpatterns in the MATCH clause, thereforeimproving the evaluation efficiency.The strong dependencebetween the WHERE and MATCH clauses lead to ashort form thatallows arestriction on properties alsointhe pattern. Thequery in Algorithm 4.2 is equivalenttothe previous one as it contains therestrictionsonpropertiesinthe “{}”brackets. [ope18] 1 MATCH (p:Person {firstName: ’John’,lastName: ’Doe’}) -[rel]- (dst) 2 RETURN p, rel, dst Listing 4.2: Query 4.1 in amore compact notation.

Nowthatwehaveseenasimple example of aquery, we will look at theevaluationof queriesand clauses and theinformation passing between clauses in moredetail. Aclause resemblesafunctionthat, analogously to SQL, operates on an inputtable and produces atableasoutput. In other words, eachclause gets an input table,applies afunctionon the valuesinthattable, and produces an output table thatinturn is the inputtable forthe nextclause.These tablesare alsocalled binding tables in Cypher as they bind variablesfrom the query to elementsinthe graph or other values. However, the definition of aqueryconsisting of clauses that operateonthese tablesleadstoaproblem,asa Cypherquery in an abstractedsense outputsatablebut takes apropertygraph as input. We can generatethe outputasatableunderthe definition of aclause, but until now eachclauseexpects atableasinputaswell. That is especiallyalso true forthe first clauseinaquery, meaning thatsuchaclause getsanemptytableasinput. The MATCH clauseisthe main representativefor aspecial group of clauses regardingthese binding tables 2,asthey usually takeanemptytableasinputand canpopulateitwith entries fromthe propertygraph. Therefore, we needsuchaclause in every query thatinteracts with thegraph, even if we simply selectall nodes in the database as does thequery in Algorithm 4.3. Thisalso highlightsthe status of patterns as an integral partofCypher. [FGG+18b] 1 MATCH (n) 2 RETURN n Listing 4.3: Asimple Cypher query thatselects all nodes.

Linear Composition and theWITH clause Apartfrom the conceptually linear flowbetween clauses in asingle query as seenuntil now, Cypher also supportsthe WITH clausethat allows foralinear composition of queries. As an example, thequery in Algorithm 4.4 startsatJohn Doeand in afirst step traverses

2If we restrict ourselves to the DQL, MATCH is the only suchclause, but there are others like MERGE thatare part of the DML.

47 4. QueryLanguageAnalysis

to all persons m connected to him. Sinceweare onlyinterestedinthe personwith the alphabeticallylastfirst name, we have to order them by theirfirst name in descending order before we can select the top result. We then want to pass thatpersonasinput to the second part of thequery,namelythe second MATCH.However, ORDER BY is a sub-clauseinCypher thathas to followaprojecting clause, meaningeither RETURN or WITH.Tothis end, we have to use the WITH clausetospecify which variables are passed on from thebinding tableofthe first part of thequery to the second MATCH. Furthermore, onlythese variablescan be used in the ORDER BY clause, that in turn is evaluated before the second MATCH as it is asub-clause of WITH.Wehighlight this dependenceofORDER BY on WITH in thealgorithm by the indentation of line 3, which is howevernot required or motivatedbythe Cypher guidelinesand should onlyhelpthe reader in thiscase. Thesecond MATCH clauseinthe query nowuses both, the input variable m it got from theprevious part and thepropertygraph,topopulate its binding table with entriesfor thevariable o and returnproperties of them. [ope18]

1 MATCH (p:Person {firstName: ’John’,lastName: ’Doe’}) -[rel]-(m:Person) 2 WITH m 3 ORDER BY m.firstName DESC LIMIT 1 4 MATCH (m) -[r]- (o:Person) 5 RETURN o.firstName, o.lastName Listing 4.4: ACypherquery thatcontains alinear composition of queriesvia the MATCH clause.

Grouping One well-known keyword from SQLhoweverismissing in Cypher, namely GROUP BY.The functionalityisinstead provided directly by the RETURN clause, as eachnon-aggregate functionisimplicitly takenaskey to group by.This canbeseeninAlgorithm 4.5 where n becomes thegroupingkey andwecountthe number of neighbors o.

1 MATCH (n) -[rel]- (o) 2 RETURN n, count(*) LIMIT 10 Listing 4.5: Asimple Cypher query showinganalternativetoSQL’s GROUP BY.

Apartfrom grouping in thelast clause, we can alsouse WITH to specify the group(s) and the aggregate operation.This allowsustowork with theresultofthe aggregation, for example by filteringout parts of theresult.Asanexample, the query in Algorithm 4.6 selects allfriends of John Doethat have at least tenoutgoing relationships 3.Itincludes some newconcepts, startingwith adirected edgewherethe directionisgiven by an arrow (“<-[]-”or“-[]->”). Note howeverthatthe data model of Cypherdoesnot support undirected edges at all,meaningthatwecannotcreate or store undirectededges. The pattern -[]- thatweusedinthe queries until nowallows us to test for theexistence of some directed relationship, regardlessofthe direction. Furthermore, thereare two anonymous edges and one anonymous node in this query.Theseare entities thathave no

3Note again that theedge label KNOWS depicts afriendship relation in thebenchmark.

48 4.1. Cypher

variable in theircorrespondingbrackets and can thereforenot be accessed later on in thequery or included in theoutput. In this scenario however, we are not interested in the edges that match the directed edge-pattern, but only in the amountofsuchedges. Cypheralsoprovidesashort form forsuchanonymous edges: “--”for edges where we do notspecifythe directionaswell as “<- -”and “-->”for directedones.The WITH clauseisneeded in this query as we want to furtherlimitour selection to friends thathave at least 10 suchedges. To this end, we groupbythe friends f and call the aggregate operation COUNT in the WITH clausethat in turn forwardsthe resulting table to the WHERE clause wherewecan use the previously computedvalues.

1 MATCH (p:Person {firstName: ’John’,lastName: ’Doe’}) -[:KNOWS]- (f:Person) -[]-> () 2 WITH f, COUNT(*) AS out 3 WHERE out >= 10 4 RETURN f.firstName, f.lastName Listing 4.6: ACypherquery thatselects all friends of JohnDoe that have at least ten outgoingrelationships.

Nowthatwehave seensome of thegeneral principlesofCypher, we will go over the queriesfrom Chapter 3and highlight advantages and disadvantages compared to other graph query languages. We took mostimplementationsofthe benchmark queriesfrom theirreference implementationsonGitHub4.Toensure the correctness of thequeries, we executedall of them on aNeo4j CommunityEdition 4.1.0database. However, unless stated otherwise,the queriesuse onlythese functions and proceduresthat are also availableinCypher 9.

4.1.1Structure Independent We have already seenhow we can formulate Query 1: either by restricting the matched nodes in the WHERE clauseasinAlgorithm 4.1 or by using the short form as in Algo- rithm 4.2. Cypheralso supports the parameterization of queriessuchthatthe usercan pass argumentstothe query.Anargumentisthen accessible by preceding its namewith adollar sign as we can seeinAlgorithm 4.7.

1 MATCH (p:Person {firstName:$firstName, lastName:$lastName}) 2 RETURN p Listing 4.7: Query 1asaCypherquery.

The implementation of Query 2inCypherisgiven in Algorithm 4.8. As mentionedinthe introductionofthe query in Chapter 3, it returns thecreationdateofagiven message and either thetextual content in case there is one, or the image file(thatisalsosaved as string).This differentiation can be seen in lines 4-6 of thealgorithm whereaCASE expression distinguishesbetween thetwo cases. To determinewhether amessage has the

4https://github.com/ldbc/ldbc_snb_implementations

49 4. QueryLanguageAnalysis

property content,Cypherprovidesthe EXISTS function. Thisfunctioncan be used on nodes and edges to testfor theexistence of properties. 1 MATCH (m:Message {id:$messageId}) 2 RETURN 3 m.creationDate AS messageCreationDate, 4 CASE EXISTS(m.content) 5 WHEN true THEN m.content 6 ELSE m.imageFile 7 END AS messageContent Listing 4.8: Query 2(IS4) as aCypherquery.

Before we takealook at theimplementation of Query 3, we have to look at Cypher’s capabilities regardingcollections suchaslists.The benchmark states that thelanguages spoken by aperson are stored in alist of strings and Cypherindeed supports lists as well as maps.Lists canbecreated simply by surrounding elementsofthe same type by squarebrackets as in [0,2,4] or by using the collect aggregation. Algorithm 4.9 depicts aCypherimplementation of Query 3. After selecting all persons in line 1, we use SIZE to get thenumberofentries in the speaks propertyofanindividual Person (whichisalist of strings). Thereafter,Cypher implicitly groupsall of thesevalues and calculates theaverage over thesizes. Apartfromthe aggregate operation AVG,Cypher alsosupports MAX, MIN, COUNT, SUM as well as functions to calculate the standard deviationand percentiles. 1 MATCH (p:Person) 2 RETURN AVG( SIZE(p.speaks )) Listing 4.9: Query 3asaCypherquery.

We have already mentionedthe collect functionand we will nowgobrieflyoverthe query in Algorithm 4.10 to giveanexample of it.The query selects all entities (edges and nodes) that are directly related to acontinent whosename starts with ’A’.Atthe endofthe first query part we collect thenames of thesecontinentsintothe list conts that is passed to the UNWIND clause. There, thelist is dissectedagainand each element in the list is passed on as cont to the remaining querypart,meaningthatthis results in one execution of thelatter MATCH for eachentry in the list. Thematches from these executions are then implicitlybrought togetherand we can access all of them in the RETURN clause. Note that thereare simplerwaystospecify this query and we onlyuse this as it allowsustoshowboth, collect and UNWIND in one query. 1 MATCH (c:Continent) 2 WHERE c.name STARTSWITH ’A’ 3 WITH collect(c.name) as conts 4 UNWIND conts AS cont 5 MATCH (c:Continent {name: cont}) -[r]- (n) 6 RETURN c, r, n Listing 4.10: ACypherquery thatselects all entities thatare directly related to a continent thatstarts with ’A’.

50 4.1. Cypher

4.1.2 Pattern Matching We have already seenthatpatterns areused as main building blocksinCypherqueries. Caused by the fact thatwehavealready given multiple examplesofqueriesthat contain simple bgps, we skip the code of Query 4and continue with the similarlooking Query 5. Algorithm 4.11 shows an implementation of this query. It includes an ORDER BY clause thatworks as expected fordeveloperscoming from SQL, meaningthatthe results are first ordered by the friendshipCreationDate in descendingorder and, on identical valuesonthatvariable,onthe personId in ascendingorder. 1 MATCH (n:Person {id:$personId}) -[r:KNOWS]- (friend) 2 RETURN 3 friend.id AS personId, friend.firstName, friend.lastName, 4 r.creationDate AS friendshipCreationDate 5 ORDER BY friendshipCreationDate DESC,toInteger(personId) ASC Listing 4.11: Query 5(IS3) as aCypherquery.

To understand the implementation of Query 6inAlgorithm 4.12, we first have to introduce the OPTIONAL MATCH clause. Similarly to MATCH,wecan specify apatterninthis clauseand the databasesystem searches for matches in thedata. However, if there is no suchmatch or onlyapartial one,meaningthatfor exampleonly some variablesofthe OPTIONAL MATCH clausecan be matched,all unmatched ones are settonull.This resemblesthe semantics of outer joininSQL. In ourexample,wefirst searchfor apattern between agiven message m,areplying comment c and itscreator p.Totest whether p knowsthe personthat created the original message m,weoptionally matchthe path from m to its creator and furthermoreoveraKNOWS edge to p.However, we do not knowifthe KNOWS edge existsasthis is exactly what we want to find out.Therefore, we can then use a CASE expression in the RETURN clause to test whethersuchamatchwas found. 1 MATCH (m:Message {id:$messageId}) <-[:REPLY_OF]- (c:Comment) 2 -[:HAS_CREATOR]-> (p:Person) 3 OPTIONALMATCH (m) -[:HAS_CREATOR]-> (a:Person)-[r:KNOWS]-(p) 4 RETURN 5 c.id, c.content, c.creationDate AS commentCreationDate, 6 p.id AS replyAuthorId, p.firstName, p.lastName, 7 CASE r 8 WHEN null THEN false 9 ELSE true 10 END AS replyAuthorKnowsOriginalMessageAuthor 11 ORDER BY commentCreationDate DESC,replyAuthorId Listing 4.12: Query 6(IS7) as aCypherquery.

Evaluation Semantics Regarding the evaluation semantics of abgp, Cypheruses relationship isomorphism as adefault.That resembles ourno-repeated-edge semantics where no twoedgevariables can be matched to the same edgeinasingle match. This avoidsapotential infinite result in patterns with variable length[AAB+18]. It is envisioned thatfutureversions of

51 4. QueryLanguageAnalysis

Cypher, i.e. Cypher10underthe openCypherproject, allow the user to switchbetween ahomomorphism-based, ano-repeated-node and thecurrentno-repeated-edge semantics on aquery-by-querybase.[GJK+18] However, we can influence thesemantics of aquerybyour choice of thequery structure even today. As an example, assume that we are searching forall friends-of-friends (of lengthexactly 2) of agiven person n.Wecan then collectall thesefriendsinalist and test whether n is in that list or not.The first example in Algorithm 4.13 contains a single MATCH clauseand,caused by the no-repeated-edgesemantics,returns false as the same edgewill not be matched to both edges in the pattern. However, the versionin Algorithm 4.14 returns true as thereare no two edges in asingle MATCH but they are split up intomultiple ones. 1 MATCH (n:Person {id:$id})-[:KNOWS]- (friend) -[:KNOWS]- (foaf) 2 WITH n, collect(foaf) as foafs 3 RETURN n IN foafs //returnsfalse Listing 4.13: Acypherquery thatreturns false as thesame edge will not be matched to both edges in the pattern.

1 MATCH (n:Person {id:$id})-[:KNOWS]- (friend) 2 MATCH (friend) -[:KNOWS]- (foaf) 3 WITH n, collect(foaf) as foafs 4 RETURN n IN foafs //returnstrue Listing 4.14: Acypherquery thatreturns true as thesame edge will be matched to the edges in both MATCH clauses.

Apartfrom that, Cypher also provides UNION and UNION ALL clauses that allow for acombination of results frommultiple queries.Regardingthe evaluationsemantics of suchqueries, Cypher uses abag semantics which is not problematic as we knowthat the evaluation of asinglequery results in afinite number of matches. [AAB+17, ARV19]

4.1.3 Path Cyphersupportsasubset of RPQsasitallows forvariablelengthpaths. These paths can be further restrictedbyasingle relationship type or adisjunctionofsuch[FGG+18b]. In the most general form, we can abbreviateanotherwise unrestricted path with three edges and fournodes by (a) -[*3]- (b).Furthermore,wecan specify an upperbound y on thelengthvia [*..y],alowerbound x via [*x..] and acombination of both via [*x..y].This canbeextended to alsoinclude edgelabels. As an example, we can search for the friends-of-friends of length3via (a)-[:KNOWS*3]-(p).Besidespaths of variable length, adisjunctionoflabelsisgiven as (a)-[:KNOWS|WORK_AT]-(b). However, Cypher doesnot supportfull composition of regular expressionsoveredge labels and its path querying functionalityistherefore notasexpressiveasRPQs. Apartfrom restrictions on paths, Cypheralsoallows paths to be assignedtovariables. As an example, we can assign apath to avariableasinAlgorithm 4.15. Thisallows

52 4.1. Cypher

us to furtherrestrictapath, for example in a WHERE clause, and to return it.Cypher providesfunctions like nodes and relationships to access theseentities on apath in order to restrict or returnthem as well as length to get thelengthofapath.

1 MATCH p=(n:Person {id:$id}) -[:KNOWS*..5]-> () 2 WHERE length(p) >2 3RETURN relationships(p), length(p) LIMIT 10 Listing 4.15:ACypherquery that demonstrates named paths (paths that are assigned to variables).

Algorithm 4.16 depictsanimplementation of thereachabilityquery7.This is an interesting query as it searches for theshortestpathbetween two nodes with arestriction on theedges thatcan be traversed. Cypher providesthe shortestPath function for exactlythat, meaning that thefunctionfinds“asingle shortest path between two nodes” [ope18, Neo20]. However, it is notspecified which path is returned when there are multiple shortestones. As we do not knowifsuchapathexists at all in our case,weuse the function in an OPTIONAL MATCH so that we can later test whether suchapath was found or not.Apart fromthe shortestPath function that returns asingle shortest path, Cypher also providesthe allShortestPaths functionthat returns all shortest paths between two nodes.

1 MATCH (person1:Person {id:$person1Id}), (person2:Person{id:$person2Id}) 2 OPTIONALMATCH path =shortestPath((person1)-[:KNOWS*]-(person2)) 3 RETURN 4 CASE path IS NULL 5 WHEN true THEN -1 6 ELSE length(path) 7 END AS shortestPathLength; Listing 4.16: Query 7asaCypherquery.

Algorithm 4.17contains an implementation of theadjacency query 8. Thequery incorpo- rates multiple concepts and clauses that we explained before and at afirstglance looks quite complicated. However, this comes mostlyfrom the factthat it starts with asimple patternthatgathersfriends up to distance 3and then proceeds by collecting further information about them in multiple steps.Asthere canbemore than one pathoflength 1to3between two persons thatknoweachother, we use the shortestPath function to get asingle one. After excluding the start personinline 2, we pass the required informationtothe next query partvia the WITH clause. Thispart is then evaluated oncefor everyfriendfrom the first part and gathersinformationabout the place the friend is livingatand optionallythe place(s) he or she studied at.Information about asingle universityisgathered in amanually created list in line 12. As aperson can have relations to none,asingleormultiple universities, we iterate over them and collect theinformationinalist using collect.The informationiscombinedinanother WITH clause, forwarded to the next query part where information aboutthe work-relationships of thefriend are added and finally returned. Thisquery shows howwecan iteratively

53 4. QueryLanguageAnalysis

gather information and that theinformationwewanttofinally return hastobepassed to theend via WITH clauses.

1 MATCH p=shortestPath((person:Person {id:$personId}) -[path:KNOWS*1..3]- ( friend:Person {firstName:$firstName})) 2 WHERE person<>friend 3 WITH friend, length(p) AS distance 4 ORDER BY distance ASC,friend.lastName ASC,toInteger(friend.id) ASC 5 MATCH (friend) -[:IS_LOCATED_IN]-> (friendCity:Place) 6 OPTIONAL MATCH (friend) -[studyAt:STUDY_AT]-> (uni:Organisation) -[: IS_LOCATED_IN]-> (uniCity:Place) 7 WITH 8 friend, 9 collect( 10 CASE uni.name 11 WHEN null THEN null 12 ELSE [uni.name, studyAt.classYear, uniCity.name] 13 END 14 ) AS unis, 15 friendCity, 16 distance 17 OPTIONAL MATCH (friend) -[workAt:WORK_AT]-> (company:Organisation) -[: IS_LOCATED_IN]-> (companyCountry:Place) 18 WITH 19 friend, 20 collect( 21 CASE company.name 22 WHEN null THEN null 23 ELSE [company.name,workAt.workFrom, companyCountry.name] 24 END 25 ) AS companies, 26 unis, 27 friendCity, 28 distance 29 RETURN friend, distance,friendCity.name, unis, companies

Listing 4.17: Query 8asaCypherquery.

We skip theimplementation of Query 9asitdoesnot provideany new insights and can be written as arelativelysimple navigational query.

Evaluation Semantics Similar to the evaluation semantics of pattern matching queries, Cypher uses ano-repeated- edge semantics whenevaluating path queries[DXWL19]. Therefore, the evaluation of suchaquerywill never takethe sameedge inapath, whichinturn avoids paths of unlimited length. Regardingpossibleoutputs of aquery, Cyphersupportseverything fromfixed arity outputs likeBooleans, single properties and entities to variable arity outputslikepathsand lists.However, we are not abletoreturn agraph in Cypher as of now, although somethinginthat regardshould be supported in Cypher10[FGG+18b].

54 4.1. Cypher

4.1.4 Data Manipulation Nowthatwehaveseenavarietyofqueries thatselectdatafrom the graph,wewill look at DMLqueries. As mentionedinChapter3,queriesthatmanipulatedataare expressedsimilarly to ones that onlyretrievedata. Thiscan be seeninthe simplified implementation of Query 10 in Algorithm 4.18 that uses thesame patterns also to create nodes and relationships.The query inserts anew personnodevia the CREATE clauseas well as arelationship labeled IS_LOCATED_IN to connect thenew node to the citythe personisliving in. All of that is doneinthe first twolines and thecodeinthe following linesthen creates avariablenumberofrelationships to already existingnodes. We use UNWIND to unroll thelist of tags, study and work information andeachsuchentry is then passed to one MATCH-CREATE combination.Asaninteresting side note, we have to include count(*) in the WITH clauses to avoid an unexpected behavior: if we pass forexample two tagsintagIds,this results in two evaluationsofthe MATCH-CREATE combination in lines 5and 6whichinturn would result in two calls of the WITH clause in line 7. By adding count(*) in thatclause, we implicitly group on the singlekey p which in turn is theonly value thatisthen passed to the next query part.

1 MATCH (c:City {id:$cityId}) 2 CREATE (p:Person {id: $personId, firstName:$fn, birthday: date($bd), creationDate: timestamp($cd), emails: $emails}) -[:IS_LOCATED_IN]-> (c) 3 WITH p 4 UNWIND $tagIds AS tagId 5 MATCH (t:Tag {id: tagId}) 6 CREATE (p) -[:HAS_INTEREST]-> (t) 7 WITH p, count(*) AS dummy1 8 UNWIND $studyAt AS s 9 MATCH (u:Organisation {id: s[0]}) 10 CREATE (p) -[:STUDY_AT {classYear: s[1]}]-> (u) 11 WITH p, count(*) AS dummy2 12 UNWIND $workAt AS w 13 MATCH (comp:Organisation{id: w[0]}) 14 CREATE (p) -[:WORKS_AT {workFrom: w[1]}]-> (comp) Listing 4.18: Simplified versionofQuery 10 (INS1) in Cypherthatinserts onlysome properties.

Apartfrom CREATE,Cypheralsoprovidesthe MERGE clausethatensures thatagiven patternexists in agraph. If the patternalready exists, it works similartoMATCH as the variablesare bound to the existing and matched entities.Otherwise,the entities are first createdand thenboundtothe variables. Cypherprovidestwo clauses to remove entities from thegraph: DELETE removesasingle node or edge whereas DETACH DELETE removesanodetogetherwith all incident edges. Regarding the removalofconnected nodesfrom the graph,stale edgesare not allowedat anytime.Therefore, aquerythat deletes anodewithout alsoremoving its relationships will fail. That alsomeansthat the removal of aconnected node will failvia the DELETE command. Using DETACH DELETE however, we are abletoremove connected nodesand

55 4. QueryLanguageAnalysis

trigger recursiveremovals via ashort query.This canbeseeninthe implementation of Query 11 in Algorithm4.19.

1 MATCH (comment:Comment {id: $commentId}) <-[:REPLY_OF*]- (replies:Comment) 2 DETACH DELETE comment, replies Listing 4.19: Query 11 (DEL7) as aCypherquery.

Apartfrom inserting and deleting nodesand edges, we can alsoupdatetheir properties, add newones or remove existingones. An exampleofsuchanupdate can be seeninAlgo- rithm 4.20 where we change an existingproperty. The SET command is alsoused to insert anew property,meaningthatCypher does nottestwhether the property classYear already existsbut simply savesitwith the new value,overwriting apotentially existing one or creatingitanew.Toremove aproperty from anodeoranedge, Cypher provides the REMOVE clause. NodesinCypher’s data model can have multiple labels where the same commands are used to update,add or remove them.Edges howevercan onlyhave one label that is setatedgecreation and cannot be updated or removedlater on. 1 MATCH (:Person {id:$personId}) -[r:STUDY_AT]-> (u:University {id:$univId}) 2 SET r.classYear=2020 Listing 4.20: Query 12 that sets (updates or adds) the property classYear.

4.1.5 Data Definition Cypherinits original versiondoesnot support aschema and “was originally conceived in adynamically typed,schema-lesscontext”[FGG+18b]thatworks on asingle graph only. That is stilltrue in the currentversion governedbyopenCypher[ope18], but other implementations like Morpheus: Cypherfor Apache Spark5, SAP HANAGraph6 and CypherinNeo4j support at least some kindsof(schema) constraintsand more than one graph.Wewill nowlookatthe constraints available inNeo4j as thesewill likely alsobeavailable in future versions of openCypherasthere exists aCypherImprovement Proposal7 forthem in openCypher. Unique node property constraints ensurethatthe valuesofapropertyonanodewith aspecific label are unique. This constrainthowever does notenforce thatthe propertyhas to existonall nodes with this label butonly that if thepropertyexists,its value is unique. Node property existenceconstraints allow us to specify thatall nodes with aspecific label have to have aspecific propertyand relationship property existence constraints are the counterpart for edges. Finally, node key constraints ensurethat aset of properties has to exist on each node with agiven label and thecombination of thepropertyvalues hastobeunique.Wecouldenforce at least parts of aschema using thelatterthree constraints. However, they are onlyavailable in the EnterpriseEditionofthe Neo4j database whereas theopensource Community Editionislimited to uniquenodepropertyconstraints.

5https://github.com/opencypher/morpheus 6https://www.sap.com/products/hana.html 7https://github.com/opencypher/openCypher/pull/166

56 4.1. Cypher

Even if we were to use Cypher as in Neo4j’sEnterprise Edition,wecouldnot implement Query13asthereisnoway to specify allowed patterns. The onlything we can do is to change theexisting relationships in away thatthey comply with the the query.Tocope with thelackofaschemaingeneral, Cypherprovidesfunctions like EXISTS,thatcan alsotestfor theexistence of patterns, or the OPTIONAL MATCH clause. However, having aschema would allow asystem to fully type checkqueries up-front andadd additional optimization potentialfor thequeryplanner and theexecution [FGG+18b].

4.1.6Further Features

• Composability: We have already seenthatCypher supports linear composition of queriesvia the WITH clause. If we look into (nested) subqueries, we are again limited to versions of Cypherdifferentfrom openCypher.There is howeveraCypher ImprovementProposal8 thattargetsversion10ofthe languageand introduces this feature in the openCypherversion. We will nowlookatanexample of acorrelated subquery as it is currentlysupported by the versionofCypher in Neo4j.Asan example, assume that we want to calculate the number of persons thatare older than John Doe. Algorithm 4.21 depictsanimplementation of this querywhere we use the “CALL {}”clause9 to evaluate the subquery.Again,wehave to use WITH to specify whichelements of the binding table from the outer query are passed to the subquery.

1 MATCH (p:Person {firstName: ’John’,lastName: ’Doe’}) 2 CALL { 3 WITH p 4 MATCH (o:Person) 5 WHERE o.birthday

However, Cypher allows us to formulate many suchqueries in away thatavoids subqueries altogether. The queryinAlgorithm 4.22 is equivalenttothe previous one but achieves this via two (uncorrelated)matches that are then relatedtoeach other in the WHERE clause. We use WITH in this implementation to group by p and countthe matches of younger persons before we return them.

1 MATCH (p:Person {firstName: ’John’,lastName: ’Doe’}) 2 MATCH (o:Person) 3 WHERE o.birthday

8https://github.com/opencypher/openCypher/pull/100 9Note thatthere is also a“CALL”clause available in both,Neo4j’s and openCypher’sversion, that invokesaprocedure. The “CALL {}”clause thatinvokesasubquery howeverisonly available in Neo4j’s version.

57 4. QueryLanguageAnalysis

4 WITH p, COUNT(o) as youngerCount 5 RETURN p.firstName, p.lastName,youngerCount Listing 4.22:Analternativeversion of thequery fromAlgorithm 4.21 that avoidsthe subquery.

Neo4j’s versionofCypher furthermore allows for existential subqueries using a combination of WHERE and EXISTS.Unlikeexplained above,queries nested through WHERE EXISTS can access variablesfrom the outer scopeevenwithout WITH.On the other hand, suchqueries canoftenbereformulatedinawaythatthey avoid existential subqueries altogetherbyusing amore sophisticated pattern, OPTIONAL MATCH or specifying therequirements in the WHERE clause.

Apartfrom that, Cypher in its currentstate is notacomposablelanguage as a query outputs atablebut takes agraph and an (empty) tableasinput. There is howeveragain aCipherImprovementProposal10 to include this in Cypher 10 via achangeofthe binding tables. Theplan is to adaptthese tablessothat they notonly include tabular information but alsoone or multiplegraphs, therefore becoming table-graphs [FGG+18b]. Thiswould enable query composability butalso requires achangeofother structures as Cypher9cannot deal with multiple graphs. There is already one implementation of Cypherthat supportsthese features,namely Morpheus,the version for Apache Spark. • Pathsasfirst-class citizens: We mentioned before thatCypher allowspaths to be matched and returned. However, this is not enough for it to treat them as first-class citizens as we cannot save paths or add propertiestothem and to the best of our knowledge,there is no attempt to change this as of now.

As we have seeninthis section, Cypher focusesheavily on patterns andprovidesasyntax thatallows forcompactqueriesinmanycases.Itprovidesamyriadofmathematical and string functions that we didnot introduceaswell as aggregating and graph-specific functions like shortestPath, nodes, labels.The versionofCypher in Neo4j also includestemporal functions for dates, times including time zones and durations as well as spatial functions. Apartfrom thesefunctions that are includedinthe language,wecan extend Cyphervia user-defined functions. These functions aredeployedintothe database and can then be called similar to Cypher functions in queries. TheNeo4j database also supports user-defined procedures thatcan takearguments and interactwith the database before returning aresult.InNeo4j,these functions and proceduresare written in Java and thecompanyprovidessome of them in bundles. AwesomeProcedures On Cypher (APOC)11 and abundle especially fordata scientists12 are examples for them. 10https://github.com/opencypher/openCypher/pull/241 11https://neo4j.com/labs/apoc/, https://github.com/neo4j-contrib/ neo4j-apoc-procedures 12https://github.com/neo4j/graph-data-science/

58 4.2. Gremlin

4.2Gremlin

UnlikeCypherthatisahigh-leveldeclarativelanguage, theGremlin query language [Tin20]isalow-level language that offersimperativeaswellasdeclarative constructs [Rod15]. Although it is afunctionallanguage at itscore,itismore imperativeinnature and focuses on graph traversals instead of pattern matching [AAB+17]. However, apart fromits focusontraversals and thereforenavigational queriesinthe imperativestyle, Gremlin providespattern matching featuresinadeclarativeconstruct [TPAV19]. From the earliestprototypes onwards, Gremlin extensively usesXPath to provide complex graph traversals [Lin18, AAB+18]. As brieflymentionedinthe introductionofGremlininChapter 2, Gremlin denotes not onlythe querylanguage butalsothe traversalmachine GTM usedinthe TinkerPop framework. Aquerywritten in the Gremlin querylanguage is compiled to generatea traversal thatisthen executed on theGTM. In the imperative traversalmode, that is alsousedinsome sense to match patterns as we will seelateron, theuser specifies so called motifs,that are traversal instructions containing multiple steps.These steps are then executed by a traverser,aninstance of a traversal,inthe GTM.Inotherwords,a single traversalhas aset of traversers attachedtoitthatwalk on agraph according to the steps fromthe query.The result of suchatraversalthen contains all locations where atraverser halted.From nowon, we mean thequery language and not thetraversal machine whenwespeak about Gremlin. [Rod15, AAB+17] Before we can examineour first query on thethird line in Algorithm 4.23, we have to createatraversalsource on agraph. Suchasource is needed to createatraversalthat in turn resemblesaquery and interacts with agraph. After connectingtosome graph and assigning it to the graph variableinline 1, we createthe traversalsource g in line 2. Thistraversalsourcecan be usednot onlyfor asingle query but forall queriesonthe graph.The queryinline3selectsall properties(values)ofJohn Doe. Similar to the linearstructureofCypher queries, aqueryinGremlin is alwaysevaluated fromleft to right.Starting fromthe traversalsource g,wefirst generateatraversalthatstarts at the verticesinthe graph via V().Intuitively,there is nowone traverser on eachvertex in the graph andthe followingsteps specify howthey proceed.Ifthe query wastoend here, meaning that it onlycontains g.V(),weget all nodesasaresult sincethere is one traverser on eachnodeafter the executionofV().Inour query in Algorithm 4.23 however, we continuewith a has() step to remove all traverserswherethe corresponding element (a node in our case) does not have the given key/value property. Therefore, we are left with traversersononly those nodes thathavea’firstName’/’John’ property. Note thatthis alsoremovesall traversers on nodes thatdonot resembleaperson as thesedo not have a firstName property. After repeating this step to also remove traversers on nodes thatdonot have the ’lastName’/’Doe’ property, we selectall propertyvalues of the node with values. Note that thecommands can be run as given in thealgorithminthe Gremlin Console, an interactive terminal that connectstothe GTM and allowsfor ad hocqueries to be

59 4. QueryLanguageAnalysis

executed. Forqueries in applicationshowever, one normally usesanimplementation of Gremlin in aprogramminglanguage and interacts with aGTM from there.Wewill go into more details regardingsuchimplementations and theirdifferencesdetaillateron. 1 gremlin> graph =TitanFactory.open() 2 gremlin> g=graph.traversal() 3 gremlin> g.V().has(’firstName’,’John’).has(’lastName’,’Doe’).values() Listing 4.23: An exampleonhow to connecttoagraph,create agraph traversalsource on thegraph and spawn atraversalinthe Gremlin Console. Thequery then selects the propertiesofnode(s)with the name John Doe.

The query in Algorithm 4.24 selects thenames of all friendsofJohn Doe. In the previous query,weremovedall traversers on nodesthatdonot resembleaperson implicitlyby requiring the property firstName.Wecan alsostate suchalabel requirement explicitly via the hasLabel() step. After thesteps in line 1are executed,weare leftwith a traverser on the node representing John Doe(assuming that thereisonly one suchnode). The next step, out(),isavertex step that allowsustotraverse overanoutgoingedge to an adjacentvertex, denoted as outgoingadjacentinGremlin.Inour query,welimit the traversaltoknows edges (edgeslabeled with “knows”).Incase thereare multiple such adjacentvertices, this stepessentially spawnsnew traversers andassigns eachofthose verticesone traverser. Apartfrom moving to outgoingadjacentvertices,Gremlin also supports in() to move to incoming adjacentvertices (adjacent vertices connectedover incoming edges) and both() to move to both, incoming and outgoingadjacentones. In the last line of this query, we select thefirstand last name of the friends of JohnDoe. 1 g.V().hasLabel(’person’).has(’firstName’,’John’).has(’lastName’,’Doe’) 2 .out(’knows’) 3 .values(’firstName’, ’lastName’) Listing 4.24: AGremlin query that selects the names of John Doe’s friends.

As we have seeninthis query,weare forcedtothink in agraph-perspectivewhen writing aqueryinGremlin [Rod15]. Untilnow,this perspectiveismostly limited to verticesas we can selectand filterthem and move fromone to another. However, we can alsomove to edges whichcan be seeninAlgorithm 4.25. Thisquery selectsamap containing all key/value pairs of outgoing knows edges of JohnDoe via valueMap().The outE() step is again avertex step butunlikethe ones mentionedbefore, movestooutgoing incident edges.Similartothe vertex steps fromabove,Gremlin alsoprovidessteps like inE() to move to incoming incident edges and bothE() to move to both, incoming and outgoingones. All steps mentioneduntil nowcan be invokedontraversers on vertices. Furthermore, steps like outV(), inV(), bothV() and otherV() can be usedon traversers on edges that allow us to move from an edge to avertex. 1 g.V().has(’firstName’,’John’).has(’lastName’,’Doe’).outE(’knows’).valueMap() Listing 4.25:AGremlin query thatselects properties of outgoing knows edges of John Doe.

60 4.2. Gremlin

All of our queriesuntil noware composed as aconcatenation of steps which lookssimilar to ageneral programming language. And indeed,Gremlin is sometimesdenoted as a “programminglanguage for property graphs”[AAB+18]. Thissets itapart from most other querylanguages,especiallyalso Cypher, and allowsusfor exampletostore the result fromone query in avariableand use it later on in adifferentquery. Algorithm 4.26 contains some examples for this whereeachline depicts asingle query.The query in the first line assigns thenodedepicting JohnDoe to the variable john.Inordertoassign it, we have to use the next() step that triggersthe evaluation.This is needed as Gremlin usesalazy evaluation on most queries. We were abletoomit this stepinthe previous queriesasqueriesentered in theGremlin Console are evaluated automatically after pressingEnter.However, thatdoesnot work when assigning aqueryresult to variables, where we have to manuallytrigger theevaluation of thequery to assign theresult.Now thatwehave areference to this node, we can use this variable in the followingqueries. As an example, we directly start fromthe node john in the query in line 3and traverse to verticesconnected over knows edges and select their firstName. Anotherarea where Gremlin differs from most query languagesisthe support for lambdas. Lambdasteps like map(), flatMap() and filter() allow for custom functions that can be usedtorestructureorfilter data.Although these steps “represent the foundational constructs of the Gremlin language”[Tin15, Tin20], they should be avoidedifpossible as they cannot be inspected and optimized by the compiler. Nonetheless, having theability to use lambdas in case there is no lambda-lessstep directly providedbyGremlin allows auser to essentially run anyfunction. Therefore, it is alsonot surprising that Gremlin is aTuring-completelanguage [WD18]. The queriesinlines3,4and 5are equivalent and while thefirst versionuses Gremlinfunctions, thelatterones use the lambdastep map(). 1 gremlin>john =g.V().has(’firstName’,’John’).has(’lastName’,’Doe’).next() 2 //the following three queries are equivalent: 3 gremlin>g.V(john).out(’knows’).values(’firstName’) 4 gremlin>g.V(john).out(’knows’).map(values(’firstName’)) 5 gremlin>g.V(john).out(’knows’).map {it.get().value(’firstName’)} Listing 4.26:Multiple Gremlin queriesthatassign and use avariableand demonstrate the use of lambdafunctions.

Each queryuntil nowstartedwith the V() step, that spawnsatraversalonthe vertices fromthe traversalsource g.Gremlin alsoallows atraversalthat startsonthe edgesvia the E() step as can be seeninAlgorithm 4.27. groupCount() is astep thatcounts howoften an object has been partofatraversaland cantherefore notonlysummarize labels but also anyotherinformationlikethe namesorages of persons. 1 g.E().label().groupCount() Listing 4.27: AGremlin traversalstarting fromthe edges that generates asummary of the edgelabels.

61 4. QueryLanguageAnalysis

We can furthermoreuse union() to combinethe resultsofmultiple traversals or query parts. As we can see in Algorithm 4.28, we can save areference to an object using as() and can later on in thequery accessitvia select().This query selects apersonwith agiven id13 and combinesthe first name of thepersonwith the number of outgoing knows edges in the union() step. The laststep, fold(), aggregates these two valuesintoalistand emits it.

1 g.V().has(’iid’,’person:933’).as(’person’). 2 union( 3 select(’person’).values(’firstName’), 4 outE(’knows’).count() 5 ).fold() Listing 4.28:Agremlinquery thatoutputs informationabout aspecific personand counts the number of outgoing edges.

All queriesinthis chapter were testedonaTitan 1.0.0 database14 thatnativelysupports the TinkerPop graph stack and thereforealsothe Gremlin query language.However, as Titan is no longer maintained (but afork of it,JanusGraph15,is),itdoesnot use the most recent version of TinkerPopand thereforeGremlin [Tin20]but the older version 3.0.1-incubating [Tin15]. As we ranall queriesonTitan, they are compatible with version 3.0.1and do not use additions that were introducedinlaterversions. An exampleofsuch an addition is the propertyMap() step that extends valuesMap() to alsoinclude the id and thelabel of an entity.However,all of thecore functionalities and steps are alsoavailable in version3.0.1.

Apartfrom the database we will brieflyexplain the workingsofthe Gremlin Console, whythereare implementationsofGremlin in multiple programming languages and which one we use in our queries. The TinkerPopprojectprovidesareference implementation of Gremlin in Java 8: Gremlin-Java.However,usersare freetoimplementavariant of Gremlin in anyother programming language as longasthe language supportsfunction composition and function nesting. Suchanimplementationprovidesthe step functions and is responsible for thetranslation of queriestoGremlin Bytecode thatinturn canbe executed on aGTM.Apart fromthe implementation in Java,TinkerPop also provides an implementation in Apache Groovy: Gremlin-Groovy.This language variant is used in the Gremlin Console.Therefore,the Gremlin queries we have seen untilnow can all be run on everyGroovy-systemthat includes the required libraries forGremlin.However, the only notabledifferencesbetween thesetwo language variantsregardsthe use of single versus double quotes forstrings(Groovyallows single quotes, Java requires double quotes), and asmallchange in syntax on anonymous entities.The followingqueriesare implemented in Gremlin-Java.Wepooled the benchmark queriesfrom twoGitHub repositories: the

13We use the property iid in our datamodel for our custom ids as id() is anativeGremlin function that accesses theidgenerated by Gremlin and not the one generated by theLDBC datagenerator. 14http://titan.thinkaurelius.com/ 15https://janusgraph.org/

62 4.2. Gremlin

original implementation from PlatformLab16 and afork and extension of this from Anil Pacaci17.

4.2.1 Structure Independent We have already seenhow aperson node with agiven first andlastname can be selected, which is exactlywhat the query in Algorithm 4.29 does. Nonetheless, we givethis as an exampleofaquerywritten in Gremlin-Java,wherethe onlydifference to queries from before is the usage of doublequotes forstringsThatmeans, thequerycan be executed directly in anyJavaapplication thatincludesthe Gremlin-Java libraries.Therefore, the usage of Gremlindiffers from the use of query languageslikeSQL in application code as SQL code is usually providedasastring thatispassed over an interface to thedatabase. In contrast,aGremlin query is implementedinthe programming language itself and compiledtoBytecode to be executed. The query also showsanextensionofthe has() step that can require anodetohave alabel (the first argument)and akey/valueproperty in asingle statement.

1 String firstname = "John"; 2 String lastname = "Doe"; 3 System.out.println(g.V().has("person", "firstName",firstname).has("lastName" ,lastname).valueMap()); Listing 4.29: Query 1asaGremlinquery in the Gremlin-Java language variant.

Anotherexample where we can see theinteraction between code written in Java and the Gremlin query is given in Algorithm 4.30. Whereas thequery itself is given in lines1-3, the result of thequery is accessedinthe followinglines where we checkwhether the post contains an imageFile or atextual content.Nonetheless, we run into alimitation of Gremlin when implementing this query as thedatamodel onlysupportsasingle label on vertices. In our case however, postsand commentsare both messagesand we labeled them either with “post, message”or“comment, message”when working with Cypher. Sincethe corresponding nodes in Gremlin canonly have asingle label, they are either labeled by “post”or“comment”, and we cannot directly search for a message. We achievethis functionalityinGremlin using the within predicate in a has() step to filternodes labeled with post or comment.Apart fromthat, the algorithm shows that accessingthe resultsfrom Java is not as straightforward as one mightthink since we need to repeatedly cast variablesoftype Object to List in order to get thefirst entryofthese lists.Onaside note,the T and P prefixes are onlyneeded in Gremlin-Java as arestriction from Java and can for example be omitted in Gremlin-Groovy.

1 Mapvalues =g.V().has(T.label,P.within("post", "comment")). 2 has("iid",messageID). 3 valueMap("iid", "creationDate", "content", "imageFile").next(); 4 List result = new LinkedList<>();

16https://github.com/PlatformLab/ldbc-snb-impls 17https://github.com/anilpacaci/graph-benchmarking

63 4. QueryLanguageAnalysis

5 result.add(((List) values.get("iid")).get(0)); 6 result.add(((List) values.get("creationDate")).get(0)); 7 if (((List)values.get("content")).get(0).equals("")) 8 result.add(((List) values.get("imageFile")).get(0)); 9 else 10 result.add(((List) values.get("content")).get(0)); Listing 4.30: Query 2(IS4) as aGremlin query in Gremlin-Java.Wecan see howresults of aquery canbeaccessed in Java.

Algorithm 4.31 contains twodifferentimplementationsofQuery 3. Forthis query,we have to access themulti-value property speaks of aperson.Gremlin’s data model allows forsuchmulti-value properties, meaning that asingle propertykey on avertex can have multiple valuesattached to it.The first version savesareference to atraversalinJava and iterates manually over thetraversaltotrigger thelazy evaluation and access the results.Inthe second version, we use twoqueries that calculate thetwo sums and then calculatethe average in Java.However,both versions compute the average in Java and not in the query directly.While Gremlin providessteps to group,countand compute the average, it is not clear howtoformulatethis with the access to amulti-value propertyin asingle query. 1 //Version 1: 2 Map map; 3 GraphTraversal> tr =g.V().hasLabel("person"). values("speaks"); 4 while(tr.hasNext()) { 5 map =tr.next(); 6 overall+=((List