Noname manuscript No. (will be inserted by the editor)
gStore: A Graph-based SPARQL Query Engine
Lei Zou · M. Tamer Ozsu¨ · Lei Chen · Xuchuan Shen · Ruizhe Huang · Dongyan Zhao
the date of receipt and acceptance should be inserted later
Abstract We address efficient processing of SPARQL 1 Introduction queries over RDF datasets. The proposed techniques, incorporated into the gStore system, handle, in a uni- The RDF (Resource Description Framework) data model form and scalable manner, SPARQL queries with wild- was proposed for modeling Web objects as part of devel- cards and aggregate operators over dynamic RDF datasets. oping the semantic web. Its use in various applications Our approach is graph-based. We store RDF data as a is increasing. large graph, and also represent a SPARQL query as a A RDF data set is a collection of (subject, property, query graph. Thus the query answering problem is con- object) triples denoted as hs, p, oi. A running example verted into a subgraph matching problem. To achieve is given in Figure 1a. In order to query RDF reposito- efficient and scalable query processing, we develop an ries, SPARQL query language [23] has been proposed index, together with effective pruning rules and efficient by W3C. An example query that retrieves the names search algorithms. We propose techniques that use this of individuals who were born on February 12, 1809 and infrastructure to answer aggregation queries. We also who died on April 15, 1865 can be specified by the fol- propose an effective maintenance algorithm to handle lowing SPARQL query (Q1): online updates over RDF repositories. Extensive exper- SELECT ?name WHERE iments confirm the efficiency and effectiveness of our {?m
Although RDF data management has been studied Extended version of paper “gStore: Answering SPARQL over the past decade, most early solutions do not scale Queries via Subgraph Matching” that was presented at 2011 to large RDF repositories and cannot answer complex VLDB Conference. queries efficiently. For example, early systems such as Lei Zou, Xuchuan Shen, Ruizhe Huang, Dongyan Zhao Jena [31], Yars2 [14] and Sesame 2.0 [6], do not work Institute of Computer Science and Technology, well over large RDF datasets. More recent works (e.g., Peking University, Beijing, China Tel.: +86-10-82529643 [2,20,33]) as well as systems, such as RDF-3x [19], x- E-mail: {zoulei,shenxuchuan,huangruizhe,zhaody}@pku.edu.cn RDF-3x [22], Hexastore [30] and SW-store [1], are de- M. Tamer Ozsu¨ signed to address scalability over large data sets. How- David R. Cheriton School of Computer Science ever, none of these address scalability along with the University of Waterloo, Waterloo, Canada following real requirements of RDF applications: Tel.: +1-519-888-4043 E-mail: [email protected] – SPARQL queries with wildcards. Similar to SQL and Lei Chen XPath counterparts, the wildcard SPARQL queries Department of Computer Science and Engineering, Hong Kong University of Science and Technology, enable users to specify more flexible query criteria in Hong Kong, China real-life applications where users may not have full Tel.: +852-23586980 knowledge about a query object. For example, we E-mail: [email protected] may know that a person was born in 1976 in a city 2
Subject Property Object y:Abraham Lincoln hasName “Abraham Lincoln” y:Abraham Lincoln bornOnDate “1809-02-12”
y:Abraham Lincoln diedOnDate “1865-04-15” 030 031 y:Abraham Lincoln bornIn y:Hodgenville KY “1926-07-01” “Female” y:Abraham Lincoln diedIn y:Washington DC bornOnDate gender y:Abraham Lincoln title “President” 009 029 diedOnDate hasName 032 y:Abraham Lincoln gender “Male” “1962-08-05” y:Marilyn Monroe “Marilyn Monroe” y:Washington DC hasName “Washington D.C.”
y:Washington DC foundingYear “1790” 010 013 014 y:Hodgenville KY hasName “Hodgenville” “Abraham Lincoln” “President” “Male” y:United States hasName “United States” hasName title gender 001 003 y:United States hasCapital y:Washington DC 011 bornOnDate bornIn hasName 017 y:United States foundingYear “1776” “1809-02-12” y:Abraham Lincoln y:Hodgenville KY “Hodgenville” y:Reese Witherspoon bornOnDate “1976-03-22” diedOnDate 025 026 012 diedIn 019 “Franklin D. Roosevelt” “Male” y:Reese Witherspoon bornIn y:New Orleans LA “1865-04-15” 002 “1776” hasName gender y:Reese Witherspoon hasName “Reese Witherspoon” y:Washington D.C. 007 020 y:Franklin Roosevelt y:Reese Witherspoon gender “Female” foundYear foundingYear “1976-03-22” 015 hasName hasCapital y:Reese Witherspoon title “Actress” 016 004 title “1790” 027 bornOnDate 005 “Washington D.C.” y:United States y:New Orleans LA foundingYear “1718” “President” y:New Orleans LA locatedIn y:United States y:Reese Witherspoon bornIn y:Franklin Roosevelt hasName “Franklin D. Roo- gender hasName 021 locatedIn locatedIn title hasName bornIn sevelt” “Female” 018 y:Franklin Roosevelt bornIn y:Hyde Park NY 022 023 006 “United States” 008 “Reese Witherspoon” y:Franklin Roosevelt title “President” “Actress” y:New Orleans LA y:Hyde Park NY y:Franklin Roosevelt gender “Male” foundingYear foundingYear y:Hyde Park NY foundingYear “1810” 024 028 y:Hyde Park NY locatedIn y:United States “1718” “1810” y:Marilyn Monroe gender “Female” y:Marilyn Monroe hasName “Marilyn Monroe” y:Marilyn Monroe bornOnDate “1926-07-01” y:Marilyn Monroe diedOnDate “1962-08-05” (a) RDF Triples (Prefix: y=http://en.wikipedia.org/wiki/) (b) RDF Graph G
Fig. 1: RDF Graph
that was founded in 1718, but we may not know the In this paper we describe gStore, which is a graph- exact birth date. In this case, we have to perform a based triple store that can answer the above discussed query with wildcards, as shown below (Q2): SPARQL queries over dynamic RDF data repositories. SELECT ?name WHERE In this context, answering a query is transformed into {?m
regex(str(?bd),“1976”) discuss the maintenance of indexes (VS∗-tree and T- index) as RDF data get updated in Section 10. We ?name ?bd “1718” study our methods by experiments in Section 11. Sec- hasName bornOnDate foundingYear tion 12 concludes this paper. Some of the additional bornIn ?m ?city material supporting the main findings reported in the paper are included in an Online Supplement. Fig. 2: Query Graph of Q2
2 Related Work than what is considered in typical graph databases by orders of magnitude. Second, the cardinality of ver- Three approaches have been proposed to store and query tex and edge labels in an RDF graph is much larger RDF data: one giant triples table, clustered property than that in traditional graph databases. For example, tables, and vertically partitioned tables. a typical dataset (i.e., the AIDS dataset) used in the existing graph database work [25,32] has 10,000 data graphs, each with an average number of 20 vertices One giant triples table. The systems in this category and 25 edges. The total number of distinct vertex la- store RDF triples in a single three-column table where bels is 62. The total size of the dataset is about 5M columns correspond to subject, property, and object bytes. However, the Yago RDF graph has about 500M (as in Figure 1a) enabling them to manipulate all RDF vertices and the total size is about 3.1GB. Therefore, triples in a uniform manner. However, this requires per- I/O cost becomes a key issue in RDF query processing. forming a large number of self-joins over this table to However, most existing subgraph query algorithms are answer a SPARQL query. Some efforts have been made memory-based. Third, SPARQL queries combine sev- to address this issue, such as, RDF-3x [19,20] and Hex- + eral attribute-like properties of the same entity; thus, astore [30], which build several clustered B -trees for they tend to contain stars as subqueries [19]. A star all permutations of s, p and o columns. query refers to the query graph in the shape of a star, formed by one central vertex and its neighbors. Property tables. There are two kinds of property tables. Contributions of this paper are the following: The first one, called a clustered property table, groups together the properties that tend to occur in the same 1. We adopt the graph model as the physical storage subjects. Each property cluster is mapped to a property scheme for RDF data. Specifically, we store RDF table. The second type is a property-class table, which data in disk-based adjacency lists. clusters the subjects with the same type of property 2. We transform an RDF graph into a data signature into one property table. graph by encoding each entity and class vertex. An index (VS∗-tree) is developed over the data signa- Vertically partitioned tables. For each property, this ap- ture graph with light maintenance overhead. proach builds a single two-column (subject, object) ta- 3. We develop a filtering rule for subgraph query over ble ordered by subject [1]. The advantage of the order- the data signature graph, which can be seamlessly ing is to perform fast merge-join during query process- embedded into our query algorithm that answers ing. However, this approach does not scale well as the SPARQL queries efficiently. number of properties increases. 4. We introduce an auxiliary structure (called T-index), Existing RDF storage systems, such as Jena [31], which is a structured organization of materialized Yars2 [14] and Sesame 2.0 [6], do not work well in large views, to speed up aggregate SPARQL queries. RDF datasets. SW-store [1], RDF-3x [19], x-RDF-3x 5. We demonstrate experimentally that the performance [22] and Hexastore [30] are designed to address scalabil- of our approach is superior to existing systems. ity, however, they only support exact SPARQL queries, The rest of this paper is organized as follows. We since they replace all literals (in RDF triples) by ids discuss the related work and preliminaries in Sections using a mapping dictionary. 2 and 3, respectively. We give an overview of our solu- Furthermore, most of existing methods do not ef- tion in Section 4. We discuss the storage and encoding ficiently handle online updates of the underlying RDF method in Section 5. We then present the VS∗-tree in- repositories. For example, in clustered property table- dex in Section 6 and an algorithm for SPARQL query based methods (such as Jena [31]), if there are up- processing in Section 7. In order to support aggregate dates to the properties in RDF triples, it is necessary queries efficiently, we develop T-index in Section 8 and to re-cluster and re-build the property tables. In SW- aggregate query processing algorithm in Section 9. We store [1], it is potentially expensive to insert data, since 4 each update requires writing to many columns. In order uals in each group. The answer to this query, R(Q3), to address this issue, it uses “overflow table + batch is given in Figure 3a (we show how to compute this write”, meaning that online updates are recorded to answer in Section 9). overflow tables that SW-store periodically scans to ma- Now consider another query (say Q4) that groups terialize the updates. Obviously, this kind of mainte- all individuals by their titles and gender and reports nance method cannot work well for applications such the number of individuals in each group. The answer to as online social networks that require real time access. this query is given in Figure 3b. Although the group- More recent x-RDF-3x [22] proposes an efficient on- by dimensions in Q4 is a subset of those in Q3, it is line maintenance algorithm, but does not support wild- not possible to get the aggregate result set R(Q4) by card or aggregate SPARQL queries. There exist some scanning R(Q3). The main reason is the nature of RDF works that discuss the possibility of storing RDF data data and the fact that RDF data tend not be struc- as a graph (e.g., [5,30]), but these approaches do not tured, and there may be subjects of the same type that address scalability. Some are based on main memory do not have the same properties. Therefore, some sub- implementations [26], while others utilize graph parti- jects that exist in a “smaller” materialized view may tioning to reduce self-joins of triple tables [33]. While not occur in a “larger” view. graph partitioning is a reasonable technique to par- allelize execution, updates to the graph may require title gender foundingYear COUNT re-partitioning unless incremental partitioning methods President Male 1718 1 are developed (which are not in these works). Actress Female 1976 1 Few SPARQL query engines consider aggregate queries, (a) Answer to Query Q3 and to the best of our knowledge only two proposals exist in literature [16,24]. Given an aggregate SPARQL title gender COUNT President Male 2 query Q, a straightforward method [16] is to transform Actress Female 1 Q into a SPARQL query Q0 without aggregation predi- (b) Answer to Query Q4 cates, find the solution to Q0 by existing query engines, then partition the solution set into one or more groups Fig. 3: Difficulty of Using Materialized Views based on rows that share the specified values, and fi- nally, compute the aggregate values for each group. Al- though it is easy for existing RDF engines to implement aggregate functions this way, the approach is problem- atic, since it misses opportunities for query optimiza- 3 Preliminaries tion. Furthermore, it has been pointed out [24] that this method may produce incorrect answers. An RDF data set is a collection of (subject, property, Seid and Mehrotra [24] study the semantics of group- object) triples hs, p, oi, where subject is an entity or by and aggregation in RDF graph and how to extend a class, and property denotes one attribute associated SPARQL to express grouping and aggregation queries. with one entity or a class, and object is an entity, a class, They do not address the physical implementation or or a literal value. According to the RDF standard, an query optimization techniques. entity or a class is denoted by a URI (Uniform Resource Finally, the RDF data tend not to be very struc- Identifier). In Figure 1, “http://en.wikipedia.org/wiki/ tured. For example, each subject of the same type do United States” is an entity, “http://en.wikipedia.org/ not need to have the same properties. This facilitates wiki/Country” is a class, and “United States” is a lit- “pay-as-you-go” data integration, but prohibits the ap- eral value. In this work, we do not distinguish between plication of classical relational approaches to speed up an “entity” and a “class” since we have the same opera- aggregate query processing. For example, materialized tions over them. RDF data can be modeled as an RDF views [13], which are commonly used to optimize query graph, which is formally defined as follows (frequently execution, may not be used easily. In relational sys- used symbols are shown in Table 1): tems, if there is a materialized view V over dimensions 1 Definition 1 A RDF graph is a four-tuple G = hV,L , (A, B, C), an aggregate query over dimensions (A, B) V E,LEi, where can be answered by only scanning view V1 rather than scanning the original table. However, this is not always 1. V = Vc ∪ Ve ∪ Vl is a collection of vertices that possible in RDF. For example, consider Q3 that groups correspond to all subjects and objects in RDF data, all individuals by their titles, gender, and founding year where Vc, Ve, and Vl are collections of class vertices, of their birth places and reports the number of individ- entity vertices, and literal vertices, respectively. 5
Table 1: Frequently-used Notations
Notation Description Notation Description G A RDF graph (Definition 1) Q A SPARQL query graph (Definition 2) v A vertex v in a SPARQL query graph (Definition 3) u a vertex in RDF graph G (Definition 3) eSig(e) an edge Signature (Definition 5) vSig(u) a vertex signature (Definition 6) RS The answer set of the SPARQL query matches CL The candidate set of the SPARQL query matches G∗ A data signature graph (Definition 7) Q∗ A query signature graph A(u) A transaction of vertex u (Definition 16) O A node in T-index (Definition 16) O.L The corresponding vertex list of node O MS(O) The corresponding aggregate set of node O
2. LV is a collection of vertex labels. The label of a 1. If vi is a literal vertex, vi and ui have the same vertex u ∈ Vl is its literal value, and the label of a literal value; vertex u ∈ Vc ∪ Ve is its corresponding URI. 2. If vi is an entity or class vertex, vi and ui have the −−−→ 3. E = {u1, u2} is a collection of directed edges that same URI; connect the corresponding subjects and objects. 3. If vi is a parameter vertex, ui should satisfy the fil- 4. LE is a collection of edge labels. Given an edge e ∈ ter constraint over parameter vertex vi if any; oth- E, its edge label is its corresponding property. erwise, there is no constraint over ui; −−−→ 4. If there is an edge from vi to vj in Q, there is also An edge u1, u2 is an attribute property edge if u2 ∈ Vl; otherwise, it is a link edge. ut an edge from ui to uj in G. If the edge label in Q is p (i.e., property), the edge from ui to uj in G has Figure 1b shows an example of an RDF graph. The the same label. If the edge label in Q is a parameter, vertices that are denoted by boxes are entity or class the edge label should satisfy the corresponding filter vertices, and the others are literal vertices. A SPARQL constraint; otherwise, there is no constraint over the query Q is also a collection of triples. Some triples in Q edge label from ui to uj in G. ut have parameters. In Q2 (in Section 1), “?m” and “?bd” are parameters, and “?bd” has a wildcard filter: FIL- Given Q2’s query graph in Figure 2, vertices (005,006, TER(regx(str(?bd),“1976”)). Figure 2 shows the query 020,023,024) in RDF graph G of Figure 1b form a match graph that corresponds to Q2. of Q2. Answering a SPARQL query is equivalent to find- ing all matches of its corresponding query graph in RDF Definition 2 A query graph is a five-tuple Q = hV Q, Q Q Q graph. LV ,E ,LE ,FLi, where Q Q Q Q Q Definition 4 An aggregate SPARQL query Q consists 1. V = Vc ∪ Ve ∪ Vl ∪ Vp is a collection of ver- tices that correspond to all subjects and objects in a of three components: Q SPARQL query, where Vp is a collection of parame- 1. Query pattern PQ is a set of triple statements that Q Q Q ter vertices, and Vc and Ve and Vl are collections form one query graph. of class vertices, entity vertices, and literal vertices 2. Group-by dimensions and measure dimensions are in the query graph Q, respectively. pre-defined object variables in query pattern PQ. Q 2. LV is a collection of vertex labels in Q. The label of 3. (Optional) HAVING condition specifies the condi- Q Q tion(s) that each result group must satisfy. ut a vertex v ∈ Vp is φ; that of a vertex v ∈ Vl is its literal value; and that of a vertex v ∈ V Q ∪ V Q is c e Figure 4 demonstrates these three components. This its corresponding URI. example shows the case where all group-by dimensions 3. EQ is a collection of edges that correspond to prop- correspond to attribute property edges (e.g., “?g”,“?t” erties in a SPARQL query. LQ is the edge labels in E and “?y” in Figure 4). This is not necessary, and in EQ. An edge label can be a property or an edge Section 9.3 we discuss more general cases. parameter. 4. FL are constraint filters, such as a wildcard con- straint. ut Measure SELECT ?g ? t ?y COUNT(?m) WHERE dimension Note that, in this paper, we do not consider SPARQL {?m
4 Overview of gStore According to this framework, two issues need to be addressed. First, the encoding technique should guar- gStore is a graph-based triple store system that can an- antee that RS ⊆ CL. Second, an efficient subgraph swer different kinds of SPARQL queries – exact queries, matching algorithm is required to find matches of Q∗ queries with wildcards and aggregate queries – over dy- over G∗. To address the first issue, we develop an encod- namic RDF data repositories. An RDF dataset is rep- ing technique (Section 5) that maps each vertex in G∗ resented as an RDF graph G and stored as an adja- to a signature. For the second issue, we design a novel cency list table (Figure 7). Then, each entity and class index structure called VS∗-tree (Section 6). VS∗-tree vertex is encoded into a bitstring (called vertex signa- is a summary graph of G∗ used to efficiently process ture). The encoding technique is discussed in Section 5. queries using a pruning strategy to reduce the search According to RDF graph’s structure, these vertex sig- space for finding matches of Q∗ over G∗ (Section 7). natures are linked to form a data signature graph G∗, VS∗-tree is also used in answering aggregate SPARQL in which each vertex corresponds to a class or an entity queries (Section 9). We first decompose an aggregate ∗ vertex in the RDF graph (Figure 5). Specifically, G is query Q into star aggregate queries Si (i = 1, ..., n), induced by all entity and class vertices in G together where each star aggregate query is formed by one ver- with the edges whose endpoints are either entity or class tex (called center) and its adjacent properties (i.e., ad- ∗ vertices. Figure 5b shows the data signature graph G jacent edges). For example, query Q3 is decomposed that corresponds to RDF graph G in Figure 1b. An in- into two star aggregate queries S1 and S2 whose graph coming SPARQL query is also represented as a query patterns are shown in Figure 6. We make use of materi- graph Q that is similarly encoded into a query signature alized views to efficiently process star aggregate queries ∗ graph Q . The encoding of query Q2 (which we will use without performing joins. For this purpose, we intro- ∗ as a running example) into a query signature graph Q2 duce T-index (Section 8), which is a trie where each is shown in Figure 5a. node O has a materialized set of tuples MS(O). A star aggregate query can be answered by grouping materi- alized sets associated with nodes in T-index. Once the v v 1 2 results R(Si) of star aggregate queries Si (i = 1, ..., n) 10000 ∗ 0010 1000 1000 0000 are obtained, we employ VS -tree to join R(Si) and find all relevant nodes for each star center. Then, based (a) Query Signature Graph Q∗ 2 on these relevant nodes, we can find the final result of aggregate queries. 001 003 009 10000 0000 0001 0001 1000 0010 0000 007 Q 00001 002 3 0010 1000 ?m bornIn foundingYear 0011 1000 v1 v2 10000 gender title 005 00100 004 008 01000 ?m foundingYear 0010 1000 1000 0001 0100 0000 v1 v2 gender title 10000 006 01000 S1 S2 1000 0100 foundingYear L (b) Data Signature Graph G∗ gender title L 1790 {012} Male President {001,007} 1718 {006} Fig. 5: Signature Graphs Female Actress {005} 1810 {008} 1776 {004} R(S1) R(S2)
∗ ∗ gender title foundingYearon Matches Finding matches of Q over G is known to be NP- Male President 1718 {005,006} hard since it is analogous to subgraph isomorphism. Female Actress 1810 {007,008} R(Q ) Therefore, we use a filter-and-evaluate strategy to re- 3 duce the search space over which we do matching. We Fig. 6: Aggregate Queries first use a false-positive pruning strategy to find a set of candidate subgraphs (denoted as CL), and then we validate these using the adjacency list to find answers gStore considers RDF data management in a dy- (denoted as RS). Reducing the search space has been namic environment, i.e., as the underlying RDF data considered in other works as well (e.g.,[25,32]). get updated, VS∗-tree and T-index are adjusted accord- 7 ingly. Therefore, we also address the index maintenance Specifically, in our implementation, we employ m dif- ∗ issues of VS -tree and T-index (Section 10). ferent string hash functions Hi (i = 1, ..., m), such as BKDR and AP hash functions [7]. For each hash func- tion Hi, we set the (Hi(eLabel) MOD M)-th bit in 5 Storage Scheme and Encoding Technique eSig(e).e to be ‘1’, where Hi(eLabel) denotes the hash function value. We develop a graph-based storage scheme for RDF data. Specifically, we store an RDF graph G using a disk- (hasName, “Abraham Lincoln”) based adjacency list. Each (class or entity) vertex u is 0010 0010 0000 1000 0010 0110 1001 represented by an adjacency list [uID, uLabel, adjList], (bornOnDate, “1809-02-12”) where uID is the vertex ID, uLabel is the corresponding 0010 0000 0010 0100 0010 0010 0100 URI, and adjList is the list of its outgoing edges and the (diedOnDate, “1865-04-15”) 0000 0010 1000 1000 1000 0010 1000 corresponding neighbor vertices. Formally, adjList(u) = (diedIn, y:Washington DC) Vertex 001 {(ei.eLabel, ei.nLabel)}, where ei is an edge adjacent 0010 1000 0000 0001 0100 0100 0010 OR 1010 1010 1010 1101 1110 0110 1111 vsig(e).e vsig(e).n to u, eLabel is ei’s edge label that corresponds to some (bornIn, y:Hodgenville KY) property, and nLabel is the vertex label of u’s neighbor 0010 0010 0000 0000 0100 0100 0011 (gender, “Male”) connected via e . When clear, we omit “e .” prefix from i i 0010 0010 0000 0000 0100 0100 0011 the specification of adjList. Figure 7 shows part of the (title, “President”) adjacency list table for the RDF graph in Figure 1b. 1010 0010 0000 0000 0100 0100 0011 eSig(u).e eSig(u).n
uID uLabel adjList 001 y:Abraham Lincoln (hasName, “Abraham Lincoln”), Fig. 8: The Encoding Technique (bornOnDate, “1809-02-12”), (diedOnDate, “1865-04-15”), (diedIn, y:Washington DC), (gender, “Male”), (title, “President”), (bornIn, y:Hodgenville KY) 002 y:Washington DC (hasName, “Washington D.C.”), In order to encode neighbor vertex label nLabel into (foundingYear, “1790”) ...... eSig(e).n, we adopt the following technique. We first Prefix: http://en.wikipedia.org/wiki/ represent nLabel by a set of n-grams [10], where an Fig. 7: Disk-based Adjacency List Table n-gram is a subsequence of n characters from a given string. For example, “1809-02-12” can be represented by a set of 3-grams: {(180),(809),(09-),...,(-12)}. Then, According to Definition 3, if vertex v (in query Q) we use a string hash function H for each n-gram g to can match vertex u (in RDF graph G), each neigh- obtain H(g). Finally, we set the (H(g) MOD N)-th bit bor vertex and each adjacent edge of v should match in eSig(e).n to be ‘1’. We discuss the settings of param- to some neighbor vertex and some adjacent edge of u. eters M, m, N and n in Section 11.2. Figure 8 shows a Thus, given a vertex u in G, we encode each of its adja- running example of edge signatures. For example, given cent edge labels and the corresponding neighbor vertex edge (hasName,“Abraham Lincoln”), we first map the labels into bitstrings. We encode query Q with the same edge label “hasName” into a bitstring of length 12, and encoding method. Consequently, the match between Q then map the vertex label “Abraham Lincoln” into a and G can be verified by simply checking the match bitstring of length 16. between corresponding encoded bitstrings. Definition 6 Given a class or entity vertex u in the Each row in the table corresponds to an entity ver- RDF graph, the vertex signature vSig(u) is formed by tex or a class vertex. Given a vertex, we encode each performing bitwise OR operations over all its adjacent of its adjacent edges e(eLabel, nLabel) into a bitstring. edge signatures. Formally, vSig(u) is defined as follows: This bitstring is called edge signature (i.e., eSig(e)). vSig(u) = eSig(e )| ... |eSig(e ) Definition 5 The edge signature of an edge adjacent 1 n to vertex u, e(eLabel, nLabel), is a bitstring, denoted as where eSig(ei) is the edge signature for edge ei adjacent eSig(e), which has two parts: eSig(e).e, eSig(e).n. The to u and “|” is the bitwise OR operation. ut first part eSig(e).e (M bits) denotes the edge label (i.e., eLabel) and the second part eSig(e).n (N bits) denotes Considering vertex 001 in Figures 1b and 7, there the neighbor vertex label (i.e., nLabel). ut are seven adjacent edges. We can encode each adjacent edge by its edge signature, as shown in Figure 8, which eSig(e).e and eSig(e).n are generated as follows. also shows the signature of vertex 001. Let |eSig(e).e| = M. Using an appropriate hash func- In computing the vertex signature we use textual tion, we set m out of M bits in eSig(e).e to be ‘1’. value of the neighbor node, not its vertex signature. For 8 example, in computing the signature of y:Abraham Lincoln Note that, each vertex u (and v) in data (and query) that has in its adjacency list (diedIn, y:Washington DC), signature graph G∗ (and Q∗) has one vertex signature we simply encode string “y:Washington DC” and use vSig(u) (and vSig(v)). For simplicity, we use u (and v) this encoding rather than the vertex signature of node to denote vSig(u) in G∗ (and vSig(v) in Q∗) when the y:Washington DC. This avoids recursion in the compu- context is clear. tation of vertex signatures. Given an RDF graph G and a query graph Q, their corresponding signature graphs are G∗ and Q∗, respec- Definition 7 Given an RDF graph G, its correspond- tively. The matches of Q over G are denoted as RS, ing data signature graph G∗ is induced by all entity and the matches of Q∗ over G∗ are denoted as CL. and class vertices in G together with link edges (the edges whose endpoints are either entity or class ver- Theorem 1 RS ⊆ CL holds. tices). Each vertex u in G∗ has its corresponding ver- tex signature vSig(u) (Definition 6) as its label. Given Proof See Part B of Online Supplements. −−−→ ∗ an edge u1, u2 in G , its edge label is also a signature, −−→ denoted as Sig(u1u2), to denote the property between u and u . ut 1 2 6 VS∗-tree We adopt the same hash function in Definition 5 to In this section, we describe VS∗-tree, which is an index define Sig(u−−−→, u ). Specifically, we set m out of M bits 1 2 structure over G∗ that can be used to answer SPARQL in Sig(u−−−→, u ) to be ‘1’ by some string hash function. 1 2 queries as described in the next section. As discussed Figure 5 shows an example of data signature graph G∗. earlier, the key problem to be addressed is how to find We also encode the query graph Q using the same matches of Q∗ (query signature graph) over G∗ (data method. Specifically, given an entity or class vertex v in signature graph) efficiently using Definition 8. A straight- Q, we encode each adjacent edge pair e(eLabel, nLabel) forward method can work as follows: first, for each ver- into a bitstring eSig(e) according to Definition 5. Note tex v ∈ Q∗, we find a list R = {u , u , ..., u }, where that, if the adjacent neighbor vertex of v is a param- i i i1 i2 in v &u = v (& is a bitwise AND operation). Then, eter vertex, we set eSig(e).n to be a signature with i ij i we perform a multiway join over these lists R to find all zeros; if the adjacent neighbor vertex of v is a pa- i matches of Q∗ over G∗ (finding CL). The first step rameter vertex and there is a wildcard constraint (e.g., (finding R ) is a classical inclusion query. regex(str(?bd),“1976”)), we only consider the substring i An inclusion query is a subset query that, given a without “wildcard” in the label. For example, in Figure set of objects with set-valued attributes, finds all ob- 2, we can only encode substring “1976” for vertex ?bd. jects containing certain attribute values. In our case, The vertex signature vSig(v) can be obtained by per- presence of elements in sets is captured in signatures. forming bitwise OR operations over all adjacent edge Thus, we have a set of signatures {s } (representing a signatures. i set of objects with set-valued attributes) and a query Given a query graph Q, we can obtain a query sig- signature q [27]. Then, an inclusion query finds all sig- nature graph Q∗ induced by all entity and class vertices natures {s } ⊆ {s }, where q&s = q. in Q together with all edges whose endpoints are also j i j entity or class vertices. Each vertex v in Q∗ is a vertex In order to reduce the search space for the inclusion + −−−→ ∗ query, S-tree [8], a height-balanced tree similar to B - signature vSig(v), and each edge v1, v2 in Q is associ- −−−→ tree, has been proposed to organize all signatures {sj}. ated with an edge signature Sig(v1, v2). Figure 5 shows ∗ Each intermediate node is formed by ORing all child Q that corresponds to query Q5. signatures in S-tree. Therefore, S-tree can be employed Definition 8 Consider a data signature graph G∗ and to support the first step efficiently, i.e., finding Ri. An ∗ example of S-tree is given in Figure 9. a query signature graph Q that has n vertices {v1, . . . , vn}. ∗ A set of n distinct vertices {u1, . . . , un} in G is said to However, S-tree cannot support the second step (i.e., be a match of Q∗ if and only if the following conditions a multiway join), which is NP-hard. Although many hold: subgraph matching methods have been proposed (e.g., [25,32]), they are not scalable to very large graphs. 1. vSig(vi)&vSig(ui) = vSig(vi), i = 1, ..., n, where Therefore, we develop VS∗-tree (vertex signature tree) ‘&’ is the bitwise AND operator. to index a large data signature graph G∗ that also sup- ∗ 2. If there is an edge from vi to vj in Q , there is also an ports the second step. VS∗-tree is a multi-resolution ∗ −−−→ −−−→ edge from ui to uj in G ; and Sig(vi, vj)&Sig(ui, uj) summary graph based on S-tree that can be used to −−−→ = Sig(vi, vj). ut reduce the search space of subgraph query processing. 9
d1 1111 1101 G1 1 edge labels are the following: bornIn → 10000, diedIn → 00001, hasCapital → 00100, and locatedIn → 01000. 2 d2 d2 k G 1 0011 1001 1101 1101 2 Finally, given two non-leaf nodes at the same level di k k and dj (where di denotes a node at the k-th level), a 3 G 3 3 3 3 −−−→ d1 0010 1001 0011 1000 d2 d3 0101 1000 1000 0101 d4 k k super-edge di , dj is introduced if and only if there is an 001 002 003 004 edge from any of di’s children to any of dj’s children. 0000 0001 0011 1000 0001 1000 1000 0001 −−−→ k k The edge label of di , dj is obtained by performing bit- 005 009 007 008 006 wise OR over all the edge labels from di’s children to 0010 1000 0010 0000 0010 1000 0001 0000 1000 0100 dj’s children. In Figure 10, since there is a super-edge from 008 to 004, we introduce a super-edge from d3 to Fig. 9: S-tree −−−→ 3 3 k k d4 (i.e., di , dj ) with signature 01000. We also introduce k a self-edge for a non-leaf node di , if and only if there Definition 9VS ∗-tree is a height balanced tree with is an edge from one of its children to another one of its 3 the following properties: children. Thus, a self-loop super-edge over d4 is also in- troduced since there is a (super-)edge from 006 to 004, 3 1. Each path from the root to any leaf node has the both of which are children of d4. same length h, which is the height of the VS∗-tree. 2. Each leaf node corresponds to a vertex in G∗ and 01000 has a pointer to it. 3 01000 3 3. Each non-leaf node has pointers to its children. d3 d4 4. The level numbers of VS∗-tree increase downward with the root at level 1. 003 004 I I 0001 1000 1000 0001 5. Each node di at level I is assigned a signature di .Sig. I I If node di is a leaf node, di .Sig is the correspond- 008 006 ∗ I ing vertex signature in G . Otherwise, di .Sig is ob- 1000 0000 1000 0100 I 01000 tained by “OR”ing signatures of di ’s children (i.e., dI .Sig = OR(dI+1.Sig, . . . , dI+1.Sig)) where dI+1 i 1 n j Fig. 10: Building Super-edges I are di ’s children and j = 1, . . . , n. 6. Each node other than the root has at least b chil- dren. B+1 7. Each node has at most B children, 2 ≥ b. 11101 I I 8. Given two nodes di and dj , there is a super-edge G1 1 −−−→ d1 1111 1101 I I di , dj if and only if there is an edge (can be a super- I 00001 01000 edge) from at least one of di ’s children to one of G2 10000 −−−→ 2 2 I I I d1 0011 1001 1101 1101 d2 dj ’s children. The edge label Sig(di , dj ) is created 00100 10000 I 01000 by “OR”ing signatures of all edge labels from di ’s 10000 I 3 children to d ’s children. G 3 3 3 3 j d1 0010 1001 0011 1000 d2 d3 0101 1000 1000 0101 d4 ∗ 9. The I-th level of the VS -tree is a summary graph, 00001 10000 01000 00100 10000 I 00100 denoted as G , which is formed by all nodes at the 001 002 003 004 00001 01000 I-th level together with all edges between them in 0000 0001 0011 1000 0001 1000 1000 0001 ∗ the VS -tree. ut 005 009 007 008 01000 006 10000 0010 1000 0010 0000 0010 1000 0001 0000 1000 0100 ∗ According to Definition 9, the leaf nodes of VS -tree 10000 correspond to vertices in G∗. An S-tree is built over these leaf nodes. Furthermore, each pair of leaf nodes Fig. 11: VS∗-tree −−−→ (u1, u2) is connected by a directed “super-edge” ui, uj ∗ if there is a directed edge from u1 to u2 in G , and an −−−→ ∗ ∗ edge signature Sig(u1, u2) is assigned to it according to Figure 11 shows the VS -tree over G of Figure 5. I Definition 7. Figure 10 illustrates the process; for ex- As defined in Definition 9, we use di .Sig to denote the I ample, a super-edge is introduced from (008) to (004) signature associated with node di . For simplicity, we I I (shown as a dashed line) since such an edge exists in also use di to denote di .Sig when the context is clear. G∗. In this figure we assume that the hash function for 10
Definition 10 Consider a query signature graph Q∗ instance. Since there is an edge from 001 to 003, we with n vertices vi (i = 1, ..., n) and a summary graph introduce edges from 001[1] to 003[1] and from 001[2] to I ∗ I G at the I-th level of VS -tree. A set of nodes {di } 003[1], as shown in Figure 12b. (i = 1, ..., n) at GI is called a summary match of Q∗ Given a vertex v in query graph, if v can match over GI , if and only if the following conditions hold: vertex u in RDF graph G, v’s neighbor vertices may match neighbors of different instances of u. Therefore, 1. vSig(v )&dI .Sig = vSig(v ), i = 1, ..., n; i i i we need to revise encoding strategy for query graph Q. 2. Given edge v−−−→, v ∈ Q∗, there exists a super-edge −−→ 1 2 −−−→ Assume that v in the query graph has m neighbors. I I I −−−→ I I −−−→ d d in G and Sig(v1, v2)&Sig(d , d ) = Sig(v1, v2). 1 2 1 2 We introduce m instances of v, i.e., v[j], j = 1, ..., m, ut and each instance has only one neighbor. According to Definition 6, we can compute the vertex signature for Note that, a summary match is not an injective func- these instances, i.e., vSig(v ). For example, v in Q∗ tion from {v } to {dI }, namely, dI can be identical to dI . [j] 1 2 i i i j (Figure 5) has three neighbors, thus, we introduce three For example, given a query signature graph Q∗ (in Fig- instances v , v , and v that corresponds to three ure 5) and a summary graph G3 of VS∗-tree (in Figure 1[1] 1[2] 1[3] 3 3 neighbors, respectively, Figure 12b. 11), we can find one summary match {(d1, d4)}. Sum- mary matches can be used to reduce the search space ∗ for subgraph search over G as we discuss next. 7 SPARQL Query Processing As we discuss in Section 7, the query algorithm uses level-by-level matching of vertex signatures of the query Given a SPARQL query Q, we first encode it into a ∗ ∗ graph Q to the nodes in VS -tree. A problem that may query signature graph Q∗, according to the method in affect the performance of the query algorithm is that Section 5. Then, we find matches of Q∗ over G∗. Finally, the vertex encoding strategy may lead to some vertex we verify if each match of Q∗ over G∗ is also a match signatures having too many “1”s. Given a vertex u, we of Q over G following Definition 3. Therefore, the key perform bitwise OR operations over all its adjacent edge issue is how to efficiently find matches of Q∗ over G∗ signatures to obtain vertex signature vSig(u) (see Def- using the VS∗-tree. inition 6). Therefore, for a vertex with a high degree, We employ a top-down search strategy over the VS∗- vSig() may be all (or mostly) 1’s, meaning that these tree to find matches. According to Theorem 2, the search vertices can match any query vertex signature. This will space at level I + 1 of the VS∗-tree is bounded by the ∗ affect the pruning power of VS -tree. summary matches at level I (level numbers increase In order to address this issue, we perform the fol- downward with the root at level 1). This allows us to lowing optimization. Given a vertex u, if the number of reduce the total search space. 1’s in vSig(u) is larger than some threshold δ, we par- ∗ tition all of u’s neighbors into n groups g1, ..., gn and Theorem 2 Given a query signature graph Q with n ∗ each group gi corresponds to one instance of vertex u, vertices {v1, . . . , vn}, a data signature graph G and the ∗ ∗ denoted as u[i]. According to Definition 6, we can com- VS -tree built over G : pute vertex signature for these instances, i.e., vSig(u ). [i] 1. Assume that n vertices {u , . . . , u } forms a match We guarantee that the number of 1’s in vSig(u )(i = 1 n [i] (Definition 8) of Q∗ over G∗. Given a summary 1, ..., n) is no larger than δ. Given a vertex u, if the graph GI in VS∗-tree, let u ’s ancestor in GI be number of 1’s in vSig(u) is no larger than δ, u has only i node dI . (dI , ..., dI ) must form a summary match one instance u . Then, we use these vertex instance i 1 n [1] (Definition 10) of Q∗ over GI . signatures of u (vSig(u ), i = 1, ..., n) instead of ver- [i] 2. If there exists no summary match of Q∗ over GI , tex signature (vSig(u)) in building VS∗-tree. Note that, there exists no match of Q∗ over G∗. these vertex instances have the same vertex ID of u that corresponds to the same vertex in RDF graph G. Given Proof See Part B of Online Supplements. 0 two instances vSig(u[i]) and vSig(u[j]) at the leaf level of the revised VS∗-tree, we introduce an edge between The basic query processing algorithm (BVS∗-Query) 0 u[i] and u[j] if and only if there is an edge between their is given in Algorithm 1. We illustrate it using a running 0 ∗ corresponding vertices u and u . example Q2 (of Figure 5). Figure 13 shows the pro- For example, vertex 001 has seven neighbors in RDF cess (pruned search space is shown as shaded). First, ∗ 1 ∗ graph G. We can decompose them into two groups g1 we find summary matches of Q2 over G in VS -tree and g2 and encode the two groups as in Figure 12a. Each and insert them into a queue H. In this case, the sum- 1 1 group corresponds to one instance, denoted as 001[1] mary match is {(d1, d1)}, which goes into queue H. and 001[2]. Assume that other vertices have only one We pop one summary match from H and expand it 11
v1[2] v v v 1[1] 0010 1000 1[3] 2[1] 0010 0000 0000 1000 0000 1000
(hasName, “Abraham Lincoln”) v1 v2 0010 0010 0000 1000 0010 0110 1001 10000 0010 1000 1000 0000 (bornOnDate, “1809-02-12”) ∗ 0010 0000 0010 0100 0010 0010 0100 001[1] Q2
(diedOnDate, “1865-04-15”) OR 0010 1010 1010 1101 1110 0110 1111 001[1] 0000 0010 1000 1000 1000 0010 1000 vsig(u).e vsig(u).n 0000 0001 (diedIn, y:Washington DC) 00001 10000 0010 1000 0000 0001 0100 0100 0010 001[2] 003[1] 10000 (bornIn, y:Hodgenville KY) 0000 0001 0001 1000 002[1] 0010 0010 0000 0000 0100 0100 0011 0011 1000 007[1] (gender, “Male”) 001[2] 0010 1000 00100 0010 0010 0000 0000 0100 0100 0011 OR 1010 0010 0000 0000 0100 0100 0011 005[1] 004[1] vsig(u).e vsig(u).n (title, “President”) 0010 1000 1000 0001 10000
1010 0010 0000 0000 0100 0100 0011 10000 006[1] 01000 01000 008[1] eSig(u).e eSig(u).n 1000 0100 0100 0000
(a) Optimized Encoding Strategy (b) Optimized Signature Graph
Fig. 12: Optimizing VS∗-tree to its child states (as given in Definition 11). Given Definition 11 Given a query signature graph Q∗ with 1 1 I I ∗ the summary match (d1, d1), its child states are formed n vertices {v1, . . . , vn}, and n nodes {d1, ..., dn} in VS - 1 1 2 2 2 2 ∗ I0 I0 by d1.children × d1.children = {d1, d2} × {d1, d2} = tree that form a summary match of Q , n nodes {d1 , ..., dn } 2 2 2 2 2 2 2 2 I I I0 {(d1, d1), (d1, d2), (d2, d1), (d2, d2)}. The set of child states form a child state of {d1, ..., dn}, if and only if di is a ∗ I I0 I0 that are summary matches of Q are called valid child child node of di , i = 1, .., n. Furthermore, if {d1 , ..., dn } ∗ I0 I0 states, and they are inserted into queue H. In this ex- is also a summary match of Q , {d1 , ..., dn } is called a 2 2 ∗ I I ample, only (d1, d2) is a summary match of Q , thus, valid child state of {d1, ..., dn}. ut we insert it into H. We continue to iteratively pop one summary match from H and repeat this process until ∗ the leaf nodes (i.e., vertices in G ) are reached. Finally, Queue H ∗ ∗ 1 1 1 1 we find matches of Q over leaf entries of VS -tree, (d1, d2) Step 1: (d1, d1) namely, the matches of Q∗ over G∗. 2 2 2 2 2 2 2 2 2 2 (d1, d2) Step 2: (d1, d1) (d1, d2) (d2, d1) (d2, d2)
∗ 3 3 3 3 3 3 3 3 3 3 Algorithm 1 Basic Query Algorithm Over VS -tree (d1, d2) Step 3: (d1, d3) (d1, d4) (d2, d3) (d2, d4) (BVS∗-Query) CL Input: a query signature graph Q∗ and a data signature (005, 006) Step 4: (001,004) (001,006) (005,004) (005,006) graph G∗ and a VS∗-tree Output: CL: All matches of Q∗ over G∗ Fig. 13: BVS∗-Query Algorithm Process 1: Set CL = φ 2: Find summary matches of Q∗ over G1, and insert into queue H 3: while (|H| > 0) do Algorithm 1 always begins by finding summary matches 4: Pop one summary match from H, denoted as SM from the root of the VS∗-tree, which can lead to a large 5: for each child state S of SM do 6: if S reaches leaf entries and S is a match of Q∗ number of intermediate summary matches. In order to then speed up query processing, we do not materialize all 7: Insert S into CL summary matches in non-leaf levels. Instead, we apply 8: if S does not reach the leaf nodes and S is a sum- ∗ a semijoin [4] like pruning strategy, i.e., pruning some mary match of Q then ∗ 9: Insert it into queue H nodes (in the VS -tree) that are not possible in any 10: return CL. summary match, and incorporate it into VS∗-query al- gorithm. ∗ Given a vertex vi in query graph Q and a summary graph GI in VS∗-tree, we try to prune nodes dI (∈ GI ) 12
∗ ∗ ∗ that cannot match vi in any summary match of Q over Algorithm 3 Find Matches of Q over G I ∗ G . At the leaf-level of the index, this shrinks the can- Input: a query signature graph Q with n vertices vi, i = ∗ didate list C(vi). Let us recall query Q2 in Figure 5. 1, ..., n Input: G∗ Vertices v1 and v2 have three {002, 005, 007} and two Input: C(vi) that are candidate vertices that may match vi. candidates {004, 006}, respectively, according to their Output: M(Q∗): all matches of Q∗ over G∗ vertex signatures. Therefore, the whole join space is 1: Set Q0 = φ 3 3 × 2 = 6. Let us consider summary graph G . If we 2: Select some vertex vi, where C(vi) is minimal among all 3 vertices in Q∗. use S-tree for pruning, we get two candidates (d1 and 0 0 0 3 3 3: Q = Q ∪ vi and M(Q ) = C(vi). d2) for v1 and one candidate (d4) for v2. There is one 4: while Q0! = Q∗ do edge from v to v whose label signature is 10000. d3 −−−−→ 1 2 2 5: for each backward edge ei = vi1 , vi2 that is adjacent 0 has one edge to candidates of v2 with an edge signa- to Q do 0 0 6: M(Q ∪ ei)=Backward(ei,M(Q )) ture of 10000. Obviously 00010&100000 6= 10000, and, 0 0 3 7: Q = Q ∪ ei therefore, d2 cannot be in the candidate list of v1. If a −−−−→ 8: for each forward edge ei = vi1 , vi2 that is adjacent to node cannot match v1, all descendent nodes of d can be Q0 do 3 0 0 pruned safely. It means that node d2 and its descendent 9: M(Q ∪ ei)=Forward(ei,M(Q )) 0 0 10: Q = Q ∪ ei nodes are pruned from candidates of v1 resulting in a 11: Set M(Q∗)=M(Q0) single candidate {005} for v1. Therefore, the join space 12: return M(Q∗) is reduced to 1 × 2 = 2. −−−−→ 0 Backward(ei = vi1 , vi2 ,M(Q )) 1: for each tuple t in M(Q0) do 0 2: If t cannot form a match of Q ∪ ei Algorithm 2 An Optimized Query Algorithm Over 3: Delete t from M(Q0) ∗ ∗ 0 0 VS -tree (VS -Query Algorithm) 4: M(Q ∪ ei) = M(Q ) 0 Input: a query signature graph Q∗ 5: return M(Q ∪ ei). ∗ −−−−→ 0 Input: a data signature graph G Forward(ei = vi1 , vi2 ,M(Q )) ∗ 0 0 Input: a VS -tree. 1: if vi1 ∈ Q ∧ vi2 ∈/ Q then Output: CL: All matches of Q∗ over G∗. 2: for each tuple t in M(Q0) do ∗ 1: for each vertex vi ∈ Q do 3: for each node d in C(vi2 ) do 1 1 ∗ 0 2: C(vi) = d1, where d1 is the root of VS -tree {C(vi) 4: if t on d is a match of Q ∪ ei then ∗ ∗ 0 are candidate vertices in G to match vi in Q } 5: Insert t on d into M(Q ∪ ei) I ∗ 0 0 3: for each level summary graph G of VS -tree I = 2, ..., h 6: if vi2 ∈ Q ∧ vi1 ∈/ Q then do {h is the height of VS∗-tree} 7: for each tuple t in M(Q0) do
4: for each C(vi), i = 1, ..., n do 8: for each node d in C(vi1 ) do 0 0 5: set C (vi) = φ 9: if d on t is a match of Q ∪ ei then I 0 6: for each child node d of each element in C(vi) do 10: Insert d on t into M(Q ∪ ei) 0 7: for each instance vi[j] of vertex vi, j = 1, ..., m 11: return M(Q ∪ ei) do I 8: if Sig(d )&sig(vi[j]) = sig(vi[j]) then 9: push dI into C(v ) i[j] in Algorithm 2. For each vertex vi in query signature 10: Set C(v ) = T C(v ) i j=1,...,m i[j] graph Q∗, we find the candidate list of vertices that can 11: for each C(v ), i = 1, ..., n do i match v , denoted as C(v ). Specifically, based on Def- 12: for each each node d in C(vi) do i i ∗ I −−−→ ∗ 0 −−→0 inition 8, given a vertex v in Q , a node d in (∈ G ) 13: if ∃vj |vi, vj ∈ Q , ∀d |d, d ∈ i I 0 ∗ G , vSig(vj )&vSig(d ) 6= vSig(vj ) then VS -tree cannot match vi, if and only if one of the fol- 14: Remove vi from C(vi) lowing conditions hold: −−−→ ∗ 0 −−→0 15: if ∃vj |vi, vj ∈ Q , ∀d |d, d ∈ −−→ 1. vSig(vi)&vSig(d) 6= vSig(vi); or GI , Sig(v−−−→, v )&Sig(d, d0) 6= Sig(v−−−→, v ) then −−→ i j i j 2. ∃v |v−−−→, v ∈ Q∗, ∀d0|d, d0 ∈ GI , vSig(v )&vSig(d0) 6= 16: Remove vi from C(vi) j i j j ∗ 17: Call Algorithm 3 to find matches of Q over C(v1) × ... × vSig(vj). −−→ −−→ C(vn), denoted as CL. −−−→ ∗ 0 0 I −−−→ 0 3. ∃vj|vi, vj ∈ Q , ∀d |d, d ∈ G , Sig(vi, vj)&Sig(d, d ) 6= 18: for each candidate match in CL do −−−→ 19: Check whether it is match of SPARQL query Q over Sig(vi, vj). RDF graph G. If so, insert it into RS. Furthermore, if a node d (∈ GI ) in VS∗-tree cannot 20: return RS. match vi (i = 1, . . . , n), its descendant nodes cannot match vi. Consequently, the descendant nodes can be pruned safely. The basic BVS-Query algorithm can be improved by reducing the candidates for each query vertex in Definition 12 Given a subgraph Q0 of Q∗, an edge ∗ −−−→ ∗ 0 Q instead of finding summary matches in non-leaf lev- e = v1, v2 in Q is called adjacent to Q if and only if ∗ 0 0 0 els. The improved algorithm, called VS -Query, is given (e∈ / Q ) ∧ (v1 ∈ Q ∨ v2 ∈ Q ). ut 13
−−−→ Definition 13 Given an adjacent edge e = v1, v2 to Thus, we have a transaction A(001) =“hasName, gen- 0 Q , e is called a backward edge if and only if (v1 ∈ der, bornOnDate, title, diedOnDate”. All transactions 0 0 Q ∧ v2 ∈ Q ). Otherwise, e is called a forward edge. ut are collected to form a transaction database DB, as shown in Figure 14a. Each entity vertex u in RDF graph Lines 6–10 of Algorithm 2 exploit the optimization G generates one transaction A(u) in DB. In each trans- we introduced (see end of Section 6) in VS∗-tree where action, properties (dimensions1) are ordered in their we divide vertices whose signatures contain too many frequency descending order in DB, where property fre- “1”s, which reduces their discriminatory power. Specifi- quency is defined as follows. ∗ cally, first, for each query vertex vi in Q , we set C(vi) = 1 I Definition 14 The frequency of a property p in a trans- {d1}. Consider the level I summary graph G . For each ∗ query vertex vi in Q , we consider each instance vi[j] of action database DB is F req(p) = |{A(u)|p ∈ A(u) ∧ I vi. For each child node d of each element in C(vi), we A(u) ∈ DB}|. ut determine whether Sig(dI )& Sig(v ) = Sig(v ). If i[j] i[j] For example, the frequencies of “hasName”, “Found so, we push dI into C(v ). When we finish considering i[j] Year”, “gender”, “bornOnDate”, “title” and “diedOn- all instances of v , we update C(v ) = T C(v ). i i j=1,...,m i[j] Date” are 7, 4, 4, 3, 3 and 3, respectively. Therefore, After finding candidates C(v ) for each query vertex i “hasName” precedes “FoundYear”, which precedes “gen- u , we perform multiway join C(v ) ... C(v ) to i 1 on on n der” in the relevant transactions. This order is impor- find matches of Q∗ (Algorithm 3). Specifically, we ap- tant since the construction of paths follows this order. ply a depth-first search strategy to find matches of Q∗ Frequency ordering leads to fewer nodes being inserted over G∗, denoted as M(Q∗). Initially, we set Q0 = φ, since there is a higher probability that more prefixes will which denotes the structure of Q∗ that has been visited be shared among different transactions, and, therefore, so far. We start a DFS over G∗ beginning with a vertex minimizes the number of materialized views that need v where C(v ) is minimal among all vertices in Q∗. We i i to be maintained. It also facilitates query processing, insert v into query Q0. Now, the matches of Q0, i.e., i as discussed in Section 9. Note that, if multiple dimen- M(Q0), are updated as M(Q0) = C(v ). For each edge i sions have the same frequency, their order is arbitrarily e adjacent to Q0, if e is a backward edge, we employ i i defined (the effect of this is experimentally studied). Backward function in Algorithm 3 to find matches of 0 0 Q ∪ ei, i.e., M(Q ∪ ei). Otherwise, we employ Forward Definition 15 Given a transaction A, the length-n pre- 0 function to find M(Q ∪ ei). Essentially, Forward func- fix of A is the first n dimensions in A. ut tion is a nested loop join process, but Backward func- tion is a selection process. Therefore, we always process Definition 16 A T-index is a trie constructed as fol- backward edges ahead of forward edges. The whole pro- lows: cess is iterated until Q0 = Q∗, and then M(Q∗) is re- 1. There is one root labeled as “root”. turned. 2. Each node O in T-index denotes a dimension. Finally, for each match of Q∗ over G∗, we check 3. Node Oj is a child of node Oi if and only if there whether it is a subgraph match of Q. If so, we insert exists at least one transaction A, where the path it into answer set RS. According to these subgraph reaching node Oi is length-n prefix of A and the matches, it is straightforward to find the matching vari- path reaching node Oj is length-(n + 1) prefix of A. able bindings for the SELECT variables in SPARQL. 4. Each node O in T-index has a vertex list O.L regis- Experiments (see Part D of Online Supplements) demon- tering the IDs of all transactions Ai, where the path strate that VS∗ performs much better than BVS∗. reaching O is a prefix of Ai. ut
Figure 14b shows an example of T-index. When in- 8 T-index serting A(001) =“hasName, gender, bornOnDate, title, diedOnDate” into the T-index, the path “O0-O1-O2- In this section, we introduce T-index, which is a struc- O3-O4-O5” is followed. 001 is registered to Oi.L where tured organization of materialized views. T-index is used i = 1, 2, 3, 4, 5. Furthermore, since “hasName” is a pre- to process aggregate SPARQL queries (Section 9). fix of seven transactions (“001,002,003,004,005,007,009”), For each entity vertex u in a RDF graph G, all dis- the corresponding vertex list to node O1 is O1.L = tinct attribute properties (Definition 1) adjacent to u {001, 002, 003, 004, 005, 007, 009}. Note that the storage are collected to form a transaction. For example, the 1 The literature on aggregation and aggregate queries fre- adjacent attribute properties of entity vertex 001 are quently refer to these attributes as dimensions. We follow the “hasName, gender, bornOnDate, title, diedOnDate”. same convention. 14
MS(O4) hasName gender bornOnDate title L Abraham Male 1865-04-15 President {001} Lincoln Reese With- Female 1976-03-22 Actress {005} erspoon root O0
{001,002,003,004,005,007,009} Vertex Entity vertex Adjacent Attribute Properties hasName foundingYear O O ID 1 9 {006,008}
001 y:Abraham Lincoln hasName, gender, bornOn- {001,005,007,009} Date, title, diedOnDate gender foundingYear O O 002 y:Washington DC hasName, foundingYear 2 8 {002,004} Dimension 003 y:Hodgenville KY hasName List DL {001,005,009} hasName:7 004 y:United States hasName, foundingYear bonOnDate title O O foundingYear:4 005 y:Reese Witherspoon hasName, gender, bornOn- 3 7 {007} Date, title gender:4 006 y:New Orleans LA foundingYear {001,005} title diedOnDate bornOnDate:3 O O 007 y:Franklin Roosevelt hasName, gender, title 4 6 {009} title:3 008 y:Hyde Park NY foundingYear diedOnDate:2 009 y:Marilyn Monroe hasName, gender, bornOn- {001} diedOnDate Date, diedOnDate O5
MS(O7) hasName gender title L Franklin D. Male President {007} Roosevelt (a) Transaction Database DB (b) T-Index
Fig. 14: T-index of the vertex lists can be optimized by a variety of en- Since T-index is a trie augmented with materialized coding techniques. We don’t discuss this tangential is- views MS(O) for each node O, we do not give its con- sue any further. struction algorithm (for completeness, this is provided in Part C of Online Supplements). Two observations are In addition to T-index, there are two associated important regarding T-index: (1) given a non-leaf node data structures: dimension list DL and materialized O in T-index that has n child nodes O ,(i = 1, ..., n), sets MS(O). Dimension list DL records all dimensions i if the path reaching at least one O is a prefix of A, the in the transaction database. When introducing a node i path reaching node O is also a prefix of A; and (b) the into T-index, according to the dimension of the node, structure of T-index does not depend on the order of we register the node to the corresponding dimension inserting transactions into the structure. in DL, similar to building an inverted index. Conse- quently, the dimensions in DL are ordered in their fre- We now discuss the materialization of MS(O) of quency descending order. each node O in T-index. Obviously, for each node O in T-index, we can acceess all entity vertices in O.L and Each node O has an aggregate set MS(O). Let the their attribute properties to build MS(O). However, dimensions along the path from the root to node O be some computation and I/O can be shared for computing (l , l , ..., l ). According to O.L, one can find all trans- 1 2 N MS(O) of different nodes. actions represented by the portion of the path reach- ing this node. MS(O) is a set of tuples that group Given a node O with n child nodes Oi,(i = 1, ..., n), these transactions based on shared values on dimen- if the path reaching at least one Oi is a prefix of one sions (l1, l2, ..., lN ). Each tuple t in MS(O) has two transaction A, the path reaching node O must also be S parts: the dimensions t.D and the vertex list t.L that a prefix of A. Thus, O.L ⊇ i Ni.L. Consequently, its stores the vertex IDs in this aggregate tuple, as shown aggregate set MS(O) can be computed from the aggre- in Figure 14b. Consider node O4 in Figure 14b. We gate sets associated with its child nodes. Therefore, we partition all transactions represented by the path “O0- propose a post-order traversal-based algorithm to ma- O1-O2-O3-O4” into two groups based on group-by di- terialize MS(O) of the T-index in Algorithm 7. Assume mensions that share specified values along (hasName, that the properties along the path reaching node O are gender, bornOnDate, title). These pre-computed sets (p1, ..., pm). Initially, MS(O) = φ. For each child node speed up aggregate SPARQL query processing, as dis- O of O, we compute MS0(O ) = Q MS(O ) i i (p1,p2,...,pn) i cussed in the next section. (Lines 3-4 in Algorithm 7). Then, we compute MS(O) = 15
S 0 i MS (Oi) (Line 7). Furthermore, for each vertex v Section 9.1. We then join R(Si)’s to compute the result S that is in MS(O).L but not in i MS(Oi).L, we need of Q, i.e., R(Q) =oni R(Si) as discussed in Section 9.2. to access the property values of vertex u on dimensions p1, ..., pn in the RDF graph (Lines 6-12). Specifically, we define function F (u) = Q u, which means 9.1 Star Aggregate Query Processing (p1,...,pn) projecting u’s adjacent properties over (p1, ..., pn) (Line 10). We insert F (u) into MS(O). If there exists some Definition 17 A Star Aggregate (SA) query (v, {p1, ... 0 0 aggregate tuple t , where t .D = F (u), we register ver- pd}, {pd+1, ...pn}) consists of a central vertex v, a set of tex ID of u in vertex list t.L (Lines 8-9). Otherwise, we group-by dimensions {p1, ...pd}, and a set of measure 0 0 0 generate a new aggregate tuple t , where t .D = F (u) dimensions {pd+1, ...pn}, where {p1, ...pd, ...pn} are all and insert vertex ID of u into t0.L (Lines 10-12). the attribute properties adjacent to v. ut
Theorem 3 Any entity vertex u in RDF graph G is Given a SA query S = (v, {p1, ...pd}, {pd+1, ...pn}), accessed once in computing aggregate sets of trie-index we answer S using T-index by Algorithm 4. Let P = by Algorithm 6. {p1, ..., pd, ...pn}. Given the set of properties P , we find their match (O1, ..., On), where Oi is a node in T-index Proof See Part B of Online Supplementals. and all nodes Oi (i = 1, ..., n) are in the same path from We illustrate the construction of T-index using an the root, and the property associated with Oi equals pi example. First, a scan of DB (Figure 14a) derives a list (Line 1 in Algorithm 4). We do not require that all of all dimensions in DB and their frequencies, and the nodes Oi (i=1,...,n) are adjacent to each other in the dimension list DL is constructed. The root of T-index path. Thus, it is possible to have multiple matches for a given P . For each match (O1, ..., On), we use O to (O0) is created and labeled as “root”. Then, we insert all transactions of DB into T-index one-by-one. denote the farthest node from the root (Line 3 in Algo- rithm 4)2. The aggregate set associated with node O is 1. The scan of the first transaction A(001) leads to the denoted as MS(O). We compute MS0(O) by project- construction of the first branch of the tree: (root, ing MS(O) over group-by dimensions {p1, ..., pd} (i.e., hasName,gender,bornOnDate,title, diedOnDate), in- MS0(O) = Q MS(O)). The MS0(O) from all (p1,p2,...,pd) serting nodes O1, O2, O3, O4 and O5. Initially, we the matches are merged to form the final result to SA set Oi.L = {001}, i = 1, ..., 5. query S, i.e., R(S) (Lines 6-7 in Algorithm 4). 2. A(002) shares a common prefix (hasName) with the existing path, and adds one new node O8 (found- Algorithm 4 SA Query Algorithm ingYear) as a child of node O1 (hasName). It also causes updating the corresponding vertex list of each Input: T-Index Input: SA queryS(v, {p1, ..., pd}, {pd+1, ..., pn}) node along the path. Output: An aggregate result set MS for a SA query Q. 3. The above process is iterated until all transactions 1: Locate all matches of properties (p1, ..., pd, ..., pn) in T- are inserted into T-index. index 2: for each match m do 4. Finally, for each node Oi in T-index, we build ag- i 3: Let Oi denote the node in match mi that is farthest gregate sets MS(Oi) by post-order traversal over T- from the root index. Specifically, we first compute MS(O ) by as- 5 4: Let MS(Oi) denote the aggregate set associated with sessing entity vertices 001 and its dimension values. node Oi 5: MS0(O ) = Q MS(O ). Then, we compute MS(O4) by merging the projec- i (p1,p2,...,pd) i 6: MS = S MS0(O ) tion of MS(O ) over dimensions (hasName,gender, i i 5 7: return MS bornOnDate,title) and assessing entity vertex 005. The process is iterated until all aggregate sets are computed. MS(O ) and MS(O ) are given in Fig- 4 7 For example, given a SA query S1 (v1, {gender, ure 14b as examples. title}, φ) in Figure 6, group-by dimensions are {gender, title} and measure dimension is ?m and we use “count” as an aggregate function (this is analogous to COUNT(*) 9 Aggregate Query Processing in SQL). There are two matches (O1,O2,O3,O4) and (O ,O ,O ) in T-index corresponding to two group- As noted in Section 4, we decompose a GA query Q into 1 2 7 by dimensions. In the first match, O4 is the farthest several SA queries {Si}, i = 1, ..., n, where each star 2 center vi is an entity vertex in Q. The result of each Note that, this is not necessarily On, since node identifiers Si (R(Si)) is computed using the approach discussed in are arbitrarily assigned to help with presentation. 16 node from the root. Since MS(O4) is an aggregate set Algorithm 5 General Aggregate (GA) Query Algo- over dimensions (hasName,gender, bornOnDate,title), rithm 0 we compute a temporary aggregate set MS (O4) on Input: A GA query Q Output: R(Q) {Query Result} dimensions (gender, title) by projecting MS(O4) over 1: Each entity vertex vi (in Q), i = 1, ..., n, together with these two dimensions. In the second match, O7 is the its adjacent attribute properties form a SA query Si farthest node from the root. Since MS(O7) is an ag- 2: for each SA query Si do gregate set over dimensions (hasName,gender,title), it 3: Call Algorithm 4 to find R(Si), i = 1, ..., n. S 0 4: Ti = t.L is also projected over (gender,title) to get MS(O7). Fi- t∈R(Si) nally, we obtain R(S ) by merging MS0(O ) and MS0(O ). 5: All entity vertices vi together with link properties be- 1 4 7 tween them form the link structure J Figure 15 illustrates the process. 6: Find all subgraph matches of J over RDF graph 7: U = {g1}, where g1 includes all subgraph matches 8: for each entity vertex vi in J, i = 1, ..., n do MS(O4) 9: set U 0 = φ hasName gender bornOnDate title L Abraham Male 1865-04-15 President {001} 10: for each group g in U do Lincoln 11: for each aggregate tuple t ∈ R(Si) do Reese Female 1976-03-22 Actress {005} 0 Witherspoon 12: Select a group g of matches MS ∈ g {MS[i] ∈ t.L and MS[i] refers to the i-th vertex in MS} 0 MS (O4) 13: Insert group g0 into U 0 gender title L 14: Set U = U 0 Male President {001} R(S1) Female Actress {005} gender title L COUNT 15: Assume that the measure dimension is associated with vi ∪ Male President {001,007} 2 0 16: for each group g in U do MS (O7) Female Actress {005} 1 gender title L 17: Find all matching vertices to vi for all matches in g Male President {007} 18: Access the measure values of these matching vertices 19: Compute the aggregate value in measure dimension in MS(O7) this group, and insert it into R(Q) hasName gender title L Franklin D. Male President {007} 20: return R(Q) Roosevelt
Fig. 15: Answering SA Query 0 0 0 0 {ui},E = {ej},Σ = {eLabel(ej)}) in RDF graph G, 0 where ui is an entity vertex in G, ej is an edge whose 0 0 endpoints are both in V , and eLabel(ej) is the label 0 0 9.2 General Aggregate Query Processing (link property) of the edge ej, G is called a subgraph match of J in RDF graph G if and only if:
At this point, all R(Si) are computed. We now discuss 1. ui ∈ Ti how to compute the final result R(Q) = i R(Si). 0 0 0 on 2. ej ∈ E ⇔ ej ∈ E and eLabel(ej) = eLabel(ej) ut
Definition 18 Let GA query Q consist of n SA queries. Consider query Q in Figure 6. The results of S1 and The link structure J of Q is a subgraph induced by all S2 and the Ti lists are shown in Figure 16a, while the star centers vi, i = 1, ..., n. Specifically, J is denoted as link structure J is shown in Figure 16b. J(V = {vi},E = {ej},Σ = {eLabel(ej)}), where ver- tex vi (1 ≤ i ≤ n) is a star center, ej (1 ≤ j ≤ m) Link Structure J foundingYear S v1 S v2 bornIn is an edge whose endpoints are both star centers, and 1 2 v1 v2 gender title R(S2) eLabel(ej) is the label (link property) of the edge ej. R(S ) 1 foundingYear L gender title L 1790 {002} ut Male President {001,007} v1 v2 1718 {006} Female Actress {005} {005} {006} 1810 {008} {007} {008} 1776 {004}
Note that, J is a connected subgraph, since all en- T1 = {001, 005, 007} tity vertices (in Q) are connected together by link prop- T2 = {002, 004, 006, 008} erties. For each R(Si), we can find a vertex list Li (a) SA Query & Answers (b) Link Structure & Matches that includes all vertices in R(Si). Specifically, we get T = S t.L, where t is an aggregate tuple in i t∈R(Si) Fig. 16: Query Decomposition R(Si). This means that all vertices in Ti are candidate matching vertices of vi. To compute R(Q), we need to find all subgraph matches of J over the RDF graph, Obviously, we can perform multiway join T1 on ... on where a subgraph match is defined as follows. Tn to find matches of J over RDF graph G. In order to speed up online performance, Ti (i = 1, ..., n) should Definition 19 Given a link structure J(V = {vi},E = be as small as possible. Note that, link structure J has 0 0 ∗ {ej},Σ = {eLabel(ej)}) in Q and a subgraph G (V = a structure analogous to the query signature graph Q , 17 since each star center vi in J corresponds to an entity peat the above process (Lines 10-14) for all groups g vertex in SPARQL query graph Q and an edge in J is an in U. Then, U 0 is the first-level partition. Obviously, S S edge between two entity vertices. Therefore, we can use g∈U {all matches in group g} = g0∈U 0 {all matches in 0 the approach described in Section 7 to compute results. group g }. Iteratively, we consider other R(Si) for other Specifically, given an aggregate query, we represent its level partitions (Lines 8-14). Finally, for each aggregate query pattern graph as Q. Then, we encode Q into a group, we compute the aggregate value in aggregated ∗ ∗ signature query graph Q . For each vertex vi in Q , dimension, and insert it into final result R(Q). Assume we can find a candidate list C(vi) by employing Lines that the measure dimension is associated with vertex ui 1-15 in Algorithm 2. Then, we update Ti = Ti ∩ C(vi). in Q. For each group g, we find a list of distinct vertices Finally, we utilize Algorithm 3 to find matches of J over matching ui. We access the measure dimension values RDF graph G. of these matching vertices and compute the aggregate For example, we find two subgraph matches of J value for this group. over RDF graph G, where Figure 16b shows the flat- Level 1 partition tened representation of these matches. Then, we need Subgraph Matches ↓ Subgraph Matches to partition these subgraph matches into one or more v1 v2 gender title v v {005} {006} 1 2 groups based on subgraph matches that share specified Male President 005 006 values in group-by dimensions, and create a new solu- {007} {008} Female Actress 007 008 tion set R(Q) that contains one tuple per aggregated (a) Step 0 (b) Step 1 group. Specifically, we try to get U = {gj}, j = 1, ..., m, where all subgraph matches in group gj share the same Level 1 Level 2 values over group-by dimensions. partition partition L1 L2 ↓ ↓ ↓ ↓ A straightforward method is to find, for each match, Subgraph Matches gender title foundingYear COUNT(v1) the corresponding entity vertices and their group-by di- v1 v2 mension values in the RDF graph, and partition these Male President 1718 005 006 1 matches into different groups based on subgraph matches Female Actress 1810 007 008 1 that share specified values in group-by dimensions. This (c) Step 1 method suffers from a large number of random I/Os. Furthermore, partitioning subgraph matches is an ex- Fig. 17: Partitioning Subgraph Matches pensive task if there are a large number of subgraph Let us recall the decomposition of Q given in Fig- matches of J over the RDF graph. 3 ure 6. We discuss how to partition all matches of J In order to improve performance, we use star aggre- into different groups based on group-by dimensions to gate query results R(Si) to partition subgraph matches, find the final result R(Q). Initially, all matches are in which helps reduce I/O accesses. Furthermore, scan- the same group g, i.e., g = {(005, 006), (007, 008)} and ning aggregate sets in T-index (in SA query algorithm) U0 = {g}, where (005, 006) is a flattened representa- requires sequential access, which is much faster. More tion of a match. Based on R(S1), we can get the first- importantly, R(Si) has partitioned subgraph matches level partition U1. Specifically, we partition matches of based on group-by dimensions associated with each ver- g into fine-grained groups. According to the first aggre- tex vi in query Q. Therefore, we use R(Si) to find fi- gate tuple t1 in R(S1) (Figure 16a), we create a new nal partitions. Initially, we assume that all subgraph group g1 = {(005, 006)}, since 005 ∈ t1.L. Similarly, we matches are in the same group g1, and set U = {g1} create another group g2 = {(007, 008)}. Consequently, (Line 7 in Algorithm 5). Then, we perform a multi- the first level partition is U1 = {g1, g2}, shown in Figure level partitioning over these subgraph matches. At the 17b. Then, based on R(S2), we can perform the second- first level, we consider group-by dimensions in R(S1). level partition U2. Specifically, we partition each group For each group g ∈ U, we partition matches in g into g (i = 1, 2) into more fine-grained groups. Figure 17c 0 i some new groups gi, i = 1, ..., m, where each new group shows the second-level partition. Finally, we compute 0 gi has matches that share the same values over group- the aggregate value for each group in U . S 0 2 by dimensions in R(S1). Obviously, g = i gi. Specifi- cally, in order to partition matches in group g, we se- quentially scan R(S1). For each aggregate tuple t ∈ 9.3 Aggregation over Link Properties 0 R(S1), we find a new group gi of subgraph matches MS, such that MS[i] ∈ t.L and MS[i] refers to the So far, we assumed that group-by dimensions always ith vertex in subgraph match MS (Lines 11-12). We correspond to attribute property edges. In this subsec- 0 0 insert these new groups gi into U (Line 13). We re- tion, we discuss how to relax this assumption to allow 18 group-by dimensions that correspond to link property 10 Handling Data Updates edges. Consider the following aggregate query Q5. As mentioned earlier, most existing RDF triple stores SELECT ?d ? c COUNT(?m) WHERE cannot support updates effectively. In this section, we {?m
Like other height balanced trees, insertion into a node 10.3.1 Dimension List DL’s order does not change that is already full will invoke a node split. The B + 1 entities of the node will be partitioned into two new Consider the insertion of a new triple hs, p, oi. If p is a nodes, where B is the maximal fanout for a node in link property, we do not need to update the T-index. VS∗-tree. We adopt a node splitting method similar to Thus, we only consider the case when p is an attribute [27]. The fundamental issue is to determine the two new property, i.e., dimension. nodes from among the B + 1 entities. For this, we need If s does not occur in the existing RDF data, we to consider each pair of B+1 entities. At each iteration, introduce a new transaction into D. We insert the new we select as the seed nodes two entities from among the transaction into one path of T-index following Defini- B + 1. We allocate the remaining entities to these two tion 16, and update the affected materialized aggregate nodes according to Equation 1. We perform this for sets. each pair of B + 1 entities and keep the entity pair that If s is already in the existing RDF graph, assume leads to the minimal value of Max(α,β), where α and that s’s existing dimensions are {p1, ..., pn} and F req(pi) β are the number of 1’s in the two nodes. > F req(p) > F req(pi+1) in dimension list DL. Again, After a node splits, we have to update the signatures in the case of equality, order is chosen arbitrarily. This and the super edges associated with the two new nodes. means that the new inserted dimension p should be in- The updates are straightforward. Node splitting invokes serted between pi and pi+1. We locate two nodes Oi and insertions over the upper level of VS∗-tree, which also On, where the path reaching node Oi (and On) has di- 3 leads to the splitting that may be propagated to the mensions (p1, ..., pi) (and (p1, ..., pi, ...pn) ). We remove root of the VS∗-tree. subject s from all materialized sets along the path be- tween nodes Oi+1 and On, where Oi+1 is a child node of O . Then, we insert dimensions (p, p , ..., p ) into 10.2.3 Deletion i i+1 n T-index from node Oi, and update the materialized sets To delete a vertex u from VS∗-tree, we find the leaf along the path. node d where u is stored, and delete u. This effects Consider the deletion of a triple hs, p, oi, where p the nodes along the path from the root to d. We adopt is an attribute property as discussed above. Assume the bottom-up strategy to update the signature of and 3 Although dimension list is a set, when the order is impor- super edges associated with the nodes. After deletion, tant, we specify them as a list enclosed in ( ). 20 that s’s existing dimensions are {p1, ..., pn} and p = define six sample SPARQL queries, as shown in the pi. We proceed as above to delete s from all affected Online Supplements. materialized sets. 4. LUBM is a synthetic benchmark [12] that adopts an ontology for the university domain, and can gener- 10.3.2 Dimension List DL’s order changes ate synthetic OWL data scalable to any arbitrary size. We vary the university number from 100 to Some triple deletions and insertions that affect dimen- 1000. The number of triples is 13 to 133 million. sion frequency will lead to changing the order of di- The total RDF file size is 1.5 to 15.1 GB. mensions in DL. Assume that two dimensions pi and p need to be swapped in DL due to inserting or delet- j 11.2 Parameter Settings ing one triple hs, p, oi. Obviously, j = i + 1, i.e., pi and p are adjacent to each other in DL. j Our coding methods and indexing structures use a num- Updates are handled in two phases. First, we ignore ber of parameters. In this subsection, we discuss how to the order change in DL and handle the updates using set up these parameters to optimize query processing. the method in Section 10.3.1. Second, we swap pi and pj in DL, and change the structure of T-index and the 11.2.1 M and m relevant materialized sets. Given a vertex u in the RDF graph, we encode each edge label (eLabel) adjacent to u into a bitstring eSig(e).e 11 Experiments with length M, and set m out of M bits to be ‘1’. We obtain vSig(u).e by performing bitwise OR over all We evaluate gStore over real RDF and synthetic datasets, eSig(e).e. Analogous to signature files, there may exist and compare it with SW-store [1], RDF-3x [19], x-RDF- the “false drop” problem [9]. Let AdjEdges(u, G) de- 3x [22], a graph-based solution GRIN [29], and two com- note all edge labels adjacent to u in G and AdjEdges(v, Q) mercial systems: BigOWLIM (www.ontotext.com/owlim/ defined similarly. If AdjEdges(v, Q) 6⊂ AdjEdges(u, G)∧ big/) and Virtuoso 6.3 (virtuoso.openlinksw.com/). The vSig(v)&vSig(u) = vSig(v), we say that a false drop results of the main experiments are reported here; more has occurred. vSig(v)&vSig(u) = v means that u is a results are given in Part D of Online Supplements. candidate match v. However, AdjEdges(v, Q) 6⊂ AdjEdges (u, G) means that u cannot match to v. Obviously, the key issue is how to reduce the number of false drops. 11.1 Datasets According to a theoretical study [18], the probabil- We use three real RDF and one benchmark datasets in ity of false drops can be quantified by the following our experiments. equation.
1. Yago (www.mpi-inf.mpg.de/yago-naga/yago/) extracts − |AdjEdges(u,G)|∗m m∗|AdjEdges(v,Q)| facts from Wikipedia and integrates them with the Pfalse drop = (1 − e M ) WordNet thesaurus. It contains about 20 million where |AdjEdges(u, G)| is u’s degree in G and |AdjEdges RDF triples and consumes 3.1GB. (v, Q)| is v’s degree in Q and M is the length of bit- For exact query evaluation, we use all SPARQL queries string, and m out of M bits are set to be ‘1’ in hash in [19] over Yago. For wildcard query evaluation, functions. According to the optimization method in Sec- we rewrite all of these queries to include wildcards. tion 6, a high degree vertex will be decomposed into Specifically, for each exact SPARQL query Q, we re- several instances and each instance’s degree is no larger place each literal vertex in Q as a wildcard vertex. In than δ. Thus, the false drop probability P in- 00 false drop this way, we can get a query Q with wildcards. We creases with |AdjEdges(u, G)|. Since |AdjEdges(u, G)| ≤ also define six aggregate queries. All sample queries − δ∗m m∗|AdjEdges(v,Q)| δ, Pfalse drop ≤ (1 − e M ) . To im- are given in Part A of Online Supplements. prove query performance, false drop probability should 2. In order to evaluate the scalability of gStore, we also be as small as possible, as the false drop will lead to un- use Yago2 [15] in our experiments. It has has more necessary I/O. For example if we wish this probability than 10 million entities and more than 180 million to be no larger than 1.0 × 10−3, the parameter setting triples. The total size of Yago2 is larger than 23 GB. can be tuned to ensure the following holds: 3. DBLP (sw.deri.org/∼aharth/2004/07/dblp/) contains a large number of bibliographic descriptions. There − δ∗m m∗|AdjEdges(v,Q)| −3 are about 8 million triples consuming 0.8GB. We Pfalse drop ≤ (1 − e M ) ≤ 1.0 × 10 21
·104 ·105 800 600 S-Q1 S-Q1 8 S-Q2 S-Q2 6 600 S-Q3 S-Q3
400 6 4 400 4
Index Size (MB) Index Size (MB) 200 2 Candidate Set Size 200 Candidate Set Size 2 0 0 50 100 150 200 250 50 100 150 200 250 29 59 97 149 199 251 29 59 97 149 199 251 N N N N (a) Yago (b) DBLP (a) Yago (b) DBLP
∗ Fig. 18: VS -tree Index Size vs. N Fig. 19: Candidate Size vs. |N|
From the query logs, we determine that the average files and load them into their own systems. The load- value of |AdjEdges(v, Q)| is 3. Therefore, in Yago and ing time is defined as the total offline processing time. DBLP datasets, we set m = 2,M = 97 and δ = 10. In The total space cost is defined as the size of the whole −3 this case, Pfalse drop ≤ 1.0 × 10 . database including the corresponding indexes. We show load time and the total space cost in Figures 20a and 20b, respectively. Since our method adopts the bitstring 11.2.2 N and n as the basic index structures, compared with other ap- proaches, gStore has the minimal total space size, as The false drop problem also exists in comparing vSig(q).n shown in Figure 20b. Figure 20a shows that our in- with vSig(v).n, but it is quite difficult to quantify the dex building time is the fastest. In other methods, they probability of false drops in this case. Therefore, we need to build several exhaustive indexes, such as RDF- adopt the following method, using the “n-gram” tech- 3x and x-RDF-3x. The index structure in GRIN is a nique where n = 3 that has been experimentally shown memory-based data structure that is not optimized for to work well [11]. I/O cost. In order to handle large graphs, GRIN has to It is clear that the larger N is, the fewer conflicts employ virtual memory for index construction, which exist among the vertex signatures. On the other hand, leads to I/O cost. large N will lead to large space cost of vertex signatures. Thus, we need to find a good trade-off for N. We use three star queries to evaluate the pruning power of the 100 3,500 encoding technique. Given a star query S, we encode its gStore gStore RDF-3X 3,000 RDF-3X 80 SW-Store SQ-Store central vertex v into a vertex signature vSig(v). We use x-RDF-3x 2,500 x-RDF-3x 60 BigOWLIM BigOWLIM X to denote the number of vertex signatures vSig(u), GRIN 2,000 GRIN ∗ 1,500 where vSig(v)&vSig(u) = vSig(v) in G . Obviously, X 40 1,000 decreases as N increases as shown in Figure 19. How- Space Cost (in MB) Offline Time (in min.) 20 ever, the decreasing trend slows down when N > 149 500 0 0 in Yago and N > 97 in DBLP. On the other hand, Yago DBLP Yago DBLP Data Set Data Set larger N leads to larger space cost of VS∗-tree. In our (a) Offline Processing Time (b) Total Space Size experiments, we set N = 149 in Yago and N = 97 in DBLP, respectively, as larger N’s values cannot lead to Fig. 20: Evaluating Offline Performance significant increase in pruning power.
11.3 Database Load Performance 11.4 Query Performance
We compare gStore with five competitors over both 11.4.1 Exact Queries Yago and DBLP datasets in terms of database load time and index size. For a fair comparison, we adopt We compare the performance of gStore with five com- the settings in [19], i.e., each dataset is first converted petitors over both Yago and DBLP datasets. The sam- into a factorized form: one file with RDF triples repre- ple queries are shown in Tables 6 and 7 in the Online sented as integer triples, and one dictionary file to map Supplements. Figure 21 shows that gStore system is from ids to literals. All methods utilize the same input much faster than other methods. The key reason is that 22
6,000 gStore 16,000 gStore 5,000 RDF-3X 14,000 RDF-3X+PF SW-Store SW-Store+PF 12,000 4,000 x-RDF-3x x-RDF-3x+PF BigOWLIM 10,000 BigOWLIM GRIN GRIN+PF 3,000 8,000
2,000 6,000 4,000 1,000 Query Response Time (in ms) Query Response2 Time (in, ms) 000
0 0 A1 A2 B1 B2 B3 C1 C2 A1 A2 B1 B2 B3 C1 C2 Queries Queries
(a) Yago (a) Yago
3,000 gStore 8,000 gStore 2,500 RDF-3X RDF-3X+FM SW-Store SW-Store+FM x-RDF-3x+FM 2,000 x-RDF-3x 6,000 BigOWLIM BigOWLIM GRIN GRIN+FM 1,500 4,000
1,000 2,000 500 Query Response Time (in ms) Query Response Time (in ms)
0 0 Q1 Q2 Q3 Q4 Q5 Q6 Q1 Q2 Q3 Q4 Q5 Q6 Queries Queries
(b) DBLP (b) DBLP
Fig. 21: Exact Query Response Time Fig. 22: Wildcard Query Response Time (FM: postfiltering) candidate lists for each query vertex are reduced greatly returned result, check if it satisfies the wildcard con- by VS∗-tree. We show the pruning power of VS∗-tree in straints. The worst performance drop for each system Figures 26 and 28b (in Online Supplements). The other are 929% for RDF-3x, 560% for SW-Store, 920% for x- ssytems do not consider the query graph structure for RDF-3x, 330% for BigOWLIM, and 4486% for GRIN. pruning. 11.5 Aggregate Query Performance
11.4.2 Wildcard Queries We compared the performance of gStore with Virtuoso 6.3 (http://virtuoso.open linksw.com/) and the CAA In order to enable comparison over wildcard queries, we algorithm [16] on both SA and GA queries. To the best adopt the post-filtering method in Section 11.1 in RDF- of our knowledge, Virtuoso is the only commercial sys- 3x, SW-store, x-RDF-3x and GRIN. Since BigOWLIM tem that can fully support group-by and aggregation has embedded full-text index, it can support wildcard following SPARQL 1.1 specification, and CAA algo- queries. The sample queries are shown in Tables 9 and rithm is the only proposal in literature that can handle 10 in the Online Supplements. Figure 22 shows query aggregate queries that follows SPARQL 1.1 semantics. response times of different methods. As would be ex- We use Yago dataset in this experiment. Six aggregate pected, all of the tested systems suffer some amount of queries that are used in our experiments are given in performance drop in executing wildcard queries. How- Table 12 in the Online Supplements. Figure 23 shows ever gStore experiences the least amount of drop, be- that our method has the best online performance. cause gStore can answer both exact and wildcard queries Generally speaking, the query performance depends |R(Q0)| in a uniform manner, making its performance more ro- on several factors, such as aggregation ratio |R(Q)| , bust. In all but one query, even with the performance the size of |R(Q0)|, the size of query graph Q and the drop, it executes wildcard queries in under one second. characteristics of group-by dimensions (whether or not However, the query performance degrades dramatically they include link properties). In Figure 23, both SA1 for other systems, since they cannot support wildcard and SA2 have one group-by dimension. SA1 (52 mil- queries directly. The only way to process these queries liseconds) is much faster than SA2 (163 milliseconds). is to first ignore wildcard constraints, and, for each Furthermore, the speedup ratios (between gStore and 23
∗ 10,000 gStore Table 2: Evaluating Updates to VS -tree in Yago (num- CAA ber of updated triples/second) Virtuoso 8,000 gStore RDF-3x x-RDF-3x 6,000 Insertion 2.2×104 1.4×104 1.2×104 4,000 Deletion (free-at-empty) 2.0×104 1.1×104 0.9×104 Deletion (merge-at-half) 1.5×104 1.1×104 0.9×104 2,000 Query Response Time (in ms)
0 SA1 SA2 SA3 GA1 GA2 GA3 Queries evaluating deletion performance, we randomly deleted 20% of the triples from VS∗-tree and T-index. Fig. 23: Aggregation Queries in Yago We report the number of triple insertions/deletions per second in gStore, and compare it with RDF-3x and x-RDF-3x in Table 2. RDF-3X had been extended Virtuoso) for SA1 (6.1) is larger than that for SA2 for updates using the deferred-indexing approach [21] (3.5). This is because |R(Q0)| is smaller in SA1 than where the updates are first recorded into differential that in SA2. The speedup ratio of SA3 (8.68) is the indexes, which are periodically merged into main in- largest among all three SA queries in Yago, since SA3 dexes. x-RDF-3x employs the similar update strategy has two edges in the query graph, which requires Vir- except for introducing “timestamp” of each triple [22]. tuoso to perform expensive join operations. We find Table 2 shows that gStore’s update time is the fastest, that SA3 is even slower than some GA queries. That is bacause VS∗-tree is a height-balance tree. The inser- because “hasFamilyName” and “hasGivenName” have tion/deletion operation has log|V (G)|’s time complex- the largest frequency in Yago, which results in large ity. RDF-3x and x-RDF-3x need to update six clus- |R(Q0)|. The results also show that gStore-Aggregation tered B+-trees. We also compare two deletion strate- B method also performs better than other alternative ap- gies (b = 1 and b = 2 ) in Table 2. Experiments show proaches on processing GA queries. Our method uses that the “free-at-empty” (b = 1) leads to faster deletion ∗ B VS -tree to find matches of the link structure J, which time, since “merge-at-half” strategy (b = 2 ) needs to uses structural pruning to reduce the search space. re-insert the children of the half-empty nodes, while it is not necessary in the free-at-empty strategy. The average running times to insert/delete one triple 11.6 Evaluating Dimension Orders in DL (without changing T-index’s structure) into/from T- index and update the corresponding aggregate sets are We evaluate the affect of the order of dimensions DL 0.02 and 0.01 msec, respectively. The average running using both Yago and LUBM datasets. Specifically, we time to swap two adjacent dimensions is 3000 msec. build T-index under three different orders: frequency- The total running time to insert/delete 10,000 triples, descending order, frequency-ascending order and ran- which include both inserting and dimension swapping dom order. The results show that frequency-descending times, are 15.20 and 12.20 msec, respectively. The num- order leads to the least number of nodes in T-index, ber of dimension swaps for insertion/deletion are 5 and the minimal total size and the minimal building time, 4, respectively. As these show, T-index’s maintenance since this order ensures that more prefixes can be shared overhead is light. When we insert/delete a triple with- among different transactions. Furthermore, the frequency- out changing T-index’s structure, we only need to insert descending order also brings the fastest query response one tuple into some aggregate sets along one path. The time, since it accesses the minimal number of nodes in operation is very cheap. If we need to swap two dimen- T-index. Complete experimental results can be found sions pi and pj, and there are n paths containing both in Part D of Online Supplements. pi and pj, there are at most 2 × n aggregate sets that need to be updated. Thus, the time for swapping two dimensions is higher. However, the chances of swapping two dimensions during the insertion/deletion is small. 11.7 Evaluating Maintenance Cost
To evaluate the overhead of VS∗-tree and T-index main- tenance, we randomly selected 80% of the triples in each 11.8 Scalability Experiments dataset to build VS∗-tree and T-index. Then, for the re- maining 20% of the triples, we performed a sequence of We evaluate the scalability of gStore against RDF-3x triple insertions using the method in Section 10. For and Virtuoso, using two large datasets: LUBM and Yago2. 24
We first report gStore’s index building performance over Table 3: Scalability of Index Building various LUBM dataset sizes in Table 3. Raw Data Offline Index Size For online query performance over LUBM, we use Data Set RDF N3 File Number of Number of Processing (KB) the set of 7 queries that have been defined and used in Size(KB) Triples Entities Time LUBM-100 1,499,131 13,402,474 2,178,865 12m23s 852,448 recent studies [3,34]. This workload was designed be- LUBM-200 3,007,385 26,691,773 4,340,588 32m11s 1,190,856 cause of the recognition that the original workload of LUBM-500 7,551,065 66,720,02 10,846,157 1h5m9s 1,640,281 LUBM-1000 15,136,798 133,553,834 21,715,108 2h10m15s 3,071,205 14 queries are very simple involving few triple patterns Yago 3,126,721 19,012,854 4,339,591 20m35s 1,305,267 (many involve two triple patterns) and are similar to Yago2 23,216,336 183,183,314 10,557,352 8h55m21s 3,071,205 each other. Furthermore, a large number of them in- volve constants that artificially increase their selectiv- Table 4: Scalability of Query Performance on LUBM ity – for example, triple patterns ?x undergraduateDe- Query Response Time (msec) Queries greeFrom http://www.University0.edu and ?x publica- LUBM-100 LUBM-1000 gStore RDF-3x Virtuoso gStore RDF-3x Virtuoso tionAuthor http://www.Department0.University0.edu/ Q1 1310 22634 2152 43832 202728 54460 Q2 236 4276 30560 1563 15008 > 30 mins AssistantProfessor0 identify a single object making their Q3 208 216 1981 1491 1737 7784 selectivity very high (with very few results). It has been Q4* 153 44 1341 680 30 1357 Q5* 129 48 312 131 12 243 argued that these queries are not representative of real Q6* 227 20 843 828 49 842 Q7 1016 960 5257 8301 14072 36848 workloads that contain implicit joins, and that they fa- ∗ indicates the query contains at least one constant entity. vor systems that use extensive indexing (such as RDF- 3x). We ran experiments with both query sets4, but we Table 5: Scalability of Query Performance on Yago2 report the results obtained for the 7 queries that are given in Table 14. The conclusions reported below hold Query Response Time (msec) Exact A1* A2* B1* B2 B3 C1 C2* for the original 14 query set as well. Queries Table 4 shows comparative online query performance gStore 251 230 2157 131 198 875 865 RDF-3x 35 26 921 289 228 219077 80 of gStore, RDF-3x and Virtuoso under small and very Virtuoso 1544 3213 23447 2777 6240 151337 23275 large LUBM configurations. gStore always performs (some- Wildcard AW1* AW2* BW1* BW2 BW3 CW1 CW2* Queries times an order of magnitude) better than Virtuoso, with gStore 3226 6122 8644 268 3197 15183 6189 the latter sometimes failing to complete within the 30 Virtuoso 3338 10109 33728 2388 21482 >30min 69031 minute window we allocated for executing queries. As ∗ indicates the query contains at least one constant entity. expected, and as noted above, RDF-3x does better than gStore when the query contains a triple pattern has by orders of magnitude in exact queries, and outper- high selectivity because it refers to a constant. Even forms Virtuoso greatly in wildcard queries. This is be- for these queries, gStore still performs very well with cause gStore utilizes the same (signature-based) prun- sub-second response time. For other queries (i.e., those ing strategy for wildcard queries, while Virtuoso adopts that involve implicit joins), gStore’s performance is su- the post-processing technique to handle them. Similar perior to RDF-3x. to LUBM results, the relative performance of gStore We also performed cross-query set evaluation to test and RDF-3x depend on the selectivity of the triple pat- the performance of systems according to the order of terns that include constants for which extensive index- triple pattern evaluation. For example, query Q2 in the ing can be exploited. RDF-3x does not support wildcard original LUBM workload (Table 15) is identical to Q1 queries, thus, we have not compared the two systems we report (Table 14) except for the different triple pat- for those queries. tern orders. For these queries, RDF-3x’s performance varies by 29–119 times, while gStore performance varies by 3 times. Naturally, both systems would benefit from 12 Conclusions optimizing join orders, but gStore performance appears to be more stable. In this paper, we described gStore, which is a graph- We also compare the three systems over the Yago2 based triple store. Our focus is on the algorithms and dataset (query sets in Tables 8 and 11). The results in data structures that were developed to answer SPARQL Table 5 demonstrate that gStore is faster than Virtuoso queries efficiently. The class of queries that gStore can handle include exact, wildcard, and aggregate queries. 4 We revised the original 14 to remove type reasoning that The performance experiments demonstrate that com- gStore does not currently support; the resulting queries re- pared to two other state-of-the-art systems that we con- turn larger result sets since there is no filtering as a result of type reasoning. For completeness, these are included in sider, gStore has more robust performance across all Online Supplements as Table 15. these query types. Other systems either do not support 25 some of these query types (e.g., none of the systems 10. L. Gravano, P. G. Ipeirotis, H. V. Jagadish, N. Koudas, support wildcard queries and only Virtuoso supports S. Muthukrishnan, L. Pietarinen, and D. Srivastava. Us- aggregate queries) or they perform considerably worse ing q-grams in a DBMS for approximate string process- ing. IEEE Data Eng. Bull., 24(4):28–34, 2001. (e.g., Virtuoso handles exact and aggregate queries, but 11. L. Gravano, P. G. Ipeirotis, N. Koudas, and D. Srivastava. is consistently worse than gStore). The techniques can Text joins in an RDBMS for web data integration. In handle dynamic RDF repositories that may be updated. Proc. 12th Int. World Wide Web Conf., pages 90–101, gStore is a fully implemented and operational system. 2003. 12. Y. Guo, Z. Pan, and J. Heflin. LUBM: a benchmark for There are many directions that we intend to follow. OWL knowledge base systems. J. Web Sem., 3(2-3):158– These include support for partitioned RDF reposito- 182, 2005. ries, parallel execution of SPARQL queries, and further 13. A. Gupta, , and V. H. Dallan Quass. Aggregate-query processing in data warehousing environments. In Proc. query optimization techniques. 21st Int. Conf. on Very Large Data Bases, pages 358– 369, 1995. 14. A. Harth, J. Umbrich, A. Hogan, and S. Decker. YARS2: 13 Acknowledgements A federated repository for querying graph structured data from the web. In Proc. 6th Int. Semantic Web Lei Zou’s work was supported by National Science Foun- Conf., pages 211–224, 2007. 15. J. Hoffart, F. M. Suchanek, K. Berberich, E. L. Kelham, dation of China (NSFC) under Grant 61370055 and G. de Melo, and G. Weikum. YAGO2: Exploring and by CCF-Tencent Open Research Fund. M. Tamer zsu’s querying world knowledge in time, space, context, and work was supported by Natural Sciences and Engineer- many languages. In Proc. 20th Int. World Wide Web ing Research Council (NSERC) of Canada under a Dis- Conf., pages 229–232, 2011. 16. E. Hung, Y. Deng, and V. S. Subrahmanian. RDF aggre- covery Grant. Lei Chen’s work was supported in part gate queries and views. In Proc. 21st Int. Conf. on Data by the Hong Kong RGC Project M-HKUST602/12, Na- Eng., pages 717–728, 2005. tional Grand Fundamental Research 973 Program of 17. T. Johnson and D. Shasha. B-trees with inserts and China under Grant 2012-CB316200, Microsoft Research deletes: Why free-at-empty is better than merge-at-half. J. Comput. Syst. Sci., 47(1):45–76, 1993. Asia Grant, and a Google Faculty Award. Dongyan 18. H. Kitagawa and Y. Ishikawa. False drop analysis of set Zhao was supported by NSFC under Grant 61272344 retrieval with signature files. IEICE Trans. on Inf. and and China 863 Project under Grant No. 2012AA011101. Syst., E80-D(6):1–12, 1997. 19. T. Neumann and G. Weikum. RDF-3X: a RISC-style en- gine for RDF. Proc. VLDB Endow., 1(1):647–659, 2008. 20. T. Neumann and G. Weikum. Scalable join processing References on very large RDF graphs. In Proc. ACM SIGMOD Int. Conf. on Management of Data, pages 627–640, 2009. 1. D. J. Abadi, A. Marcus, S. Madden, and K. Hollenbach. 21. T. Neumann and G. Weikum. The RDF-3X engine for SW-Store: a vertically partitioned DBMS for semantic scalable management of RDF data. VLDB J., 19(1):91– web data management. VLDB J., 18(2):385–406, 2009. 113, 2010. 2. D. J. Abadi, A. Marcus, S. Madden, and K. J. Hollen- 22. T. Neumann and G. Weikum. X-RDF-3X: Fast querying, bach. Scalable semantic web data management using ver- high update rates, and consistency for RDF databases. tical partitioning. In Proc. 33rd Int. Conf. on Very Large Proc. VLDB Endow., 1(1):256–263, 2010. Data Bases, pages 411–422, 2007. 23. J. P´erez,M. Arenas, and C. Gutierrez. Semantics and 3. M. Atre, V. Chaoji, M. J. Zaki, and J. A. Hendler. Matrix complexity of SPARQL. ACM Trans. Database Syst., “bit” loaded: a scalable lightweight join query processor 34(3):16:1–16:45, 2009. for RDF data. In Proc. 19th Int. World Wide Web Conf., 24. D. Y. Seid and S. Mehrotra. Grouping and aggregate pages 41–50, 2010. queries over semantic web databases. In Proc. Int. Conf. 4. P. A. Bernstein and D.-M. W. Chiu. Using semi-joins to on Semantic Computing, pages 775–782, 2007. solve relational queries. J. ACM, 28(1):25–40, 1981. 25. D. Shasha, J. T.-L. Wang, and R. Giugno. Algorith- 5. V. B¨onstr¨om, A. Hinze, and H. Schweppe. Storing RDF mics and applications of tree and graph searching. In as a graph. In Proc. 1st Latin American Web Congress, Proc. 21st ACM Symp. on Principles of Database Sys- pages 27–36, 2003. tems, pages 39–52, 2002. 6. J. Broekstra, A. Kampman, and F. van Harmelen. 26. M. Stocker, A. Seaborne, A. Bernstein, C. Kiefer, and Sesame: A generic architecture for storing and querying D. Reynolds. SPARQL basic graph pattern optimization RDF and RDF schema. In Proc. 1st Int. Semantic Web using selectivity estimation. In Proc. 17th Int. World Conf., pages 54–68, 2002. Wide Web Conf., pages 595–604, 2008. 7. T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein. 27. E. Tousidou, P. Bozanis, and Y. Manolopoulos. Introduction to Algorithms. The MIT Press, 2001. Signature-based structures for objects with set-valued at- 8. U. Deppisch. S-tree: A dynamic balanced signature index tributes. Inf. Syst., 27(2):93–121, 2002. for office retrieval. In Proc. 9th Int. ACM SIGIR Conf. 28. E. Tousidou, A. Nanopoulos, and Y. Manolopoulos. Im- on Research and Dev. in Inf. Retr., pages 77–87, 1986. proved methods for signature-tree construction. Comput. 9. C. Faloutsos and S. Christodoulakis. Signature files: An J., 43(4):301–314, 2000. access method for documents and its analytical perfor- 29. O. Udrea, A. Pugliese, and V. S. Subrahmanian. GRIN: mance evaluation. ACM Trans. Inf. Syst., 2(4):267–288, A graph based RDF index. In Proc. 22nd National Conf. 1984. on Artificial Intelligence, pages 1465–1470, 2007. 26
30. C. Weiss, P. Karras, and A. Bernstein. Hexastore: sextu- ple indexing for semantic web data management. Proc. VLDB Endow., 1(1):1008–1019, 2008. 31. K. Wilkinson, C. Sayers, H. A. Kuno, and D. Reynolds. Efficient RDF storage and retrieval in Jena2. In Proc. 1st Int. Workshop on Semantic Web and Databases, pages 131–150, 2003. 32. X. Yan, P. S. Yu, and J. Han. Graph indexing: A frequent structure-based approach. In Proc. ACM SIGMOD Int. Conf. on Management of Data, pages 335–346, 2004. 33. Y. Yan, C. Wang, A. Zhou, W. Qian, L. Ma, and Y. Pan. Efficient indices using graph partitioning in RDF triple stores. In Proc. 25th Int. Conf. on Data Eng., pages 1263–1266, 2009. 34. P. Yuan, P. Liu, H. Jin, W. Zhang, and L. Liu. TripleBit: A fast and compact system for large scale RDF data. Proc. VLDB Endow., 6(7):517–528, 2013. 1
Online Supplements Table 8: Exact SPARQL Queries in Yago2
A1 SELECT ?gn ?fn WHERE { ?p hasGiven- a oriented fact query, i.e., finding scientists A. Queries Used in Experiments Name ?gn. ?p hasFamilyName ?fn . ?p type star from Switzerland with a doctoral advisor from
Table 10: SPARQL Queries with Wildcards in DBLP Table 13: Aggregation SPARQL Queries in LUBM
Q1 SELECT ?x1 WHERE {?x1 foaf:name ?a.?x1 rdf:type A wildcard query with two star subqueries, Sample Queries foaf:Person.?x2 dc:creator ?x1.?x2 dc:identifier finding Jiawei Han’s VLDB papers. SA1 SELECT ?p COUNT(?m) WHERE {?m
Q1 SELECT ?x ?y ?z WHERE { ?z ub:subOrganizationOf ?y . ?y This query has large input and low selectiv- rdf:type ub:University . ?z rdf:type ub:Department . ?x ity. Furthermore, it is a triangular pattern of Table 11: SPARQL Queries with Wildcards in Yago2 ub:memberOf ?z . ?x rdf:type ub:GraduateStudent . ?x relationships between the objects. ub:undergraduateDegreeFrom ?y . } Q2 SELECT ?x WHERE { ?x rdf:type ub:Course . ?x ub:name ?y . This query has large input and low selectivity. } AW1 SELECT ?gn ?fn WHERE { ?p hasGivenName ?gn. ?p a oriented fact query, i.e., finding scientists Q3 SELECT ?x ?y ?z WHERE { ?x rdf:type The query is the same with S-Q1 except that hasFamilyName ?fn . ?p type star ?w1. ?p wasBornIn ?city from Switzerland with a doctoral advisor from ub:Undergraduate-Student. ?y rdf:type ub:University . ?z the student is an undergraduate student. The . ?p hasAcademicAdvisor ?a . ?city isLocatedIn ?w2. ?a Germany rdf:type ub:Department . ?x ub:memberOf ?z . ?z output size is 0. wasBornIn ?city2 . ?city2 isLocatedIn transitive ?w3 . ub:subOrganizationOf ?y . ?x ub:undergraduateDegreeFrom ?y . FILTER (regex(?w1,“scientist”) && regex(?w2, “Switzer- } land”) && regex(?w3,“Germany”)) } Q4 SELECT ?x WHERE { ?x ub:worksFor
0 0 0 0 neighbors of v also have the matching vertices in G. (F (v1), ..., F (vm)) forms a subgraph match of {v1, ..., vm} i ∗ Thus, according to the encoding strategy in Definition in a data signature graph G . ut 0 0 0 6, we know that vSig(ui)&vSig(vi) = vSig(vi). Thus, 3
∗ Theorem 2 Given a query signature graph Q with n Oi and {pi, ..., pk} be A(u)’s dimensions. If {l1, ..., lNi } ∗ vertices {v1, . . . , vn}, a data signature graph G and ⊂ {p1, ..., pk}, then we need to add new nodes to ex- VS∗-tree built over G∗: tend the path (Lines 5-6). Specifically, we introduce a new node O as a child of node O to denote the 1. Assume that n vertices {u , . . . , u } forms a match j i 1 n (n + 1)-th dimension of A(u). Iteratively, we introduce (Definition 8) of Q∗ over G∗. Given a summary (|A(u)| − n) new nodes (i.e., nodes corresponding to di- graph GI in VS∗-tree, let u ’s ancestor in GI be i mensions {p , ..., p }\{l , ..., l }) to extend the path node dI . (dI , ..., dI ) must form a summary match 1 k 1 Ni i 1 n from node O . We register these newly introduced nodes (Definition 10) of Q∗ over GI . i into the corresponding dimensions in DL (Line 7). Fur- 2. If there exists no summary match of Q∗ over GI , thermore, we update vertex lists of each node along the there exists no match of Q∗ over G∗. path (Line 9). We iterate and insert all transactions
Proof (1) According to Definition 8, vSig(vi)& vSig(ui) = into T-index (Lines 3-10). I I vSig(vi). Since di is an ancestor of ui, vSig(ui)&vSig(di ) = vSig(ui). Therefore, we can conclude that vSig(vi)& I vSig(di ) = vSig(vi). ∗ Algorithm 6 T-index and Materialized Aggregate Sets If there is an edge from vi to vj in Q , there is also ∗ −−−→ −−−→ Input: Transaction Database DB an edge from ui to uj in G ; and Sig(vi, vj)&Sig(ui, uj) −−−→ I I Input: RDF graph G = Sig(vi, vj). Since di and dj are ancestors of ui and uj, Output: T-tree and aggregate sets associated with each respectively, according to VS∗-tree’s structure, we know −−−→ node. −−−→ I I −−−→ 1: Scan DB to find all dimensions and build dimension list that Sig(ui, uj)&Sig(di , dj ) = Sig(ui, uj). Therefore, −−−→ DL. −−−→ I I −−−→ we can conclude that Sig(vi, vj)&Sig(di , dj ) = Sig(vi, vj). 2: Introduce a root node into T-tree In summary, according to Definition 10, we know 3: for each transaction A(u) in DB do I I ∗ I 4: Find a node Oi in T-tree, where the path reaching Oi that (d1, ..., dn) is a summary match of Q in G . (i.e.,{N0, ..., Ni}) is the maximal prefix of A(u). Let (2) The second claim in Theorem 2 can be proved {l1, ..., lNi } be dimensions along path {N0, ..., Ni}. according to the first claim by the contradiction. ut 5: if {l1, ..., lNi } ⊂ {p1, ..., pk}, where {p1, ..., pk} are A(u)’s dimensions then
Theorem 3 Any entity vertex u in RDF graph G is 6: Introduce |A(u) − n| nodes {p1, ..., pk}\{l1, ..., lNi } accessed once in computing aggregate sets of trie-index to extend the path from node Oi. 7: Register these new introduce nodes into the corre- by Algorithm 6. sponding dimensions in DL 8: Record the ID of transaction A(u) into vertex lists Proof According to T-index’s structure, each entity ver- Oi.L, where i = 0, ..., k tex u is only in one path of T-index. Assume that u cor- 9: for each child Oi of the root do responds to node O in T-index. The dimension values 10: Call Function PostOrderVisit(Oi) (Algorithm 7) of u only need to be accessed when computing MS(O). Aggregate sets of O’s ancestor nodes can be computed from its children nodes without accessing the original data. ut
Algorithm 7 Function: PostOrderVisit(O) C. T-index Construction and Maintenance 1: for each child node Oi of O do 2: Call Function PostOrderVisit(Oi). Construction 3: for each child Oi of O do 0 Q 4: compute MS (Oi) = MS(Oi). (p1,p2,...,pn) S 0 The construction algoirthm of T-index is given in Al- 5: MS(O) = M (Oi) i S gorithm 6. Initially, a scan of DB derives a list of all 6: for each vertex u in O.L but not in i Ni.L do 7: F 0(u) = Q u dimensions, and the dimension list DL is created. We (p1,...,pn) 8: if there exists some aggregate tuple t0, where t0.D = introduce a root node into T-index (Lines 1-2 in Al- F 0(u) then gorithm 6). When we insert a transaction A(u) into 9: Register vertex ID of u in vertex list t.L T-index, we find a node Oi, where the path reaching 10: else 11: Generate a new aggregate tuple t0, where t0.D = Oi is the maximal prefix of A(u) (Line 4). The maxi- F 0(u) and insert vertex ID of u into t0.L mal prefix means that (1) the path reaching node Oi is 12: Insert t0 into MS(O) a length-n prefix of A(u) and (2) there exists no path that is a length-(n + 1) prefix of A(u). Let {l1, ..., lNi } be the dimensions along the path from the root to node 4
MS(O4) Maintenance hasName gender bornOnDate title L Abraham Male 1865-04-15 President {001} Lincoln Reese With- Female 1976-03-22 Actress {005} Updates of the RDF data requires efficient maintenance erspoon Franklin D. Male 1882-01-30 President {007} of T-index. Obviously, updates can be modeled as a Roosevelt root sequence of triple deletions and triple insertions. As O0 mentioned earlier, all dimensions are ordered in their {001,002,003,004,005,007,009} hasName foundingYear O O frequency descending order in dimension list DL to im- 1 9 {006,008}
{001,005,007,009} prove SA query evaluation performance. However, RDF gender foundingYear O O data updates may change the order of dimensions in 2 8 {002,004} Dimension List DL {001,005,007, 009} hasName:7 DL requiring special care during index maintenance. bonOnDate O3 foundingYear:4 Therefore, we consider updates of T-index in two cases gender:4 {001,005,007} title diedOnDate bornOnDate:4 based on whether or not the order of dimensions in DL O O 4 6 {009} title:3 changes. diedOnDate:2 {001} diedOnDate O5 Dimension List DL’s order does not change Fig. 24: Addition of Triple hy:Franklin D. Roosevelt, Consider the insertion of a new triple hs, p, oi. If p is a bornOnDate,“1882-01-30”i link property, we do not need to update the T-index. Thus, we only consider the case when p is an attribute property, i.e., dimension. “O2 −O3 −O4”. Path “O2 −O7” is deleted, since the up- If s does not occur in the existing RDF data, we dated aggregate sets in O7 are empty. Figure 24 shows introduce a new transaction into D. We insert the new the updated T-index after inserting the triple. transaction into one path of T-index following Defini- Suppose now that we need to delete a triple hs, p, oi, tion 16. Accordingly, we need to update the material- where p is an attribute property as discussed above. As- ized aggregate sets MS(Oi) along the path. The de- sume that s’s existing dimensions are {p1, ..., pn} and tailed steps are the same as Lines 4-9 in Algorithm 6. p = pi, i.e., p and pi are the same dimension. We If s is already in the existing RDF graph, assume locate two nodes Oi and On, where the path reach- that s’s existing dimensions are {p1, ..., pn} and F req(pi) ing node Oi (and On) has dimensions (p1, ..., pi) (and > F req(p) > F req(pi+1) in dimension list DL. Again, (p1, ..., pi, ...pn)). We remove subject s from all mate- in the case of equality, order is chosen arbitrarily. This rialized sets along the path between nodes Oi+1 and means that the new inserted dimension p should be in- On. Then, we insert dimensions (pi+1, ..., pn) into T- serted between pi and pi+1. We locate two nodes Oi and index from node Oi, and update the materialized sets On, where the path reaching node Oi (and On) has di- along the path. Again T-index itself does not need to 5 mensions (p1, ..., pi) (and (p1, ..., pi, ...pn) ). We remove be modified. subject s from all materialized sets along the path be- tween nodes Oi+1 and On, where Oi+1 is a child node Dimension List DL’s order changes of Oi. Then, we insert dimensions (p, pi+1, ..., pn) into T-index from node Oi, and update the materialized sets Some triple deletions and insertions that affect dimen- along the path. sion frequency will lead to changing the order of di- Consider inserting a triple mensions in DL. Assume that two dimensions pi and hy:Franklin D. Roosevelt, bornOnDate,“1882-01-30”i pj need to be swapped in DL due to inserting or delet- into RDF triple table T shown in Figure 1a, where ing one triple hs, p, oi. Obviously, j = i + 1, i.e., pi and 6 y:Franklin D. Roosevelt pj are adjacent to each other in DL . corresponds to vertex 007. Updates are handled in two phases. First, we ignore Although inserting the triple changes the frequency the order change in DL and handle the updates using of dimension “bornOnDate”, it does not lead to chang- the method in Section 13. Second, we swap pi and pj ing the order in DL. 007’s dimensions are (hasName, in DL, and change the structure of T-index and the gender, title). According to dimension order, the in- relevant materialized sets. serted triple should be placed between “gender” and We focus on the second phase. There are only three “title”. Therefore, we delete 007 from path “O2 − O7” categories of paths that can be affected by swapping in the original T-index. Then, we insert 007 into path 6 Assume that some dimensions pi, pi+1, ..., pj−1 pj have 5 Although dimension list is a set, when the order is impor- the same frequency. If we insert one triple hs, p, oi, we need tant, we specify them as a list enclosed in ( ). to swap adjacent dimensions several times. 5
MS(O4) MS(O6) hasName gender bornOnDate title L hasName gender bornOnDate diedOnDate L MS(O4) MS(O4) Abraham Male 1865-04-15 President {001} Marilyn Female 1926-06-01 1962-0805 {009} hasName gender bornOnDate title L hasName gender bornOnDate diedOnDate L Lincoln Monroe Abraham Male 1865-04-15 President {001} Marilyn Female 1926-06-01 1962-0805 {001} root O Lincoln Monroe 0 root O0 {001,002,003,004,005,007,009} hasName foundingYear O1 O9 {001,002,003,004,005,007,009} {006,008} hasName foundingYear O1 O9 {001,005,007,009} {006,008} gender foundingYear O2 O8 {001,005,007,009} {002,004} Dimension gender foundingYear List DL O O 2 8 {002,004} Dimension {001,005,007} hasName:7 title bornOnDate List DL 0 00 O3 O foundingYear:4 {001,005,007, 009} hasName:7 3 {009} bonOnDate title gender:4 O O foundingYear:4 3 {7005,007} {001} title:3 bornOnDate diedOnDate gender:4 0 O O6 bornOnDate:2 {001,005,007} 4 {009} title diedOnDate bornOnDate:2 diedOnDate:2 O O 4 6 {009} title:3 {001} diedOnDate diedOnDate:2 O {001} 5 diedOnDate O5 0 MS(O3) hasName gender title L MS(O3) MS(O7) Reese Female Actress {005} hasName gender bornOnDate L hasName gender title L Witherspoon Abraham Male 1985-04-15 {005} Reese Female Actress {005} Franklin D. Male President {007} Lincoln Witherspoon Roosevelt Marilyn Female 1926-07-01 {009} Franklin D. Male President {007} Abraham Male President {001} Monroe Roosevelt Lincoln
(a) Phase 1 (b) Phase 2
Fig. 25: Deletion of triple hy:Reese Witherspoon, bornOnDate, “1976-03-22”i
pi and pj: (1) path has both dimensions pi and pj, inal T-index, which is a case belonging to the third cate- 0 i.e., path has dimensions (p1, .., pi, pj, ..., pm); (2) path gory, we merge node Oj and Om. Specifically, we remove 0 shares a prefix (p1, ..., pi) with a path in the first cat- Om and move Om’s child nodes to be Oi’s child nodes. 0 0 egory; and (3) path shares a prefix (p1, ..., pi−1) with Then, we compute MS(Oi) = MS(Oi) ∪ MS(Om). a path in the first category and pi−1’s child is pj. We Specifically, for each aggregate tuple t in MS(Om), we 0 0 discuss below how to update the three categories of af- check if there exists an aggregate tuple t in MS(Oi), fected path. where t.D = t0.D. If so, we set t0.L = t0.L ∪ t.L. Other- 0 We first locate all paths in the first category, i.e., wise, we insert t into MS(Oi) directly. find all paths that have both dimensions pi and pj. Con- Consider deleting the triple hy:Reese Witherspoon, sider one such path H with dimensions (p1, ..., pi, pj, ..., pn). bornOnDate, “1976-03-22”i from the RDF triple table Nodes Oi and Oj have dimensions pi and pj, respec- T shown in Figure 1a. This changes the order of “ti- 0 0 tively. We rename Oi as Oi and Oj as Oj. Moreover, tle” and “bornOnDate” in dimension list DL, since the 0 0 we set Oi’s corresponding dimension to be pj and Oj’s frequency of “bornOnDate” is changed to 2. We first corresponding dimension to be pi. We compute aggre- assume that the order does not change, and we delete 0 gated set MS(Oi) by projecting MS(Oj) over dimen- the triple using the method in Section 13. Figure 25a sions (p , ..., p , p ), i.e., MS(O0) = Q 1 i−1 j i (p1,...,pi−1,pj ) shows the updated T-index after the first phase. MS(O ) and set MS(O0 ) = MS(O ), since they have j j j Now, we swap “title” and “bornOnDate”. Path H1 the same group by dimensions. =“O0− O1 − O2 − O3 − O4 − O5” is a path in the first If node Oi has a branch (i.e. Oi has other child nodes category, since O3 and O4’s corresponding dimensions in addition to Oj) in the original T-index, which is ex- are “bornOnDate” and “title”. Path H2 =“O0 − O1 − actly the case belonging to the second category, we in- O2 − O3 − O6” is a path in the second category, since 00 troduce a node Oi as a child of Oi−1, where Oi−1 is a it shares prefix (O0 − O1 − O2 − O3) with H1. Path parent of Oi in the original T-index, and set the dimen- H3=“O0 − O1 − O2 − O7” is one path in the third cate- 00 sion in Oi as pi. We move all child nodes of Oi except gory, since it shares prefix “O0 − O1 −O2” with H1 and 00 for Oj to be children of Oi . We compute aggregate set O2’s child node’s dimension is “title”. 00 Q MS(Oi ) = MS(Oi) \ MS(Oj). Specifically, (p1 ,...,pi) In the first step of phase 2, we first find the path for each aggregate tuple t in MS(Oi), we check whether “O0 − O1 − O2 − O3 − O4”, where the dimensions in 0 Q there exists an aggregate tuple t ∈ MS(Oj), (p1 ,...,pi) O3 and O4 are “bornOnDate” and “title”, respectively. 0 0 where t.D = t .D. If there exists such an aggregate Then, we rename O4 as O4 with dimension “bornOn- 00 0 tuple, we generate a new aggregate tuple t , where Date”, and rename O3 as O3 with dimension “title”. 00 00 0 00 0 0 Q t .D = t.D, and t .L = t.L \ t .L, and insert t into MS(O4) = MS(O4) and MS(O3) = (hasName,gender,title) 00 00 MS(Oi ). Otherwise, we insert t into MS(Oi ) directly. MS(O4). Since transaction 009 does not have dimen- 00 00 If node Oi−1 has a child Om whose corresponding sion “title”, we introduce a new node O3 , and MS(O3 ) = Q dimension is pj, where Oi−1 is a parent of Oi in the orig- MS(O3)− (hasName,gender,bornOnDate) MS(O4). Finally, 6
0 we merge nodes O7 with node O3, since they share the BVS∗-Query VS∗-Query same prefixes. Figure 25b shows the final updated T- 4,000 index. 3,000
D. Additional Experiments 2,000
Effect of Pruning Power 1,000 Theorem 1 shows that CL (matches of Q∗ over G∗) is Query Response Time (in ms) 0 a subset of RS (matches of Q over G). Figure 26 shows A1 A2 B1 B2 B3 C1 C2 both |CL| and |RS| of queries over Yago dataset. We Queries find that |CL| < 3 × |RS|, which indicates the low cost Fig. 27: BVS∗-Query vs VS∗-query of the verification process in our method.
2. Therefore, VS∗-query algorithm in our gStore system RS 800 is much faster than S-tree+Join method, as shown in CL Figure 28a. Given a sample query in our experiment in 600 Figure 28b, we show the candidate sizes for each vertex ∗ vi in Q using S-tree and VS -tree for pruning, respec- 400 tively. Obviously, VS∗-tree provides stronger pruning power. Thus, VS∗-query algorithm is much faster than 200 S-tree+Join. Query Response Time (in ms)
0 A1 A2 B1 B2 B3 C1 C2 Queries 6,000 ?x y:hasGivenName ?x5 S-tree+Join ?x1 y:hasFamilyName ?x6 5,000 gStore ?x1 rdf:type
S-tree+Join VS. gStore Tables 16 and 17 depict the effect of different δ values on the number of 1’s at each node of the VS∗-tree. The In order to find matches of Q∗ over G∗, a straightfor- lower the number of 1’s, the better is the pruning power. ward method can work as follows: for each vertex vi in ∗ Q , we can employ S-tree to find Ri = {ui1 , ..., uin }, 13.0.1 Evaluating Dimension Orders in DL ∗ where vSig(uij )&vSig(vi) = vSig(vi) and uij ∈ G . Then, according to the structure of Q∗, we join these In this subsection, we evaluate the affect of the order ∗ ∗ lists Ri to find matches of Q over G . A key problem of dimensions DL using both Yago and LUBM datasets is that |Ri| may be very large. Consequently, it is quite (aggregate LUBM queries are given in Table 13 in On- expensive to join Ri. According to our experiments in line Supplements). Table 18 shows that the descending Yago, |Ri| >1000 in many queries. Different from S- frequency order leads to the least number of nodes in T- tree+Join method, |Ri| is shrunk according to Theorem index, the minimal total size and the minimal building 7
gStore RDF-3X SW-Store x-RDF-3x 6,000 BigOWLIM GRIN 5,000
4,000
3,000
2,000
Query Response1 Time (in, ms) 000
0 Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Queries
Fig. 29: Exact Queries over LUBM Dataset
Table 16: Yago Data – No. of 1’s at Various Levels Table 17: DBLP Data – No. Of 1’s at Various Levels
δ 1st level 2nd level 3rd level δ 1st level 2nd level 3rd level 80 221 211 181 80 183 139 103 70 191 172 159 70 161 120 91 60 181 161 151 60 153 115 82 50 173 155 145 50 140 109 75 40 167 153 142 40 138 105 73 30 165 149 138 30 135 102 68 time, since this order ensures that more prefixes can Table 19: Evaluating Dimension Orders in DL be shared among different transactions. Furthermore, Query Response Time (msec) Number of Accessed Nodes Dimension we find that T-index in LUBM has a small number of Yago LUBM Yago LUBM Order nodes. This is because LUBM data is more structured SA1 SA2 SA3 SA1 SA2 SA3 SA1 SA2 SA3 SA1 SA2 SA3 Frequency than Yago data, since all RDF triples are generated Decreas- 52 163 951 19 28 160 4 5 3 2 1 3 following an ontology in LUBM. The structured data ing Frequency causes most entities to have almost equal dimensions; Increas- 150 236 1223 31 45 235 34 72 8 3 2 3 ing thus, they introduce few branches in T-index, i.e., the Random 67 200 1103 27 37 189 7 36 5 2 2 2 number of nodes is small.
that SA query on T-index using descending frequency Table 18: Evaluating T-index order has the fastest response time, since it accesses the
Construction Time (sec) Index size (MB) Node # minimal number of nodes in T-index. T-index Dimension Furthermore, our experiments show that the ran- Datasets T- Aggregate Total (including Aggregate Total T- Order index Sets Time vertex lists in Sets Size index dom choice of the order for dimensions with the same each node Frequency frequency does not affect online performance, because Decreas- 28 41 69 17 158 175 672 only a few dimensions have the same frequency, and T- Yago ing Frequeny index’s size and node numbers are similar in different Increas- 35 58 93 19 210 229 810 ing arbitrarily defined orders. Random 32 52 80 18 180 198 750 Frequency Decreas- 48 24 72 16 249 265 15 Additional Scalability Experiments LUBM ing Frequeny Increas- 53 36 89 20 287 307 22 ing The full set of scalability experiments involving gStore, Random 50 31 81 18 271 289 16 RDF-3x, and Virtuoso are given in Table 20. The first and last columns are identical to what is reported in Table 4 in Section 11.8; they are included here for com- We evaluate the online performance of different di- pleteness. mension orders in answering SA queries. Table 19 shows 8
Table 20: Scalability of Query Performance on LUBM
Query Response Time (msec) Queries LUBM-100 LUBM-200 LUBM-500 LUBM-1000 gStore RDF-3x Virtuoso gStore RDF-3x Virtuoso gStore RDF-3x Virtuoso gStore RDF-3x Virtuoso Q1 1310 22634 2152 4146 44589 11653 24691 99231 28252 43832 202728 54460 Q2 236 4276 30560 393 7595 71807 857 7084 >30 mins 1563 15008 > 30 mins Q3 208 216 1981 297 480 2175 688 860 4868 1491 1737 7784 Q4* 153 44 1341 162 52 1358 311 6 1357 680 30 1357 Q5* 129 48 312 163 15 312 246 4 328 131 12 243 Q6* 227 20 843 367 29 827 726 20 827 828 49 842 Q7 1016 960 5257 1786 2901 8861 4859 5513 19546 8301 14072 36848 Note that ∗ means that the query contains at least one constant entity.