Reachability Queries in Very Large Graphs: A Fast Refined Online Search Approach Renê R. Veloso1, Loïc Cerf2, Wagner Meira Jr2, Mohammed J. Zaki3,4 1Universidade Est. Montes Claros, Brazil 2Universidade Federal de Minas Gerais, Belo Horizonte MG, Brazil 3Qatar Computing Research Institute, Doha, Qatar 4Rensselaer Polytechnic Institute, Troy NY, USA [email protected], {lcerf,meira}@dcc.ufmg.br, [email protected] ABSTRACT A key problem in many graph-based applications is the need to know, given a directed graph G and two vertices u, v ∈ G, whether there is a path between u and v, i. e., if u reaches v. This problem is particularly challenging in the case of very large real-world graphs. A common approach is the pre- processing of the graphs, in order to produce an efficient index structure, which allows fast access to the reachability Figure 1: Reachability query tradeoffs information of the vertices. However, the majority of exist- ing methods can not handle very large graphs. We propose, in this paper, a novel indexing method called FELINE (Fast rEfined onLINE search), which is inspired by Dominance task even harder and demands the development of efficient Graph Drawing. FELINE creates an index from the graph methods to handle these graphs. representation in a two-dimensional plane, which provides A common problem in several graph-based applications is reachability information in constant time for a significant to verify whether a vertex is reachable from another. Let G V,E V portion of queries. Experiments demonstrate the efficiency = ( ) be a directed graph, with being its set of E V 2 of FELINE compared to state-of-the-art approaches. vertices and ⊆ its set of edges. A reachability query r(u, v) asks whether a vertex v ∈ V is reachable from a vertex u ∈ V , i. e., whether there is a path from u to v in G Categories and Subject Descriptors (see Definition 1). This decision problem is often found in H.2 [Database Management]: Systems; G.2.2 [Discrete social networks analysis, where, for instance, it is necessary Mathematics]: Graph Theory—graph labeling, graph algo- to learn whether there is a relationship between two entities, rithms, path and circuit problems for security reasons, to provide conditional access to shared resources or in business intelligence [2, 13]. Reachability is General Terms also useful to determine the visibility between websites for link structure analysis [25]. Algorithms, Performance Definition 1 (Reachability). Keywords 2 Reachability queries, online search, graph indexing ∀(u, v) ∈ V , v is reachable from u, denoted r(u, v), u = v 1. INTRODUCTION if and only if or . Developing scalable methods for the analysis of large sets ∃(u, w) ∈ E ∧ r(w, v) of graphs, including graphs that model biological and chem- ical structures is a challenging task. Further, the continuous Definition 1 shows the reachability relation r. This reach- growth of the size of the graphs, mainly in the web context, ability relation trivially is reflexive and transitive. It is not where they can reach tens of millions of nodes, makes such a antisymmetric, hence not a partial order, because G may contain cycles. That is why the approaches first turn G acyclic, i. e., they build the graph G′ = (V ′,E′) by con- densation (folding every strongly connected component of G into one single vertex in V ′ and retaining in E′ the edges be- tween those components). Tarjan’s algorithm [30] is used to identify the strongly connected components, i. e., the func- (c) 2014, Copyright is with the authors. Published in Proc. 17th Inter- tion scc : V → V ′. Its time complexity is O(|V | + |E|). By national Conference on Extending Database Technology (EDBT), March using the acyclic graph G′, the answer to the reachability 24-28, 2014, Athens, Greece: ISBN 978-3-89318065-3, on OpenProceed- query from u ∈ V to v ∈ V (in G) is the same as the answer ings.org. Distribution of this paper is permitted under the terms of the Cre- ′ ′ ative Commons license CC-by-nc-nd 4.0 to the reachability query from scc(u) ∈ V to scc(v) ∈ V 511 10.5441/002/edbt.2014.46 (in G′). However the reachability relation in G′ now is a 2. RELATED WORK partial order. As mentioned, Jin et al. [18] divides the existing ap- We follow the convention of existing approaches and as- proaches into three classes: Transitive Closure Compression, sume that all input graphs have been transformed into their ′ Hop Labeling and Refined Online Search. The methods of corresponding DAGs, i. e., we will refer the graph G as the the first class [1, 8, 21, 26, 33, 34] compress the transitive DAG G, omitting the application of function scc(u) to every closure of edges to assign to each vertex u a reachability com- vertex u ∈ V . Thus, we will discuss methods for reachability pressed set. Thus, we determine the reachability between only on DAGs. two vertices by verifying whether the destination vertex is As explained by Yildirim et al. [35], there are two basic in the reachability compressed set of the origin vertex. The approaches to answer reachability queries on DAGs, which second class is called Hop Labeling [10, 11, 16, 20, 27] and are in two extremes of a spectrum (see Figure 1 extracted its methods use intermediate vertices to encode the reach- from [35]). The first approach (left side) is to pre-compute ability between two other vertices. In general, each node and store the full transitive closure of edges, which allows maintains a list of intermediate vertices that it can reach, constant time queries, but requires a quadratic space com- and a list of intermediate vertices that can reach it. Con- plexity, making it infeasible to maintain the index in the sequently, to answer the queries, a join operation is made case of very large graphs. The second approach (right side) between the lists to determine the occurrence of vertices in is to employ a DFS or BFS search to verify the reachability, common. In the third class, as proposed by [7, 28, 31, 35], starting from vertex u to vertex v. This approach requires the approaches are based on the use of online searches. In O(|V | + |E|) time for each query, which is often unaccept- this case, they use a scheme for labeling the vertices in order able. There are some alternate approaches between these to prune aggressively the search space, minimizing the num- two basic ones, including ours, which aim to obtain a small ber of vertices to be expanded. However, the original graph index that is able to answer most queries in O(1) time. must be maintained in the memory to enable the searches, In a recent work, Jin et al. [18] divide the existing alter- which is usually in DFS fashion. Thus, these approaches nate approaches into three classes: Transitive Closure Com- have the advantage of not requiring prior computation of pression, Hop Labeling and Refined Online Search. How- the transitive closure in the construction of the index, and ever, the same work states that the manipulation of large have a more intuitive index construction step, which allows real-world graphs is not supported by the majority of the its application to very large graphs, and the query costs, in methods. In this paper, we demonstrate that it is possible the worst case, grow proportionally to the size of the graph. to create a new and efficient reachability index for very large Despite all the existing approaches, Yildirim et al. [35] graphs, that is simple to understand and easy to implement. showed in their work that very large graphs are not sup- This new index is inspired by Weak Dominance Drawing and ported by the vast majority of them. For example, the re- exposes a nice connection to graph drawing literature [3, 4, cent PATHTREE [19], that had its scalability contested [18, 14, 23]. 28, 35], since it performed well just on small graphs. In- Our reachability index represents non-planar DAGs in a deed, we found in the literature only three methods that two dimensional plane by assigning to each vertex u a unique can handle graphs with more than 100,000 edges, the Nuu- 2 coordinate (i. e., a pair of integers (xu,yu)) in the N plane, tila’s INTERVAL [26, 33], GRAIL [35], FERRARI [28] and preserving the reachability relations between most pairs of TF-Label [9]. vertices. Graphically, we translate reachability to the re- Nuutila’s INTERVAL [26, 33] extracts the complete tran- lation “being at the upper-right”, that is, both coordinates sitive closure of the graph, and uses some lists of interval rep- of a vertex v, reachable from another vertex u, need to be resentation to compress any contiguous vertex segment. For greater than or equal to the respective coordinates of u, instance, if the transitive closure of vertex u is the set of ver- necessarily. When this geometrical relation holds, we veri- tices identified by the integers 1, 2, 3, 4, 6, 7, 8, 9, 11, 12, it can fy the reachability through a search in a reduced part of be compressed into three intervals: [1, 4],[6, 9] and [11, 12]. A the graph, identified from the index. When it does not, the bit-vector compression method, called PWAH (Partitioned non-reachability is decided in constant time. Word Aligned Hybrid compression scheme), is used to com- Based on this index, we present in this paper a novel in- press these lists of intervals, allowing efficient operation with dexing method called FELINE (Fast rEfined onLINE search).