Fast Approximate Nearest Neighbor Search with the Navigating Spreading-Out Graph

Fast Approximate Nearest Neighbor Search with the Navigating Spreading-Out Graph

Fast Approximate Nearest Neighbor Search With The Navigating Spreading-out Graph ∗ Cong Fu, Chao Xiang, Changxu Wang, Deng Cai The State Key Lab of CAD&CG, College of Computer Science, Zhejiang University, China Alibaba-Zhejiang University Joint Institute of Frontier Technologies Fabu Inc., Hangzhou, China ffc731097343, changxu.mail, [email protected]; [email protected] ABSTRACT Approximate nearest neighbor search (ANNS) has been Approximate nearest neighbor search (ANNS) is a funda- a hot topic over decades and provides fundamental support mental problem in databases and data mining. A scalable for many applications in data mining, databases, and in- ANNS algorithm should be both memory-efficient and fast. formation retrieval [2, 10, 12, 23, 37, 42]. For sparse discrete Some early graph-based approaches have shown attractive data (like documents), the nearest neighbor search can be theoretical guarantees on search time complexity, but they carried out efficiently on advanced index structures (e.g., all suffer from the problem of high indexing time complexity. inverted index [35]). For dense continuous vectors, various Recently, some graph-based methods have been proposed solutions have been proposed such as tree-structure based to reduce indexing complexity by approximating the tradi- approaches [2,6,8,17,24,36], hashing-based approaches [18, tional graphs; these methods have achieved revolutionary 20, 23, 32, 40], quantization-based approaches [1, 19, 26, 39], performance on million-scale datasets. Yet, they still can and graph-based approaches [3, 21, 33, 41]. Among them, not scale to billion-node databases. In this paper, to further graph-based methods have shown great potential recently. improve the search-efficiency and scalability of graph-based There are some experimental results showing that the graph- methods, we start by introducing four aspects: (1) ensur- based methods perform much better than some popular al- ing the connectivity of the graph; (2) lowering the average gorithms from other types in the commonly used Euclidean out-degree of the graph for fast traversal; (3) shortening Space [2, 7, 15, 27, 33, 34]. The reason may be that these the search path; and (4) reducing the index size. Then, methods cannot express the neighbor relationship as well as we propose a novel graph structure called Monotonic Rela- the graph-based methods and they tend to check much more tive Neighborhood Graph (MRNG) which guarantees very points in neighbor-subspaces than the graph-based methods low search complexity (close to logarithmic time). To fur- to reach the same accuracy [39]. Thus, their search time ther lower the indexing complexity and make it practical complexity involves large factors exponential in the dimen- for billion-node ANNS problems, we propose a novel graph sion and leads to inferior performance [22]. structure named Navigating Spreading-out Graph (NSG) by Nearest neighbor search via graphs has been studied for decades [3,13,25]. Given a set of points S in the d-dimensional approximating the MRNG. The NSG takes the four aspects d into account simultaneously. Extensive experiments show Euclidean space E , a graph G is defined as a set of edges that NSG outperforms all the existing algorithms signifi- connecting these points (nodes). The edge pq defines a cantly. In addition, NSG shows superior performance in the neighbor-relationship between node p and q. Various con- E-commercial scenario of Taobao (Alibaba Group) and has straints are proposed on the edges to make the graphs suit- been integrated into their billion-scale search engine. able for ANNS problem. These graphs are now referred to as the Proximity Graphs [25]. Some proximity graphs like De- PVLDB Reference Format: launay Graphs (or Delaunay Triangulation) [4] and Mono- Cong Fu, Chao Xiang, Changxu Wang, Deng Cai. Fast Approx- tonic Search Networks (MSNET) [13] ensure that from any imate Nearest Neighbor Search With The Navigating Spreading- node p to another node q, there exists a path on which the out Graph. PVLDB, 12(5): 461-474, 2019. DOI: https://doi.org/10.14778/3303753.3303754 intermediate nodes are closer and closer to q [13]. However, the computational complexity needed to find such a path is not given. Other works like Randomized Neighborhood 1. INTRODUCTION Graphs [3] guarantee polylogarithmic search time complex- ∗ ity. Empirically, the average length of greedy-routing paths Corresponding author. grows polylogarithmically with the data size on the Navi- gable Small-World Networks (NSWN) [9, 29]. However, the time complexity of building these graphs is very high (at This work is licensed under the Creative Commons Attribution- least O(n2)), which is impractical for massive problems. NonCommercial-NoDerivatives 4.0 International License. To view a copy Some recent graph-based methods try to address this prob- of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0/. For any use beyond those covered by this license, obtain permission by emailing lem by designing approximations for the graphs. For exam- [email protected]. Copyright is held by the owner/author(s). Publication rights ple, GNNS [21], IEH [27], and Efanna [15] are based on licensed to the VLDB Endowment. the kNN graph, which is an approximation of the Delau- Proceedings of the VLDB Endowment, Vol. 12, No. 5 nay Graph. NSW [33] approximates the NSWN, FANNG ISSN 2150-8097. [7] approximates the Relative Neighborhood Graphs (RNG) DOI: https://doi.org/10.14778/3303753.3303754 461 Algorithm 1 Search-on-Graph(G, p, q, l) MSNET. Based on this, we propose a novel graph, Require: graph G, start node p, query point q, candidate pool MRNG, which ensures a close-logarithmic search com- size l plexity in expectation. Ensure: k nearest neighbors of q 1: i=0, candidate pool S = ; 2. To further improve the efficiency and scalability of 2: S.add(p) 3: while i < l do graph-based ANNS methods, we consider four aspects 4: i =the index of the first unchecked node in S of the graph: ensuring connectivity, lowering the av- 5: mark pi as checked erage out-degree, shortening the search path, and re- 6: for all neighbor n of pi in G do ducing the index size. Motivated by these, we design 7: S.add(n) a close approximation of the MRNG, called Navigat- 8: end for ing Spreading-out Graph (NSG), to address the four 9: sort S in ascending order of the distance to q 10: If S.size() > l, S.resize(l) aspects simultaneously. The indexing complexity is 11: end while reduced significantly compared to the MRNG and is 12: return the first k nodes in S practical for massive problems. Extensive experiments show that our approach outperforms the state-of-the- art methods in search performance with the smallest memory usage among graph-based methods. [38], and Hierarchical NSW (HNSW) [34] is proposed to take advantage of properties of the Delaunay Graph, the 3. The NSG algorithm is also tested on the E-commercial NSWN, and the RNG. Moreover, the hierarchical structure search scenario of Taobao (Alibaba Group). The algo- of HNSW enables multi-scale hopping on different layers. rithm has been integrated into their search engine for These approximations are mainly based on intuition and billion-node search. generally lack rigorous theoretical support. In our experi- mental study, we find that they are still not powerful enough for billion-node applications, which are in great demand to- 2. PRELIMINARIES day. To further improve the search-efficiency and scalability d We use E to denote the Euclidean space under the l2 of graph-based methods, we start with how ANNS is per- norm. The closeness of any two points p; q is defined as the formed on a graph. Despite the diversity of graph indices, l distance, δ(p; q), between them. almost all graph-based methods [3,7,13,21,27,33] share the 2 same greedy best-first search algorithm (given in Algorithm 2.1 Problem Setting 1), we refer to it as the search-on-graph algorithm below. Algorithm 1 tries to reach the query point with the fol- Various applications in information retrieval and database lowing greedy process. For a given query q, we are required management of high-dimensional data can be abstracted as to retrieve its nearest neighbors from the dataset. Given a the nearest neighbor search problem in high-dimensional starting node p, we follow the out-edges to reach p's neigh- space. The Nearest Neighbor Search (NNS) problem is de- bors, and compare them with q to choose one to proceed. fined as follows [20]: The choosing principle is to minimize the distance to q, and Definition 1 (Nearest Neighbor Search). Given a the new iteration starts from the chosen node. We can see finite point set S of n points in space Ed, preprocess S to that the key to improve graph-based search is to shorten efficiently return a point p 2 S which is closest to a given the search path formed by the algorithm and reduce the query point q. out-degree of the graph (i.e., reduce the number of choices of each node). Intuitively, to improve graph-based search we This naturally generalizes to the K Nearest Neighbor need to: (1) Ensure the connectivity of the graph to make Search when we require the algorithm to return K points sure the query (or the nearest neighbors of the query) is (are) (K > 1) which are the closest to the query point. The reachable; (2) Lower the average out-degree of the graph and approximate version of the nearest neighbor search problem (3) shorten the search path to lower the search time complex- (ANNS) can be defined as follows [20]: ity; (4) Reduce the index size (memory usage) to improve scalability.

View Full Text

Details

  • File Type
    pdf
  • Upload Time
    -
  • Content Languages
    English
  • Upload User
    Anonymous/Not logged-in
  • File Pages
    14 Page
  • File Size
    -

Download

Channel Download Status
Express Download Enable

Copyright

We respect the copyrights and intellectual property rights of all users. All uploaded documents are either original works of the uploader or authorized works of the rightful owners.

  • Not to be reproduced or distributed without explicit permission.
  • Not used for commercial purposes outside of approved use cases.
  • Not used to infringe on the rights of the original creators.
  • If you believe any content infringes your copyright, please contact us immediately.

Support

For help with questions, suggestions, or problems, please contact us