Pattern Recognition 40 (2007) 1042–1056 www.elsevier.com/locate/pr A Riemannian approach to graph embedding Antonio Robles-Kellya,∗, Edwin R. Hancockb aNICTA, Building 115, ANU, ACT 0200, Australia1 bDepartment of Computer Science, University of York, York YO1 5DD, UK Received 20 April 2006; accepted 24 May 2006 Abstract In this paper, we make use of the relationship between the Laplace–Beltrami operator and the graph Laplacian, for the purposes of embedding a graph onto a Riemannian manifold. To embark on this study, we review some of the basics of Riemannian geometry and explain the relationship between the Laplace–Beltrami operator and the graph Laplacian. Using the properties of Jacobi fields, we show how to compute an edge-weight matrix in which the elements reflect the sectional curvatures associated with the geodesic paths on the manifold between nodes. For the particular case of a constant sectional curvature surface, we use the Kruskal coordinates to compute edge weights that are proportional to the geodesic distance between points. We use the resulting edge-weight matrix to embed the nodes of the graph onto a Riemannian manifold. To do this, we develop a method that can be used to perform double centring on the Laplacian matrix computed from the edge-weights. The embedding coordinates are given by the eigenvectors of the centred Laplacian. With the set of embedding coordinates at hand, a number of graph manipulation tasks can be performed. In this paper, we are primarily interested in graph-matching. We recast the graph-matching problem as that of aligning pairs of manifolds subject to a geometric transformation. We show that this transformation is Pro-crustean in nature. We illustrate the utility of the method on image matching using the COIL database. ᭧ 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. Keywords: Graph embedding; Riemannian geometry; Combinatorial Laplacian 1. Introduction corresponding to the isometric mapping of these pairwise distances onto an n-dimensional Euclidean space. The problem of embedding relational structures onto a Once the nodes of a graph have been embedded on a Riemannian manifold for the purposes of representation manifold, then the graph can be manipulated by applying or visualisation is an important one in computer science geometric methods to the embedded points. For instance, by which arises in parameterisation of three-dimensional data altering the curvature of the manifold, then the graph may [1], multidimensional scaling (MDS) [2] and graph drawing be deformed to emphasise different aspects of the set of [3]. In general, this problem is clearly one of a combina- adjacency relations, and this may be useful for the purposes torial nature which has been traditionally solved by view- of visualisation. The problem of locating a spanning tree ing the edge-weights for the graph as distances between becomes that of searching for a geodesic path. Finally, the pairs of nodes. The embedding coordinates are then those problem of graph-matching can be transformed into that of point-set alignment. In general, of course, the manifold on which the graph is embedded can have a rather complex structure. In this ∗ Corresponding author. Tel.: +61 261251384. paper, our aim is to explore whether a simple embedding E-mail addresses: [email protected] that restricts the manifold to be of constant sectional curva- (A. Robles-Kelly), [email protected] (E.R. Hancock). 1 National ICT Australia is funded by the Australian Governments Back- ture can yield useful results. The choice of the sign of the ing Australia’s Ability initiative, in part through the Australian Research sectional gives considerable freedom, and, as we will see Council. later, the embeddings that result from the choice of negative 0031-3203/$30.00 ᭧ 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.patcog.2006.05.031 A. Robles-Kelly, E.R. Hancock / Pattern Recognition 40 (2007) 1042–1056 1043 curvature and a hyperbolic geometry are very different to methods [14], and exploit information conveyed by the those that result from a positive curvature and an elliptic eigenvalues and eigenvectors of the adjacency matrix. There geometry. have been successful attempts to use spectral methods for Although we also investigate the problems of relational both structural graph-matching [15], and for point-pattern deformation and spanning tree recovery as applications of matching [16,17]. For instance Umeyama [15] has devel- our embedding method, in this paper we focus mainly on oped a method for finding the permutation matrix which the problem of matching. Hence, in the remainder of this best matches pairs of weighted graphs of the same size, introductory section we review the literature related to em- by using a singular value decomposition of the adjacency bbeding graphs onto manifolds, and then focus on how the matrices. Scott and Longuet-Higgins [16], on the other graph-matching problem can be posed as one of embedding. hand, align point-sets by performing singular value decom- Finally, we detail the elements of our contribution. position on a point association weight matrix. Shapiro and Brady [17] have reported a correspondence method which 1.1. Related literature relies on measuring the similarity of the eigenvectors of a Gaussian point-proximity matrix. Although Umeyama’s In the mathematics literature, there is a considerable body algorithm [15] is elegant in its formulation and can deal with of work aimed at understanding how graphs can be embed- both weighted or unweighted graphs, it cannot be applied ded on a manifold so as to minimise a measure of distortion. to graphs which contain different numbers of nodes and for Broadly speaking there are three ways in which the problem weighted graphs the method is susceptible to weight errors. can be addressed. First, the graph can be interpolated by a One way to overcome these problems is to cast the problem surface whose genus is determined by the number of nodes, of recovering correspondences in a statistical setting using edges and faces of the graph. Second, the graph can be inter- the EM algorithm [18]. However, the resulting algorithm is polated by a hyperbolic surface which has the same pattern time consuming because of its iterative character. of geodesic (internode) distances as the graph [4]. Third, a Spectral methods can be viewed as embedding the nodes manifold can be constructed whose triangulation is the sim- of a graph in a space spanned by the eigenvectors of the ad- plicial complex of the graph [5]. A review of methods for jacency matrix. In the case of the Umeyama algorithm [15], efficiently computing distance via embedding is presented matching is effected by finding the transformation matrix in the recent paper of Hjaltason and Samet [6]. that best aligns the embedded points. Shapiro and Brady In the pattern analysis community, there has recently been [17] algorithm finds correspondences by seeking the closest renewed interest in the use of embedding methods moti- embedded points. Kosinov and Caelli [19] have improved vated by graph theory. One of the best known of these is this method by allowing for scaling in the eigenspace. The ISOMAP [7]. Here a neighbourhood ball is used to convert resulting graph-matching methods effectively involve locat- data-points into a graph, and Dijkstra’s algorithm is used to ing correspondences using a simple closest point alignment compute the shortest (geodesic) distances between nodes. principal. However, there is clearly scope for using more so- By applying MDS to the matrix of geodesic distances the phisticated alignment methods. The task of matching point manifold is reconstructed. The resulting algorithm has been patterns is of pivotal importance in a number of problems demonstrated to locate well-formed manifolds for a number in image analysis and pattern recognition. The problem is of complex data sets. Related algorithms include locally lin- to find matches between pairs of point-sets when there is ear embedding which is a variant of PCA that restricts the noise, geometric distortion and structural corruption. The complexity of the input data using a nearest neighbour graph problem arises in shape analysis [20], motion analysis [21] [8], and the Laplacian eigenmap that constructs an adjacency and stereo reconstruction [17]. Alignment is concerned weight matrix for the data-points and projects the data onto with the recovery of the geometric transformation that the principal eigenvectors of the associated Laplacian ma- minimises an error functional. If the transformation is a trix (the degree matrix minus the weight matrix) [9]. Hein simple isometry, then one of the most elegant and effec- et al. [10] have established the degree of point-wise consis- tive methods is to use Procrutes alignment [22,23]. This tency for graph Laplacians with data-dependent weights to a involves co-centring and scaling the point-sets so that they weighted Laplace operator. Collectively, these methods are have the same variance. The rotation matrix is found by sometimes referred to as manifold learning theory. performing singular value decomposition on the point cor- Embedding methods can also be used to transform the relation matrix, and taking the outer product of the singular graph-matching problem into one of point-pattern align- vectors. ment. There is a considerable literature on the problem of graph-matching, and many contrasting methods including 1.2. Contribution search [11], relaxation [12] and optimisation [13] have been successfully used. However, the main challenge in graph- Our aim in this paper is to seek an embedding of the matching is how to deal with differences in node and edge nodes of a graph which allows matching to be effected us- structure. One of the most elegant recent approaches to ing simple point-pattern matching methods.