Generalized Median Graphs: Theory and Applications∗ Lopamudra Mukherjee1 Vikas Singh1 Jiming Peng2 Jinhui Xu1 Michael J. Zeitz3 Ronald Berezney3 1Department of Computer Science & Eng. 2Industrial & Enterprise Systems Eng. State University of New York at Buffalo University of Illinois at Urbana-Champaign {lm37,vsingh,jinhui}@cse.buffalo.edu [email protected] 3Department of Biological Sciences, State University of New York at Buffalo {mjzeitz,berezney}@buffalo.edu Abstract ods, graph cuts and minimum spanning trees. In many applications, results from classical graph theory can We study the so-called Generalized Median graph be directly adapted to derive efficient solutions; an ap- problem where the task is to to construct a prototype propriate graph construction that encodes available in- (i.e., a ‘model’) from an input set of graphs. The formation presents the main challenge here. In other problem finds applications in many vision (e.g., object cases, the construction strategy is somewhat simpler. recognition) and learning problems where graphs are in- However, issues such as distortion and noise propagate creasingly being adopted as a representation tool. Exist- into the graph representations. For example, we may ing techniques for this problem are evolutionary search have multiple representations of the same object (or based; in this paper, we propose a polynomial time algo- scene) based on the number of observations (or read- rithm based on a linear programming formulation. We ings taken). We are then faced with the task of building present an additional bi-level method to obtain solu- a single composite model describing the scene in the tions arbitrarily close to the optimal in non-polynomial best possible way. Problems of this form are broadly time (in worst case). Within this new framework, one known as Prototype Learning, and require learning a can optimize edit distance functions that capture simi- model of a class given several of its members. The Gen- larity by considering vertex labels as well as the graph eralized Median Graph problem asks following ques- structure simultaneously. In context of our motivating tion: given a set of graphs, what is a median graph application, we discuss experiments on molecular im- (not necessarily from the input set) that is a good rep- age analysis problems - the methods will provide the resentative (or prototype) for the set? basis for building a topological map of all pairs of the human chromosome. We are particularly interested in this problem in the context of applications in biological image analysis – specifically, in building a topological map of chromo- 1. Introduction some organization in the human cell nucleus. A proper understanding of the chromosomal organization will Graphs are a natural choice in applications where lead to insights into developmental changes and gene information regarding object-to-object relationships regulation and interaction; an all important focus of needs to be encoded. They serve as an invariant struc- ongoing research is on its relationship to the organi- ture descriptor in object recognition and for low level zation and mutations in the genome [5, 14]. Given image representations. Recent literature in vision in- sets of nuclear images exhibiting chromosomal organi- cludes several examples of graph theoretic algorithms zation, the task is to derive a “model” that is a good for some classical problems like segmentation, image representation of the organizational relationship among denoising and stereo using ideas such as spectral meth- chromosomes. The objective effectively reduces to find- ∗Research was supported in part by NSF grants CCF- ing a median graph for a set of attributed graphs from 0546509, IIS-0713489 and NIH grant GM 072131-23. individual nuclear images. In addition, the general- ized median graph problem is also a natural formaliza- based or heuristic approaches; no combinatorial solu- tion for many problems arising in computer vision and tion paradigms are known (except the special case of structural pattern recognition as well as specific graph certain types of trees [15]). learning problems arising in drug design [6]. While the problem of generalized median graphs 1.1. Problem Description is still somewhat young, a strongly related problem, called the graph isomorphism problem, has been exten- Let a labeled undirected graph G be given as G = sively studied by the computer vision and theoretical (V, E, fv, fe), where computer science communities. The earliest papers on • V is the vertex set, E is the edge set this topic are due to Corneil and Gotlieb [4] and Ull- mann [17]. The status of the problem is interesting – • L is the set of node labels, L is the set of edge V E no polynomial time algorithms are known, at the same labels, time, a formal proof of NP-hardness is also unknown. • fv : V → LV is a mapping from nodes to labels or However, a number of approaches exist for solving var- weights, and ious special cases of this problem. We will avoid an • fe : E → LE is a mapping from edges to labels or exhaustive discussion (see [16] for details) but will fo- weights. cus only on a subset of algorithms that are relevant for putting the remainder of this paper in context. Let G = (G1,G2,...,Gn) be a collection of graphs in arbitrary orientation with these properties In [18], Umeyama proposed an algorithm for graph matching based on eigen decomposition of the adja- • ∀Gi = (Vi,Ei, fv, fe), Vi ⊆ V and Ei ⊆ E. cency matrix of a graph. The technique is efficient • No restriction on the uniqueness of the vertex labels in practice but is only applicable for adjacency ma- of Vi, i.e., fv(ui) = fv(vi), ui, vi ∈ Vi is permissible trices with no repeat eigen values. The algorithm by (and likely). Almohamad and Duffuaa in [2] also employs adjacency matrices for weighted graph matching optimization. It • No restriction on the cardinality of the graphs in uses a Linear Program (LP, for short) formulation that G, i.e., |Gi| 6= |Gj|, Gi,Gj ∈ G is permissible (and nicely exploits the relationship between permutation likely). matrices and graph isomorphism as follows. ¯ The median graph, G, for G = {G1,...,Gn} must T minimize the sum of distances as follows. min kAG0 − PAG1 P k, (2) n where A and A denote the adjacency matrices of ¯ X ˆ G0 G1 G = arg min d(G, Gi) Gi ∈ G, (1) Gˆ the two edge weighted graphs and P denotes a permu- i=1 tation matrix applied to one of the graphs. Hence, the where d(·, ·) is an appropriately defined ‘distance’ func- graph matching problem reduces to finding a P that tion. In the simplest case, d(·, ·) can be the cost of the minimizes the difference of one graph (say, AG0 ) with fewest edit operations required to ‘convert’ one graph the permuted version of the other (say, P (AG1 )). to the other. Alternative definitions of distance may The algorithm in [2] cannot be directly applied to reflect a similarity measure between a pair of graphs, the isomorphism problem on general graphs with un- as we will see shortly. When G¯ ∈ G, the median graph equal number of vertices or vertex labeled graphs. The is the set median; if we waive this requirement, we get more general case of vertex labeled graphs with weighted the generalized median graph problem. edges adds a new realm of complexity to the problem. To see this, let us consider the factors contributing to 1.2. Previous works the edit cost in general graphs. Normally, ‘edits’ are Generalized Median graphs were recently introduced performed on nodes and edges and can be broadly clas- by Jiang, M¨unger, and Bunke [10]. Their solution in- sified as insertion and deletion of nodes (fv(u)), edges volved a genetic search based algorithm; however, the (fe(e)) and substitution of nodes (dv(fv(u1), fv(u2))) work was significant because it provided a link be- and edges (de(fe(e1), fe(e2))). The problem of gener- tween problems of model construction (given a set) alized graph isomorphism becomes rather ill-posed for and median graphs. A subsequent paper by Hlaoui the case where the costs (for nodes and edges) are not and Wang [8] relied on certain application-specific hy- defined in the same metric space. To motivate this ar- potheses where local choices that improved the objec- gument, let us consider an illustrative example. In Fig. tive function were iteratively picked. The reader may 1, we seek to match graphs G0 and G1 in a minimal edit notice that despite the fact that the median graph cost sense. Assuming vertex substitution has unit cost problem allows a precise combinatorial graph theoretic (Hamming distance) and edge replacement costs are in definition, both existing techniques are soft computing L1 or L2 space, the matching will favor mapping 4abc in G0 to 4def in G1 instead of 4abc. Clearly, the cordance with the edit cost definitions, but in most ap- matching is a trade-off between competing influences. plications, the semantic meaning of a vertex in a graph is as important as its structural relationship with other nodes. For example, in matching graphs that represent chemical compounds (e.g., Lewis structure), should we match a ‘C’ (carbon) to a ‘H’ (hydrogen) simply be- cause they share bonds with the same number (degree) of atoms? Figure 1: Matching with labels and weighted edges. The analysis above indicates that using edit distance as a cost function in many applications yields a bi- For the special case of graphs with labeled vertices ased weighted matching where a degree mismatch has a and unweighted edges, Justice and Hero [11] very re- higher penalty. Therefore, we must somehow re-weight cently proposed a modification of the algorithm in [2] the cost of replacing vertices to reflect the vertex labels to define the Graph Edit Distance in terms of an In- as well as the associated edge information (structure) teger Linear Programming formulation.