Inference and Sampling of K33-free Ising Models Valerii Likhosherstov 1 Yury Maximov 1 2 Michael Chertkov 1 2 3 Abstract one of the most significant contributions in theoretical and We call an Ising model tractable when it is possi- mathematical physics of the 20th century. Then, Kac and ble to compute its partition function value (statisti- Ward (1952) showed in the case of a finite square lattice cal inference) in polynomial time. The tractability that the problem of the partition function computation is also implies an ability to sample configurations reducible to a determinant. Kasteleyn (1963) generalized of this model in polynomial time. The notion the results to the case of an arbitrary inhomogeneous interac- of tractability extends the basic case of planar tion Ising model over an arbitrary planar graph. Kasteleyn’s zero-field Ising models. Our starting point is to construction was based on mapping of the Ising model to a describe algorithms for the basic case, computing perfect matching (PM) model with specially defined weights partition function and sampling efficiently. Then, over a modified graph. Kasteleyn’s construction was also we extend our tractable inference and sampling based on the so-called Pfaffian orientation, which allows algorithms to models whose triconnected compo- counting of PMs by finding a single Pfaffian (or determi- nents are either planar or graphs of O(1) size. In nant) of a matrix. Fisher (1966) simplified Kasteleyn’s particular, it results in a polynomial-time infer- construction such that the modified graph remained pla- nar. Transition to PM is fruitful because it extends planar ence and sampling algorithms for K33 (minor)- free topologies of zero-field Ising models—a gen- zero-field Ising model inference to models embedded on eralization of planar graphs with a potentially un- a torus (Kasteleyn, 1963) and, in fact, on any surface of bounded genus. small (orientable) genus g, but with a price of the additional, multiplicative, and exponential in genus, 4g, factor in the algorithm’s run time (Gallucio & Loebl, 1999). 1. Introduction A parallel way of reducing the planar zero-field Ising model Computing the partition function of the Ising model is gener- to a PM problem consists of constructing a so-called ex- ally intractable, even an approximate solution in the special panded dual graph (Bieche et al., 1980; Barahona, 1982; anti-ferromagnetic case of arbitrary topology would have Schraudolph & Kamenetsky, 2009). This approach is more colossal consequences in the complexity theory (Jerrum & natural and interpretable because there is a one-to-one cor- Sinclair, 1993). Therefore, a question of interest—rather respondence between spin configurations and PMs on the than addressing the general case—is to look after tractable expanded dual graph. An extra advantage of this approach families of Ising models. In the following, we briefly review is that the reduction allows one to develop an exact efficient tractability related to planar graphs and graphs embedded in sampling. Based on linear algebra and planar separator surfaces of small genus. theory (Lipton & Tarjan, 1979), Wilson introduced an al- gorithm (1997) that allows one to sample PMs over planar 3 Related work. Onsager (1944) gave a closed-form solution graphs in O(N 2 ) time. The algorithms were implemented for the partition function in the case of a homogeneous inter- in (Thomas & Middleton, 2009; 2013) for the Ising model action Ising model over an infinite two-dimensional square sampling, however, the implementation was limited to only grid without a magnetic field. This result has opened an ex- the special case of a square lattice. In (Thomas & Middleton, citing era of phase transition discoveries, which is arguably 2009) a simple extension of the Wilson’s algorithm to the 1Skolkovo Institute of Science and Technology, Moscow, Rus- case of bounded genus graphs was also suggested, again g sia 2Theoretical Division and Center for Nonlinear Studies, Los with the 4 factor in complexity. Notice that imposing zero Alamos National Laboratory, Los Alamos, NM, USA 3Graduate field condition is critical, as otherwise, the Ising model over Program in Applied Mathematics, University of Arizona, Tuc- a planar graph is NP-hard (Barahona, 1982). On the other son, AZ, USA. Correspondence to: Valerii Likhosherstov <va- hand, even in the case of zero magnetic field Ising models [email protected]>. over general graphs are difficult (Barahona, 1982). th Proceedings of the 36 International Conference on Machine Contribution. In this manuscript, we discuss tractability Learning, Long Beach, California, PMLR 97, 2019. Copyright 2019 by the author(s). related to the Ising model with zero magnetic fields over Inference and Sampling of K33-free Ising Models 0 graphs more general than planar. Our construction is related V g). Let H = (VH ;EH ) be a graph. Then H is a sub- to graphs characterized in terms of their excluded minor graph of G, if VH ⊆ V; EH ⊆ E. Vertex v 2 V is an property. Planar graphs are characterized by excluded K5 articulation point of G, if G(V n fvg) is disconnected. G is minor and K33 minor (Wagner’s theorem (Diestel, 2006), biconnected if there are no articulation points in G. Bicon- Chapter 4.4). Therefore, instead of attempting to generalize nected component is a maximal subgraph of G without an from planar to graphs embedded into surfaces of higher articulation point. genus, it is natural to consider generalizations associated The graph G is planar if it can be drawn on a plane without with a family of graphs excluding K minor or K minor. 5 33 edge intersections. The corresponding drawing is referred In this manuscript, we show that K33-free zero-field Ising to as planar embedding of G. When no ambiguity arises, models are tractable in terms of inference and sampling we do not distinguish planar graph G from its embedding. 3 and give a tight asymptotic bound, O(N 2 ), for both oper- A set E0 ⊆ E is called a perfect matching (PM) of G, if ations. For that purpose, we use graph decomposition into edges of E0 are disjoint and their union equals V . PM(G) triconnected components—the result of recursive splitting denotes the set of all PMs of G. K denotes a complete by pairs of vertices, disconnecting the graph. Indeed, the p (normal) graph on p vertices, and K denotes a utility K -free graphs are simple to work with because their tri- 33 33 graph. Triple bond is a graph of two vertices and three edges connected components are either planar or K graphs (Hall, 5 between them. Multiple bond is a graph of two vertices and 1943). Therefore, the essence of our construction is to de- at least three edges between them. compose the inference task in Ising over a K33-free graph into a sequential dynamic programming evaluation over pla- nar or K5 graphs in the spirit of (Straub et al., 2014). Notice 3. Problem Setup that the triconnected classification of the tractable zero-field Let G = (V; E) be a normal graph, jV j = N. For each Ising models is complementary to the aforementioned small v 2 V , define a random binary variable (a spin) s 2 genus classification. We illustrate the difference between the v {−1; +1g, S = (s ; :::; s ). Subscript i will be used as two classifications with an explicit example of a tractable v1 vN shorthand for v , for brevity, thus S = (s ; :::; s ). For problem over a graph with genus growing linearly with i 1 N each e 2 E, define a pairwise interaction Je 2 R. We graph size. N associate assignment X = (x1; :::; xN ) 2 {−1; +1g to Structure. The manuscript is organized as follows. Sec- vector S with probability as follows: tions2 and3, respectively, establish notations and pose 0 1 problems of inference and sampling. Section4 presents 1 X (S = X) = exp J x x ; (1) transition from the zero-field Ising model to an equivalent P Z @ e v wA tractable perfect matching (PM) model. This provides a e=fv;wg2E 3 description of a O(N 2 ) inference and sampling method in where planar models, which is new (to the best of our knowledge), 0 1 and it sets the stage for what follows. Section5 discusses a X X Z = exp @ JexvxwA : scheme for polynomial inference and sampling in zero-field X∈{−1;+1gN e=fv;wg2E models over graphs with triconnected components that are either planar or of O(1) size. Section6 applies this scheme The probability distribution (1) defines the so-called zero- to K33-free zero-field Ising models, resulting in tight asymp- field (or pairwise) Ising model, and Z is called the partition totic bounds, which appear to be equivalent to those in the function (PF) of the zero-field Ising (ZFI) model. Notice planar case. Section7 describes benchmarks justifying cor- that P(S = X) = P(S = −X). rectness and efficiency of our algorithm. Technical proofs of statements given throughout the manuscript can be found Given a ZFI model, our goal is to find Z (inference) and in the supplementary material. draw samples from the model efficiently. 2. Definitions and Notations 4. Reducing Planar ZFI Model to PM Model Let V = fv1; :::; vjV jg be a finite set of vertices, a multiset In this section, we consider a special case of planar graph G E consisting of e ⊆ V , jej = 2 be edges, then we call and introduce a transition from the ZFI model to the perfect G = (V; E) a graph. We call G normal, if E is a set (i.e., matching (PM) model on a different planar graph. there are no multiple edges in G).