Markov Logic Networks

Matthew Richardson ([email protected]) and Pedro Domingos ([email protected]) Department of Computer Science and Engineering, University of Washington, Seattle, WA 98195-250, U.S.A.

Abstract. We propose a simple approach to combining first-order logic and probabilistic graphical models in a single representation. A Markov logic network (MLN) is a first-order knowledge base with a weight attached to each formula (or clause). Together with a set of constants representing objects in the domain, it specifies a ground Markov network containing one feature for each possible grounding of a first-order formula in the KB, with the corresponding weight. Inference in MLNs is performed by MCMC over the minimal subset of the ground network required for answering the query. Weights are efficiently learned from relational databases by iteratively optimizing a pseudo-likelihood measure. Optionally, additional clauses are learned using inductive logic programming techniques. Experiments with a real-world database and knowledge base in a university domain illustrate the promise of this approach.

Keywords: Statistical relational learning, Markov networks, Markov random fields, log-linear models, graphical models, first-order logic, satisfiability, inductive logic programming, knowledge-based model construction, Markov chain Monte Carlo, pseudo-likelihood, link prediction

1. Introduction

Combining probability and first-order logic in a single representation has long been a goal of AI. Probabilistic graphical models enable us to efficiently handle uncertainty. First-order logic enables us to compactly represent a wide variety of knowledge. Many (if not most) applications require both. Interest in this problem has grown in recent years due to its relevance to statistical relational learning (Getoor & Jensen, 2000; Getoor & Jensen, 2003; Diet- terich et al., 2003), also known as multi-relational data mining (Dzeroskiˇ & De Raedt, 2003; Dzeroskiˇ et al., 2002; Dzeroskiˇ et al., 2003; Dzeroskiˇ & Blockeel, 2004). Current proposals typically focus on combining probability with restricted subsets of first-order logic, like Horn clauses (e.g., Wellman et al. (1992); Poole (1993); Muggleton (1996); Ngo and Haddawy (1997); Sato and Kameya (1997); Cussens (1999); Kersting and De Raedt (2001); Santos Costa et al. (2003)), frame-based systems (e.g., Friedman et al. (1999); Pasula and Russell (2001); Cumby and Roth (2003)), or database query languages (e.g., Taskar et al. (2002); Popescul and Ungar (2003)). They are often quite complex. In this paper, we introduce Markov logic networks (MLNs), a representation that is quite simple, yet combines probability and first-order logic with no restrictions other than finiteness of the domain. We develop

mln.tex; 26/01/2006; 19:24; p.1 2 Richardson and Domingos efficient algorithms for inference and learning in MLNs, and evaluate them in a real-world domain. A Markov logic network is a first-order knowledge base with a weight attached to each formula, and can be viewed as a template for constructing Markov networks. From the point of view of probability, MLNs provide a compact language to specify very large Markov networks, and the ability to flexibly and modularly incorporate a wide range of domain knowledge into them. From the point of view of first-order logic, MLNs add the ability to soundly handle uncertainty, tolerate imperfect and contradictory knowledge, and reduce brittleness. Many important tasks in statistical relational learning, like collective classification, link prediction, link-based clustering, social network modeling, and object identification, are naturally formulated as instances of MLN learning and inference. Experiments with a real-world database and knowledge base illustrate the benefits of using MLNs over purely logical and purely probabilistic approaches. We begin the paper by briefly reviewing the fundamentals of Markov networks (Section 2) and first-order logic (Section 3). The core of the paper introduces Markov logic networks and algorithms for inference and learning in them (Sections 4–6). We then report our experimental results (Section 7). Finally, we show how a variety of statistical relational learning tasks can be cast as MLNs (Section 8), discuss how MLNs relate to previous approaches (Section 9) and list directions for future work (Section 10).

2. Markov Networks

A Markov network (also known as Markov random ﬁeld) is a model for the joint distribution of a set of variables X = (X1, X2, . . . , Xn) ∈ X (Pearl, 1988). It is composed of an undirected graph G and a set of potential functions φk. The graph has a node for each variable, and the model has a potential function for each clique in the graph. A potential function is a non- negative real-valued function of the state of the corresponding clique. The joint distribution represented by a Markov network is given by

1 P (X =x) = φ (x ) (1) Z k {k} Yk where x{k} is the state of the kth clique (i.e., the state of the variables that appear in that clique). Z, known as the partition function, is given by Z = x∈X k φk(x{k}). Markov networks are often conveniently represented as log-linear models, with each clique potential replaced by an exponentiated P Q weighted sum of features of the state, leading to

mln.tex; 26/01/2006; 19:24; p.2 Markov Logic Networks 3

1 P (X =x) = exp w f (x) (2)  j j  Z j X   A feature may be any real-valued function of the state. This paper will focus on binary features, fj(x) ∈ {0, 1}. In the most direct translation from the potential-function form (Equation 1), there is one feature corresponding to each possible state x{k} of each clique, with its weight being log φk(x{k}). This representation is exponential in the size of the cliques. However, we are free to specify a much smaller number of features (e.g., logical functions of the state of the clique), allowing for a more compact representation than the potential-function form, particularly when large cliques are present. MLNs will take advantage of this. Inference in Markov networks is #P-complete (Roth, 1996). The most widely used method for approximate inference in Markov networks is Markov chain Monte Carlo (MCMC) (Gilks et al., 1996), and in particular Gibbs sampling, which proceeds by sampling each variable in turn given its Markov blanket. (The Markov blanket of a node is the minimal set of nodes that renders it independent of the remaining network; in a Markov network, this is simply the node’s neighbors in the graph.) Marginal probabilities are computed by counting over these samples; conditional probabilities are computed by running the Gibbs sampler with the conditioning variables clamped to their given values. Another popular method for inference in Markov networks is belief propagation (Yedidia et al., 2001). Maximum-likelihood or MAP estimates of Markov network weights can- not be computed in closed form, but, because the log-likelihood is a concave function of the weights, they can be found efﬁciently using standard gradient- based or quasi-Newton optimization methods (Nocedal & Wright, 1999). Another alternative is iterative scaling (Della Pietra et al., 1997). Features can also be learned from data, for example by greedily constructing conjunctions of atomic features (Della Pietra et al., 1997).

3. First-Order Logic

A ﬁrst-order knowledge base (KB) is a set of sentences or formulas in ﬁrst- order logic (Genesereth & Nilsson, 1987). Formulas are constructed using four types of symbols: constants, variables, functions, and predicates. Con- stant symbols represent objects in the domain of interest (e.g., people: Anna, Bob, Chris, etc.). Variable symbols range over the objects in the domain. Function symbols (e.g., MotherOf) represent mappings from tuples of objects to objects. Predicate symbols represent relations among objects in the domain (e.g., Friends) or attributes of objects (e.g., Smokes). An inter-

mln.tex; 26/01/2006; 19:24; p.3 4 Richardson and Domingos pretation specifies which objects, functions and relations in the domain are represented by which symbols. Variables and constants may be typed, in which case variables range only over objects of the corresponding type, and constants can only represent objects of the corresponding type. For example, the variable x might range over people (e.g., Anna, Bob, etc.), and the constant C might represent a city (e.g., Seattle). A term is any expression representing an object in the domain. It can be a constant, a variable, or a function applied to a tuple of terms. For example, Anna, x, and GreatestCommonDivisor(x, y) are terms. An atomic formula or atom is a predicate symbol applied to a tuple of terms (e.g., Friends(x, MotherOf(Anna))). Formulas are recursively constructed from atomic formulas using logical connectives and quantifiers. If F1 and F2 are formulas, the following are also formulas: ¬F1 (negation), which is true iff F1 is false; F1 ∧ F2 (conjunction), which is true iff both F1 and F2 are true; F1 ∨ F2 (disjunction), which is true iff F1 or F2 is true; F1 ⇒ F2 (implication), which is true iff F1 is false or F2 is true; F1 ⇔ F2 (equivalence), which is true iff F1 and F2 have the same truth value; ∀x F1 (universal quantification), which is true iff F1 is true for every object x in the domain; and ∃x F1 (existential quantification), which is true iff F1 is true for at least one object x in the domain. Parentheses may be used to enforce precedence. A positive literal is an atomic formula; a negative literal is a negated atomic formula. The formulas in a KB are implicitly conjoined, and thus a KB can be viewed as a single large formula. A ground term is a term containing no variables. A ground atom or ground predicate is an atomic formula all of whose arguments are ground terms. A possible world or Herbrand interpretation assigns a truth value to each possible ground atom. A formula is satisfiable iff there exists at least one world in which it is true. The basic inference problem in first-order logic is to determine whether a knowledge base KB entails a formula F , i.e., if F is true in all worlds where KB is true (denoted by KB |= F ). This is often done by refutation: KB entails F iff KB∪¬F is unsatisfiable. (Thus, if a KB contains a contradiction, all formulas trivially follow from it, which makes painstaking knowledge engineering a necessity.) For automated inference, it is often convenient to convert formulas to a more regular form, typically clausal form (also known as conjunctive normal form (CNF)). A KB in clausal form is a conjunction of clauses, a clause being a disjunction of literals. Every KB in first-order logic can be converted to clausal form using a mechanical sequence of steps.1 Clausal form is used in resolution, a sound and refutation-complete inference procedure for first-order logic (Robinson, 1965).

1 This conversion includes the removal of existential quantifiers by Skolemization, which is not sound in general. However, in finite domains an existentially quantified formula can simply be replaced by a disjunction of its groundings.

mln.tex; 26/01/2006; 19:24; p.4 Markov Logic Networks 5

Inference in first-order logic is only semidecidable. Because of this, knowledge bases are often constructed using a restricted subset of first-order logic with more desirable properties. The most widely-used restriction is to Horn clauses, which are clauses containing at most one positive literal. The Prolog programming language is based on Horn clause logic (Lloyd, 1987). Prolog programs can be learned from databases by searching for Horn clauses that (approximately) hold in the data; this is studied in the field of inductive logic programming (ILP) (Lavracˇ & Dzeroski,ˇ 1994). Table I shows a simple KB and its conversion to clausal form. Notice that, while these formulas may be typically true in the real world, they are not always true. In most domains it is very difficult to come up with non-trivial formulas that are always true, and such formulas capture only a fraction of the relevant knowledge. Thus, despite its expressiveness, pure first-order logic has limited applicability to practical AI problems. Many ad hoc extensions to address this have been proposed. In the more limited case of propositional logic, the problem is well solved by probabilistic graphical models. The next section describes a way to generalize these models to the first-order case.

4. Markov Logic Networks

A first-order KB can be seen as a set of hard constraints on the set of possible worlds: if a world violates even one formula, it has zero probability. The basic idea in MLNs is to soften these constraints: when a world violates one formula in the KB it is less probable, but not impossible. The fewer formulas a world violates, the more probable it is. Each formula has an associated weight that reflects how strong a constraint it is: the higher the weight, the greater the difference in log probability between a world that satisfies the formula and one that does not, other things being equal.

DEFINITION 4.1. A Markov logic network L is a set of pairs (Fi, wi), where Fi is a formula in first-order logic and wi is a real number. Together with a finite set of constants C = {c1, c2, . . . , c|C|}, it defines a Markov network ML,C (Equations 1 and 2) as follows:

1. ML,C contains one binary node for each possible grounding of each predicate appearing in L. The value of the node is 1 if the ground atom is true, and 0 otherwise.

2. ML,C contains one feature for each possible grounding of each formula Fi in L. The value of this feature is 1 if the ground formula is true, and 0 otherwise. The weight of the feature is the wi associated with Fi in L.

mln.tex; 26/01/2006; 19:24; p.5 6 Richardson and Domingos () Ca eight 1.1 1.5 2.3 1.1 0.7 W and ) , z , () x , ( ) ) y y ( ( Fr ∨ Sm Sm Smokes ) ¬ ∨ z ) , ) ∨ for x y x ) ( ( ( x () ( ) Sm Fr Sm x Sm ( ¬ Sm ¬ ∨ , ∨ ∨ ∨ Ca () )) orm ) ) ) x ∨ F y y y ( , ) , , g x x x x , ( ( ( ( x ( Fr Sm Fr Fr Friends ¬ Clausal ¬ Fr ¬ ¬ for ) z , x ( )) short y Fr ( is ) Sm () ⇒ ) ) x Fr ( z ⇔ , ) Sm y x ( ( ⇒ Fr MLN. Sm ( ) ∧ )) x y ) ( and , ⇒ y x , ) Ca ( x y Logic ( , Fr base ⇒ x Fr ( y ) ∃ x z Fr ( ( ∀ ¬ y y ( Sm ∀ ∀ wledge x x x x First-Order ∀ ∀ ∀ ∀ kno either e. . does. friends. ﬁrst-order a are smok friends, cancer of neither are or . people friends e () causes of Example people smok o I. tw Cancer both If Smoking English Friends Friendless able for T

mln.tex; 26/01/2006; 19:24; p.6 Markov Logic Networks 7

The syntax of the formulas in an MLN is the standard syntax of first-order logic (Genesereth & Nilsson, 1987). Free (unquantified) variables are treated as universally quantified at the outermost level of the formula. An MLN can be viewed as a template for constructing Markov networks. Given different sets of constants, it will produce different networks, and these may be of widely varying size, but all will have certain regularities in structure and parameters, given by the MLN (e.g., all groundings of the same formula will have the same weight). We call each of these networks a ground Markov network to distinguish it from the first-order MLN. From Defini- tion 4.1 and Equations 1 and 2, the probability distribution over possible worlds x specified by the ground Markov network ML,C is given by

1 1 ni(x) P (X =x) = exp wini(x) = φi(x{i}) (3) Z i ! Z i X Y where ni(x) is the number of true groundings of Fi in x, x{i} is the state wi (truth values) of the atoms appearing in Fi, and φi(x{i}) = e . Notice that, although we defined MLNs as log-linear models, they could equally well be defined as products of potential functions, as the second equality above shows. This will be the most convenient approach in domains with a mixture of hard and soft constraints (i.e., where some formulas hold with certainty, leading to zero probabilities for some worlds). The graphical structure of ML,C follows from Definition 4.1: there is an edge between two nodes of ML,C iff the corresponding ground atoms appear together in at least one grounding of one formula in L. Thus, the atoms in each ground formula form a (not necessarily maximal) clique in ML,C . Figure 1 shows the graph of the ground Markov network defined by the last two formulas in Table I and the constants Anna and Bob. Each node in this graph is a ground atom (e.g., Friends(Anna, Bob)). The graph contains an arc between each pair of atoms that appear together in some grounding of one of the formulas. ML,C can now be used to infer the probability that Anna and Bob are friends given their smoking habits, the probability that Bob has cancer given his friendship with Anna and whether she has cancer, etc. Each state of ML,C represents a possible world. A possible world is a set of objects, a set of functions (mappings from tuples of objects to objects), and a set of relations that hold between those objects; together with an interpretation, they determine the truth value of each ground atom. The following assumptions ensure that the set of possible worlds for (L, C) is finite, and that ML,C represents a unique, well-defined probability distribution over those worlds, irrespective of the interpretation and domain. These assumptions are quite reasonable in most practical applications, and greatly simplify the use of MLNs. For the remaining cases, we discuss below the extent to which each one can be relaxed.

mln.tex; 26/01/2006; 19:24; p.7 8 Richardson and Domingos

Friends(A,B)

Friends(A,A) Smokes(A) Smokes(B) Friends(B,B)

Cancer(A) Cancer(B) Friends(B,A)

Figure 1. Ground Markov network obtained by applying the last two formulas in Table I to the constants Anna(A) and Bob(B).

ASSUMPTION 1. Unique names. Different constants refer to different objects (Genesereth & Nilsson, 1987).

ASSUMPTION 2. Domain closure. The only objects in the domain are those representable using the constant and function symbols in (L, C) (Gene- sereth & Nilsson, 1987).

ASSUMPTION 3. Known functions. For each function appearing in L, the value of that function applied to every possible tuple of arguments is known, and is an element of C.

This last assumption allows us to replace functions by their values when grounding formulas. Thus the only ground atoms that need to be considered are those having constants as arguments. The infinite number of terms constructible from all functions and constants in (L, C) (the “Herbrand universe” of (L, C)) can be ignored, because each of those terms corresponds to a known constant in C, and atoms involving them are already represented as the atoms involving the corresponding constants. The possible groundings of a predicate in Definition 4.1 are thus obtained simply by replacing each variable in the predicate with each constant in C, and replacing each function term in the predicate by the corresponding constant. Table II shows how the groundings of a formula are obtained given Assumptions 1–3. If a formula contains more than one clause, its weight is divided equally among the clauses, and a clause’s weight is assigned to each of its groundings. Assumption 1 (unique names) can be removed by introducing the equality predicate (Equals(x, y), or x = y for short) and adding the necessary axioms to the MLN: equality is reflexive, symmetric and transitive; for each unary predicate P, ∀x∀y x = y ⇒ (P(x) ⇔ P(y)); and similarly for higher-order predicates and functions (Genesereth & Nilsson, 1987). The resulting MLN will have a node for each pair of constants, whose value is 1 if the constants represent the same object and 0 otherwise; these nodes will be connected to

mln.tex; 26/01/2006; 19:24; p.8 Markov Logic Networks 9

Table II. Construction of all groundings of a ﬁrst-order formula under Assumptions 1–3.

function Ground(F , C) inputs: F , a formula in ﬁrst-order logic C, a set of constants output: GF , a set of ground formulas calls: CNF (F, C), which converts F to conjunctive normal form, replacing existentially quantiﬁed formulas by disjunctions of their groundings over C F ← CNF (F, C) GF = ∅ for each clause Fj ∈ F Gj = {Fj } for each variable x in Fj for each clause Fk(x) ∈ Gj Gj ← (Gj \ Fk(x)) ∪ {Fk(c1), Fk(c2), . . . , Fk(c|C|)}, where Fk(ci) is Fk(x) with x replaced by ci ∈ C GF ← GF ∪ Gj for each ground clause Fj ∈ GF repeat for each function f(a1, a2, . . .) all of whose arguments are constants Fj ← Fj with f(a1, a2, . . .) replaced by c, where c = f(a1, a2, . . .) until Fj contains no functions return GF

each other and to the rest of the network by arcs representing the axioms above. Notice that this allows us to make probabilistic inferences about the equality of two constants. We have successfully used this as the basis of an approach to object identification (see Subsection 8.5). If the number u of unknown objects is known, Assumption 2 (domain closure) can be removed simply by introducing u arbitrary new constants. If u is unknown but finite, Assumption 2 can be removed by introducing a distribution over u, grounding the MLN with each number of unknown objects, and umax u computing the probability of a formula F as P (F ) = u=0 P (u)P (F |ML,C ), where M u is the ground MLN with u unknown objects. An infinite u L,C P requires extending MLNs to the case |C| = ∞. Let HL,C be the set of all ground terms constructible from the function symbols in L and the constants in L and C (the “Herbrand universe” of (L, C)). Assumption 3 (known functions) can be removed by treating each element of HL,C as an additional constant and applying the same procedure used to remove the unique names assumption. For example, with a function G(x) and constants A and B, the MLN will now contain nodes for G(A) = A, G(A) = B, etc. This leads to an infinite number of new constants, requiring the corresponding extension of MLNs. However, if we restrict the level of nesting to some maximum, the resulting MLN is still finite.

mln.tex; 26/01/2006; 19:24; p.9 10 Richardson and Domingos

To summarize, Assumptions 1–3 can be removed as long the domain is finite. We believe it is possible to extend MLNs to infinite domains (see Jaeger (1998)), but this is an issue of chiefly theoretical interest, and we leave it for future work. In the remainder of this paper we proceed under Assumptions 1– 3, except where noted. A first-order KB can be transformed into an MLN simply by assigning a weight to each formula. For example, the clauses and weights in the last two columns of Table I constitute an MLN. According to this MLN, other things being equal, a world where n friendless people are non-smokers is e(2.3)n times less probable than a world where all friendless people smoke. Notice that all the formulas in Table I are false in the real world as universally quantified logical statements, but capture useful information on friendships and smoking habits, when viewed as features of a Markov network. For example, it is well known that teenage friends tend to have similar smoking habits (Lloyd-Richardson et al., 2002). In fact, an MLN like the one in Table I succinctly represents a type of model that is a staple of social network analysis (Wasserman & Faust, 1994). It is easy to see that MLNs subsume essentially all propositional probabilistic models, as detailed below.

PROPOSITION 4.2. Every probability distribution over discrete or ﬁnite- precision numeric variables can be represented as a Markov logic network.

Proof. Consider first the case of Boolean variables (X1, X2, . . . , Xn). Define a predicate of zero arity Rh for each variable Xh, and include in the MLN L a formula for each possible state of (X1, X2, . . . , Xn). This formula is a conjunction of n literals, with the hth literal being Rh() if Xh is true in the state, and ¬Rh() otherwise. The formula’s weight is log P (X1, X2, . . . , Xn). (If some states have zero probability, use instead the product form (see Equa- tion 3), with φi() equal to the probability of the ith state.) Since all predicates in L have zero arity, L defines the same Markov network ML,C irrespective of C, with one node for each variable Xh. For any state, the corresponding formula is true and all others are false, and thus Equation 3 represents the original distribution (notice that Z = 1). The generalization to arbitrary discrete variables is straightforward, by defining a zero-arity predicate for each value of each variable. Similarly for finite-precision numeric variables, by noting that they can be represented as Boolean vectors. 2

Of course, compact factored models like Markov networks and Bayesian networks can still be represented compactly by MLNs, by deﬁning formulas

mln.tex; 26/01/2006; 19:24; p.10 Markov Logic Networks 11 for the corresponding factors (arbitrary features in Markov networks, and states of a node and its parents in Bayesian networks).2 First-order logic (with Assumptions 1–3 above) is the special case of MLNs obtained when all weights are equal and tend to inﬁnity, as described below.

PROPOSITION 4.3. Let KB be a satisﬁable knowledge base, L be the MLN obtained by assigning weight w to every formula in KB, C be the set of constants appearing in KB, Pw(x) be the probability assigned to a (set of) possible world(s) x by ML,C, XKB be the set of worlds that satisfy KB, and F be an arbitrary formula in ﬁrst-order logic. Then:

−1 1. ∀x ∈ XKB limw→∞ Pw(x) = |XKB| ∀x 6∈ XKB limw→∞ Pw(x) = 0

2. For all F , KB |= F iff limw→∞ Pw(F ) = 1.

Proof. Let k be the number of ground formulas in ML,C . By Equation 3, if kw (k−1)w x ∈ XKB then Pw(x) = e /Z, and if x 6∈ XKB then Pw(x) ≤ e /Z. Thus all x ∈ XKB are equiprobable and limw→∞ P (X \ XKB)/P (XKB) ≤ −w limw→∞(|X \XKB|/|XKB|)e = 0, proving Part 1. By definition of entailment, KB |= F iff every world that satisfies KB also satisfies F . Therefore, letting XF be the set of worlds that satisfy F , if KB |= F then XKB ⊆ XF and Pw(F ) = x∈XF Pw(x) ≥ Pw(XKB). Since, from Part 1, limw→∞ P (X ) = 1, this implies that if KB |= F then lim P (F ) = 1. The w KB P w→∞ w inverse direction of Part 2 is proved by noting that if limw→∞ Pw(F ) = 1 then every world with non-zero probability in the limit must satisfy F , and this includes every world in XKB. 2

In other words, in the limit of all equal infinite weights, the MLN represents a uniform distribution over the worlds that satisfy the KB, and all entailment queries can be answered by computing the probability of the query formula and checking whether it is 1. Even when weights are finite, first-order logic is “embedded” in MLNs in the following sense. Assume without loss of generality that all weights are non-negative. (A formula with a negative weight w can be replaced by its negation with weight −w.) If the knowledge base composed of the formulas in an MLN L (negated, if their weight is negative) is satisfiable, then, for any C, the satisfying assignments are the modes of the distribution represented by ML,C. This is because the modes are the worlds x with maximum i wini(x) (see Equation 3), and this expression is maximized when all groundings of all formulas are true (i.e., the KB is P 2 While some conditional independence structures can be compactly represented with directed graphs but not with undirected ones, they still lead to compact models in the form of Equation 3 (i.e., as products of potential functions).

mln.tex; 26/01/2006; 19:24; p.11 12 Richardson and Domingos satisfied). Unlike an ordinary first-order KB, however, an MLN can produce useful results even when it contains contradictions. An MLN can also be obtained by merging several KBs, even if they are partly incompatible. This is potentially useful in areas like the Semantic Web (Berners-Lee et al., 2001) and mass collaboration (Richardson & Domingos, 2003). It is interesting to see a simple example of how MLNs generalize first- order logic. Consider an MLN containing the single formula ∀x R(x) ⇒ S(x) with weight w, and C = {A}. This leads to four possible worlds: {¬R(A), ¬S(A)}, {¬R(A), S(A)}, {R(A), ¬S(A)}, and {R(A), S(A)}. From Equa- tion 3 we obtain that P ({R(A), ¬S(A)}) = 1/(3ew + 1) and the probability of each of the other three worlds is ew/(3ew + 1). (The denominator is the partition function Z; see Section 2.) Thus, if w > 0, the effect of the MLN is to make the world that is inconsistent with ∀x R(x) ⇒ S(x) less likely than the other three. From the probabilities above we obtain that P (S(A)|R(A)) = 1/(1 + e−w). When w → ∞, P (S(A)|R(A)) → 1, recovering the logical entailment. In practice, we have found it useful to add each predicate to the MLN as a unit clause. In other words, for each predicate R(x1, x2, . . .) appearing in the MLN, we add the formula ∀x1, x2, . . . R(x1, x2, . . .) with some weight wR. The weight of a unit clause can (roughly speaking) capture the marginal distribution of the corresponding predicate, leaving the weights of the non- unit clauses free to model only dependencies between predicates. When manually constructing an MLN or interpreting a learned one, it is useful to have an intuitive understanding of the weights. The weight of a formula F is simply the log odds between a world where F is true and a world where F is false, other things being equal. However, if F shares variables with other formulas, as will typically be the case, it may not be possible to keep those formulas’s truth values unchanged while reversing F ’s. In this case there is no longer a one-to-one correspondence between weights and probabilities of formulas.3 Nevertheless, the probabilities of all formulas collectively determine all weights, if we view them as constraints on a maximum entropy distribution, or treat them as empirical probabilities and learn the maximum likelihood weights (the two are equivalent) (Della Pietra et al., 1997). Thus a good way to set the weights of an MLN is to write down the probability with which each formula should hold, treat these as empirical frequencies, and learn the weights from them using the algorithm in Section 6. Conversely, the weights in a learned MLN can be viewed as collectively encoding the empirical formula probabilities.

3 This is an unavoidable side-effect of the power and ﬂexibility of Markov networks. In Bayesian networks, parameters are probabilities, but at the cost of greatly restricting the ways in which the distribution may be factored. In particular, potential functions must be conditional probabilities, and the directed graph must have no cycles. The latter condition is particularly troublesome to enforce in relational extensions (Taskar et al., 2002).

mln.tex; 26/01/2006; 19:24; p.12 Markov Logic Networks 13

The size of ground Markov networks can be vastly reduced by having typed constants and variables, and only grounding variables to constants of the same type. However, even in this case the size of the network may be extremely large. Fortunately, many inferences do not require grounding the entire network, as we see in the next section.

5. Inference

MLNs can answer arbitrary queries of the form “What is the probability that formula F1 holds given that formula F2 does?” If F1 and F2 are two formulas in ﬁrst-order logic, C is a ﬁnite set of constants including any constants that appear in F1 or F2, and L is an MLN, then

P (F1|F2, L, C) = P (F1|F2, ML,C ) P (F ∧ F |M ) = 1 2 L,C P (F2|ML,C )

x∈X ∩X P (X =x|ML,C ) = F1 F2 (4) P (X =x|M ) P x∈XF2 L,C P where XFi is the set of worlds where Fi holds, and P (x|ML,C ) is given by Equation 3. Ordinary conditional queries in graphical models are the special case of Equation 4 where all predicates in F1, F2 and L are zero-arity and the formulas are conjunctions. The question of whether a knowledge base KB entails a formula F in first-order logic is the question of whether P (F |LKB, CKB,F ) = 1, where LKB is the MLN obtained by assigning infinite weight to all the formulas in KB, and CKB,F is the set of all constants appearing in KB or F . The question is answered by computing P (F |LKB, CKB,F ) by Equation 4, with F2 = True. Computing Equation 4 directly will be intractable in all but the smallest domains. Since MLN inference subsumes probabilistic inference, which is #P-complete, and logical inference, which is NP-complete even in finite domains, no better results can be expected. However, many of the large number of techniques for efficient inference in either case are applicable to MLNs. Because MLNs allow fine-grained encoding of knowledge, including context- specific independences, inference in them may in some cases be more efficient than inference in an ordinary graphical model for the same domain. On the logic side, the probabilistic semantics of MLNs facilitates approximate inference, with the corresponding potential gains in efficiency. In principle, P (F1|F2, L, C) can be approximated using an MCMC algorithm that rejects all moves to states where F2 does not hold, and counts the number of samples in which F1 holds. However, even this is likely to be too

mln.tex; 26/01/2006; 19:24; p.13 14 Richardson and Domingos

Table III. Network construction for inference in MLNs.

function ConstructNetwork(F1, F2, L, C) inputs: F1, a set of ground atoms with unknown truth values (the “query”) F2, a set of ground atoms with known truth values (the “evidence”) L, a Markov logic network C, a set of constants output: M, a ground Markov network calls: MB(q), the Markov blanket of q in ML,C G ← F1 while F1 6= ∅ for all q ∈ F1 if q 6∈ F2 F1 ← F1 ∪ (MB(q) \ G) G ← G ∪ MB(q) F1 ← F1 \ {q} return M, the ground Markov network composed of all nodes in G, all arcs between them in ML,C , and the features and weights on the corresponding cliques

slow for arbitrary formulas. Instead, we provide an inference algorithm for the case where F1 and F2 are conjunctions of ground literals. While less general than Equation 4, this is the most frequent type of query in practice, and the algorithm we provide answers it far more efﬁciently than a direct applica- tion of Equation 4. Investigating lifted inference (where queries containing variables are answered without grounding them) is an important direction for future work (see Jaeger (2000) and Poole (2003) for initial results). The algorithm proceeds in two phases, analogous to knowledge-based model construction (Wellman et al., 1992). The ﬁrst phase returns the minimal subset M of the ground Markov network required to compute P (F1|F2, L, C). The algorithm for this is shown in Table III. The size of the network returned may be further reduced, and the algorithm sped up, by noticing that any ground formula which is made true by the evidence can be ignored, and the corresponding arcs removed from the network. In the worst case, the network contains O(|C|a) nodes, where a is the largest predicate arity in the domain, but in practice it may be much smaller. The second phase performs inference on this network, with the nodes in F2 set to their values in F2. Our implementation uses Gibbs sampling, but any inference method may be employed. The basic Gibbs step consists of sampling one ground atom given its Markov blanket. The Markov blanket of a ground atom is the set of ground atoms that appear in some grounding of a formula with it. The probability of a ground atom Xl when its Markov blanket Bl is in state bl is

mln.tex; 26/01/2006; 19:24; p.14 Markov Logic Networks 15

P (Xl =xl|Bl =bl) (5) exp( wifi(Xl =xl, Bl =bl)) = fi∈Fl exp( fi∈Fl wifi(Xl =P0, Bl =bl)) + exp( fi∈Fl wifi(Xl =1, Bl =bl)) P P where Fl is the set of ground formulas that Xl appears in, and fi(Xl = xl, Bl = bl) is the value (0 or 1) of the feature corresponding to the ith ground formula when Xl = xl and Bl = bl. For sets of atoms of which exactly one is true in any given world (e.g., the possible values of an attribute), blocking can be used (i.e., one atom is set to true and the others to false in one step, by sampling conditioned on their collective Markov blanket). The estimated probability of a conjunction of ground literals is simply the fraction of samples in which the ground literals are true, after the Markov chain has converged. Because the distribution is likely to have many modes, we run the Markov chain multiple times. When the MLN is in clausal form, we minimize burn-in time by starting each run from a mode found using MaxWalkSat, a local search algorithm for the weighted satisfiability problem (i.e., finding a truth assignment that maximizes the sum of weights of satisfied clauses) (Kautz et al., 1997). When there are hard constraints (clauses with infinite weight), MaxWalkSat finds regions that satisfy them, and the Gibbs sampler then samples from these regions to obtain probability estimates.

6. Learning

We learn MLN weights from one or more relational databases. (For brevity, the treatment below is for one database, but the generalization to many is trivial.) We make a closed world assumption (Genesereth & Nilsson, 1987): if a ground atom is not in the database, it is assumed to be false. If there are n possible ground atoms, a database is effectively a vector x = (x1, . . . , xl, . . . , xn) where xl is the truth value of the lth ground atom (xl = 1 if the atom appears in the database, and xl = 0 otherwise). Given a database, MLN weights can in principle be learned using standard methods, as follows. If the ith formula has ni(x) true groundings in the data x, then by Equation 3 the derivative of the log-likelihood with respect to its weight is

∂ 0 0 log Pw(X =x) = ni(x) − Pw(X =x ) ni(x ) (6) ∂wi 0 Xx 0 0 0 where the sum is over all possible databases x , and Pw(X =x ) is P (X =x ) computed using the current weight vector w = (w1, . . . , wi, . . .). In other words, the ith component of the gradient is simply the difference between the number of true groundings of the ith formula in the data and its expectation

mln.tex; 26/01/2006; 19:24; p.15 16 Richardson and Domingos according to the current model. Unfortunately, counting the number of true groundings of a formula in a database is intractable, even when the formula is a single clause, as stated in the following proposition (due to Dan Suciu).

PROPOSITION 6.1. Counting the number of true groundings of a ﬁrst-order clause in a database is #P-complete in the length of the clause.

Proof. Counting satisfying assignments of propositional monotone 2-CNF is #P-complete (Roth, 1996). This problem can be reduced to counting the number of true groundings of a ﬁrst-order clause in a database as follows. Consider a database composed of the ground atoms R(0, 1), R(1, 0) and R(1, 1). Given a monotone 2-CNF formula, construct a formula Φ that is a conjunction of predicates of the form R(xi, xj), one for each disjunct xi ∨ xj appearing in the CNF formula. (For example, (x1∨x2)∧(x3∨x4) would yield R(x1, x2) ∧ R(x3, x4).) There is a one-to-one correspondence between the satisfying assignments of the 2-CNF and the true groundings of Φ. The latter are the false groundings of the clause formed by disjoining the negations of all the R(xi, xj), and thus can be counted by counting the number of true groundings of this clause and subtracting it from the total number of groundings. 2

In large domains, the number of true groundings of a formula may be counted approximately, by uniformly sampling groundings of the formula and checking whether they are true in the data. In smaller domains, and in our experiments below, we use an efficient recursive algorithm to find the exact count. A second problem with Equation 6 is that computing the expected number of true groundings is also intractable, requiring inference over the model. Fur- ther, efficient optimization methods also require computing the log-likelihood itself (Equation 3), and thus the partition function Z. This can be done approximately using a Monte Carlo maximum likelihood estimator (MC-MLE) (Geyer & Thompson, 1992). However, in our experiments the Gibbs sampling used to compute the MC-MLEs and gradients did not converge in reasonable time, and using the samples from the unconverged chains yielded poor results. A more efficient alternative, widely used in areas like spatial statistics, social network modeling and language processing, is to optimize instead the pseudo-likelihood (Besag, 1975)

n ∗ Pw(X =x) = Pw(Xl =xl|MBx(Xl)) (7) lY=1 where MBx(Xl) is the state of the Markov blanket of Xl in the data. The gradient of the pseudo-log-likelihood is

mln.tex; 26/01/2006; 19:24; p.16 Markov Logic Networks 17

n ∂ ∗ log Pw(X =x) = [ni(x) − Pw(Xl =0|MBx(Xl)) ni(x[Xl=0]) ∂wi Xl=1 −Pw(Xl =1|MBx(Xl)) ni(x[Xl=1])] (8) where ni(x[Xl=0]) is the number of true groundings of the ith formula when we force Xl = 0 and leave the remaining data unchanged, and similarly for ni(x[Xl=1]). Computing this expression (or Equation 7) does not require inference over the model. We optimize the pseudo-log-likelihood using the limited-memory BFGS algorithm (Liu & Nocedal, 1989). The computation can be made more efﬁcient in several ways:

− The sum in Equation 8 can be greatly sped up by ignoring predicates that do not appear in the ith formula.

− The counts ni(x), ni(x[Xl=0]) and ni(x[Xl=1]) do not change with the weights, and need only be computed once (as opposed to in every iteration of BFGS).

− Ground formulas whose truth value is unaffected by changing the truth

value of any single literal may be ignored, since then ni(x) = ni(x[Xl=0])

= ni(x[Xl=1]). In particular, this holds for any clause which contains at least two true literals. This can often be the great majority of ground clauses.

To combat overfitting, we penalize the pseudo-likelihood with a Gaussian prior on each weight. Inductive logic programming (ILP) techniques can be used to learn additional clauses, refine the ones already in the MLN, or learn an MLN from scratch. We use the CLAUDIEN system for this purpose (De Raedt & De- haspe, 1997). Unlike most other ILP systems, which learn only Horn clauses, CLAUDIEN is able to learn arbitrary first-order clauses, making it well suited to MLNs. Also, by constructing a particular language bias, we are able to direct CLAUDIEN to search for refinements of the MLN structure. In the future we plan to more fully integrate structure learning into MLNs, by generalizing techniques like Della Pietra et al.’s (1997) to the first-order realm, as done by MACCENT for classification problems (Dehaspe, 1997).

7. Experiments

We tested MLNs using a database describing the Department of Computer Science and Engineering at the University of Washington (UW-CSE). The

mln.tex; 26/01/2006; 19:24; p.17 18 Richardson and Domingos domain consists of 12 predicates and 2707 constants divided into 10 types. Types include: publication (342 constants), person (442), course (176), project (153), academic quarter (20), etc. Predicates include: Professor(person), Student(person), Area(x, area) (with x ranging over publications, per- sons, courses and projects), AuthorOf(publication, person), AdvisedBy (person, person), YearsInProgram(person, years), CourseLevel (course, level), TaughtBy(course, person, quarter), TeachingAssi- stant(course, person, quarter), etc. Additionally, there are 10 equality predicates: SamePerson(person, person), SameCourse(course, course), etc. which always have known, fixed values that are true iff the two arguments are the same constant. Using typed variables, the total number of possible ground atoms (n in Section 6) was 4,106,841. The database contained a total of 3380 tuples (i.e., there were 3380 true ground atoms). We obtained this database by scrap- ing pages in the department’s Web site (www.cs.washington.edu). Publica- tions and AuthorOf relations were obtained by extracting from the Bib- Serv database (www.bibserv.org) all records with author fields containing the names of at least two department members (in the form “last name, first name” or “last name, first initial”). We obtained a knowledge base by asking four volunteers to each provide a set of formulas in first-order logic describing the domain. (The volunteers were not shown the database of tuples, but were members of the department who thus had a general understanding about it.) Merging these yielded a KB of 96 formulas. The complete KB, volunteer instructions, database, and algorithm parameter settings are online at http://www.cs.washington.edu/ai/mln. Formulas in the KB include statements like: students are not professors; each student has at most one advisor; if a student is an author of a paper, so is her advisor; advanced students only TA courses taught by their advisors; at most one author of a given publication is a professor; students in Phase I of the Ph.D. program have no advisor; etc. Notice that these statements are not always true, but are typically true. For training and testing purposes, we divided the database into five sub- databases, one for each area: AI, graphics, programming languages, systems, and theory. Professors and courses were manually assigned to areas, and other constants were iteratively assigned to the most frequent area among other constants they appeared in some tuple with. Each tuple was then assigned to the area of the constants in it. Tuples involving constants of more than one area were discarded, to avoid train-test contamination. The sub-databases contained, on average, 521 true ground atoms out of a possible 58457. We performed leave-one-out testing by area, testing on each area in turn using the model trained from the remaining four. The test task was to predict the AdvisedBy(x, y) predicate given (a) all others (All Info) and (b) all others except Student(x) and Professor(x) (Partial Info). In both cases, we

mln.tex; 26/01/2006; 19:24; p.18 Markov Logic Networks 19 measured the average conditional log-likelihood of all possible groundings of AdvisedBy(x, y) over all areas, drew precision/recall curves, and computed the area under the curve. This task is an instance of link prediction, a problem that has been the object of much interest in statistical relational learning (see Section 8). All KBs were converted to clausal form. Timing results are on a 2.8Ghz Pentium 4 machine.

7.1. SYSTEMS

In order to evaluate MLNs, which use logic and probability for inference, we wished to compare with methods that use only logic or only probability. We were also interested in automatic induction of clauses using ILP techniques. This subsection gives details of the comparison systems used.

7.1.1. Logic One important question we aimed to answer with the experiments is whether adding probability to a logical knowledge base improves its ability to model the domain. Doing this requires observing the results of answering queries using only logical inference, but this is complicated by the fact that computing log-likelihood and the area under the precision/recall curve requires real- valued probabilities, or at least some measure of “confidence” in the truth of each ground atom being tested. We thus used the following approach. For a given knowledge base KB and set of evidence atoms E, let XKB∪E be the set of worlds that satisfy KB ∪ E. The probability of a query atom q is then |XKB∪E∪q| defined as P (q) = , the fraction of XKB E in which q is true. |XKB∪E | ∪ A more serious problem arises if the KB is inconsistent (which was indeed the case with the KB we collected from volunteers). In this case the denominator of P (q) is zero. (Also, recall that an inconsistent knowledge base trivially entails any arbitrary formula). To address this, we redefine XKB∪E to be the set of worlds which satisfies the maximum possible number of ground clauses. We use Gibbs sampling to sample from this set, with each chain initialized to a mode using WalkSat. At each Gibbs step, the step is taken with probability: 1 if the new state satisfies more clauses than the current one (since that means the current state should have 0 probability), 0.5 if the new state satisfies the same number of clauses (since the new and old state then have equal probability), and 0 if the new state satisfies fewer clauses. We then use only the states with maximum number of satisfied clauses to compute probabilities. Notice that this is equivalent to using an MLN built from the KB and with all equal infinite weights.

mln.tex; 26/01/2006; 19:24; p.19 20 Richardson and Domingos

7.1.2. Probability The other question we wanted to answer with these experiments is whether existing (propositional) probabilistic models are already powerful enough to be used in relational domains without the need for the additional represen- tational power provided by MLNs. In order to use such models, the domain must first be propositionalized by defining features that capture useful information about it. Creating good attributes for propositional learners in this highly relational domain is a difficult problem. Nevertheless, as a tradeoff between incorporating as much potentially relevant information as possible and avoiding extremely long feature vectors, we defined two sets of propositional attributes: order-1 and order-2. The former involves characteristics of individual constants in the query predicate, and the latter involves characteristics of relations between the constants in the query predicate. For the order-1 attributes, we defined one variable for each (a, b) pair, where a is an argument of the query predicate and b is an argument of some predicate with the same value as a. The variable is the fraction of true groundings of this predicate in the data. Some examples of first-order attributes for AdvisedBy(Matt, Pedro) are: whether Pedro is a student, the fraction of publications that are published by Pedro, the fraction of courses for which Matt was a teaching assistant, etc. The order-2 attributes were defined as follows: for a given (ground) query predicate Q(q1, q2, . . . , qk), consider all sets of k predicates and all assignments of constants q1, q2, . . . , qk as arguments to the k predicates, with exactly one constant per predicate (in any order). For instance, if Q is Advised- By(Matt, Pedro) then one such possible set would be {TeachingAssistant ( , Matt, ), TaughtBy( , Pedro, )}. This forms 2k attributes of the example, each corresponding to a particular truth assignment to the k predicates. The value of an attribute is the number of times, in the training data, the set of predicates have that particular truth assignment, when their unassigned arguments are all filled with the same constants. For example, consider filling the above empty arguments with “CSE546” and “Autumn0304”. The resulting set, {TeachingAssistant(CSE546, Matt, Autumn0304), TaughtBy (CSE546, Pedro, Autumn0304)} has some truth assignment in the training data (e.g., {True,True}, {True,False}, . . .). One attribute is the number of such sets of constants that create the truth assignment {True,True}, another for {True,False} and so on. Some examples of second-order attributes gener- ated for the query AdvisedBy(Matt, Pedro) are: how often Matt is a teaching assistant for a course that Pedro taught (as well as how often he is not), how many publications Pedro and Matt have coauthored, etc. The resulting 28 order-1 attributes and 120 order-2 attributes (for the All Info case) were discretized into five equal-frequency bins (based on the training set). We used two propositional learners: Naive Bayes (Domingos & Paz- zani, 1997) and Bayesian networks (Heckerman et al., 1995) with structure

mln.tex; 26/01/2006; 19:24; p.20 Markov Logic Networks 21 and parameters learned using the VFBN2 algorithm (Hulten & Domingos, 2002) with a maximum of four parents per node. The order-2 attributes helped the naive Bayes classiﬁer but hurt the performance of the Bayesian network classiﬁer, so below we report results using the order-1 and order-2 attributes for naive Bayes, and only the order-1 attributes for Bayesian networks.

7.1.3. Inductive logic programming Our original knowledge base was acquired from volunteers, but we were also interested in whether it could have been developed automatically using inductive logic programming methods. As mentioned earlier, we used CLAUDIEN to induce a knowledge base from data. CLAUDIEN was run with: local scope; minimum accuracy of 0.1; minimum coverage of on; maximum complexity of 10; and breadth-first search. CLAUDIEN’s search space is defined by its language bias. We constructed a language bias which allowed: a maximum of three variables in a clause; unlimited predicates in a clause; up to two non-negated appearances of a predicate in a clause, and two negated ones; and use of knowledge of predicate argument types. To minimize search, the equality predicates (e.g., SamePerson) were not used in CLAUDIEN, and this improved its results. Besides inducing clauses from the training data, we were also interested in using data to automatically refine the knowledge base provided by our volunteers. CLAUDIEN does not support this feature directly, but it can be emulated by an appropriately constructed language bias. We did this by, for each clause in the KB, allowing CLAUDIEN to (1) remove any number of the literals, (2) add up to v new variables, and (3) add up to l new literals. We ran CLAUDIEN for 24 hours on a Sun-Blade 1000 for each (v, l) in the set {(1, 2), (2, 3), (3, 4)}. All three gave nearly identical results; we report the results with v = 3 and l = 4.

7.1.4. MLNs Our results compare the above systems to Markov logic networks. The MLNs were trained using a Gaussian weight prior with zero mean and unit variance, and with the weights initialized at the mode of the prior (zero). For optimization, we used the Fortran implementation of L-BFGS from Zhu et al. (1997) and Byrd et al. (1995), leaving all parameters at their default values, and with a convergence criterion (ftol) of 10−5. Inference was performed using Gibbs sampling as described in Section 5, with ten parallel Markov chains, each initialized to a mode of the distribution using MaxWalkSat. The number of Gibbs steps was determined using the criterion of DeGroot and Schervish (2002, pp. 707 and 740-741). Sampling continued until we reached a conﬁ- dence of 95% that the probability estimate was within 1% of the true value in at least 95% of the nodes (ignoring nodes which are always true or false). A minimum of 1000 and maximum of 500,000 samples was used, with one

mln.tex; 26/01/2006; 19:24; p.21 22 Richardson and Domingos sample per complete Gibbs pass through the variables. Typically, inference converged within 5000 to 100,000 passes. The results were insensitive to variation in the convergence thresholds.

7.2. RESULTS

7.2.1. Training with MC-MLE Our initial system used MC-MLE to train MLNs, with ten Gibbs chains, and each ground atom being initialized to true with the corresponding first-order predicate’s probability of being true in the data. Gibbs steps may be taken quite quickly by noting that few counts of satisfied clauses will change on any given step. On the UW-CSE domain, our implementation took 4-5 ms per step. We used the maximum across all predicates of the Gelman criterion R (Gilks et al., 1996) to determine when the chains had reached their stationary distribution. In order to speed convergence, our Gibbs sampler preferentially samples atoms that were true in either the data or the initial state of the chain. The intuition behind this is that most atoms are always false, and sampling repeatedly from them is inefficient. This improved convergence by approximately an order of magnitude over uniform selection of atoms. Despite these optimizations, the Gibbs sampler took a prohibitively long time to reach a reasonable convergence threshold (e.g., R = 1.01). After running for 24 hours (approximately 2 million Gibbs steps per chain), the average R value across training sets was 3.04, with no one training set having reached an R value less than 2 (other than briefly dipping to 1.5 in the early stages of the process). Considering this must be done iteratively as L-BFGS searches for the minimum, we estimate it would take anywhere from 20 to 400 days to complete the training, even with a weak convergence threshold such as R = 2.0. Experiments confirmed the poor quality of the models that resulted if we ignored the convergence threshold and limited the training process to less than ten hours. With a better choice of initial state, approximate counting, and improved MCMC techniques such as the Swendsen-Wang algorithm (Edwards & Sokal, 1988), MC-MLE may become practical, but it is not a viable option for training in the current version. (Notice that during learning MCMC is performed over the full ground network, which is too large to apply MaxWalkSat to.)

7.2.2. Training with pseudo-likelihood In contrast to MC-MLE, pseudo-likelihood training was quite fast. As dis- cussed in Section 6, each iteration of training may be done quite quickly once the initial clause and ground atom satisfiability counts are complete. On average (over the five test sets), finding these counts took 2.5 minutes. From there, training took, on average, 255 iterations of L-BFGS, for a total of 16 minutes.

mln.tex; 26/01/2006; 19:24; p.22 Markov Logic Networks 23

7.2.3. Inference Inference was also quite quick. Inferring the probability of all AdvisedBy(x, y) atoms in the All Info case took 3.3 minutes in the AI test set (4624 atoms), 24.4 in graphics (3721), 1.8 in programming languages (784), 10.4 in systems (5476), and 1.6 in theory (2704). The number of Gibbs passes ranged from 4270 to 500,000, and averaged 124,000. This amounts to 18 ms per Gibbs pass and approximately 200,000–500,000 Gibbs steps per second. The average time to perform inference in the Partial Info case was 14.8 minutes (vs. 8.3 in the All Info case).

7.2.4. Comparison of systems We compared twelve systems: the original KB (KB); CLAUDIEN (CL); CLAU- DIEN with the original KB as language bias (CLB); the union of the original KB and CLAUDIEN’s output in both cases (KB+CL and KB+CLB); an MLN with each of the above KBs (MLN(KB), MLN(CL), MLN(KB+CL), and MLN(KB+CLB)); naive Bayes (NB); and a Bayesian network learner (BN). Add-one smoothing of probabilities was used in all cases. Table IV summarizes the results. Figure 2 shows precision/recall curves for all areas (i.e., averaged over all AdvisedBy(x, y) atoms), and Figures 3 to 7 show precision/recall curves for the ﬁve individual areas. MLNs are clearly more accurate than the alternatives, showing the promise of this approach. The purely logical and purely probabilistic methods often suffer when inter- mediate predicates have to be inferred, while MLNs are largely unaffected. Naive Bayes performs well in AUC in some test sets, but very poorly in others; its CLLs are uniformly poor. CLAUDIEN performs poorly on its own, and produces no improvement when added to the KB in the MLN. Using CLAUDIEN to reﬁne the KB typically performs worse in AUC but better in CLL than using CLAUDIEN from scratch; overall, the best-performing logical method is KB+CLB, but its results fall well short of the best MLNs’. The general drop-off in precision around 50% recall is attributable to the fact that the database is very incomplete, and only allows identifying a minority of the AdvisedBy relations. Inspection reveals that the occasional smaller drop-offs in precision at very low recalls are due to students who graduated or changed advisors after co-authoring many publications with them.

8. Statistical Relational Learning Tasks

Many SRL tasks can be concisely formulated in the language of MLNs, allowing the algorithms introduced in this paper to be directly applied to them. In this section we exemplify this with five key tasks: collective classification, link prediction, link-based clustering, social network modeling, and object identification.

mln.tex; 26/01/2006; 19:24; p.23 24 Richardson and Domingos

0.8 MLN(KB) MLN(KB+CL) 0.6 KB KB+CL CL 0.4 NB BN Precision 0.2