Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 2015) Anytime Inference in Probabilistic Logic Programs with TP -Compilation Jonas Vlasselaer, Guy Van den Broeck, Angelika Kimmig, Wannes Meert, Luc De Raedt Department of Computer Science KU Leuven, Belgium fi[email protected] Abstract also been addressed using lazy clause generation [Aziz et al., 2015], but only for exact inference. Existing techniques for inference in probabilistic The key contribution of this paper is -compilation, a logic programs are sequential: they first compute TP novel inference technique for probabilistic definite clause the relevant propositional formula for the query of programs that interleaves construction and compilation of the interest, then compile it into a tractable target rep- propositional formula for WMC (steps (1) and (2)). Our sec- resentation and finally, perform weighted model ond contribution, and formal basis of -compilation, is the counting on the resulting representation. We pro- TP Tc operator that generalizes the T operator from logic pose T -compilation, a new inference technique P P P programming [Van Emden and Kowalski, 1976] to explicitly based on forward reasoning. T -compilation pro- P construct the formula. At any point, the WMC of the current ceeds incrementally in that it interleaves the knowl- formula provides a lower bound on the true probability, and edge compilation step for weighted model counting we thus realize an anytime algorithm. with forward reasoning on the logic program. This leads to a novel anytime algorithm that provides As in the purely logical setting, forward reasoning with the hard bounds on the inferred probabilities. Fur- TcP operator allows one to answer multiple queries in par- allel, which is not supported by earlier anytime PLP algo- thermore, an empirical evaluation shows that TP - compilation effectively handles larger instances of rithms based on backward reasoning [Poole, 1993; De Raedt complex real-world problems than current sequen- et al., 2007]. Furthermore, forward reasoning naturally han- tial approaches, both for exact and for anytime ap- dles cyclic dependencies. This avoids the need for additional proximate inference. propositions for breaking cycles and simplifies the compila- tion step. Finally, our approach is amenable to online infer- ence. That is, when clauses are added to or deleted from the 1 Introduction program, TP -compilation can update the already compiled Research on combining probability and logic for use with re- formulas, which can cause significant savings compared to lational data has contributed many probabilistic logic pro- restarting inference from scratch. While forward reasoning gramming (PLP) languages and systems such as PRISM is common in sampling-based inference approaches in prob- [Sato, 1995], ICL [Poole, 2008], PITA [Riguzzi and Swift, abilistic programming, e.g., [Milch et al., 2005; Goodman et 2011] and ProbLog [De Raedt et al., 2007; Fierens et al., al., 2008; Gutmann et al., 2011], these do not provide guaran- 2013]. Inference algorithms for PLP often rely on a three teed lower or upper bounds on the probability of the queries. step procedure: (1) transform the dependency structure of We obtain an efficient realization of the TcP opera- the logic program and the queries into a propositional for- tor by representing formulas as Sentential Decision Dia- mula, (2) compile this formula into a tractable target repre- grams (SDD) [Darwiche, 2011], which efficiently support sentation, and (3) compute the weighted model count (WMC) incremental formula construction and WMC. While our ap- [Chavira and Darwiche, 2008] of the compiled formula. Step proach can easily be extended to handle stratified negation, (1) is shared by exact inference [Fierens et al., 2013] and ap- for ease of presentation we focus on definite clause pro- proximation techniques based on compiling selected subfor- grams which cover real-world applications such as biolog- mulas or sampling [Renkens et al., 2014; Poon and Domin- ical and social networks and web-page classification tasks. gos, 2006]. It is well-known that this step is computation- An empirical evaluation in these domains demonstrates that ally expensive or even prohibitive for highly cyclic logic pro- TP -compilation outperforms state-of-the-art sequential ap- grams, as additional propositions are needed to break every proaches on these problems with respect to time, space and cycle [Fierens et al., 2013]. This limits the applicability of the quality of results. sequential approach in real-world domains with cyclic depen- The paper is organized as follows. We review the necessary dencies, such as gene interaction networks, social networks background in Section 2. Section 3 and 4 formally introduce and the web. The most common solution is to use an approx- the TcP operator and corresponding algorithms. We discuss imate (simplified) logic program. Recently, the problem has experimental results in Section 5 and conclude in Section 6. 1852 2 Background On our example, this results in: We review the basics of (probabilistic) logic programming. I 0 = ; 2.1 Logical Inference ∆I 1 = fe(b; a); e(b; c); e(a; c); e(c; a)g A definite clause program, or logic program for short, is ∆I 2 = fp(b; a); p(b; c); p(a; c); p(c; a)g a finite set of definite clauses, also called rules. A defi- ∆I 3 = fp(a; a); p(c; c)g nite clause is a universally quantified expression of the form ∆I 4 = ; h :− b1; :::; bn where h and the bi are atoms and the comma denotes conjunction. The atom h is called the head of the 1 S i TP (;) = i ∆I is the least Herbrand model as given above. clause and b1; :::; bn the body.A fact is a clause that has true as its body and is written more compactly as h. If an 2.2 Probabilistic-Logical Inference expression does not contain variables it is ground. Most probabilistic logic programming languages (e.g. ICL, A Let be the set of all ground atoms that can be constructed PRISM, ProbLog) are based on Sato’s distribution seman- from the constants, functors and predicates in a logic pro- tics [Sato, 1995]. In this paper, we use ProbLog as it is the P Herbrand interpretation P gram .A of is a truth value as- simplest of these languages. 2 A signment to all a , and is often written as the subset of A ProbLog program P consists of a set R of rules and a set true atoms (with all others being false), or as a conjunc- F of probabilistic facts; an example is given in Figure 1. As tion of atoms. A Herbrand interpretation satisfying all rules common, we assume that no probabilistic fact unifies with a P Herbrand model in the program is a . The model-theoretic rule head. A ProbLog program specifies a probability distri- least Her- semantics of a logic program is given by its unique bution over its Herbrand interpretations, also called possible brand model, that is, the set of all ground atoms 2 A that a worlds. Every grounding fθ of a probabilistic fact p :: f in- are entailed by the logic program, written P j . = a dependently takes value true (with probability p) or false As running example we use a logic program that models a (with probability 1 − p). For ease of notation, we assume that graph (see Figure 1). The facts represent the edges between F is ground. two nodes in the graph (we ignore the probabilities at this A total choice C ⊆ F assigns a truth value to every moment) and the rules define whether there is a path between (ground) probabilistic fact, and the corresponding logic pro- two nodes. Abbreviating predicate names by initials, the least gram C [R has a unique least Herbrand model; the proba- Herbrand model is given by f e(b; a); e(b; c); e(a; c); e(c; a); bility of this model is that of C. Interpretations that do not g p(b; a); p(b; c); p(a; c); p(c; a); p(a; a); p(c; c) . correspond to any total choice have probability zero. The probability of a query q is then the sum over all total choices whose program entails q: 0.4 :: edge(b, a). 0.5 :: edge(b, c). a b X Y Y 0.8 :: edge(a, c). 0.7 :: edge(c, a). Pr(q) := pi · (1 − pi) : (1) path(X, Y ):- edge(X, Y ). C⊆F:C[Rj=q f 2C f 2FnC c i i path(X, Y ):- edge(X, Z), path(Z, Y ). As enumerating all total choices entailing the query is infea- sible, state-of-the-art ProbLog inference [Fierens et al., 2013] Figure 1: A (probabilistic) logic program modeling a graph. reduces the problem to that of weighted model counting. For a formula λ over propositional variables V and a weight func- The task of logical inference is to determine whether a pro- tion w(·) assigning a real number to every literal for an atom gram P entails a given atom, called query. The two most in V , the weighted model count is defined as common approaches to inference are backward reasoning X Y Y or SLD-resolution, which starts from the query and reasons WMC(λ) := w(a) · w(:a) : (2) back towards the facts [Nilsson and Maluszynski, 1995], and I⊆V :Ij=λ a2I a2V nI forward reasoning, which starts from the facts and derives new knowledge using the immediate consequence operator The reduction sets w(fi) = pi and w(:fi) = 1 − pi for TP [Van Emden and Kowalski, 1976]. probabilistic facts pi :: fi, and w(a) = w(:a) = 1 else. For a query q, it constructs a formula λ0 such that for every total Let P be a ground logic pro- Definition 1 (TP operator) choice C ⊆ F, C [ fλ0g j= q $ C [ R j= q.