Stable and Supported Semantics in Continuous Vector Spaces
Total Page:16
File Type:pdf, Size:1020Kb
Proceedings of the 17th International Conference on Principles of Knowledge Representation and Reasoning (KR 2020) Main Track Stable and Supported Semantics in Continuous Vector Spaces Yaniv Aspis1 , Krysia Broda1 , Alessandra Russo1 , Jorge Lobo1;2;3 1Imperial College London 2ICREA 3Universitat Pompeu Fabra fyaniv.aspis17, k.broda, [email protected], [email protected] Abstract et al. 2018). These approaches are appealing as they al- low for symbolic reasoning and learning to be carried out We introduce a novel approach for the computation of stable over fuzzy and noisy data in a natural manner. But they are and supported models of normal logic programs in continu- limited to performing approximate inference in the context ous vector spaces by a gradient-based search method. Specif- ically, the application of the immediate consequence operator of classical semantics. Approximating the reasoning pro- of a program reduct can be computed in a vector space. To do cess risks to make the learned semantics strongly dependent this, Herbrand interpretations of a propositional program are on the training data rather than on the given symbolic pro- embedded as 0-1 vectors in RN and program reducts are rep- gram. Reasoning only in the context of classical semantics resented as matrices in RN×N . Using these representations makes the approaches less suited to capture common-sense we prove that the underlying semantics of a normal logic pro- reasoning which often requires a non-monotonic semantics. gram is captured through matrix multiplication and a differ- To date, stable and supported reasoning have not been tack- entiable operation. As supported and stable models of a nor- led in neural-symbolic literature. mal logic program can now be seen as fixed points in a con- Yet, working in vector spaces offers unique advantages. tinuous space, non-monotonic deduction can be performed Recent advancements in GPU hardware allows for very ef- using an optimisation process such as Newton’s method. We report the results of several experiments using synthetically ficient computation of linear algebraic operations, such as generated programs that demonstrate the feasibility of the ap- matrix and vector multiplication. Linear algebra offers a proach and highlight how different parameter values can af- variety of algorithms such as matrix decomposition and nu- fect the behaviour of the system. merical optimisation that logical inference may benefit from. In addition, it may facilitate neural-symbolic learning by al- lowing a systematic method for translating symbolic logic 1 Introduction programs to vector spaces and back. One wonders, then, if Stable and supported models are two widely used ap- it is possible to extend exact logical inference to continuous proaches to defining the semantics of a logic program in space while maintaining stable and supported semantics. the presence of default negation (Marek and Subrahmanian Work supporting such an approach has recently been 1992). Stable semantics is at the heart of Answer Set Pro- taken by Sakama et al. (2017; 2018). The authors proposed gramming, a declarative paradigm for knowledge represen- a matrix representation of definite and normal programs that tation, reasoning and learning geared towards computation- preserves their semantics with respect to matrix multiplica- ally hard problems (Lifschitz 2008). Typically, one en- tion and application of a non-continuous operation. The re- codes a problem as a set of clauses representing background sulting algorithm allows for the computation of stable mod- knowledge, constraints or choices. The clauses form a logic els in a vector space, but the representation is still discrete. program whose models under stable semantics correspond to In this work, we lay the foundations for a novel method solutions of the original problem. To discover these stable to compute supported and stable models in continuous vec- models, an answer set solver would perform a search over a tor spaces using differentiable operations. We extend the discrete space of program interpretations, typically involv- above approach to continuous vector space while preserv- ing trial-and-error, conflict analysis and backtracking. Sup- ing two-valued semantics, allowing gradient-based methods ported models are a superset of stable models that are char- for computation of supported and stable models. Our main acterized through the Clark completion of a program (Clark contributions are: 1978). • Presenting a discrete matrix representation of proposi- The recent popularity of deep learning approaches has tional normal program reducts. sparked interest in performing symbolic reasoning in con- tinuous vector spaces rather than discrete. Approaches in- • Demonstrating how to extend this discrete representation clude defining a learning task for a neural network for learn- to continuous space and proving under what conditions ing models (Minervini et al. 2019), programs (Evans and two-valued semantics is preserved. Grefenstette 2017) or the reasoning process itself (Selsam • Presenting a novel algorithm for gradient-based computa- 59 Proceedings of the 17th International Conference on Principles of Knowledge Representation and Reasoning (KR 2020) Main Track tion of supported and stable models based on the above Supported model and Stable model semantics are among representation. these. • Systematic evaluation of this algorithm to show the feasi- A supported model M of a program P is a model of P where for every p 2 M there exists a clause r 2 P such that bility over a class of programs containing negative loops, r r r r r positive loops and choice. p = h , fb1; :::; bnr g ⊆ M and fc1; :::; cmr g\M = ;. This definition is equivalent to the characterization of supported The novel method we present here can naturally be extended models as classical models of the Clark Completion of a pro- in the future to support neural-symbolic architectures of rea- gram (Clark 1978; Marek and Subrahmanian 1992). In the soning and learning in the presence of default negation, and program above both fp; qg and fp; q; tg are supported mod- this is our main motivation. els. So, the LHM(P ) of a definite program P is a minimal The structure of this paper is as follows: Section 2 cov- supported model of P . Consider now this other program: ers the necessary background and definitions for logic pro- gramming under supported and stable semantics. Section 3 p q (3) presents and justifies our method, including a matrix-based q p characterization of program reducts, its generalization into The empty set is a supported and minimal model, but fp; qg continuous vector space and a gradient-based search algo- is also a supported model. This is due to the “positive loop” rithm for the computation of a normal program’s supported arising from the set of clauses fp q; q pg, so that models. Section 4 details the results of experiments testing fp; qg are deduced to be true only if fp; qg is assumed. One the ability of the algorithm to compute supported and stable can formalise the concept of positive loops using the concept models for synthetic programs containing both positive and of the atom dependency graph G(P ), defined as a pair of negative loops. Section 5 covers related work and section 6 atoms and arcs: concludes the paper. r r r G(P ) = (BP ; f(p; q)j9r 2 P; h = q; p 2 B [ C g) (4) 2 Background For an edge (p; q), if p 2 Br then the edge is referred to as r We consider propositional programs over a finite alphabet positive, while if p 2 C then it is negative.A positive loop is a cycle in G(P ) made of only positive edges. We call a Σ = fp1; p2; p3; :::g. The elements of Σ are called proposi- tional variables or atoms. A normal rule, or clause, r is of cycle with a negative edge a negative loop. If we consider the form: now the following program: r r r r r r r p not p h b1; b2; :::; bnr ; not c1; not c2; :::; not cmr (1) p q (5) r r r r r where h , bi , cj , for 1 ≤ i ≤ n , 1 ≤ j ≤ m , are atoms r r r r q p in Σ. h is referred to as the head, b1; b2; :::; bnr (resp. r r r r c1; c2; :::; cmr ) as (collectively) the positive body, B , (resp. the empty set is no longer a model, as it does not satisfy the negative body, Cr) of the rule. We denote jBrj = nr the first clause, but fp; qg is still a supported model, and and jCrj = mr.A definite clause r is a normal clause where also happens to be minimal, still due to the positive loop, to mr = 0.A fact is a definite rule where nr = 0.A normal which somebody might object. To capture more closely the program P is a finite set of normal rules. If all rules of a notion of a non-monotonic, default semantics, the notion of program are definite, it is referred to as a definite program. stable model semantics is often used instead. The Herbrand Base, BP , of a program P is the set of all A stable model of a normal program P is defined us- propositional variables that appear in P . We denote jBP j = ing the notion of program reduct (Gelfond and Lifschitz N. A Herbrand interpretation I of a program P is a subset 1988). Given a program P and an Herbrand interpretation M of BP .A model M of a program P is an interpretation of P M ⊆ BP , the program reduct P is constructed from P r r r where for every clause r 2 P , if fb1; b2; :::; bnr g ⊆ M and by firstly removing any rule whose negative body contains r r r r r fc1; c2; :::; cmr g \ M = ; then h 2 M.A minimal model an atom ci 2 M, and (2) removing the negative body from M of P is a model of P such that no proper subset of M is the remaining rules.