From: AAAI-97 Proceedings. Copyright © 1997, AAAI (www.aaai.org). All rights reserved.

eyond imizing C

Tom Costello Dept of Computer Science, Stanford, CA 94305 costelloQcs.Stanford.EDU

Abstract Problem, the Stanford Murder Mystery, the Ferry Boat Problem, and many other problems were posed as counter examples to proposed solutions. This led researchers to look for ways to validate their theories. One of the first ways was proposed by Lin and Shoham (1995), where they proposed reducing a non-monotonic theory to a monotonic theory to show correctness. Lifschitz and Gelfond suggested reducing reasoning about action to well understood models like automata, and developed a language A well suited to describing automata as they arise in action. This work has been greatly extended, and many solutions have been shown equivalent under certain assumptions. Kartha studied the relationship between various dif- ferent proposals, and showed that under certain syn- tactic assumptions, that many solutions were equiva- lent to those solutions based on A. Lifschitz has considered certain properties, like stability under definitional extension, and restricted monotonicity, that he claims all approaches to the frame problem should obey. Sandewall (Sandewall 1994) proposed justifying non-monotonic treatments by reducing them to dynamical systems. We propose another way of studying non-monotonic treatments of actions, though the methodology is not restricted to actions. We consider what elaborations, Hntroduetion or changes, the treatment allows, in terms of conjoining information. Reasoning about action has been one of the favorite In this paper we shall keep to as simple a language domains for non-monotonic reasoning. One of the as possible. We choose the situation calculus, which we problems with non-monotonic approaches to the frame define in a later section. Many authors have added ex- problem, or characterizing the default that actions have tra predicates to the situation calculus. We avoid doing as few effects as possible, has been judging the cor- this, as we firstly are interested in what can be done rectness, scope and appropriateness of the various so- without extra machinery, and secondly as we believe lutions. This dilemma first arose in terms of extra that many of the uses of extra predicates are unneces- minimal models-Lifschitz’s preconditions, or the bet- sary. In particular, we do not use a causes predicate, ter known Yale Shooting Problem, showed that naive rather we use McCarthy and Hayes effect axioms, that ideas gave weaker solutions than expected. do not necessitate a new predicate. This was not the end however, as new solutions arose, new problems, on which the new solutions gave We first introduce the situation calculus, and then unintuitive results, were developed. The Stolen Car sketch what adding and removing effects and precondi- tions means. We give some examples why adding pre- _Copyright 01997, American Association for Artificial conditions is natural. We then present two formalisms, Intelligence (www.aaai.org). All rights reserved. in terms of circumscription. One, first considered in

448 KNOWLEDGE REPRESENTATION (Costello 1997), a 11ows adding effects and removing effect. Consider our previous example, we would state preconditions. The new proposal allows adding effects that moving a block sometimes causes its location to and adding preconditions. We show that these two pro- change by writing, posals treat preconditions in naturally opposite ways. S.lholds(at(A, I), S)A We show that the new proposal can capture the usual holds(at(A, I), result(move(A, I), s)). challenge problems, and new problems that allow the addition of preconditions. We now have a problem, how can we now specify preconditions. We can write frame axioms. The Situation Calculus Vs.lholds(clear(A), s) A lholds(at(A, I), s) + To look at the problem of reasoning about change we lholds(at(A, I), result(move(A, I), s)) need a logical language to represent time, events, and the properties that hold in the world. We use the sit- This corresponds to adding a clause to the left hand uation calculus. The paper that introduced situations side of an effect axiom, that states that the block must (McCarthy and Hayes 1969) defined them as follows, be clear to be moved. In a later section we show that “A situation s is the complete state of the universe at this style of representation can capture the usual prob- an instant of time”. The basic mechanism used in the lems in reasoning about action. calculus to define a new situation is the result function, It is important to notice that if we represent pre- conditions in this way then the sentences describing a sI = result(e, s). smaller set of preconditions do not logically imply the In this formula s is a situation, and e is an event, and sentences we would write to represent a larger set of s’ is a new situation that results when e occurs. A preconditions. This differs from the what occurs with function or predicate of a situation is called a fluent. effect axioms. There, we saw that writing that being clear was a suficient precondition implied that being Effect Axioms clear and the gripper arm being empty was suficient. Effect axioms are the usual way of representing the We now represent what are necessary preconditions. effects of actions in the situation calculus. If we wish Thus adding the precondition that the gripper arm is to say that moving a clear block A to a location I, empty, using the formula, causes it to be in location I, we write, Vs.lholds(empty(Arm), s) A lholds(at(A, I), s) + V’s Z.holds(clear(A), s) + lholds(at(A, I), result(move(A, I), s)) holds(at(A, I), result(move(A, I), s)). now gives a logically different theory. The major change we have made is that our new notation allows We note that this notation demands that we list the us to list necessary conditions rather than suficient preconditions explicitly. If we wish to add another pre- conditions. This will allow us to add preconditions, in condition, for instance, that the gripper arm is empty, contrast to the previous method of using effect axioms we would need to replace the above sentence with, that did not allow adding preconditions by conjoining Y’s Z.holds(clear(A), s) A holds(empty(Arm), s) + on sentences. holds(at(A, I), result(move(A, I), s)). Non-monotonic Completion However, note that the second sentence is implied by As we mentioned earlier, a certain problem arises when the first. Thus, we can remove a precondition by we formalize domains. Assume there is an axiom that adding the first sentence, as the two sentences together states what happens to the location of a block when an are logically equivalent to the first. We cannot add action mowe is performed. For concreteness we assume a precondition by conjoining the second sentence- that moving a block A, from 1 to I’ changes its position, adding it has no logical effect. from Z to I’, if the mowe action succeeds. If another property, COZOUT,is added, the axiom Another Approach which described what effect the action had on the loca- If we wish to make assumptions about adding precondi- tion does not give us any information on what happens tions, we need a way of expressing the effects of actions to blocks. The usual intent is that colour is unaffected. that does not assume that we explicitly list sufficient V’a: c s.holds(colour(z, c), s) -+ conditions. holds(colour(z, c), result(move(z, I), s)) We can state that the action A, may cause the fluent F to change from false to true, by asserting, To express all the fluents that do not change, an axiom would be needed for each action/fluent pair. These ax- Js.lholds(F, s) A holds(F, result(A, s)). ioms stating that fluents do not change are called frame We note that this allows us to add more effects of action axioms. Having to specify all this lack of change seems by conjoining on sentences. We can describe the effects extravagant, and more importantly it seems unneces- of actions by stating that they sometimes have that sary to have to write these out individually. It seems

REASONING ABOUT ACTION 449 that it should be possible to generate these frame ax- a situation, then it occurs after all situations that agree ioms automatically. The fram> problem is the problem with the first situation on all fluents. of characterizing these frame axioms. We now need a way of stating that actions have ef- We now note that in our new proposal, the problem fects. We wish to say that, under some unspecified is not characterizing frame axioms, as we list these preconditions, an action a causes the fluent f to stop explicitly2, but rather adding the correct effect axioms. holding. One possible way of writing this is,

Non-monotonic Logic %.holds(f, s) A lholds(f, result(a, s)). We cannot hope that classical logic will generate the We note this is a very different way of expressing frame axioms for us, as classical logic is monotonic and that an action has an effect than by adding an effect will not infer any sentence that might be denied by axiom. An effect axiom gives sufficient conditions to a an additional fact. Thus we will have to use a logic change to occur. Here we merely say that the change without this property, a non-monotonic logic. happened once. As we are dealing with deterministic Non-monotonic logics are logics which can make de- change this means that there is some condition that fault assumptions which can be withdrawn later. We allows the change to happen. are looking at an assumption, “the tendency of facts to We now want to state some preconditions for the endure over time” (Hanks and McDermott 1986). This action a changing the fluent f from true to false. We seems to be exactly the sort of problem at which non- can write that f’ holding is a precondition for a to monotonic reasoning should excel. In this paper we use change f, by writing: circumscription, a non-monotonic logic introduced by Vs.lholds(f’, s) + (holds(f, s) E holds(f, result(a, s))) McCarthy (1986). We write Circ(A; a; KP) for the sen- tence of second order logic that minimizes the formulas This is also very different to the way we specify pre-

450 KNOWLEDGE REPRESENTATION Definition: 2 The second minimization should be done at lower The consequences of a theory A in the language of the priority than the first, as we do not want to introduce situation calculus, under the frame assumption, writ- an effect of an action merely so that we can have fewer are the consequences of, preconditions for that change. Thus what we do is to ten bframe, minimize 1, and then to minimize 2. This uses the Circ ( A A 4 A 3; ubl, ub:! ) . encoding of priority as iterated circumscription intro- duced by Lifschitz (1985). On simple examples this agrees with the proposals This is our solution to the precondition problem- of Baker (1991), Lifschitz and Kartha,( 1995) Lin and minimize the changes that actions make, and then as- Reiter(1994) , Haugh( 1987) (Sandewall 1994) etc. It sume that these changes occur as often as possible. differs in example that include disjunctions of effect ax- We now need unique name axioms for fluents and ioms with domain constraints, or observations. These actions and our function sit, issues are explored in detail in a paper by the author currently under review. UNA[~] A UNA[z] A UNA[sit]. (3) We now have two ways of describing action. We first look at simple examples to see how we use the first We have introduced a function sit here, which maps sets of fluents to situations. This exists to guarantee method. We then analyze the elaboration tolerance of that the domain of situation will be large enough. Cir- the two proposals. cumscription does not compare structures that have different domains, thus in the absence of an assertion Examples that there are more than one situation, our solution The standard examples against which any solution to will have models with only one situation. the frame problem must be tested are the Yale Shoot- For simplicity we look at the case where we have ing Problem, (YSP) and the Stanford Murder Mystery, deterministic change, that is, what is true at the next an explanation problem, essentially the YSP where we situation is fully described given the fluents that holds reason backwards in time. at the current situation, and the action that occurs. The Language of the Situation Calculus

Vufss’.(Vlf’.holds(f’, s’) E holds(f’, s)) + The situation calculus of order n, L~i,, is a language hold@, result(a, s)) EZ holds(f, result(a, s’)) (4) of nth order logic with equality, with three disjoint sorts: fluent names, actions, and situations3. It con- tains exactly one predicate constant, holds, in addition Definition: 1 to equality, one function constant result(u, s) which re- Given a theory A in the language of the situation cal- turns a situation, a function sit(g) from predicates of culus, the consequences of A under the precondition fluents to situations, and one situation constant SO, default, written A kp 4, are the consequences of the plus a set of fluent constants 7, and a set of action following circumscription. constants E. When we wish to specify the action and Circ(Circ(A A 4 A 3; 1; holds); 2; holds) fluent constants we write LFit (f, E).

The Frame Problem The Yale Shooting Problem We now consider a solution to the frame problem intro- The Yale Shooting Problem (YSP) is the problem of correctly predicting what happens in the following sce- duced by the author. We define abl and uba as syntac- nario. An individual named Fred is known to be alive, tic sugar, so that conjunctions of ub’s correspond to in- and a gun is known to be loaded initially. We know stances of change formulas, that is effect axioms, with that shooting causes an individual to not be alive if the the extra proviso that the fluent affected is changed. gun is loaded. We then want to infer that Fred will be More correctly, every change formula is equivalent to dead after we wait, then shoot. In some solutions to a conjunction of ub’s. the frame problem there are multiple minimal models, some where the gun becomes unloaded during the wait ubl (a, f, g) *&i action, some where the shoot action kills Fred. ‘ds.(V’f’.g(f’) z holds(f’, s)) A holds(f, s) + We axiomatize the Yale shooting problem as follows: lholds(f, result(u, s)), %.holds(alive, s) A lholds(alive, result(shoot, s)) Vs.lholds(loaded, s) --+ (holds(alive, s) + uba(u, f, g) *g (5) Vs.(Vlf’.g(f’) E holds(f’, s)) A lholds(f, s) + holds( alive, result (shoot, s))) (6) holds(f, result(u, s)). holds(alive, SO) A holds(loaded, SO) Now that we have written our change formulas as (7) instantiations of a pair of second order logic sentences 3We use s with primes and indices to range over situa- we can minimize all change formulas by minimizing tions, and Q and f to range over actions and fluent names, these sentences. and g to range over unary predicates of fluents.

REASONING ABOUT ACTION 451 Proposition: 2 The sentence that we should infer is: We note that the if the theory A is the conjunc- lholds(alive, result (shoot, result (wait, SO)))) tion of 6, 8, over the language Lii, ({loaded, alive}, {wait, shoot}) then

Proposition: 1 A kP holds(loaded, SO) We note that the if the theory A is the conjunc- tion of 6, 7, over the language L~i,({loaded,alive}, Roof: The proof is similar to the previous proposi- {wait, shoot}) then tion. We get the same results for our minimization. We get there is a change from alive to dead exactly in A bp -Iholds( alive, result (shoot, result (wait, SO)))) situations where the gun is loaded and Fred is alive. We can also derive that there is no change when the Proof: When we carry out the first minimization, as gun is not loaded. This suffices to show that the gun per 1, we find that, was loaded initially. !ls.holds(f, s) A lholds(f, result(a, s)) E f = alive A a = shoot An extra precondition 13s.lholds( f, s) A holds( f, result (a, s)) We now give an example of adding a precondition, something impossible in axiomatizations using effect This yields, axioms. We add the precondition that to kill someone Vs. f # alive V a # shoot -+ by shooting, they must not have a bullet proof vest. holds(f, s) -+ holds(f, result(u, s)) We write this by saying that there is no change if they Vs.lholds( f, s) -+ lholds( f, result (a, s)). are wearing a vest, Vs.holds(vest, s) A holds(alive, s) + We now minimize the preconditions for an action suc- holds(alive, result (shoot, s)) (9) ceeding, as per 2. This gives,

(Vs.(Vf’.holds(f’, s) E g(f’)) + Proposition: 3 (holds( f, s) G holds( f, result (a, s)))) G We note that the if the theory A is the conjunction (lg(alive) A f = loaded A a = shoot)V of 6, 7, 9, over the language Lii,({loaded, alive, vest}, a # shoot V f # loaded {wait, shoot}) then

This is clearly minimal, if it is satisfiable, as it is en- lholds(alive, result (shoot, result (wait, SO)))) E tailed by 6. It is consistent, as the intended model A +P lholds(vest , SO) satisfies it. From the above sentence we can derive that SO Proof: This is again similar to the proof of the first agrees with result(wait, SO) for all fluents. From the proposition. The first minimization is the same. The initial condition that the gun is loaded and that Fred second adds the vest clause in the second line. We in- is alive we get that holds(loaded, result(wait, SO)) and stantiate g with what held at the first situation as be- holds(alive, result (wait, SO)). From this and the above fore. When we simplify we have a holds(vest, s) clause. sentence, instantiating g with Xf .holds( f, SO), we get We carry out the same reasoning using determinism to that derive that there is a change, modulo the vest clause.

Zls.(Vf’holds(f’, s) -. holds(f’, sO))A We note that in previous axiomatizations using effect holds(alive, s) A lholds( alive, result (shoot, s)) axioms, adding

Finally, using the axiom of deterministic change, 4, Vs.lholds(vest, s) A holds(loaded, s) + we can derive that all situations that agree with the lholds( alive, result (shoot, s)) existentially quantified sentence above have the re- quired change. This includes result (wait, SO). Thus would have no effect, as this is a consequence of the the change from alive to dead occurs when we shoot in axiom, this situation, as required.l Vs.holds(loaded, s) + lholds(alive, result (shoot, s)) Explanation Elaboration Tolerance We now consider reasoning backwards in time. If we In this section we now define elaboration tolerance for know that after waiting and shooting that Fred was theories of action. We show that the usual frame as- dead, and that he was alive initially, we should derive sumption only allows adding effects and removing pre- that the gun must have been loaded initially. conditions, while the precondition assumption we in- holds( alive, SO) A troduce allows adding effects and adding preconditions. lholds(alive, result (shoot, result (wait, SO)))) (8) We can now define adding effects, and adding and removing preconditions.

452 KNOWLEDGE REPRESENTATION Definition: 3 Aclsnowledgments A model !-Z of the language Lf,,(f, a) has the possible This research is supported in part by ARPA/Rome effect of the action A E ii sometimes changing F E 7 if Laboratory planning initiative- under grant (ONR) U 1 !ls.l(holds(F, s) E holds(F, result(A, s))). N00014-94-1-0775. I am grateful to John McCarthy ” A model U has more effects that a model %3, if the and Anna Patterson who gave useful criticism on ear- set of possible effects of fu is a superset of the possible lier drafts. effects of %. References This states that an action has a possible effect if it ever Baker, A. B. 1991. Nonmonotonic reasoning in the changes the value of that fluent. framework of situation calculus. Artificial Intelligence 49:5-23. D&nit ion: 4 Let P, N be two disjoint subsets of 7. A model U of Costello, T. 1997. Non-monotonicity and Change. the language Lii,(f, a) has the preconditions P, N for PhD thesis, Stanford University. the action A E E changing F E f if Ginsberg, M. L., and D. E. Smith. 1988. Reasoning about action II: The qualification problem. Artificial holds(JP, s) v 1 AfEN lholds(f, s) + Q-l& Intelligence 35(3):311-342. ’ ’ (holds(F, s) G holds(F, result(A, s))). Hanks, S., and D. McDermott. 1986. Default reason- A model U has more preconditions that a model %, ing, nonmonotonic logics and the frame problem. In if the set of preconditions of rU is a superset of the Proc. National Conference on Artificial Intelligence precondition of %. (AAAI ‘86), 328-333, Philadelphia.

This states that a precondition, defined by a set of flu- Haugh, B. A. 1987. Simple causal minimization for ents that must hold, and a set of fluents whose nega- temporal persistence and projection. In Proc. Na- tion must hold, is a precondition for an action having tional Conference on Artificial Intelligence (AAA I an effect, if whenever the precondition if false, then the ‘87). action does not have that effect. Kartha, G. N., and V. Lifschitz. 1995. A simple formalization of actions using circumscription. In Theorem: 4 Proc. Thirteenth International Joint Conference on Let T be a theory in Lsit, and o a sentence in Lsit. Let Artificial Intelligence (IJCA I ‘95). the non-monotonic consequences of T under F=J,,,, be Lifschitz, V. 1985. Computing Circumscription. In Tl, let the non-monotonic consequences of TA cx under Proc. Ninth International Joint Conference on Arti- rame be T2. Then, every model of T’ has as many t=/ ficial Intelligence (IJCAI ‘85), 121-127. effects as some model of Tr , and every model of 3”i has as many preconditions as some model of Tz. Lifschitz, V. 1987. Formal Theories of Action. In Let the non-monotonic consequences of T under kp F. Brown (Ed.), The Frame Problem in Artificial In- be T’, let the non-monotonic consequences of T A cy telligence, 121-127. Morgan Kaufmann. under kp be T!. The every model of T4 has as many Lin, F., and R. Reiter. 1994. State Constraints Revis- effects as some model of Ts, and every model of T4 has ited. Journal of Logic and Computation 4(5):655-678. as many preconditions as some model of Ta. Lin, F., and U. Shoham. 1995. Provably Correct The elaborations that are possible under bframe, are Theories of Action. Journal of the ACM 42(2):293- adding effects, and removing preconditions. In con- 320. trast we can add both preconditions and effects using McCarthy, J. 1986. Applications of Circumscription the precordition assumption. to Formalizing Commonsense Knowledge. Artificial Intelligence 28:89-116. McCarthy, J., and P. IIayes. 1969. Some Philosoph- Adding preconditions is natural, at least as natural ical Problems From the Standpoint of Artificial In- as removing preconditions. We have shown that it is telligence. In D. Michie (Ed.), Machine Intelligence possible to axiomatize change so that both effects and 4, 463-502. Edinburgh, UK: Edinburgh University preconditions can be added. We have shown that usual Press. axiomatization allow adding effects and removing pre- Sandewall, E. 1994. Features and FZuents. Oxford conditions. University Press. We suggest that this method of considering what elaborations can be made to a theory is a useful tool for investigating non-monotonic phenomena. Consid- ering the elaboration tolerance of a non-monotonic so- lution gives a clear statement of the applicability of the method.

REASONING ABOUT ACTION 453