Journal of Arti cial Intelligence Research 9 1998 1{36 Submitted 1/98; published 8/98 The Computational Complexity of Probabilistic Planning Michael L. Littman [email protected] Department of Computer Science, Duke University Durham, NC 27708-0129 USA Judy Goldsmith [email protected] Department of Computer Science, University of Kentucky Lexington, KY 40506-0046 USA Martin Mundhenk [email protected] FB4 - Theoretische Informatik, Universitat Trier D-54286 Trier, GERMANY Abstract We examine the computational complexity of testing and nding small plans in proba- bilistic planning domains with b oth at and prop ositional representations. The complexity of plan evaluation and existence varies with the plan typ e sought; we examine totally ordered plans, acyclic plans, and lo oping plans, and partially ordered plans under three natural de nitions of plan value. We show that problems of interest are complete for a PP PP variety of complexity classes: PL, P, NP, co-NP, PP,NP , co-NP , and PSPACE. In PP the pro cess of proving that certain planning problems are complete for NP ,weintro duce PP a new basic NP -complete problem, E-Majsat, which generalizes the standard Bo olean satis ability problem to computations involving probabilistic quantities; our results suggest that the development of go o d heuristics for E-Majsat could b e imp ortant for the creation of ecient algorithms for a wide variety of problems. 1. Intro duction Recent work in arti cial-intelligence planning has addressed the problem of nding e ec- tive plans in domains in which op erators have probabilistic e ects Drummond & Bresina, 1990; Mansell, 1993; Drap er, Hanks, & Weld, 1994; Ko enig & Simmons, 1994; Goldman & Bo ddy, 1994; Kushmerick, Hanks, & Weld, 1995; Boutilier, Dearden, & Goldszmidt, 1995; Dearden & Boutilier, 1997; Kaelbling, Littman, & Cassandra, 1998; Boutilier, Dean, & Hanks, 1998. Here, an \e ective" or \successful" plan is one that reaches a goal state with sucient probability. In probabilistic propositional planning , op erators are sp eci ed in a Bayes network or an extended STRIPS-like notation, and the planner seeks a recip e for cho osing op erators to achieve a goal con guration with some user-sp eci ed probability. This problem is closely related to that of solving a Markov decision pro cess Puterman, 1994 when it is expressed in a compact representation. In previous work Goldsmith, Lusena, & Mundhenk, 1996; Littman, 1997a, we exam- ined the complexity of determining whether an e ective plan exists for completely observable domains; the problem is EXP-complete in its general form and PSPACE-complete when lim- ited to p olynomial-depth plans. A p olynomial-depth, or p olynomial-horizon, plan is one that takes at most a p olynomial numb er of actions b efore terminating. For these results, c 1998 AI Access Foundation and Morgan Kaufmann Publishers. All rights reserved. Littman, Goldsmith & Mundhenk plans are p ermitted to b e arbitrarily large ob jects|there is no restriction that a valid plan need haveany sort of compact p olynomial-size representation. Because they place no restrictions on the size of valid plans, these earlier results are not directly applicable to the problem of nding valid plans. It is p ossible, for example, that for a given planning domain, the only valid plans require exp onential space and exp onential time to write down. Knowing whether or not such plans exist is simply not very imp ortant b ecause they are intractable to express. In the present pap er, we consider the complexity of a more practical and realistic problem|that of determining whether or not a plan exists in a given restricted form and of a given restricted size. The plans we consider take several p ossible forms that have b een used in previous planning work: totally ordered plans, partially ordered plans, totally ordered conditional plans, and totally order lo oping plans. In all cases, we limit our attention to plans that can b e expressed in size b ounded by a p olynomial in the size of the sp eci cation of the problem. This way, once we determine that a plan exists, we can use this information to try to write it down in a reasonable amount of time and space. In the deterministic planning literature, several authors have addressed the computa- tional complexity of determining whether a valid plan exists, of determining whether a plan exists of a given cost, and of nding the valid plans themselves under a variety of assump- tions Chapman, 1987; Bylander, 1994; Erol, Nau, & Subrahmanian, 1995; Backstrom, 1995; Backstrom & Neb el, 1995. These results provide lower b ounds hardness results for analogous probabilistic planning problems since deterministic planning is a sp ecial case. In deterministic planning, optimal plans can b e represented by a simple sequence of op erators a totally ordered plan. In probabilistic planning, a go o d conditional plan will often p er- form b etter than any totally ordered unconditional plan; therefore, we need to consider the complexity of the planning pro cess for a richer set of plan structures. For ease of discussion, we only explicitly describ e the case of planning in completely observable domains. This means that the state of the world is known at all times during plan execution, in spite of the uncertainty of state transitions. We know that the state of the system is sucient information for cho osing actions optimally Puterman, 1994, however, representing such a universal plan is often impractical in prop ositional domains in which the size of the state space is exp onential in the size of the domain representation. For this reason, we consider other typ es of plan structures based on simple nite-state machines. Because the typ e of plans we consider do not necessarily use the full state of the system to makeevery decision, our results carry over to partially observable domains, although wedo not explore this fact in detail in the presentwork. The computational problems we lo ok at are complete for a variety of complexity classes ranging from PL probabilistic logspace to PSPACE. Two results are deserving of sp ecial mention b ecause they concern problems closely related to ones b eing actively addressed by arti cial-intelligence researchers; rst, the problem of evaluating a totally ordered plan 1 in a compactly represented planning domain is PP-complete. A compactly represented P PP 1. The class PP is closely related to the somewhat more familiar P; To da 1991 showed that P =P . Roughly sp eaking, this means that P and PP are equally p owerful when used as oracles. The counting class P has already b een recognized by the arti cial-intelligence community as an imp ortant complexity class in computations involving probabilistic quantities, such as b elief-network inference Roth, 1996. 2 Complexity of Probabilistic Planning planning domain is one that is describ ed bya two-stage temp oral Bayes network Boutilier et al., 1998 or similar notation. Second, the problem of determining whether a valid totally ordered plan exists for a PP compactly represented planning domain is NP -complete. Whereas the class NP can be thought of as the set of problems solvable by guessing the answer and checking it in p olyno- PP mial time, the class NP can b e thought of as the set of problems solvable by guessing the answer and checking it using a probabilistic p olynomial-time PP computation. It is likely PP that NP characterizes many problems of interest in the area of uncertainty in arti cial intelligence; this pap er and earlier work Goldsmith et al., 1996; Mundhenk, Goldsmith, & Allender, 1997a; Mundhenk, Goldsmith, Lusena, & Allender, 1997b give initial evidence of this. 1.1 Planning-Domain Representations A probabilistic planning domain M = hS ;s ; A;t;Gi is characterized by a nite set of states 0 S , an initial state s 2 S , a nite set of op erators or actions A, and a set of goal states 0 G S . The application of an action a in a state s results in a probabilistic transition 0 0 to a new state s according to the probability transition function t, where ts; a; s is the 0 probability that state s is reached from state s when action a is taken. The ob jectiveisto cho ose actions, one after another, to move from the initial state s to one of the goal states 0 2 with probability ab ove some threshold . The state of the system is known at all times fully observable and so can b e used to cho ose the action to apply. We are concerned with two main representations for planning domains: at represen- tations, which enumerate states explicitly, and propositional representations sometimes called compact, structured, or factored representations, which view states as assignments to a set of Bo olean state variables or prop ositions. Prop ositional representations can rep- resent many domains exp onentially more compactly than can at representations. In the at representation, the transition function t is represented by a collection of 3 jSjjSj matrices, one for each action. In the prop ositional representation, this typ e of jS j jS j matrix would b e huge, so the transition function must b e expressed another way. In the probabilistic planning literature, two p opular representations for prop ositional plan- ning domains are probabilistic state-space op erators PSOs Kushmerick et al., 1995 and two-stage temp oral Bayes networks 2TBNs Boutilier et al., 1995. Although these repre- sentations di er in the typ e of planning domains they can express naturally Boutilier et al., 1998, they are computationally equivalent; a planning domain expressed in one represen- tation can b e converted in p olynomial time to an equivalent planning domain expressed in the other with at most a p olynomial increase in representation size Littman, 1997a.