Journal of Articial Intelligence Research Submitted published The Computational Complexity of Probabilistic Planning Michael L Littman mlittmancsdukeedu Department of Computer Science Duke University Durham NC USA Judy Goldsmith goldsmitcsengrukyedu Department of Computer Science University of Kentucky Lexington KY USA Martin Mundhenk mundhenktiunitrierde FB Theoretische Informatik Universitat Trier D Trier GERMANY Abstract We examine the computational complexity of testing and nding small plans in proba bilistic planning domains with b oth at and prop ositional representations The complexity of plan evaluation and existence varies with the plan typ e sought we examine totally ordered plans acyclic plans and lo oping plans and partially ordered plans under three natural denitions of plan value We show that problems of interest are complete for a PP PP variety of complexity classes PL P NP coNP PP NP coNP and PSPACE In PP the pro cess of proving that certain planning problems are complete for NP we intro duce PP a new basic NP complete problem EMajsat which generalizes the standard Bo olean satisability problem to computations involving probabilistic quantities our results suggest that the development of go o d heuristics for EMajsat could b e imp ortant for the creation of ecient algorithms for a wide variety of problems Intro duction Recent work in articialintelligence planning has addressed the problem of nding eec tive plans in domains in which op erators have probabilistic eects Drummond Bresina Mansell Drap er Hanks Weld Ko enig Simmons Goldman Bo ddy Kushmerick Hanks Weld Boutilier Dearden Goldszmidt Dearden Boutilier Kaelbling Littman Cassandra Boutilier Dean Hanks Here an eective or successful plan is one that reaches a goal state with sucient probability In probabilistic propositional planning op erators are sp ecied in a Bayes network or an extended STRIPSlike notation and the planner seeks a recip e for cho osing op erators to achieve a goal conguration with some usersp ecied probability This problem is closely related to that of solving a Markov decision pro cess Puterman when it is expressed in a compact representation In previous work Goldsmith Lusena Mundhenk Littman a we exam ined the complexity of determining whether an eective plan exists for completely observable domains the problem is EXPcomplete in its general form and PSPACEcomplete when lim ited to p olynomialdepth plans A p olynomialdepth or p olynomialhorizon plan is one that takes at most a p olynomial numb er of actions b efore terminating For these results c AI Access Foundation and Morgan Kaufmann Publishers All rights reserved Littman Goldsmith Mundhenk plans are p ermitted to b e arbitrarily large ob jectsthere is no restriction that a valid plan need have any sort of compact p olynomialsize representation Because they place no restrictions on the size of valid plans these earlier results are not directly applicable to the problem of nding valid plans It is p ossible for example that for a given planning domain the only valid plans require exp onential space and exp onential time to write down Knowing whether or not such plans exist is simply not very imp ortant b ecause they are intractable to express In the present pap er we consider the complexity of a more practical and realistic problemthat of determining whether or not a plan exists in a given restricted form and of a given restricted size The plans we consider take several p ossible forms that have b een used in previous planning work totally ordered plans partially ordered plans totally ordered conditional plans and totally order lo oping plans In all cases we limit our attention to plans that can b e expressed in size b ounded by a p olynomial in the size of the sp ecication of the problem This way once we determine that a plan exists we can use this information to try to write it down in a reasonable amount of time and space In the deterministic planning literature several authors have addressed the computa tional complexity of determining whether a valid plan exists of determining whether a plan exists of a given cost and of nding the valid plans themselves under a variety of assump tions Chapman Bylander Erol Nau Subrahmanian Backstrom Backstrom Neb el These results provide lower b ounds hardness results for analogous probabilistic planning problems since deterministic planning is a sp ecial case In deterministic planning optimal plans can b e represented by a simple sequence of op erators a totally ordered plan In probabilistic planning a go o d conditional plan will often p er form b etter than any totally ordered unconditional plan therefore we need to consider the complexity of the planning pro cess for a richer set of plan structures For ease of discussion we only explicitly describ e the case of planning in completely observable domains This means that the state of the world is known at all times during plan execution in spite of the uncertainty of state transitions We know that the state of the system is sucient information for cho osing actions optimally Puterman however representing such a universal plan is often impractical in prop ositional domains in which the size of the state space is exp onential in the size of the domain representation For this reason we consider other typ es of plan structures based on simple nitestate machines Because the typ e of plans we consider do not necessarily use the full state of the system to make every decision our results carry over to partially observable domains although we do not explore this fact in detail in the present work The computational problems we lo ok at are complete for a variety of complexity classes ranging from PL probabilistic logspace to PSPACE Two results are deserving of sp ecial mention b ecause they concern problems closely related to ones b eing actively addressed by articialintelligence researchers rst the problem of evaluating a totally ordered plan in a compactly represented planning domain is PPcomplete A compactly represented P PP The class PP is closely related to the somewhat more familiar P To da showed that P P Roughly sp eaking this means that P and PP are equally p owerful when used as oracles The counting class P has already b een recognized by the articialintelligence community as an imp ortant complexity class in computations involving probabilistic quantities such as b eliefnetwork inference Roth Complexity of Probabilistic Planning planning domain is one that is describ ed by a twostage temp oral Bayes network Boutilier et al or similar notation Second the problem of determining whether a valid totally ordered plan exists for a PP compactly represented planning domain is NP complete Whereas the class NP can b e thought of as the set of problems solvable by guessing the answer and checking it in p olyno PP mial time the class NP can b e thought of as the set of problems solvable by guessing the answer and checking it using a probabilistic p olynomialtime PP computation It is likely PP that NP characterizes many problems of interest in the area of uncertainty in articial intelligence this pap er and earlier work Goldsmith et al Mundhenk Goldsmith Allender a Mundhenk Goldsmith Lusena Allender b give initial evidence of this PlanningDomain Representations A probabilistic planning domain M hS s A t G i is characterized by a nite set of states S an initial state s S a nite set of op erators or actions A and a set of goal states G S The application of an action a in a state s results in a probabilistic transition to a new state s according to the probability transition function t where ts a s is the probability that state s is reached from state s when action a is taken The ob jective is to cho ose actions one after another to move from the initial state s to one of the goal states with probability ab ove some threshold The state of the system is known at all times fully observable and so can b e used to cho ose the action to apply We are concerned with two main representations for planning domains at represen tations which enumerate states explicitly and propositional representations sometimes called compact structured or factored representations which view states as assignments to a set of Bo olean state variables or prop ositions Prop ositional representations can rep resent many domains exp onentially more compactly than can at representations In the at representation the transition function t is represented by a collection of jS j jS j matrices one for each action In the prop ositional representation this typ e of jS j jS j matrix would b e huge so the transition function must b e expressed another way In the probabilistic planning literature two p opular representations for prop ositional plan ning domains are probabilistic statespace op erators PSOs Kushmerick et al and twostage temp oral Bayes networks TBNs Boutilier et al Although these repre sentations dier in the typ e of planning domains they can express naturally Boutilier et al they are computationally equivalent a planning domain expressed in one represen tation can b e converted in p olynomial time to an equivalent planning domain expressed in the other with at most a p olynomial increase in representation size Littman a In this work we fo cus on a prop ositional representation called the sequentialeects tree representation ST Littman a which is a syntactic variant of TBNs with conditional probability tables represented as trees Boutilier et al This