Automated (AI) Planning

Planning by state-space search Automated (AI) Planning Progression Planning tasks & Search Regression Search algorithms for planning

Uninformed search

Carmel Domshlak Heuristic search State-space search

Automated (AI) Planning

Planning by state-space search: one of the big success stories of AI state-space search many planning algorithms based on state-space search Introduction (we’ll see some other algorithms later, though) Classification Progression

will be the focus of this and the following topics Regression we assume prior knowledge of basic search algorithms Search algorithms for uninformed vs. informed planning systematic vs. local Uninformed search

background on search: Russell & Norvig, Artificial Heuristic Intelligence – A Modern Approach, chapters 3 and 4 search Satisficing or optimal planning?

Automated (AI) Planning Must carefully distinguish two different problems: Planning by satisficing planning: any solution is OK state-space search (although shorter solutions typically preferred) Introduction optimal planning: plans must have shortest possible length Classification Progression

Regression

Both are often solved by search, but: Search algorithms for details are very different planning Uninformed almost no overlap between good techniques for satisficing search

planning and good techniques for optimal planning Heuristic search many problems that are trivial for satisficing planners are impossibly hard for optimal planners Planning by state-space search

Automated (AI) Planning

How to apply search to planning? many choices to make! Planning by state-space search Introduction Choice 1: Search direction Classification progression: forward from initial state to goal Progression Regression

regression: backward from goal states to initial state Search algorithms for bidirectional search planning

Uninformed search

Heuristic search Planning by state-space search

Automated (AI) Planning

How to apply search to planning? many choices to make! Planning by state-space search Introduction Choice 2: Search space representation Classification search nodes are associated with states Progression Regression

search nodes are associated with sets of states Search algorithms for planning

Uninformed search

Heuristic search Planning by state-space search

Automated (AI) Planning

How to apply search to planning? many choices to make! Planning by state-space search Introduction Choice 3: Search algorithm Classification uninformed search: Progression depth-first, breadth-first, iterative depth-first, . . . Regression Search algorithms for heuristic search (systematic): planning ∗ ∗ ∗ greedy best-first, A , Weighted A , IDA ,... Uninformed search heuristic search (local): Heuristic hill-climbing, simulated annealing, beam search, . . . search Planning by state-space search

Automated (AI) Planning

How to apply search to planning? many choices to make! Planning by state-space search Introduction Choice 4: Search control Classification heuristics for informed search algorithms Progression Regression

pruning techniques: invariants, symmetry elimination, Search helpful actions pruning, . . . algorithms for planning

Uninformed search

Heuristic search Search-based satisficing planners

Automated (AI) Planning

Planning by state-space FF (Hoffmann & Nebel, 2001) search Introduction search direction: forward search Classification search space representation: single states Progression Regression search algorithm: enforced hill-climbing (informed local) Search algorithms for heuristic: FF heuristic (inadmissible) planning pruning technique: helpful actions (incomplete) Uninformed search

Heuristic one of the best satisficing planners search Search-based optimal planners

Automated (AI) Planning

Planning by HHH state-space Fast Downward + h (Helmert, Haslum & Hoffmann, 2007) search Introduction search direction: forward search Classification search space representation: single states Progression Regression ∗ search algorithm:A (informed systematic) Search algorithms for heuristic: merge-and-shrink abstractions (admissible) planning

Uninformed pruning technique: none search

Heuristic one of the best optimal planners search Our plan for the next lectures

Automated (AI) Planning

Choices to make: Planning by state-space 1 search direction: progression/regression/both search Introduction this chapter Classification Progression 2 search space representation: states/sets of states Regression

this chapter Search algorithms for 3 search algorithm: uninformed/heuristic; systematic/local planning this chapter Uninformed search 4 search control: heuristics, pruning techniques Heuristic following chapters search Planning by forward search: progression

Automated (AI) Planning

Planning by Progression: Computing the successor state appo(s) of a state state-space s with respect to an operator o. search Progression Progression planners find solutions by forward search: Overview Example start from initial state Regression Search iteratively pick a previously generated state and progress it algorithms for through an operator, generating a new state planning Uninformed solution found when a goal state generated search Heuristic pro: very easy and efficient to implement search Search space representation in progression planners

Automated (AI) Planning Two alternative search spaces for progression planners: Planning by 1 search nodes correspond to states state-space search when the same state is generated along different paths, Progression it is not considered again (duplicate detection) Overview pro: fast Example con: memory intensive (must maintain closed list) Regression Search 2 search nodes correspond to operator sequences algorithms for planning different operator sequences may lead to identical states Uninformed (transpositions) search

pro: can be very memory-efficient Heuristic con: much wasted work (often exponentially slower) search first alternative usually preferable Progression planning example (depth-first search)

Automated (AI) Planning

I Planning by state-space search

Progression Overview Example Regression

Search algorithms for planning

Uninformed search

Heuristic G search Progression planning example (depth-first search)

Automated (AI) Planning

I Planning by state-space search

Progression Overview Example Regression

Search algorithms for planning

Uninformed search

Heuristic G search Progression planning example (depth-first search)

Automated (AI) Planning

I Planning by state-space search

Progression Overview Example Regression

Search algorithms for planning

Uninformed search

Heuristic G search Progression planning example (depth-first search)

Automated (AI) Planning

I Planning by state-space search

Progression Overview Example Regression

Search algorithms for planning

Uninformed search

Heuristic G search Progression planning example (depth-first search)

Automated (AI) Planning

I Planning by state-space search

Progression Overview Example Regression

Search algorithms for planning

Uninformed search

Heuristic G search Progression planning example (depth-first search)

Automated (AI) Planning

I Planning by state-space search

Progression Overview Example Regression

Search algorithms for planning

Uninformed search

Heuristic G search Progression planning example (depth-first search)

Automated (AI) Planning

I Planning by state-space search

Progression Overview Example Regression

Search algorithms for planning

Uninformed search

Heuristic G search Progression planning example (depth-first search)

Automated (AI) Planning

I Planning by state-space search

Progression Overview Example Regression

Search algorithms for planning

Uninformed search

Heuristic G search Progression planning example (depth-first search)

Automated (AI) Planning

I Planning by state-space search

Progression Overview Example Regression

Search algorithms for planning

Uninformed search

Heuristic G search Progression planning example (depth-first search)

Automated (AI) Planning

I Planning by state-space search

Progression Overview Example Regression

Search algorithms for planning

Uninformed search

Heuristic G search Forward search vs. backward search

Automated (AI) Planning

Going through a transition graph in forward and backward Planning by state-space directions is not symmetric: search forward search starts from a single initial state; Progression backward search starts from a set of goal states Regression Overview Example when applying an operator o in a state s in forward STRIPS 0 direction, there is a unique successor state s ; Search 0 algorithms for if we applied operator o to end up in state s , planning there can be several possible predecessor states s Uninformed search most natural representation for backward search in planning Heuristic associates sets of states with search nodes search Planning by backward search: regression

Automated (AI) Planning

Regression: Computing the possible predecessor states regro(S) of a set of states S with respect to the last operator o that was Planning by state-space applied. search Regression planners find solutions by backward search: Progression Regression start from set of goal states Overview Example iteratively pick a previously generated state set and STRIPS Search regress it through an operator, generating a new state set algorithms for planning

solution found when a generated state set includes the Uninformed initial state search Heuristic search Pro: can handle many states simultaneously Con: basic operations complicated and expensive Search space representation in regression planners

Automated (AI) Planning

Planning by state-space identify state sets with logical formulae: search Progression

search nodes correspond to state sets Regression Overview each state set is represented by a logical formula: Example φ represents {s ∈ S | s |= φ} STRIPS Search algorithms for many basic search operations like detecting duplicates are planning

NP-hard or coNP-hard Uninformed search

Heuristic search φ1 = regr−→(G) φ3 φ2 φ1 G φ2 = regr−→(φ1) φ3 = regr−→(φ2),I |= φ3

Regression planning example (depth-first search)

Automated (AI) Planning

Planning by state-space search

Progression I Regression Overview Example STRIPS Search algorithms for planning

Uninformed search

Heuristic search

G φ1 = regr−→(G) φ3 φ2 φ1 φ2 = regr−→(φ1) φ3 = regr−→(φ2),I |= φ3

Regression planning example (depth-first search)

Automated (AI) Planning G

Planning by state-space search

Progression I Regression Overview Example STRIPS Search algorithms for planning

Uninformed search

Heuristic search

G φ3 φ2 φ2 = regr−→(φ1) φ3 = regr−→(φ2),I |= φ3

Regression planning example (depth-first search)

Automated (AI) Planning φ1 = regr−→(G) φ1 G

Planning by state-space search

Progression I Regression Overview Example STRIPS Search algorithms for planning

Uninformed search

Heuristic search

G φ3

φ3 = regr−→(φ2),I |= φ3

Regression planning example (depth-first search)

Automated (AI) Planning φ1 = regr−→(G) φ2 φ1 G φ = regr (φ ) 2 −→ 1 Planning by state-space search

Progression I Regression Overview Example STRIPS Search algorithms for planning

Uninformed search

Heuristic search

G Regression planning example (depth-first search)

Automated (AI) Planning φ1 = regr−→(G) φ3 φ2 φ1 G φ = regr (φ ) 2 −→ 1 Planning by φ = regr (φ ),I |= φ state-space 3 −→ 2 3 search

Progression I Regression Overview Example STRIPS Search algorithms for planning

Uninformed search

Heuristic search

G Regression for STRIPS planning tasks

Automated (AI) Planning Definition (STRIPS planning task) Planning by A planning task is a STRIPS planning task if all operators are state-space search STRIPS operators and the goal is a conjunction of literals. Progression

Regression Overview Regression for STRIPS planning tasks is very simple: Example Goals are conjunctions of literals l ∧ · · · ∧ l . STRIPS 1 n Search algorithms for First step: Choose an operator that makes some of planning l1, . . . , ln true and makes none of them false. Uninformed search

Second step: Remove goal literals achieved by the Heuristic operator and add its preconditions. search Outcome of regression is again conjunction of literals. Automated (AI) Planning

Choices to make: Planning by state-space 1 search direction: progression/regression/both search above Progression Regression 2 search space representation: states/sets of states Search above algorithms for planning 3 search algorithm: uninformed/heuristic; systematic/local Nodes and states Search for this chapter planning Uninformed 4 search control: heuristics, pruning techniques search Heuristic next chapters search Search

Automated (AI) Planning

Planning by state-space search

Progression Search algorithms are used to find solutions (plans) for Regression transition systems in general, not just for planning tasks. Search algorithms for Planning is one application of search among many. planning Nodes and states Search for planning Uninformed search

Heuristic search Required ingredients for search

Automated (AI) Planning

A general search algorithm can be applied to any transition Planning by state-space system for which we can define the following three operations: search init(): generate the initial state Progression Regression

is-goal(s): test if a given state is a goal state Search algorithms for succ(s): generate the set of successor states of state s, planning Nodes and states along with the operators through which they are reached Search for (represented as pairs ho, s0i of operators and states) planning Uninformed Together, these three functions form a search space (a very search Heuristic similar notion to a transition system). search Search for planning: progression

Automated (AI) Planning

Planning by Let Π = hV,I,O,Gi be a planning task. state-space search

Search space for progression search Progression states: all states of Π (assignments to V ) Regression Search init()= I algorithms for planning 0 0 succ(s)= {ho, s i | o ∈ O, s = app (s)} Nodes and states o Search for ( planning true if s |= G is-goal(s)= Uninformed false otherwise search Heuristic search Classification of search algorithms

Automated (AI) Planning uninformed search vs. heuristic search:

uninformed search algorithms only use the basic Planning by state-space ingredients for general search algorithms search heuristic search algorithms additionally use heuristic Progression functions which estimate how close a node is to the goal Regression Search algorithms for systematic search vs. local search: planning Nodes and states systematic algorithms consider a large number of search Search for planning nodes simultaneously Uninformed local search algorithms work with one (or a few) candidate search Heuristic solutions (search nodes) at a time search not a black-and-white distinction; there are crossbreeds (e. g., enforced hill-climbing) Classification: what works where in planning?

Automated (AI) Planning uninformed vs. heuristic search:

Planning by For satisficing planning, heuristic search vastly state-space outperforms uninformed algorithms on most domains. search For optimal planning, the difference is less pronounced. An Progression Regression efficiently implemented uninformed algorithm is not easy Search to beat in most domains. (But doable! We’ll see that algorithms for planning later.) Nodes and states Search for planning systematic search vs. local search: Uninformed search

For satisficing planning, the most successful algorithms are Heuristic somewhere between the two extremes. search For optimal planning, systematic algorithms are required. Uninformed search algorithms Less relevant for planning, yet not irrelevant

Automated (AI) Planning

Planning by state-space Popular uninformed systematic search algorithms: search breadth-first search Progression depth-first search Regression Search algorithms for iterated depth-first search planning

Uninformed Popular uninformed local search algorithms: search Heuristic random walk search Heuristic search algorithms: systematic

Automated (AI) Planning Heuristic search algorithms are the most common and Planning by overall most successful algorithms for classical planning. state-space search

Progression

Popular systematic heuristic search algorithms: Regression greedy best-first search Search algorithms for A∗ planning ∗ Uninformed weighted A search ∗ Heuristic IDA search Heuristics depth-first branch-and-bound search Systematic search breadth-first heuristic search Local search ... Heuristic search algorithms: local

Automated (AI) Planning Heuristic search algorithms are the most common and Planning by overall most successful algorithms for classical planning. state-space search

Progression

Popular heuristic local search algorithms: Regression hill-climbing Search algorithms for enforced hill-climbing planning Uninformed beam search search Heuristic tabu search search Heuristics genetic algorithms Systematic search simulated annealing Local search ... Heuristic search: idea

Automated (AI) Planning

Planning by state-space distance estimate search

Progression distance estimate goal Regression init Search algorithms for distance estimate planning Uninformed search

distance estimate Heuristic search Heuristics Systematic search Local search Required ingredients for heuristic search

Automated (AI) Planning

A heuristic search algorithm requires one more operation Planning by state-space in addition to the definition of a search space. search

Progression Definition (heuristic function) Regression

Let Σ be the set of nodes of a given search space. Search algorithms for A heuristic function or heuristic (for that search space) is a planning function h :Σ → N0 ∪ {∞}. Uninformed search

Heuristic The value h(σ) is called the heuristic estimate or heuristic search value of heuristic h for node σ. It is supposed to estimate the Heuristics Systematic search distance from σ to the nearest goal node. Local search What exactly is a heuristic estimate?

Automated (AI) Planning What does it mean that h “estimates the goal distance”? For most heuristic search algorithms, h does not need to Planning by state-space have any strong properties for the algorithm to work search (= be correct and complete). Progression However, the efficiency of the algorithm closely relates to Regression Search how accurately h reflects the actual goal distance. algorithms for planning ∗ For some algorithms, like A , we can prove strong formal Uninformed relationships between properties of h and properties of the search Heuristic algorithm (optimality, dominance, run-time for bounded search error, . . . ) Heuristics Systematic search For other search algorithms, “it works well in practice” is Local search often as good an analysis as one gets. Heuristics applied to nodes or states?

Automated (AI) Planning Most texts apply heuristic functions to states, not nodes. Planning by This is slightly less general than our definition: state-space Given a state heuristic h, we can define an equivalent node search heuristic as h0(σ) := h(state(σ)). Progression The opposite is not possible. (Why not?) Regression Search There is good justification for only allowing state-defined algorithms for heuristics: why should the estimated distance to the goal planning Uninformed depend on how we ended up in a given state s? search Heuristic We call heuristics which don’t just depend on state(σ) search pseudo-heuristics. Heuristics Systematic search In practice there are sometimes good reasons to have the Local search heuristic value depend on the generating path of σ Perfect heuristic

Automated (AI) Planning

Planning by state-space Let Σ be the set of nodes of a given search space. search Progression

Definition (optimal/perfect heuristic) Regression The optimal or perfect heuristic of a search space is the Search ∗ algorithms for heuristic h which maps each search node σ to the length of a planning shortest path from state(σ) to any goal state. Uninformed search ∗ Heuristic Note: h (σ) = ∞ iff no goal state is reachable from σ. search Heuristics Systematic search Local search Properties of heuristics

Automated (AI) Planning

Planning by A heuristic h is called state-space search ∗ safe if h (σ) = ∞ for all σ ∈ Σ with h(σ) = ∞ Progression goal-aware if h(σ) = 0 for all goal nodes σ ∈ Σ Regression ∗ Search admissible if h(σ) ≤ h (σ) for all nodes σ ∈ Σ algorithms for planning 0 0 consistent if h(σ) ≤ h(σ ) + 1 for all nodes σ, σ ∈ Σ Uninformed such that σ0 is a successor of σ search Heuristic search Relationships? Heuristics Systematic search Local search Greedy best-first search

Automated (AI) Planning Greedy best-first search (with duplicate detection)

open := new min-heap ordered by (σ 7→ h(σ)) Planning by state-space open.insert(make-root-node(init())) search

closed := ∅ Progression

while not open.empty(): Regression σ = open.pop-min() Search if state(σ) ∈/ closed: algorithms for closed := closed ∪ {state(σ)} planning Uninformed if is-goal(state(σ)): search

return extract-solution(σ) Heuristic for each ho, si ∈ succ(state(σ)): search 0 Heuristics σ := make-node(σ, o, s) Systematic 0 search if h(σ ) < ∞: Local search open.insert(σ0) return unsolvable Properties of greedy best-first search

Automated (AI) Planning

Planning by state-space one of the three most commonly used algorithms for search

satisficing planning Progression complete for safe heuristics (due to duplicate detection) Regression Search suboptimal unless h satisfies some very strong algorithms for assumptions (similar to being perfect) planning Uninformed invariant under all strictly monotonic transformations of h search (e. g., scaling with a positive constant or adding a Heuristic search constant) Heuristics Systematic search Local search A∗

∗ Automated A (with duplicate detection and reopening) (AI) Planning open := new min-heap ordered by (σ 7→ g(σ) + h(σ)) open.insert(make-root-node(init())) Planning by state-space closed := ∅ search

distance := ∅ Progression

while not open.empty(): Regression

σ = open.pop-min() Search if state(σ) ∈/ closed or g(σ) < distance(state(σ)): algorithms for planning closed := closed ∪ {state(σ)} Uninformed distance(σ) := g(σ) search

if is-goal(state(σ)): Heuristic return extract-solution(σ) search Heuristics for each ho, si ∈ succ(state(σ)): Systematic search σ0 := make-node(σ, o, s) Local search if h(σ0) < ∞: open.insert(σ0) return unsolvable 2+6 1+2 6 2+7 7

5 2+5 5 1+3 3+5 8 2+2 4+8 3+1

A∗ example Example

Automated (AI) Planning

Planning by state-space search

Progression

Regression 0+3 G 3 Search algorithms for I planning Uninformed search

Heuristic search Heuristics Systematic search Local search 2+6

6 2+7 7

5 2+5 5 3+5 8 2+2 4+8 3+1

A∗ example Example

Automated (AI) Planning

Planning by 1+2 state-space search 2 Progression Regression 0+3 G Search algorithms for planning I 3 Uninformed 1+3 search Heuristic search Heuristics Systematic search Local search 5 2+5 5 3+5 8 2+2 4+8 3+1

A∗ example Example

Automated (AI) Planning 2+6

Planning by 1+2 state-space 6 search 2+7 Progression 7 Regression 0+3 G Search algorithms for planning I 3 Uninformed 1+3 search Heuristic search Heuristics Systematic search Local search 5 3+5 8

4+8 3+1

A∗ example Example

Automated (AI) Planning 2+6

Planning by 1+2 state-space 6 search 2+7 Progression 7 Regression 0+3 G Search 5 algorithms for I 2+5 planning Uninformed 1+3 2 search Heuristic search 2+2 Heuristics Systematic search Local search 8

4+8

A∗ example Example

Automated (AI) Planning 2+6

Planning by 1+2 state-space 6 search 2+7 Progression 7 Regression 0+3 G Search 5 algorithms for I 2+5 planning 5 Uninformed 1+3 3+5 search 1 Heuristic search 2+2 Heuristics Systematic search Local search

3+1 Terminology for A∗

Automated (AI) Planning

Planning by state-space f value of a node: defined by f(σ) := g(σ) + h(σ) search generated nodes: nodes inserted into open at some point Progression Regression

expanded nodes: nodes σ popped from open for which the Search algorithms for test against closed and distance succeeds planning reexpanded nodes: expanded nodes for which Uninformed state(σ) ∈ closed upon expansion (also called reopened search Heuristic nodes) search Heuristics Systematic search Local search Properties of A∗

Automated the most commonly used algorithm for optimal planning (AI) Planning rarely used for satisficing planning complete for safe heuristics (even without duplicate Planning by state-space detection) search optimal if h is admissible and/or consistent (even without Progression duplicate detection) Regression Search never reopens nodes if h is consistent algorithms for planning

Uninformed Implementation notes: search Heuristic in the heap-ordering procedure, it is considered a good search idea to break ties in favour of lower h values Heuristics Systematic search can simplify algorithm if we know that we only have to Local search deal with consistent heuristics common, hard to spot bug: test membership in closed at the wrong time Weighted A∗

∗ Automated Weighted A (with duplicate detection and reopening) (AI) Planning open := new min-heap ordered by (σ 7→ g(σ) + W · h(σ)) open.insert(make-root-node(init())) Planning by state-space closed := ∅ search

distance := ∅ Progression

while not open.empty(): Regression

σ = open.pop-min() Search if state(σ) ∈/ closed or g(σ) < distance(state(σ)): algorithms for planning closed := closed ∪ {state(σ)} Uninformed distance(σ) := g(σ) search

if is-goal(state(σ)): Heuristic return extract-solution(σ) search Heuristics for each ho, si ∈ succ(state(σ)): Systematic search σ0 := make-node(σ, o, s) Local search if h(σ0) < ∞: open.insert(σ0) return unsolvable Properties of weighted A∗

Automated (AI) Planning + The weight W ∈ R0 is a parameter of the algorithm. Planning by for W = 0, behaves like breadth-first search state-space search for W = 1, behaves like A∗ Progression

for W → ∞, behaves like greedy best-first search Regression

Search algorithms for Properties: planning Uninformed one of the three most commonly used algorithms for search

satisficing planning Heuristic ∗ search for W > 1, can prove similar properties to A , replacing Heuristics Systematic optimal with bounded suboptimal: generated solutions are search Local search at most a factor W as long as optimal ones Hill-climbing

Automated (AI) Planning Hill-climbing Planning by state-space σ := make-root-node(init()) search

forever: Progression

if is-goal(state(σ)): Regression

return extract-solution(σ) Search 0 algorithms for Σ := { make-node(σ, o, s) | ho, si ∈ succ(state(σ)) } planning 0 σ := an element of Σ minimizing h (random tie breaking) Uninformed search

Heuristic can easily get stuck in local minima where immediate search Heuristics Systematic improvements of h(σ) are not possible search many variations: tie-breaking strategies, restarts Local search Enforced hill-climbing

Automated Enforced hill-climbing: procedure improve (AI) Planning

def improve(σ0): Planning by queue := new fifo-queue state-space search queue.push-back(σ0) closed := ∅ Progression while not queue.empty(): Regression σ = queue.pop-front() Search algorithms for if state(σ) ∈/ closed: planning

closed := closed ∪ {state(σ)} Uninformed search if h(σ) < h(σ0): return σ Heuristic search for each ho, si ∈ succ(state(σ)): Heuristics 0 Systematic σ := make-node(σ, o, s) search queue.push-back(σ0) Local search fail

breadth-first search for more promising node than σ0 Enforced hill-climbing (ctd.)

Automated (AI) Planning Enforced hill-climbing

σ := make-root-node(init()) Planning by state-space while not is-goal(state(σ)): search

σ := improve(σ) Progression

return extract-solution(σ) Regression

Search algorithms for one of the three most commonly used algorithms for planning Uninformed satisficing planning search can fail if procedure improve fails (when the goal is Heuristic search unreachable from σ0) Heuristics Systematic search complete for undirected search spaces (where the Local search successor relation is symmetric) if h(σ) = 0 for all goal nodes and only for goal nodes