Simon Fraser University School of Computing Science Lecture #7: Constructive Search Methods ☞ Introduction • Historical perspective from Artificial Intelligence. • AI = Knowledge-based Search. • Nondeterministic programming (PLANNER, ABSYS, Conniver, Prolog) 2Remember a binary CSP is equivalent to satisfying ∃ ∈ , ..., ∃ ∈ a1 D1 an Dn ∧ ... ∧ ∧ ∧ ∧ ∧ P1(a1) Pn(an) P12(a1,a2) P13(a1,a3) ... Pn-1,n(an-1,an). • Constructing a solution is an obvious approach. October 2, 1998 Copyright © 1997 by Bill Havens 1 of 52 Simon Fraser University School of Computing Science Constructive Methods for solving CSPs: 1. British Museum Algorithm 2. Generate and Test 3. Chronological Backtracking 4. Intelligent Backtracking (various) 5. Network Consistency Algorithms (various) 6. Hybrid approaches (various) October 2, 1998 Copyright © 1997 by Bill Havens 2 of 52 Simon Fraser University School of Computing Science British Museum Algorithm • Worst search algorithm possible • Works but never used! • Not guaranteed to terminate (incomplete). • Apocryphal organization to British Museum collection. BritishMuseum (D) For some random tuple (a1,..., an) ∈ D1 × ... × Dn, If P1(a1) ∧ ... ∧ Pn(an) ∧ P12(a1,a2) ∧ P13(a1,a3) ∧ ... ∧ Pn-1,n(an-1,an) then Halt (a1,..., an) end. 2What’s wrong with this algorithm? October 2, 1998 Copyright © 1997 by Bill Havens 3 of 52 Simon Fraser University School of Computing Science Generate and Test Approach ☞ Overview • Better search algorithm • Works fine and frequently used! (for small problems) n • Complexity: O[d ] where d = |Di| and n = |V|. • Very simple to implement (nested “for-loops” approach) • Algorithm . October 2, 1998 Copyright © 1997 by Bill Havens 4 of 52 Simon Fraser University School of Computing Science Generate&Test (V, D) For some value a1 ∈ D1 do, For some value a2 ∈ D2 do, . For some value an ∈ Dn do, If P1(a1) ∧ ... ∧ Pn(an) ∧ P12(a1,a2) ∧ P13(a1,a3) ∧ ... ∧ Pn-1,n(an-1,an) then Halt (a1, a2, ... , an) end. 2The C-hackers favourite search algorithm ! October 2, 1998 Copyright © 1997 by Bill Havens 5 of 52 Simon Fraser University School of Computing Science Chronological Backtracking ☞ Overview • Most popular search algorithm (Prolog + most Expert Systems Shells) • More formally Depth-First Search (more later). • Also slow! • Complexity: O[kn] for k < d • AB-Satisfy called initially: AB-Satisfy(V, D, ()) • Algorithm . October 2, 1998 Copyright © 1997 by Bill Havens 6 of 52 Simon Fraser University School of Computing Science AB-Satisfy (V, D, X) If null(V) then Halt(X) Let Vi = first(V) Let Di = first(D) For each value ai ∈ Di do, if P1(a1)∧ ... ∧P1i(a1,ai)∧ ... ∧Pji(aj,ai) for j < i then AB-Satisfy (rest(V), rest(D), cons(ai, X)) Return false. end. 2So why is Chronological Backtracking used in modern applications (e.g. Prolog)? October 2, 1998 Copyright © 1997 by Bill Havens 7 of 52 Simon Fraser University School of Computing Science Backtracking Example ☞ Map Colouring again red green 4 blue ≠ ≠ red 3 red green ≠ green blue 1 blue ≠ ≠ 2 red green blue m Can AB-Satisfy colour this map? • Transcript of recursive calls . October 2, 1998 Copyright © 1997 by Bill Havens 8 of 52 Simon Fraser University School of Computing Science Transcript nV D X Vi ai Comments 0{V1V2V3V4}{D1D2D3D4}{} V1 r Initial call 1{V2V3V4}{D2D3D4} {r} V2 r ~(V1=r, V2=r) ≠ 1V2 gV1 V2 2{V3V4}{D3D4} {g,r} V3 r ~(V1=r, V3=r) 2V3 g ~(V2=g, V3=g) ≠ ≠ 2V3 bV1 V2 V3 3{V4}{D4} {b,g,r} V4 r ~(V1=r, V4=r) ≠ ≠ 3V4 gV1 V4, V3 V4 4 {} {} {g,b,g,r} done m Note: only shallow backtracking in this example. October 2, 1998 Copyright © 1997 by Bill Havens 9 of 52 Simon Fraser University School of Computing Science Tree Search Algorithms ☞ Introduction • Search problems can frequently be visualized as searching a tree of nodes. • Each node represents a problem state • Each branch represents a search choice. • The root represents the start node of the search. • Some leaves of the tree represent goal states. • All other leaf nodes are failures. October 2, 1998 Copyright © 1997 by Bill Havens 10 of 52 Simon Fraser University School of Computing Science Example: October 2, 1998 Copyright © 1997 by Bill Havens 11 of 52 Simon Fraser University School of Computing Science • Map Colouring example again as tree search · V1 = r V1 = g V1 = b · · · · · · ·················· V2 = r V2 = g V2 = b ··· V3 = r ·· V3 = r V3 = g V3 = b ··· V4 = r ·· V4 = rV4 = gV4 = b October 2, 1998 Copyright © 1997 by Bill Havens 12 of 52 Simon Fraser University School of Computing Science Generality of Tree Search ☞ Many problems can be represented as Tree Search • Map Colouring (above) • Route Planning (eg- Travelling Salesman Problems) • Resource Allocation (scheduling) • Puzzles (eg- Crosswords) m CSPs in general can be represented as search trees as follows: • The root of the tree corresponds to zero variables being instantiated. • Each level in the tree corresponds to another variable being instantiated. October 2, 1998 Copyright © 1997 by Bill Havens 13 of 52 Simon Fraser University School of Computing Science • The branches of each node correspond to value assignments to the variable. • The leaves of the tree correspond to consistent assignments of all variables (and hence solutions). 2Note: the chronological backtrack procedure called AB-Satisfy constructs and searches a tree of variable assignments as above. October 2, 1998 Copyright © 1997 by Bill Havens 14 of 52 Simon Fraser University School of Computing Science Generalized Tree Search Algorithm ☞ Overview • Tree search can be viewed as variations on a general method. • L = an initial list of nodes to be search (usually one root node). GeneralSearch(L) If null(L) then return(failure). Let n = some node removed from L. If GoalNode(n) then return(n) else GeneralSearch(append(children(n), L))). end. October 2, 1998 Copyright © 1997 by Bill Havens 15 of 52 Simon Fraser University School of Computing Science • Note that GeneralSearch is an iterative algorithm (c.f. AB- Satisfy) • Variations in search method focus on selection of which node to examine in step 2 above. • We will consider both Blind and Heuristic Search Methods. ☞ Blind Search Methods: 1. Breadth-First Search 2. Depth-First Search 3. Iterative Deepening Reference: M. Ginsberg (1993) Essentials of Artificial Intelligence, Morgan Kauf- mann, San Francisco. October 2, 1998 Copyright © 1997 by Bill Havens 16 of 52 Simon Fraser University School of Computing Science Breadth-First Search ☞ Introduction • A basic type of blind search (i.e. no heuristic information available). • Explores the search tree uniformly one level at a time. • Every node at depth d is explored before any node at depth d+1. • Initially called with a list L of start nodes. October 2, 1998 Copyright © 1997 by Bill Havens 17 of 52 Simon Fraser University School of Computing Science Algorithm Breadth-FirstSearch(L) If null(L) then return(failure). Let n = first(L). Let L = rest(L). If GoalNode(n) then return(n) else Breadth-FirstSearch(append(L, children(n)))). end. • Note that every node already on list L will be explored before any of the newly added children to L. October 2, 1998 Copyright © 1997 by Bill Havens 18 of 52 Simon Fraser University School of Computing Science Breadth-First Search Order • Given the search tree below [from Ginsberg] Figure 3.1 • Breadth-FirstSearch provides the following search order (assuming the children of a node are expanded left-to-right) Figure 3.2 October 2, 1998 Copyright © 1997 by Bill Havens 19 of 52 Simon Fraser University School of Computing Science ☞ Complexity • Let b = branching factor of the tree (for CSPs, b = domain size) • Let d = depth of the tree (to a goal node). • Space Complexity is O[bd/2] since list L must contain on average 1/2 of the complete fringe of the tree constructed to depth d. • Time Complexity is likewise approximately O[bd/2] since the algorithm must spend time constructing the fringe of the tree. ☞ Properties • Sound procedure for searching arbitrary (infinite trees). Why? • Finds shortest solution path possible. October 2, 1998 Copyright © 1997 by Bill Havens 20 of 52 Simon Fraser University School of Computing Science Depth-First Search ☞ Overview • Most popular form of blind search (e.g. AB-Satisfy above). • Search tree is examined from the top-down. • Every descendent of a node is explored before any of its siblings. • Initially called with a list L of start nodes. October 2, 1998 Copyright © 1997 by Bill Havens 21 of 52 Simon Fraser University School of Computing Science Algorithm Depth-FirstSearch(L) If null(L) then return(failure). Let n = first(L). Let L = rest(L). If GoalNode(n) then return(n) else Depth-FirstSearch(append(children(n), L))). end. October 2, 1998 Copyright © 1997 by Bill Havens 22 of 52 Simon Fraser University School of Computing Science Depth-First Search Order • The search order for the previous search tree is: Figure 3.4 ☞ Complexity of Depth-First Search • Space Complexity is O[d] since the algorithm need only remember the path from the root to the current node at depth d. • Time Complexity is O[bd/2] since the algorithm on average will explore 1/2 of the entire search space. • See Ginsberg[93] for a more detailed discussion. October 2, 1998 Copyright © 1997 by Bill Havens 23 of 52 Simon Fraser University School of Computing Science Properties of Depth-First Search • Cannot search infinite trees. • Cannot find shortest path solution. • Much better space complexity than Breadth-First Search. October 2, 1998 Copyright © 1997 by Bill Havens 24 of 52 Simon Fraser University School of Computing Science Iterative Deepening Search m Is it possible to have best properties of Breadth- First and Depth-First Search methods? ☞ Overview • Space requirements of Depth-First Search.