Real World Search Problems
CS 331: Artificial Intelligence Uninformed Search
1 2
Assumptions About Our Simpler Search Problems Environment • Fully Observable • Deterministic • Sequential • Static • Discrete • Single-agent
3 4
Search Problem Formulation Example: Oregon Portland S = {Coos Bay, Newport,
A search problem has 5 components: McMinnville Corvallis, Junction City, Eugene, Medford, Albany, 1. A finite set of states S Salem 2. A non-empty set of initial states I S Initial State Lebanon, Salem, 3. A non-empty set of goal states G S Portland, McMinnville} Albany Lebanon I = {Corvallis} 4. A successor function succ(s) which takes a state Newport Corvallis s as input and returns as output the set of states G={Medford} you can reach from state s in one step. Coos Junction 5. A cost function cost(s,s’) which returns the non- Bay City Succ(Corvallis)={Albany, negative one-step cost of travelling from state s Eugene Newport, McMinnville, to s’. The cost function is only defined if s’ is a Junction City} successor state of s. Goal State Medford Cost(s,s’) = 1 for all transitions
5 6
1 Results of a Search Problem Search Tree
• Solution Corvallis Start with Initial State Path from initial state to goal state
Junction Corvallis Eugene Medford City • Solution quality Path cost (3 in this case) • Optimal solution Lowest path cost among all solutions (In this case, we found the optimal solution)
7 8
Search Tree Search Tree
Corvallis Is initial state the goal? Corvallis Apply Successor() function • Yes, return solution • No, apply Successor() Junction Junction McMinnville Albany Newport McMinnville Albany Newport City function City Queue McMinnville These nodes have not been Albany expanded yet. Call them the Junction City fringe. We’ll put them in a queue. Newport 9 10
Search Tree Search Tree
Corvallis Queue Corvallis Queue Albany Albany Junction City Junction City Junction Junction McMinnville Albany Newport McMinnville Albany Newport City Newport City Newport Portland Portland Portland Portland
Now remove a node from the queue. If it’s a goal Things to note: state, return the solution. Otherwise, call Successor() on it, and put the results in the queue. • Order in which you expand nodes (in this Repeat. example, we took the first node in the queue) 11 • Avoid repeated states
2 Tree-Search Pseudocode Tree-Search Pseudocode
Note: Goal test happens after we grab a node off the queue.
13 14
Tree-Search Pseudocode Uninformed Search
• No info about states other than generating successors and recognizing goal states Why are these parent node backpointers are • Later on we’ll talk about informed search – important? can tell if a non-goal state is more promising than another
15 16
Evaluating Uninformed Search Complexity
• Completeness 1. Branching factor (b) – maximum number of Is the algorithm guaranteed to find a solution successors of any node when there is one? 2. Depth (d) of the shallowest goal node • Optimality 3. Maximum length (m) of any path in the search Does it find the optimal solution? space • Time complexity How long does it take to find a solution? Time Complexity: number of nodes generated during • Space complexity search How much memory is needed to perform the search Space Complexity: maximum number of nodes stored in memory
17 18
3 Uninformed Search Algorithms Breadth-First Search
• Breadth-first search • Expand all nodes at a given depth before • Uniform-cost search any nodes at the next level are expanded • Depth-first search • Implement with a FIFO queue • Depth-limited search • Iterative Deepening Depth-first Search • Bidirectional search
19 20
Breadth First Search Example Breadth First Search Example
A A A A
B C B C B C B C
D E F G D E F G D E F G D E F G
Not yet reached Closed (expanded) nodes Not yet reached Closed (expanded) nodes
Open nodes (on the fringe) Current node to be expanded Open nodes (on the fringe) Current node to be expanded
21 22
Evaluating BFS Evaluating BFS
Complete? Complete? Yes provided branching factor is finite Optimal? Optimal?
Time Complexity Time Complexity Space Complexity Space Complexity
23 24
4 Evaluating BFS Evaluating BFS
Complete? Yes provided branching factor is Complete? Yes provided branching factor is finite finite Optimal? Yes if step costs are identical Optimal? Yes if step costs are identical
Time Complexity Time Complexity b+b2+b3+…+bd+(bd+1-b)= O(bd+1) Space Complexity Space Complexity
25 26
Evaluating BFS Uniform-cost Search
Complete? Yes provided branching factor is • What if step costs are not equal? finite • Recall that BFS expands the shallowest node Optimal? Yes if step costs are identical • Now we expand the node with the lowest path cost Time Complexity b+b2+b3+…+bd+(bd+1-b)= • Uses priority queues O(bd+1) Space Complexity O(bd+1) Note: Gets stuck if there is a zero-cost action leading back to the same state. For completeness and optimality, we require the cost of every step to be ≥ Exponential time and space complexity make BFS impractical for all but the smallest problems 27 28
Evaluating Uniform-cost Search Evaluating Uniform-cost Search
Complete? Complete? Yes provided branching factor is finite and step costs ≥ for small positive Optimal? Optimal?
Time Complexity Time Complexity
Space Complexity Space Complexity
29 30
5 Evaluating Uniform-cost Search Evaluating Uniform-cost Search
Complete? Yes provided branching factor is Complete? Yes provided branching factor is finite and step costs ≥ for small finite and step costs ≥ for small positive positive Optimal? Yes Optimal? Yes
Time Complexity Time Complexity O(b1+floor(C*/)) where C* is the cost of the optimal solution Space Complexity Space Complexity
31 32
Evaluating Uniform-cost Search Depth-first Search
Complete? Yes provided branching factor is • Expands the deepest node in the current finite and step costs ≥ for small fringe of the search tree positive • Implemented with a LIFO queue Optimal? Yes
Time Complexity O(b1+floor(C*/)) where C* is the cost of the optimal solution Space Complexity O(b1+floor(C*/)) where C* is the cost of the optimal solution
33 34
Depth-first Search Example Depth-first Search Example
A A A A A A
B C B C B C B C B C B C
D E F G D E F G D E F G D E F G D E F G D E F G
H I J K L M N O H I J K L M N O H I J K L M N O H I J K L M N O H I J K L M N O H I J K L M N O
A A A A A A
B C B C B C B C B C B C
D E F G D E F G D E F G D E F G D E F G D E F G
H I J K L M N O H I J K L M N O H I J K L M N O H I J K L M N O H I J K L M N O H I J K L M N O
Not yet reached Expanded nodes on current path Current node to be Not yet reached Expanded nodes on current path Current node to be expanded expanded Expanded nodes with no Expanded nodes with no On fringe but M Goal state On fringe but M Goal state unexpanded descendants in the fringe (can be unexpanded descendants in the fringe (can be removed from memory) removed from memory)
6 Evaluating Depth-first Search Evaluating Depth-first Search
Complete? Complete? Yes on finite graphs. No if there is an infinitely long path with no Optimal? solutions. Optimal? Time Complexity
Space Complexity Time Complexity Space Complexity
37 38
Evaluating Depth-first Search Evaluating Depth-first Search
Complete? Yes on finite graphs. No if there is Complete? Yes on finite graphs. No if there is an infinitely long path with no an infinitely long path with no solutions. solutions.
Optimal? No (Could expand a much longer Optimal? No (Could expand a much longer path than the optimal one first) path than the optimal one first) Time Complexity Time Complexity O(bm)
Space Complexity Space Complexity
39 40
Evaluating Depth-first Search Depth-limited Search
Complete? Yes on finite graphs. No if there is • Solves infinite path problem by using an infinitely long path with no predetermined depth limit l solutions. • Nodes at depth l are treated as if they have no successors Optimal? No (Could expand a much longer path than the optimal one first) • Can use knowledge of the problem to Time Complexity O(bm) determine l (but in general you don’t know this in advance) Space Complexity O(bm)
41 42
7 Evaluating Depth-limited Search Evaluating Depth-limited Search
Complete? Complete? No (If shallowest goal node beyond depth limit) Optimal? Optimal?
Time Complexity Time Complexity
Space Complexity Space Complexity
43 44
Evaluating Depth-limited Search Evaluating Depth-limited Search
Complete? No (If shallowest goal node Complete? No (If shallowest goal node beyond depth limit) beyond depth limit) Optimal? No (If depth limit > depth of Optimal? No (If depth limit > depth of shallowest goal node and we shallowest goal node and we expand a much longer path than expand a much longer path than the optimal one first) the optimal one first) Time Complexity Time Complexity O(bl)
Space Complexity Space Complexity
45 46
Iterative Deepening Depth-first Evaluating Depth-limited Search Search Complete? No (If shallowest goal node • Do DFS with depth limit 0, 1, 2, … until a beyond depth limit) goal is found Optimal? No (If depth limit > depth of • Combines benefits of both DFS and BFS shallowest goal node and we expand a much longer path than the optimal one first) Time Complexity O(bl)
Space Complexity O(bl)
47 48
8 Iterative Deepening Depth-first IDDFS Example Search Example Limit = 3
Limit = 0 A A A A A B C B C B C D E F G D E F G D E F G Limit = 1 A A A A H I J K L M N O H I J K L M N O H I J K L M N O B C B C B C B C
Limit = 2 A A A A B C B C B C B C A A A D E F G D E F G D E F G D E F G B C B C B C D E F G D E F G D E F G A A A A H I J K L M N O H I J K L M N O H I J K L M N O B C B C B C B C D E F G D E F G D E F G D E F G 49 50
IDDFS Example Evaluating Iterative Deepening Depth-first Search Limit = 3 (Continued)
A A A Complete? B C B C B C D E F G D E F G D E F G Optimal? H I J K L M N O H I J K L M N O H I J K L M N O Time Complexity
A A A Space Complexity B C B C B C D E F G D E F G D E F G H I J K L M N O H I J K L M N O H I J K L M N O
51 52
Evaluating Iterative Deepening Evaluating Iterative Deepening Depth-first Search Depth-first Search
Complete? Yes provided branching factor is Complete? Yes provided branching factor is finite finite Optimal? Optimal? Yes if the path cost is a nondecreasing function of the depth of the node Time Complexity Time Complexity
Space Complexity Space Complexity
53 54
9 Evaluating Iterative Deepening Evaluating Iterative Deepening Depth-first Search Depth-first Search
Complete? Yes provided branching factor is Complete? Yes provided branching factor is finite finite Optimal? Yes if the path cost is a Optimal? Yes if the path cost is a nondecreasing function of the nondecreasing function of the depth of the node depth of the node Time Complexity O(bd) Time Complexity O(bd)
Space Complexity Space Complexity O(bd)
55 56
Isn’t Iterative Deepening Wasteful? Is Iterative Deepening Wasteful? Total # of nodes generated by iterative • Actually, no! Most of the nodes are at the bottom level, doesn’t matter that upper levels are generated deepening: multiple times. (d)b + (d-1)b2 +… + (1)bd = O(bd+1) • To see this, add up the 4th column below: Total # of nodes generated by BFS:
Depth # of # of times Total # of nodes generated at 2 d d+1 d+1 nodes generated depth d b + b +… + b + b - b = O(b ) 1 b d (d)b 2 b2 d-1 (d-1)b2 In general, iterative deepening is the preferred : : : : uninformed search method when there is a large d bd 1 (1)bd search space and the depth of the solution is not known 57 58
Bidirectional Search Bidirectional Search
• Run one search forward from the initial state • Needs an efficiently computable • Run another search backward from the goal Predecessor() function • Stop when the two searches meet in the middle • What if there are several goal states? – Create a new dummy goal state whose predecessors are the actual goal states • Difficult when the goal is an abstract description like “no queen attacks another queen”
59 60
10 Evaluating Bidirectional Search Evaluating Bidirectional Search
Complete? Complete? Yes provided branching factor is finite and both directions use BFS
Optimal? Optimal?
Time Complexity Time Complexity
Space Complexity Space Complexity
61 62
Evaluating Bidirectional Search Evaluating Bidirectional Search
Complete? Yes provided branching factor is Complete? Yes provided branching factor is finite and both directions use BFS finite and both directions use BFS
Optimal? Yes if the step costs are all Optimal? Yes if the step costs are all identical and both directions use identical and both directions use BFS BFS Time Complexity Time Complexity O(bd/2)
Space Complexity Space Complexity
63 64
Evaluating Bidirectional Search Avoiding Repeated States
Complete? Yes provided branching factor is • Tradeoff between space and time! finite and both directions use BFS • Need a closed list which stores every expanded node (memory requirements could make search Optimal? Yes if the step costs are all infeasible) identical and both directions use • If the current node matches a node on the closed BFS list, discard it (ie. discard the newly discovered path) Time Complexity O(bd/2) • We’ll refer to this algorithm as GRAPH-SEARCH Space Complexity O(bd/2) (At least one search tree • Is this optimal? Only for uniform-cost search or must be kept in memory for the breadth-first search with constant step costs. membership check) 65 66
11 GRAPH-SEARCH Things You Should Know
• How to formalize a search problem • How BFS, UCS, DFS, DLS, IDS and Bidirectional search work • Whether the above searches are complete and optimal plus their time and space complexity • The pros and cons of the above searches
67 68
12