Cooperative and Adaptive Algorithms Lecture 6 Allaa (Ella) Hilal, Spring 2017 May, 2017 1 Minute Quiz (Ungraded) • Select if these statement are true (T) or false (F): Statement T/F Reason Uniform-cost search is a special case of Breadth- first search Breadth-first search, depth- first search and uniform- cost search are special cases of best- first search. A* is a special case of uniform-cost search. ECE457A, Dr. Allaa Hilal, Spring 2017 2 1 Minute Quiz (Ungraded) • Select if these statement are true (T) or false (F): Statement T/F Reason Uniform-cost search is a special case of Breadth- first F • Breadth- first search is a special case of Uniform- search cost search when all step costs are equal. Breadth-first search, depth- first search and uniform- T • Breadth-first search is best-first search with f(n) = cost search are special cases of best- first search. depth(n); • depth-first search is best-first search with f(n) = - depth(n); • uniform-cost search is best-first search with • f(n) = g(n). A* is a special case of uniform-cost search. F • Uniform-cost search is A* search with h(n) = 0. ECE457A, Dr. Allaa Hilal, Spring 2017 3 Informed Search Strategies Hill Climbing Search ECE457A, Dr. Allaa Hilal, Spring 2017 4 Hill Climbing Search • Tries to improve the efficiency of depth-first. • Informed depth-first algorithm. • An iterative algorithm that starts with an arbitrary solution to a problem, then attempts to find a better solution by incrementally changing a single element of the solution. • It sorts the successors of a node (according to their heuristic values) before adding them to the list to be expanded. • If the change produces a better solution, an incremental change is made to the new solution, repeating until no further improvements can be found. ECE457A, Dr. Allaa Hilal, Spring 2017 5 Hill Climbing on a surface of states Height Defined by Evalua;on Func;on f(n) ECE457A, Dr. Allaa Hilal, Spring 2017 6 Hill Climbing Search • Looks one step ahead to determine if any successor is better than the current state; if there is, move to the best successor. • Rule: Ø If there exists a successor s for the current state n such that • h(s) < h(n) and • h(s) ≤ h(t) for all the successors t of n, Ø Then move from n to s. Otherwise, halt at n. ECE457A, Dr. Allaa Hilal, Spring 2017 7 Hill Climbing Search • Similar to Greedy search in that it uses h(), but does not allow backtracking or jumping to an alternative path since it doesn’t “remember” where it has been. “Like climbing Everest in thick fog with amnesia” • Corresponds to Beam search with a beam width of 1 (i.e., the maximum size of the nodes list is 1). • Not complete since the search will terminate at "local minima," "plateaus," and "ridges." ECE457A, Dr. Allaa Hilal, Spring 2017 8 Hill Climbing Search ECE457A, Dr. Allaa Hilal, Spring 2017 9 Hill Climbing Search • Hill climbing can be applied to the travelling salesman problem. • It is easy to find an initial solution that visits all the cities but will be very poor compared to the optimal solution. • The algorithm starts with such a solution and makes small improvements to it, such as switching the order in which two cities are visited. • Eventually, a much shorter route is likely to be obtained ECE457A, Dr. Allaa Hilal, Spring 2017 10 Hill Climbing Search Nodes traversed: A,C,F, K A h=7 B C h=6 h=3 D E h=20 h=5 F G h=50 h=2 H I h=5 h=7 J K h=4 h=0 L P Q h=48 h=8 O P Q h=8 11 h=45 h=9 Hill climbing example 2 8 3 1 2 3 start 1 6 4 h = -4 goal 8 4 h = 0 7 5 7 6 5 -5 -5 -2 2 8 3 1 2 3 1 4 h = -3 8 4 h = -1 7 6 5 7 6 5 -3 -4 2 3 2 3 h = -3 1 8 4 1 8 4 h = -2 7 6 5 7 6 5 -4 f (n) = -(number of tiles out of place) 12 Informed Search Strategies Heuristics ECE457A, Dr. Allaa Hilal, Spring 2017 13 Types of Heuristics • Perfect heuristic: If h(n) = h*(n) for all n, then only the nodes on the optimal solution path will be expanded. So, no extra work will be performed. • Null heuristic: If h(n) = 0 for all n, then this is an admissible heuristic and A* acts like Uniform-Cost Search. • Better heuristic: If h1(n) < h2(n) <= h*(n) for all non-goal nodes, then h2 is a better heuristic than h1 • If A1* uses h1, and A2* uses h2, then every node expanded by A2* is also expanded by A1*. • In other words, A1 expands at least as many nodes as A2*. • We say that A2* is better informed than A1*. • The closer h is to h*, the fewer extra nodes that will be expanded ECE457A, Dr. Allaa Hilal, Spring 2017 14 Types of Heuristics 5 8 1 2 3 4 2 1 4 5 6 7 3 6 7 8 n goal h1(n) = number of misplaced tiles = 6 is admissible h2(n) = sum of distances of each tile to goal = 13 is admissible Let h1 and h2 be two admissible and consistent heuristics such that for all nodes N: h1(n) ≤ h2(n). Then, every node expanded by A* using h2 is also expanded by A* using h1. h2 is more informed than h1 h1(n) < h2(n) both are admissible, h2 is more informed than h1 ECE457A, Dr. Allaa Hilal, Spring 2017 15 Breaking Ties 5 8 1 2 3 4 2 1 4 5 6 7 3 6 7 8 Goal 5 8 5 2 8 5 8 4 2 1 4 1 4 2 1 7 3 6 7 3 6 7 3 6 h1=6 h1=5 h1=6 h2=12 h2=12 h2=12 16 How to obtain admissible heuristics? • Can be derived by relaxing the problem • If the rules of the 8-puzzle are relaxed so a tile can move anywhere, then h1 gives the shortest cost to the goal. • If the rules are relaxed so the tile can move to any adjacent square, then h2 gives shortest cost. ECE457A, Dr. Allaa Hilal, Spring 2017 17 Game Playing as Search MiniMax Strategy ECE457A, Dr. Allaa Hilal, Spring 2017 18 Game Playing As Search • In games, there is an opponent. • Need to search for best move and wait for the opponent response to search again from the new board configuration for best move. • Time is limited to find goal in each search • In games, the objective is not only to find the best way to the goal but also beat the opponent ECE457A, Dr. Allaa Hilal, Spring 2017 19 Types of Games • Perfect information: • Each player has complete information about the opponent’s position and available choices • Deterministic: chess, checkers • Chance: Backgammon, monopoly • Imperfect information: • Each player does not have complete information about the opponent’s position and available choices • Chance: poker, bridge • Two players or multiple players ECE457A, Dr. Allaa Hilal, Spring 2017 20 Two Players, perfect information • Player and Opponent. • Ideally, the player expands the game tree taking into consideration all the possible moves of the opponent till end of the game with leaf nodes as win, lose, draw. ECE457A, Dr. Allaa Hilal, Spring 2017 21 Tic Tac Toe – Let’s play (1 min) ECE457A, Dr. Allaa Hilal, Spring 2017 22 Max Min strategy • Player – MAX Opponent – MIN • Commonly used with zero sum games (whenever, one player wins, the other one loses). • Minimax principle: A principle for decision-making by which, when presented with two various and conflicting strategies, one should determine and use the strategy that will minimize the maximum losses that could occur. • Given a game tree, the optimal strategy can be determined by examining the minimax value of each node, which we write as MINIMAX- VALUE(^). ECE457A, Dr. Allaa Hilal, Spring 2017 23 Max Min strategy • The minimax value of a node is the utility of being in the corresponding state, assuming that both players play optimally from there to the end of the game. • The minimax value of a terminal state is just its utility. • Label each level in game tree with MAX (player) and MIN (opponent) • Label leaves with evaluation of player • Go through the game tree • if father node is MAX then label the node with the maximal value of its successors • if father node is MIN then label the node with the minimal value of its successors ECE457A, Dr. Allaa Hilal, Spring 2017 24 Max Min for Tic Tac Toe 25 Game tree Current board: X’s move 26 Game tree Current board: X’s move How many nodes? 27 Limited Depth and Evaluation Function • Expanding the complete game tree is only feasible for simple games. • Tic Tac Toe has average branching factor b = 5 and average (moves) depth d = 9. • Total states of 9! => 362,880 states in the game tree • Checkers has b =35 and d =100. • Taking into consideration symmetrical and repeated states the game tree may be even less. • Still for checkers 1040. • For Chess, it is only feasible to explore a limited depth of the game tree and to use a board evaluation function to estimate the worth of this node to the player. ECE457A, Dr. Allaa Hilal, Spring 2017 28 Evaluation Function Current board: O’s move • Evaluation function f(n) measures “goodness” of board configuration n.