Game Playing Reasons to Study Games: Modeling Strategic and Adversary Problems Is of General Interest (E.G
Total Page:16
File Type:pdf, Size:1020Kb
Why study games? Games playing: ideal world of hostile agents attempting to diminish one’s well being. Game Playing Reasons to study games: Modeling strategic and adversary problems is of general interest (e.g. economic situations). Handling opponents introduces uncertainty and requires contingency plans. Problems are usually complex and very often viewed as an indicator of intelligence. Characteristics: Well-formalized problems: clear description of the environment. Common-sense knowledge is not required. Rules are fixed. Number of nodes in the tree might be high, but memorizing the past is not needed. Why Study Games? Game Playing as Search Games offer: Playing a game involves searching for the best move. ¾ Intellectual Engagement ¾ Abstraction Board games clearly involve notions like start state, ¾ Representability ¾ Performance Measure goal state, operators, etc. We can this usefully import problem solving techniques that we have already met. Not all games are suitable for AI research. We will restrict ourselves to 2 person perfect information There are nevertheless important differences from board games. standard search problems. Special Characteristics of Game Playing Search AI and game playing Main differences are uncertainties introduced by Game playing (especially chess and Till now we assumed the situation is not going to change checkers) was the first test application of AI while we search. However …. Presence of an opponent. One do not know what the It involves a different type of search opponent will do until he/she does it. Game playing programs must solve the contingency problem. problem than we have considered up to now Complexity. Most interesting games are simply too – a solution is not a path, but simply the complex to solve by exhaustive means. Chess, for next move example, has an average branching factor of 35. Uncertainty also arises from not having the resources to The best move depends on what the compute a move which is guaranteed to be the best. opponent might do (adversary search) 1 Two-player games: motivation Two-player games Previous heuristics and search procedures are Search tree for each player remains the same only useful for single-player games Even levels i are moves of player A Odd levels i+1 are moves of player B no notion of turns: one or more cooperative agents Each player searches for a goal (different for does not take into account adversarial moves each) at their level Games are ideal to explore adversarial Each player evaluates the states according to strategies their heuristic function well-defined, abstract rules A’s best move brings B to the worst state most formulated as search problems A searches for its best move assuming B will really hard combinatorial problems -- chess!! also search for its best move MinMax search strategy Typical case Search for A’s best next move, so that no 2-person game matter what B does (in particular, choosing its Players alternate moves best move) A will be better off Zero-sum: one player’s loss is the other’s gain At each step, evaluate the value of all Perfect information: both players have access to descendants: take the maximum if it is A’s complete information about the state of the game. turn, or the minimum if it is B’s turn No information is hidden from either player. No chance (e.g., using dice) involved We need the estimated values d moves ahead Examples: Tic-Tac-Toe, Checkers, Chess, Go, Nim, generate all nodes to level d (BFS) Othello propagate Min-Max values up from leafs Not: Bridge, Solitaire, Backgammon, ... How to play a game Ingredients of 2-Person Games A way to play such a game is to: Players: We call them Max and Min. Consider all the legal moves you can make Initial State: Includes board position and whose turn it is. Compute the new position resulting from each move Evaluate each resulting position and determine which is Operators: These correspond to legal moves. best Terminal Test: A test applied to a board position which Make that move determines whether the game is over. In chess, for Wait for your opponent to move and repeat example, this would be a checkmate or stalemate situation. Key problems are: Utility Function: A function which assigns a numeric Representing the “board” value to a terminal state. For example, in chess the Generating all legal next boards outcome is win (+1), lose (-1) or draw (0). Note that by Evaluating a position convention, we always measure utility relative to Max. 2 Normal and Game Search Problem Chess as a First Choice Normal search problem: Max searches for a sequence It provides proof that a machine can actually do something of moves yielding a winning position and then makes that was thought to require intelligence. the first move in the sequence. It has simple rules. Game search problem: Clearly, this is not feasible in in a game situation where Min's moves must be taken into The world state is fully accessible to the program. consideration. Max must devise a strategy which leads to a winning position no matter what moves Min makes. The computer representation can be correct in every relevant detail. Some games Complexity of Searching • tic-tac-toe The presence of an opponent makes the decision problem • checkers more complicated. •Go Games are usually much too hard to solve. • Othello •chess Games penalize inefficiency very severely. • poker • bridge Things to Come… Perfect Decisions in Two-Person Games Perfect Decisions in Two-Player Games Imperfect Decisions Alpha-Beta Pruning Games That Include an Element of Chance 3 Games as Search Problem Two Player Game Some games can normally be defined in the form of a tree. Two players: Max and Min Objective of both Max and Min to optimize winnings Branching factor is usually an average of the possible Max must reach a terminal state with the highest utility number of moves at each node. Min must reach a terminal state with the lowest utility Game ends when either Max and Min have reached a This is a simple search problem: a player must search this terminal state search tree and reach a leaf node with a favorable outcome. upon reaching a terminal state points maybe awarded or sometimes deducted Search Problem Revisited Game Playing - Minimax Simple problem is to reach a favorable terminal state Game Playing Problem Not so simple... An opponent tries to thwart your every move Max must reach a terminal state with as high a utility as possible regardless of Min’s moves 1944 - John von Neumann outlined a search Max must develop a strategy that determines best possible method (Minimax) that maximised your move for each move Min makes. position whilst minimising your opponents Game Playing – Example Game Playing - Minimax Nim (a simple game) Starting with 7 tokens, the game is small Start with a single pile of tokens enough that we can draw the entire game At each move the player must select a pile and divide the tokens into two non-empty, non-equal piles tree + The “game tree” to describe all possible + games follows: + 4 7 Game Playing - Minimax 6-1 5-2 4-3 Conventionally, in discussion of minimax, 5-1-1 4-2-1 3-2-2 3-3-1 have two players “MAX” and “MIN” 4-1-1-1 3-2-1-1 2-2-2-1 The utility function is taken to be the utility for MAX 3-1-1-1-1 2-2-1-1-1 Larger values are better for MAX” 2-1-1-1-1-1 Game Playing – Nim Game Playing – Minimax Remember that larger values are taken to be better for Basic idea of minimax: MAX Assume that use a utility function of Player MAX is going to take the best move 1 = a win for MAX available 0 = a win for MIN Will select the next state to be the one with We only compare values, “larger or smaller”, so the the highest utility actual sizes do not matter in other games might use {+1,0,-1} for {win,draw,lose}. Hence, value of a MAX node is the MAXIMUM of the values of the next possible states i.e. the maximum of its children in the search tree Game Playing – Minimax Game Playing – Minimax Summary Player MIN is going to take the best move available A “MAX” move takes the best move for MAX – for MIN so takes the MAX utility of the children i.e. the worst available for MAX Will select the next state to be the one with the lowest A “MIN” move takes the best for min – hence utility the worst for MAX – so takes the MIN utility of recall, higher utility values are better for MAX and the children so worse for MIN Hence, value of a MIN node is the MINIMUM of the Games alternate in play between MIN and values of the next possible states MAX i.e. the minimum of its children in the search tree 5 MIN 1 7 11 1 Game Playing – Use of Minimax MAX 6-1 5-2 4-3 The Min node has value +1 0101 MIN 5-1-1 4-2-1 3-2-2 3-3-1 All moves by MIN lead to a state of value 1 +1 for MAX MAX 0 4-1-1-1 3-2-1-1 2-2-2-1 0 MIN cannot avoid losing MIN 3-1-1-1-1 0 2-2-1-1-1 1 From the values on the tree one can read off the best moves for each player MAX 2-1-1-1-1-1 0 (loss for MAX) make sure you know how to extract these best moves (“perfect lines of play”) Game Playing – Bounded Game Playing – Bounded Minimax Minimax For real games, search trees are much The terminal states are no longer a definite win/loss actually they are really a definite win/draw/loss but bigger and deeper than Nim with reasonable computer resources we cannot determine which Have to heuristically/approximately evaluate the Cannot possibly evaluate the entire tree quality of the positions of the states Evaluation of the utility function is expensive if it is not Have to put a bound on the depth of the a clear win or loss search Game Playing – Bounded Minimax MAX 1 A Next Slide: MIN 1 B -3 C Artificial example of minimax bounded B Utility values of “terminal” positions obtained Evaluate “terminal position” after all possible by an evaluation function moves by MAX (The numbers are invented, and just to illustrate the working of minimax) = terminal position = agent = opponent 6 Game Playing – Bounded Minimax MAX 1 A Example of minimax with bounded depth MIN 1 B -3 C Evaluate “terminal position” after all possible moves in the order: 1.