Game-Playing

Adversarial Search: Game-playing

Adversarial search

Game playing

Minimax Search

Adversarial Search: Game-playing Alpha-Beta Minimax

Minimax assumptions

Augmenting UMaine COS 470/570 – Introduction to AI minimax Spring 2019 Games of chance Looking ahead Adversarial Search: Game-playing

Adversarial search

Game playing

Minimax Search

Alpha-Beta Minimax

Adversarial search Minimax assumptions

Augmenting minimax

Games of chance

Looking ahead Adversarial Non-solipsistic search Search: Game-playing

I So far: search involves only one agent Adversarial search I Many times ≥ 1: Game playing Minimax Search I Game playing Alpha-Beta I NLP dialogue systems Minimax I AI assistants Minimax I Multiagent systems assumptions Augmenting I Also: maybe think of “the world” as another agent minimax

I Your agent takes action x... Games of chance

I . . . world “takes action” y Looking ahead I Why? Spontaneous events/processes, uncertainty, incomplete knowledge. . .

I Can’t plan, then act – unpredictable world/agents ⇒ Adversarial search errors Game playing Minimax Search I Shouldn’t plan too far ahead Alpha-Beta I May not be able to predict every possibility Minimax

I Uncertainty Minimax assumptions I Maybe too many possibilities Augmenting I Other agent(s): minimax Games of chance I May be cooperative. . . Looking ahead I . . . or they may be out to get you! I Pessimistic: model real world this way

I ⇒ adversarial search

I This section: game playing, some game theory

I Later: MAS, real-world planning/acting

Adversarial search

Game playing

Minimax Search

Alpha-Beta Game playing Minimax Minimax assumptions

Augmenting minimax

Games of chance

Looking ahead – e.g., 40 I ~10 nodes in chess’ search space (Nilsson) 22 8 I 1 expansion/ns ⇒ 10 centuries (universe: 10 centuries old)

Adversarial Game playing Search: Game-playing

I Here: Turn-taking games, not FPS, etc. Adversarial search I Similar to search: Game playing Minimax Search I Initial state, operators known Alpha-Beta I Goal well-deﬁned, if intensional Minimax I Heuristics often possible for distance to goal Minimax assumptions I Differences: Augmenting I Adversarial search minimax

I Can’t take moves back (generally) Games of chance

I Can’t usually plan/search to goal Looking ahead

Adversarial Game playing Search: Game-playing

I Can’t take moves back (generally) Games of chance

I Can’t usually plan/search to goal – e.g., Looking ahead 40 I ~10 nodes in chess’ search space (Nilsson)

I Can’t take moves back (generally) Games of chance

I Can’t usually plan/search to goal – e.g., Looking ahead 40 I ~10 nodes in chess’ search space (Nilsson) 22 8 I 1 expansion/ns ⇒ 10 centuries (universe: 10 centuries old)

I Current state: your move Adversarial search Game playing I Apply all possible operators ⇒ child states Minimax Search I Child state: opponent’s move Alpha-Beta Minimax I Apply all opponent’s operators Minimax I Continue to some depth cut-off assumptions Augmenting I At cut-off: apply heuristic function to leaves minimax

I Pick initial move to maximize worth of move Games of chance Looking ahead

I Agent wants to maximize value Adversarial search I Opponent wants to maximize board’s value to itself Game playing Minimax Search I Assume zero-sum game Alpha-Beta I ∴ opponent wants to minimize worth of board Minimax I Every turn: player’s options based only on current Minimax assumptions board Augmenting minimax

Games of chance

Looking ahead

I Heuristic for games = static evaluation function Adversarial search

I Win: +∞, loss -∞ Game playing I Various heuristics for other states Minimax Search

Alpha-Beta I Don’t care about cost to get there (usually) Minimax I Player: want ↑SE, opponent ↓SE Minimax assumptions

Augmenting minimax

Games of chance

Looking ahead

Adversarial search

Game playing

Minimax Search

Alpha-Beta Minimax

Minimax Search Minimax assumptions

Augmenting minimax

Games of chance

Looking ahead Adversarial Minimax search procedure Search: Game-playing

I Apply all operators to generate tree down to some Adversarial search level Game playing Minimax Search I Use static eval function for each leaf Alpha-Beta I Propagate values up the tree Minimax

I Select min/max depending on level Minimax assumptions I Select best move at top node Augmenting minimax I Use depth-ﬁrst search: Games of chance

I Game trees can be huge: don’t want to store nodes Looking ahead I Board doesn’t depend on other boards ⇒ don’t need others in memory

max Adversarial search

Game playing min Minimax Search Alpha-Beta Minimax

Minimax max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing min Minimax Search Alpha-Beta Minimax

Minimax max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing min Minimax Search Alpha-Beta Minimax

Minimax max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing min Minimax Search Alpha-Beta Minimax

Minimax max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing min Minimax Search Alpha-Beta Minimax

Minimax max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing min Minimax Search Alpha-Beta Minimax

Minimax 7 max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing min Minimax Search Alpha-Beta Minimax

Minimax 7 max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing min Minimax Search Alpha-Beta Minimax

Minimax 7 max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing min Minimax Search Alpha-Beta Minimax

Minimax 7 8 max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing 7 min Minimax Search Alpha-Beta Minimax

Minimax 7 8 max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing 7 min Minimax Search Alpha-Beta Minimax

Minimax 7 8 max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing 7 min Minimax Search Alpha-Beta Minimax

Minimax 7 8 0 max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing 7 min Minimax Search Alpha-Beta Minimax

Minimax 7 8 0 max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing 7 min Minimax Search Alpha-Beta Minimax

Minimax 7 8 0 2 max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing 7 0 min Minimax Search Alpha-Beta Minimax

Minimax 7 8 0 2 max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing 7 0 min Minimax Search Alpha-Beta Minimax

Minimax 7 8 0 2 max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

MAX

Adversarial search

Game playing MIN Minimax Search

Alpha-Beta MAX Minimax

Minimax 8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2 16 6 4 assumptions

Augmenting minimax

Games of chance

Looking ahead

MAX

Adversarial search

Game playing MIN Minimax Search

Alpha-Beta 8 MAX Minimax

Minimax 8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2 16 6 4 assumptions

Augmenting minimax

Games of chance

Looking ahead

MAX

Adversarial search

Game playing MIN Minimax Search

Alpha-Beta 8 MAX Minimax

Minimax 8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2 16 6 4 assumptions

Augmenting minimax

Games of chance

Looking ahead

MAX

Adversarial search

Game playing MIN Minimax Search

Alpha-Beta 8 9 MAX Minimax

Minimax 8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2 16 6 4 assumptions

Augmenting minimax

Games of chance

Looking ahead

MAX

Adversarial search

Game playing MIN Minimax Search

Alpha-Beta 8 9 4 MAX Minimax

Minimax 8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2 16 6 4 assumptions

Augmenting minimax

Games of chance

Looking ahead

MAX

Adversarial search

Game playing MIN 4 Minimax Search

Alpha-Beta 8 9 4 MAX Minimax

Minimax 8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2 16 6 4 assumptions

Augmenting minimax

Games of chance

Looking ahead

MAX

Adversarial search

Game playing MIN 4 Minimax Search

Alpha-Beta 8 9 4 MAX 5 Minimax

Minimax 8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2 16 6 4 assumptions

Augmenting minimax

Games of chance

Looking ahead

MAX

Adversarial search

Game playing MIN 4 Minimax Search

Alpha-Beta 8 9 4 MAX 5 9 Minimax

Minimax 8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2 16 6 4 assumptions

Augmenting minimax

Games of chance

Looking ahead

MAX

Adversarial search

Game playing MIN 4 Minimax Search

Alpha-Beta 8 9 4 6 MAX 5 9 Minimax

Minimax 8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2 16 6 4 assumptions

Augmenting minimax

Games of chance

Looking ahead

MAX

Adversarial search

Game playing MIN 4 5 Minimax Search

Alpha-Beta 8 9 4 6 MAX 5 9 Minimax

Minimax 8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2 16 6 4 assumptions

Augmenting minimax

Games of chance

Looking ahead

MAX

Adversarial search

Game playing MIN 4 5 Minimax Search

3 Alpha-Beta 8 9 4 6 MAX 5 9 Minimax

Minimax 8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2 16 6 4 assumptions

Augmenting minimax

Games of chance

Looking ahead

MAX

Adversarial search

Game playing MIN 4 5 Minimax Search

3 Alpha-Beta 8 9 4 6 9 MAX 5 9 Minimax

Minimax 8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2 16 6 4 assumptions

Augmenting minimax

Games of chance

Looking ahead

MAX

Adversarial search

Game playing MIN 4 5 Minimax Search

3 Alpha-Beta 8 9 4 6 9 16 MAX 5 9 Minimax

Minimax 8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2 16 6 4 assumptions

Augmenting minimax

Games of chance

Looking ahead

MAX

Adversarial search

Game playing 3 MIN 4 5 Minimax Search

3 Alpha-Beta 8 9 4 6 9 16 MAX 5 9 Minimax

Minimax 8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2 16 6 4 assumptions

Augmenting minimax

Games of chance

Looking ahead

MAX

Adversarial search

Game playing 3 MIN 4 5 Minimax Search

3 Alpha-Beta 8 9 4 6 9 16 MAX 5 9 Minimax

Minimax 8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2 16 6 4 assumptions

Augmenting minimax

Games of chance

Looking ahead

Adversarial search

Game playing

Minimax Search

Alpha-Beta Minimax

Minimax assumptions

Augmenting minimax

Games of chance

Looking ahead

10 5 7 -1 -2 -3 3 -4 -5 1 3 5 7 2 1 1 -3 4 4 5 6 6 5 4 8 20 -100

Adversarial search

Game playing

Minimax Search

Alpha-Beta max Minimax

Minimax assumptions min Augmenting minimax

Games of chance max Looking ahead

10 5 7 -1 -2 -3 3 -4 -5 1 3 5 7 2 1 1 -3 4 4 5 6 6 5 4 8 20 -100

Adversarial search

Game playing

Minimax Search

Alpha-Beta max Minimax

Minimax assumptions min Augmenting minimax

Games of chance max 10 -1 3 Looking ahead

10 5 7 -1 -2 -3 3 -4 -5 1 3 5 7 2 1 1 -3 4 4 5 6 6 5 4 8 20 -100

Adversarial search

Game playing

Minimax Search

Alpha-Beta max Minimax

Minimax assumptions min Augmenting -1 minimax Games of chance max 10 -1 3 Looking ahead

10 5 7 -1 -2 -3 3 -4 -5 1 3 5 7 2 1 1 -3 4 4 5 6 6 5 4 8 20 -100

Adversarial search

Game playing

Minimax Search

Alpha-Beta max Minimax

Minimax assumptions min Augmenting -1 4 minimax Games of chance max 10 -1 3 5 7 4 Looking ahead

10 5 7 -1 -2 -3 3 -4 -5 1 3 5 7 2 1 1 -3 4 4 5 6 6 5 4 8 20 -100

Adversarial search

Game playing

Minimax Search

Alpha-Beta max Minimax

Minimax assumptions min Augmenting -1 4 6 minimax Games of chance max 10 -1 3 5 7 4 6 6 Looking ahead20

10 5 7 -1 -2 -3 3 -4 -5 1 3 5 7 2 1 1 -3 4 4 5 6 6 5 4 8 20 -100

Adversarial search

Game playing

Minimax Search

Alpha-Beta max Minimax

Minimax assumptions min Augmenting -1 4 6 minimax Games of chance max 10 -1 3 5 7 4 6 6 Looking ahead20

10 5 7 -1 -2 -3 3 -4 -5 1 3 5 7 2 1 1 -3 4 4 5 6 6 5 4 8 20 -100

function MINIMAX(board,player,cutoff,depth=0) Adversarial search

Inputs: a board + T/F (player/opponent) + Game playing

cutoff + current depth Minimax Search Returns: move Alpha-Beta if depth = cutoff then Minimax Minimax Return STATIC-EVAL(board) assumptions else if player = T then Augmenting minimax Return argmax(STATIC-EVAL,CHILDREN(board)) Games of chance

else Looking ahead Return argmin(STATIC-EVAL,CHILDREN(board))

rtificial A ntelligence Copyright © 2017 UMaine School of Computing and Information Science I Adversarial Minimax program 1 Search: Game-playing (defun minimax (node &key (depth 5) (which :maximizer)) (cond ((zerop depth) ;at max ply Adversarial search (static-eval node)) Game playing (t Minimax Search (let (best-value best-child Alpha-Beta current-value) Minimax (loop for child in (children node) do Minimax assumptions ;; call minimax on each child: (setq current-value Augmenting minimax (minimax child :depth (1- depth) :which (if (eql which :maximizer) Games of chance :minimizer Looking ahead :maximizer))) ;; keep track of best node: (when (or (null best-value) (funcall (if (eql which :maximizer) #’> #’<) current-value best-value)) (setq best-value current-value best-child child))) (values best-value best-child))))) rtificial A ntelligence Copyright © 2017 UMaine School of Computing and Information Science I Adversarial Minimax program 2: Search: Game-playing (defun minimax (board &key (depth 0) (cutoff 5) (player? t)) (cond ((= depth cutoff) (static-eval board)) Adversarial search (t (multiple-value-bind (child value) Game playing (if player? Minimax Search (argmax #’(lambda (b) Alpha-Beta (minimax b Minimax :depth (1+ depth) Minimax :cutoff cutoff assumptions :player? (not player?))) Augmenting (children board)) minimax (argmin #’(lambda (b) (minimax b Games of chance :depth (1+ depth) Looking ahead :cutoff cutoff :player? (not player?))) (children board)))) (if (= depth 0) (move child) value))))

rtificial A ntelligence Copyright © 2017 UMaine School of Computing and Information Science I Adversarial Minimax program 2 (argmax): Search: Game-playing (defmacro argmax (fcn things &optional best) ‘(argcmp ,fcn ,things #’> ,best)) Adversarial search (defmacro argmin (fcn things &optional best) Game playing ‘(argcmp ,fcn ,things #’< ,best)) Minimax Search

Alpha-Beta (defun argcmp (fcn things cmp &optional best) Minimax (if (null things) Minimax (values (cadr best) (car best)) assumptions (let ((val (funcall fcn (car things)))) Augmenting (when (or (null best) (funcall cmp val (car best))) minimax (setq best (list val (car things)))) (argcmp fcn (cdr things) cmp best)))) Games of chance Looking ahead

rtificial A ntelligence Copyright © 2017 UMaine School of Computing and Information Science I Adversarial Minimax program 3: Search: Game-playing (defun minimax (board &key (depth 0) (cutoff 5) (player? t)) (cond ((= depth cutoff) (static-eval board)) Adversarial search (t (multiple-value-bind (child value) Game playing (argcmp #’(lambda (b) Minimax Search (minimax b Alpha-Beta :depth (1+ depth) Minimax :cutoff cutoff Minimax :player? (not player?))) assumptions (children board) Augmenting (if player? #’> #’<))) minimax (if (= depth 0) (move child) Games of chance value)))) Looking ahead

rtificial A ntelligence Copyright © 2017 UMaine School of Computing and Information Science I Adversarial Minimax program 4: Search: Game-playing (defun minimax (board &key (depth 0) (cutoff 5) (player? t)) (when (= depth cutoff) (return (static-eval board))) (multiple-value-bind (child value) Adversarial search (argcmp #’(lambda (b) Game playing (minimax b :depth (1+ depth) Minimax Search :cutoff cutoff Alpha-Beta :player? (not player?))) Minimax (children board) Minimax (if player? #’> #’<)) assumptions (if (= depth 0) Augmenting (move child) minimax value))) Games of chance

Looking ahead

Adversarial search

Game playing

Minimax Search

Alpha-Beta Alpha-Beta Minimax Minimax Minimax assumptions

Augmenting minimax

Games of chance

Looking ahead max

min ≤ -1

max 10 -1 ≥ 3

10 5 7 -1 -2 -3 3 -4 -5 1 3

Adversarial Problem Search: Game-playing

I Game trees can be very large Adversarial search Game playing I Is there any way to prune the space? Minimax Search I I.e., are there moves that no one will choose? Alpha-Beta Minimax

Minimax assumptions

Augmenting minimax

Games of chance

Looking ahead

I Game trees can be very large Adversarial search Game playing I Is there any way to prune the space? Minimax Search I I.e., are there moves that no one will choose? Alpha-Beta Minimax max Minimax assumptions

Augmenting minimax

Games of chance min Looking ahead ≤ -1

max 10 -1 ≥ 3

10 5 7 -1 -2 -3 3 -4 -5 1 3

max Adversarial search

Game playing min Minimax Search Alpha-Beta Minimax

Minimax max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing min Minimax Search Alpha-Beta Minimax

Minimax >= 7 max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing min Minimax Search Alpha-Beta Minimax

Minimax = 7 max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing <=7 min Minimax Search Alpha-Beta Minimax

Minimax = 7 max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing <=7 min Minimax Search Alpha-Beta Minimax

Minimax = 7 >=8 max assumptions Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

max Adversarial search

Game playing <=7 min Minimax Search Alpha-Beta Minimax

Minimax = 7 >=8 max assumptions prune Augmenting minimax here Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

Game playing

Minimax Search =7 min Alpha-Beta Minimax

Minimax assumptions = 7 >=8 max Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

Game playing

Minimax Search =7 min Alpha-Beta Minimax

Minimax assumptions = 7 >=8 >=0 max Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

Game playing

Minimax Search =7 min Alpha-Beta Minimax

Minimax assumptions = 7 >=8 =0 max Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

Game playing

Minimax Search =7 <=0 min Alpha-Beta Minimax

Minimax assumptions = 7 >=8 =0 max Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

Minimax Search

=7 <=0 min Alpha-Beta Minimax prune Minimax assumptions = 7 >=8 =0 max Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

Game playing

Minimax Search =7 =0 min Alpha-Beta Minimax

Minimax assumptions = 7 >=8 =0 max Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

Game playing

Minimax Search =7 =0 min Alpha-Beta Minimax

Minimax assumptions = 7 >=8 =0 max Augmenting minimax

Games of chance

Looking ahead 7 6 8 5 0 -2 -6 2

Adversarial search

Game playing

MAX Minimax Search

Alpha-Beta Minimax

Minimax MIN assumptions

Augmenting minimax MAX Games of chance

8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2Looking16 ahead6 4 .

Adversarial search

Game playing

MAX Minimax Search

Alpha-Beta Minimax

Minimax MIN <=8 assumptions

Augmenting minimax MAX =8 Games of chance

8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2Looking16 ahead6 4 .

Adversarial search

Game playing

MAX Minimax Search

Alpha-Beta Minimax

Minimax MIN <=8 assumptions

Augmenting >9 minimax MAX =8 Games of chance

8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2Looking16 ahead6 4 .

Adversarial search

Game playing

MAX Minimax Search

Alpha-Beta Minimax

Minimax MIN <=8 assumptions

Augmenting >9 >=2 minimax MAX =8 Games of chance

8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2Looking16 ahead6 4 .

Adversarial search

Game playing

MAX Minimax Search

Alpha-Beta Minimax

Minimax MIN <=8 assumptions

Augmenting >9 =4 minimax MAX =8 Games of chance

8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2Looking16 ahead6 4 .

Adversarial search

Game playing

MAX Minimax Search >=4 Alpha-Beta Minimax

Minimax MIN =4 assumptions

Augmenting >9 =4 minimax MAX =8 Games of chance

8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2Looking16 ahead6 4 .

Adversarial search

Game playing

MAX Minimax Search >=4 Alpha-Beta Minimax

Minimax MIN =4 assumptions

Augmenting >9 =4 >=1 minimax MAX =8 Games of chance

8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2Looking16 ahead6 4 .

Adversarial search

Game playing

MAX Minimax Search >=4 Alpha-Beta Minimax

Minimax MIN =4 assumptions

Augmenting >9 =4 >=3 minimax MAX =8 Games of chance

8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2Looking16 ahead6 4 .

Adversarial search

Game playing

MAX Minimax Search >=4 Alpha-Beta Minimax

<=5 Minimax MIN =4 assumptions

Augmenting >9 =4 =5 minimax MAX =8 Games of chance

8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2Looking16 ahead6 4 .

Adversarial search

Game playing

MAX Minimax Search >=4 Alpha-Beta Minimax

<=5 Minimax MIN =4 assumptions

Augmenting >9 =4 =5 >=3 minimax MAX =8 Games of chance

8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2Looking16 ahead6 4 .

Adversarial search

Game playing

MAX Minimax Search >=4 Alpha-Beta Minimax

<=5 Minimax MIN =4 assumptions

Augmenting >9 =4 =5 >=9 minimax MAX =8 Games of chance

8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2Looking16 ahead6 4 .

Adversarial search

Game playing

MAX Minimax Search >=4 Alpha-Beta Minimax

<=5 Minimax MIN =4 assumptions

Augmenting >9 =4 =5 >=9 minimax MAX =8 Games of chance

8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2Looking16 ahead6 4 .

Adversarial search

Game playing

MAX Minimax Search >=4 Alpha-Beta Minimax

=5 Minimax MIN =4 assumptions

Augmenting >9 =4 =5 >=9 >=6 minimax MAX =8 Games of chance

8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2Looking16 ahead6 4 .

Adversarial search

Game playing

MAX Minimax Search >=5 Alpha-Beta Minimax

=5 Minimax MIN =4 assumptions

Augmenting >9 =4 =5 >=9 >=6 minimax MAX =8 Games of chance

8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2Looking16 ahead6 4 .

Adversarial search

Game playing

MAX Minimax Search >=5 Alpha-Beta Minimax

=5 <=3 Minimax MIN =4 assumptions

Augmenting >9 =4 =5 >=9 >=6 =3 minimax MAX =8 Games of chance

8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 2Looking16 ahead6 4 .

Adversarial search

Game playing

MAX Minimax Search >=5 Alpha-Beta Minimax

=5 <=3Minimax MIN =4 assumptions

Augmenting >9 =4 =5 >=9 >=6 =3 minimax MAX =8 Games of chance . 8 7 2 9 1 6 2 4 1 1 3 5 3 9 2 6 5 2 1 2 3 9 7 Looking2 16 ahead6 4

I Keep track of two values: Adversarial search

I α: Maximum value the maximizer (player) can expect Game playing I β: Minimum value the minimizer (opponent) can Minimax Search

expect Alpha-Beta Minimax α = −∞, β = +∞ I Initialize to worst case for each: Minimax assumptions I Update: Augmenting I Maximizer: α = max of α, value of recurring on minimax

children Games of chance

I Minimizer: β = min of β, value of recurring on Looking ahead children I If at any point in the update β ≤ α

I Maximizer: then opponent has a better move than this I Minimizer: then player has a better move than this I Either way, no point continuing: prune

Alpha-Beta min … Minimax … Minimax assumptions

Augmenting max minimax Games of chance

Looking ahead

10 5 7 -1 -2 -3 3 -4 -5

Alpha-Beta min … Minimax α β … Minimax - + assumptions ∞ ∞ Augmenting max minimax Games of chance

Looking ahead

10 5 7 -1 -2 -3 3 -4 -5

Alpha-Beta min … Minimax α β … Minimax - + assumptions ∞ ∞ Augmenting max minimax α β Games of chance - ∞ + ∞ Looking ahead

10 5 7 -1 -2 -3 3 -4 -5

Alpha-Beta min … Minimax α β … Minimax - + assumptions ∞ ∞ Augmenting max minimax α β Games of chance 10 + ∞ Looking ahead

10 5 7 -1 -2 -3 3 -4 -5

Alpha-Beta min … Minimax α β … Minimax - + assumptions ∞ ∞ Augmenting max minimax α β Games of chance 10 + ∞ Looking ahead

10 5 7 -1 -2 -3 3 -4 -5

Alpha-Beta min … Minimax α β … Minimax - + assumptions ∞ ∞ Augmenting max minimax α β Games of chance 10 + ∞ Looking ahead

10 5 7 -1 -2 -3 3 -4 -5

Alpha-Beta min … Minimax α β … Minimax - 10 assumptions ∞ Augmenting max minimax α β Games of chance 10 + ∞ Looking ahead

10 5 7 -1 -2 -3 3 -4 -5

Alpha-Beta min … Minimax α β … Minimax - 10 assumptions ∞ Augmenting max minimax α β Games of chance - ∞ 10 Looking ahead

10 5 7 -1 -2 -3 3 -4 -5

Alpha-Beta min … Minimax α β … Minimax - 10 assumptions ∞ Augmenting max minimax α β Games of chance -1 10 Looking ahead

10 5 7 -1 -2 -3 3 -4 -5

Alpha-Beta min … Minimax α β … Minimax - 10 assumptions ∞ Augmenting max minimax α β Games of chance -1 10 Looking ahead

10 5 7 -1 -2 -3 3 -4 -5

Alpha-Beta min … Minimax α β … Minimax - 10 assumptions ∞ Augmenting max minimax α β Games of chance -1 10 Looking ahead

10 5 7 -1 -2 -3 3 -4 -5

Alpha-Beta min … Minimax α β … Minimax - -1 assumptions ∞ Augmenting max minimax α β Games of chance -1 10 Looking ahead

10 5 7 -1 -2 -3 3 -4 -5

Alpha-Beta min … Minimax α β … Minimax - -1 assumptions ∞ Augmenting max minimax α β Games of chance Looking ahead - ∞ -1

10 5 7 -1 -2 -3 3 -4 -5

Alpha-Beta min … Minimax α β … Minimax - -1 assumptions ∞ Augmenting max minimax α β Games of chance Looking ahead 3 -1

10 5 7 -1 -2 -3 3 -4 -5

Alpha-Beta min … Minimax α β … Minimax - -1 assumptions ∞ Augmenting max minimax α β Games of chance Looking ahead 3 -1

10 5 7 -1 -2 -3 3 -4 -5

Alpha-Beta min … Minimax α β … Minimax - -1 assumptions ∞ Augmenting max minimax α β Games of chance Looking ahead 3 -1

10 5 7 -1 -2 -3 3 -4 -5

Alpha-Beta min … Minimax α β … Minimax - -1 assumptions ∞ Augmenting max minimax Games of chance

Looking ahead

10 5 7 -1 -2 -3 3 -4 -5

[From Russell & Norvig] Adversarial search

Game playing

Minimax Search

Alpha-Beta Minimax

Minimax assumptions

Augmenting minimax

Games of chance

Looking ahead

rtificial A ntelligence Copyright © 2017 UMaine School of Computing and Information Science I √ d I Effective branching factor: (b*) , ∴ b* = b I Also: search twice as deep, same amount of time I So what is average cost?

I Who knows? But. . . I Closer to b.c. than w.c.: “nature proving unusually beneﬁcient” (Winston)

Adversarial Effectiveness of α–β pruning Search: Game-playing

Games of chance

Looking ahead

I Who knows? But. . . I Closer to b.c. than w.c.: “nature proving unusually beneﬁcient” (Winston)

Adversarial Effectiveness of α–β pruning Search: Game-playing

d I Minimax: O(b ) worst case Adversarial search Game playing I α–β minimax: same w.c. Minimax Search I Best case: Alpha-Beta I If nodes are in best-to-worst order Minimax d/2 I ⇒ O(b ) Minimax assumptions I What does this mean? √ d Augmenting I Effective branching factor: (b*) , ∴ b* = b minimax Games of chance

Looking ahead

I Who knows? But. . . I Closer to b.c. than w.c.: “nature proving unusually beneﬁcient” (Winston)

Adversarial Effectiveness of α–β pruning Search: Game-playing

Looking ahead

Adversarial Effectiveness of α–β pruning Search: Game-playing

I Who knows? But. . .

I Who knows? But. . . I Closer to b.c. than w.c.: “nature proving unusually beneﬁcient” (Winston)

Adversarial search

Game playing

Minimax Search

Alpha-Beta Minimax assumptions Minimax Minimax assumptions

Augmenting minimax

Games of chance

Looking ahead Adversarial Assumptions of (α–β) minimax Search: Game-playing

I Well-deﬁned operators Adversarial search Game playing I Zero-sum, symmetric: good for me, bad for you by same amount Minimax Search Alpha-Beta I Opponent is rational, infallible Minimax Minimax I Static evaluation is accurate assumptions Augmenting minimax

Games of chance

Looking ahead

Adversarial search

Game playing

Minimax Search

Alpha-Beta Augmenting minimax Minimax Minimax assumptions

Augmenting minimax

Games of chance

Looking ahead Adversarial Handling “horizon” effect Search: Game-playing

I Horizon effect: Adversarial search

I Static eval at depth d = v Game playing 0 0 I But just beyond d, SE ⇒ v v or v v Minimax Search I How to handle? Alpha-Beta Minimax I Wait for quiescence Minimax I If notice SE oscillating as near depth. . . assumptions I . . . may be in middle of move sequence that takes a Augmenting loss ﬁrst (e.g., piece exchange) minimax Games of chance I Continue searching a little more – maybe will stabilize Looking ahead I Secondary search: choose most promising node, continue search

I Maybe have standard “canned” move sequences Adversarial search Game playing I E.g., standard openings in chess Minimax Search I Avoids/short-circuits search entirely early/late in Alpha-Beta process Minimax Minimax assumptions

Augmenting minimax

Games of chance

Looking ahead

I What if all moves ⇒ loss (or very poor SE)? Adversarial search Game playing I Pick a move that could result in win. . . Minimax Search I . . . and hope opponent makes a mistake Alpha-Beta Minimax

Minimax assumptions

Augmenting minimax

Games of chance

Looking ahead

I Problem: Adversarial search

I Suppose we have timed moves (e.g., chess) Game playing I Want to use up all time Minimax Search

I Don’t want to be caught without move, though Alpha-Beta Minimax I Solution: Minimax I Iterative deepening assumptions I Search to depth 1, then 2, then. . . Augmenting minimax I At any time, always have last best move “on tap” Games of chance

I Example of an anytime algorithm Looking ahead

Adversarial search

Game playing

Minimax Search

Alpha-Beta Minimax

Games of chance Minimax assumptions

Augmenting minimax

Games of chance

Looking ahead Adversarial Games of Chance Search: Game-playing

I Many games: chance (or stochastic) element Adversarial search

I Cards, dice, spinners, etc. Game playing I Uncertainty about opponent’s rationality/utility Minimax Search

function Alpha-Beta Minimax I How to modify minimax? Minimax I First iteration level is easy: roll the dice (e.g.), then assumptions minimax Augmenting minimax I What about opponent’s move? Games of chance I What about player’s future moves? Looking ahead

I Add chance nodes to game tree at points of Adversarial search uncertainty Game playing I Branches: Minimax Search Alpha-Beta I Different outcomes Minimax

I Tag with probability Minimax I E.g., roll two dice: assumptions

I Chance node has 11 branches Augmenting 1 2 3 4 5 6 5 4 3 2 1 minimax I Tagged: 36 , 36 , 36 , 36 , 36 , 36 , 36 , 36 , 36 , 36 , 36 Games of chance

I Explore tree as in minimax, but: Looking ahead I Chance node ⇒ weighted average of children I Weights: probabilities

Maximizer Adversarial search Game playing Maximizer’s -0.5 -4.1 Minimax Search moves -1.3 -0.37 -3.7 Alpha-Beta -2.3 Minimax 1/6 1/6 Opponent’s Minimax 2.01 1/6 1/6 -3.2 assumptions roll 1/6 1/6 0.35 1/6 3.83 -2.3 Augmenting -6 minimax Minimizer 3.07 Games of chance

Looking ahead Minimizer’s 12.3 6.2 moves 3.83 6.3 5.3 3.93 4.5 1/6 1/6 1/6 1/6 Player’s 1/6 1/6 roll 3 1/6 -4 4 10 7 6 -3 rtificial A ntelligence Copyright © 2017 UMaine School of Computing and Information Science I Adversarial Algorithm (w/o α-β cutoffs) Search: Game-playing

1: function EXPECTIMINIMAX(state,depth) 2: if depth = 0 then Adversarial search 3: return state & STATICEVAL(state) Game playing 4: else if maximizer then Minimax Search 5: for child in CHILDREN(state) do 6: best = argmax(best, EXPECTIMINIMAX(child,depth -1) Alpha-Beta Minimax 7: return best 8: else if minimizer then Minimax assumptions 9: for child in CHILDREN(state) do 10: best = argmin(best, EXPECTIMINIMAX(child,depth -1) Augmenting minimax 11: return best & value(best) 12: else Games of chance 13: for each chance value do Looking ahead 14: sum = sum + value of state returned by 15: EXPECTIMINIMAX(UPDATE(state,chance value)) 16: return state & sum ÷ # chance values

I Not solely contingent on utilities Adversarial search Game playing I May predict, but not get, a win (etc.) d d Minimax Search I Time: O(b c ) ← c = # chance values Alpha-Beta Minimax

Minimax assumptions

Augmenting minimax

Games of chance

Looking ahead

I Some people consider game-playing a metaphor for Adversarial search real-world problem solving Game playing Minimax Search I With opponents, it’s clear. . . Alpha-Beta I . . . but can also treat the world as an opponent Minimax Minimax assumptions

Augmenting minimax

Games of chance

Looking ahead

I Many researchers use game theory to make Adversarial search

decisions Game playing

I Think of it as a generalization of minimax Minimax Search

I Payoff matrices: play the role of static evaluation Alpha-Beta function Minimax Minimax Player 2 assumptions

Augmenting p minimax l c d Games of chance a y 3 1 Looking ahead a e 4 2 r 4 2 b 1 1 3

Adversarial search

Game playing

Minimax Search

Alpha-Beta Looking ahead Minimax Minimax assumptions

Augmenting minimax

Games of chance

Looking ahead Adversarial Looking ahead: Deep learning Search: Game-playing

I Deep reinforcement learning: AlphaGo, AlphaZero Adversarial search I So why minimax? Game playing Minimax Search I Simple Alpha-Beta I Doesn’t require enormous computational bandwidth Minimax to train Minimax I Some insights about dealing with more complex assumptions scenarios than games Augmenting minimax

Games of chance

Looking ahead