CS 730/730W/830: Intro AI Heuristic Search CSPs 1 handout: slides asst 1 milestone was due Wheeler Ruml (UNH) Lecture 4, CS 730 – 1 / 19 EOLQs Heuristic Search CSPs Wheeler Ruml (UNH) Lecture 4, CS 730 – 2 / 19 Heuristic Search ■ Comparison ■ Heuristics ■ Other Algorithms ■ Search Algorithms ■ Take 2 CSPs Heuristic Search Wheeler Ruml (UNH) Lecture 4, CS 730 – 3 / 19 Comparison Heuristic Search Algorithm Time Space Complete Admissible ■ Comparison m ■ Depth-first b bm If m ≥ d No Heuristics d d ■ Other Algorithms Breadth-first b b Yes ifopscost1 ■ Search Algorithms d d ■ Take 2 Uniform-cost b b Yes Yes d CSPs IDDFS b bd Yes Yes Greedy depends No A* depends ≈ time if h adm. if h adm. branching factor b maximum depth m solution depth d Wheeler Ruml (UNH) Lecture 4, CS 730 – 4 / 19 Heuristics Heuristic Search Simplified problem must give lower bound on original! ■ Comparison ■ Heuristics 1. Relaxation: fewer and/or weaker constraints ■ Other Algorithms ■ Search Algorithms ■ Sometime efficient closed form ■ Take 2 CSPs 2. Abstraction: simplify token identity ■ Smaller search space Want highest value ■ If h1(n) ≤ h2(n) for all n, h2 dominates h1 Wheeler Ruml (UNH) Lecture 4, CS 730 – 5 / 19 Other Shortest-path Algorithms ■ Heuristic Search IDA* ■ Comparison ■ SMA*, IE ■ Heuristics ■ Other Algorithms ■ RBFS ■ Search Algorithms ■ ■ Take 2 RTA*, LRTA* CSPs Course projects! Wheeler Ruml (UNH) Lecture 4, CS 730 – 6 / 19 Search Algorithms ■ Heuristic Search Uninformed methods ■ Comparison ■ Heuristics ◆ Depth-first ■ Other Algorithms ◆ Breadth-first ■ Search Algorithms ■ Take 2 ◆ Uniform-cost CSPs ■ Informed methods ◆ Greedy ◆ A* ■ Bounding memory ◆ Iterative deepening ◆ Beam search Wheeler Ruml (UNH) Lecture 4, CS 730 – 7 / 19 Search Algorithms, Take 2 ■ Heuristic Search Depth-first ■ Comparison ■ Heuristics ◆ Depth-first ■ Other Algorithms ■ Search Algorithms ■ ‘Best-first’ ■ Take 2 ◆ CSPs Breadth-first ◆ Uniform-cost ◆ Greedy ◆ A* ■ Bounding memory ◆ Iterative deepening ◆ Beam search Wheeler Ruml (UNH) Lecture 4, CS 730 – 8 / 19 Heuristic Search CSPs ■ Other Problems ■ Types of Problems ■ Backtracking ■ Break ■ Forward Checking ■ ‘Heuristics’ ■ Example Results ■ MAC Constraint Satisfaction Problems ■ Other Algorithms ■ EOLQs Wheeler Ruml (UNH) Lecture 4, CS 730 – 9 / 19 Some Other Search Problems Heuristic Search Map coloring: Given a map of n countries and a set of k CSPs colors, color every country differently from its neighbors. ■ Other Problems n-queens : Given an n × n chessboard, arrange n queens so ■ Types of Problems ■ Backtracking that none is attacking another. ■ Break ■ Forward Checking ■ ‘Heuristics’ ■ Example Results What algorithm would you use? ■ MAC ■ Other Algorithms ■ EOLQs Wheeler Ruml (UNH) Lecture 4, CS 730 – 10 / 19 Types of Search Problems ■ Heuristic Search Shortest-path (vacuum, tile puzzle, M&C) CSPs ◆ ■ Other Problems given operators and their costs ■ Types of Problems ◆ want least-cost path to a goal ■ Backtracking ◆ ■ Break goal depth/cost unknown ■ Forward Checking ■ ‘Heuristics’ ■ Constraint satisfaction (map coloring, n-queens) ■ Example Results ■ MAC ◆ any goal is fine ■ Other Algorithms ◆ ■ EOLQs fixed depth ◆ explicit constraints on partial solutions Wheeler Ruml (UNH) Lecture 4, CS 730 – 11 / 19 Chronological Backtracking Heuristic Search Do not expand any partial solution that violates a constraint. CSPs ■ Other Problems ■ Types of Problems ■ Backtracking ■ Break ■ Forward Checking ■ ‘Heuristics’ ■ Example Results ■ MAC ■ Other Algorithms ■ EOLQs Wheeler Ruml (UNH) Lecture 4, CS 730 – 12 / 19 Break ■ Heuristic Search office hours CSPs ■ asst 1 ■ Other Problems ■ Types of Problems ◆ milestone: ■ Backtracking ◆ ■ Break you know it all now ■ Forward Checking ◆ don’t forget: beauty, write-up ■ ‘Heuristics’ ■ Example Results ■ blog entries due on Wednesdays, 8am, 400 words ■ MAC ■ Other Algorithms ■ projects (Apr 2), UROP (Mar 1) ■ EOLQs Wheeler Ruml (UNH) Lecture 4, CS 730 – 13 / 19 Forward Checking Heuristic Search When assigning a variable, remove the conflicting values for all CSPs connected variables. Backtrack on domain wipeout. ■ Other Problems ■ Types of Problems ■ Backtracking Arc consistency: for every value in the domain of x, there exists ■ Break ■ Forward Checking a value in the domain of y that satisfies all the constraints. ■ ‘Heuristics’ ■ Example Results ■ MAC ■ Other Algorithms ■ EOLQs Wheeler Ruml (UNH) Lecture 4, CS 730 – 14 / 19 Heuristics for CSPs Heuristic Search Variable choice: choose most constrained variable (smallest CSPs domain) ■ Other Problems ■ Types of Problems ■ want to keep tree small, failing quickly ■ Backtracking ■ Break Value choice: try least constraining value first (fewest ■ Forward Checking ■ ‘Heuristics’ removals) ■ Example Results ■ MAC ■ might as well succeed sooner if possible ■ Other Algorithms ■ EOLQs Wheeler Ruml (UNH) Lecture 4, CS 730 – 15 / 19 Example Results Heuristic Search BT FC FC+MCV CSPs USA > 1M 2K 60 ■ Other Problems ■ Types of Problems n-Queens > 40M > 40M 820K ■ Backtracking Zebra 3.9M 35K 500 ■ Break ■ Forward Checking Random1 420K 26K 2K ■ ‘Heuristics’ Random2 940K 77K 15K ■ Example Results ■ MAC ■ Other Algorithms ■ EOLQs Wheeler Ruml (UNH) Lecture 4, CS 730 – 16 / 19 Maintaining Arc Consistency Heuristic Search Ensure every value for x has a legal value in all neighbors y. If CSPs one doesn’t, remove it and ensure consistency of all y. ■ Other Problems ■ Types of Problems ■ Backtracking ■ Break ■ Forward Checking ■ ‘Heuristics’ ■ Example Results ■ MAC ■ Other Algorithms ■ EOLQs Wheeler Ruml (UNH) Lecture 4, CS 730 – 17 / 19 Maintaining Arc Consistency Heuristic Search Ensure every value for x has a legal value in all neighbors y. If CSPs one doesn’t, remove it and ensure consistency of all y. ■ Other Problems ■ Types of Problems while Q is not empty ■ Backtracking ■ Break (x, y) ← pop Q ■ Forward Checking ■ ‘Heuristics’ if revised(x, y) then ■ Example Results if x’s domain is now empty, return failure ■ MAC ■ Other Algorithms for every other neighbor z of x ■ EOLQs push (z,x) on Q revise(x, y) revised← false foreach v in x’s domain if no value in domain of y is compatible with v remove v from x’s domain revised← true return revised Wheeler Ruml (UNH) Lecture 4, CS 730 – 17 / 19 Other Algorithms for CSPs ■ Heuristic Search (Conflict-directed) Backjumping CSPs ■ Dynamic backtracking ■ Other Problems ■ ■ Types of Problems Randomized restarting ■ Backtracking ■ Break ■ Forward Checking ■ ‘Heuristics’ Course projects! ■ Example Results ■ MAC ■ Other Algorithms ■ EOLQs Wheeler Ruml (UNH) Lecture 4, CS 730 – 18 / 19 EOLQs Heuristic Search Please write down the most pressing question you have about CSPs the course material covered so far and put it in the box on your ■ Other Problems way out. ■ Types of Problems ■ Backtracking ■ Break Thanks! ■ Forward Checking ■ ‘Heuristics’ ■ Example Results ■ MAC ■ Other Algorithms ■ EOLQs Wheeler Ruml (UNH) Lecture 4, CS 730 – 19 / 19.