Search Agents Ben Sapp 9/19/2011 A simple reflex agent [Galvani, 1791] Agent sensors Environment What the world is like now Transi@on rules: What ac@on should I state -> ac@on do now? actuators A model-based reflex agent Agent sensors Environment What the world state is like now Transi@on rules: What ac@on should I state -> ac@on do now? actuators What’s missing? Happiness this way Life ✔ Liberty ✔ Pursuit of Happiness ✗ A model-based, goal-based agent Agent sensors What the world state is like now Environment what my ac@ons What it will be like do to the world if I do ac@on a What ac@on should I goals do now? actuators A simple goal-oriented agent input: current_state, goal, world, turn if turn == 0: ac@on_sequence = search(current_state, goal, world) else: do(ac@on_sequence[turn]) turn 0 turn > 0 rawr! hmm…ok, got it Problem-solving agent: assump@ons • Environment is sta+c • Environment is observable • States and ac@ons are discrete or discre+zable • Ac@ons and environment are determinis+c Well-defined problems • State space – How big is it? – Is it redundant? • Acons – Possibly indexed by state, a.k.a. a successor func@on get_successors(state) • Ini@al state • Goal state or goal test • Step cost: map (state,ac@on) pair to a real valued cost Example: path planning Find the shortest path from point A (ant) to point B (food) • states? ant • successor funcon? • goal test? • step cost? food Example: tour planning Find the shortest path visi+ng every node. • states? • successor funcon? • goal test? • step cost? 13,509 ci@es in the US with populaon > 500 Solved op@mally in 2003 at Rice, using 44 Pen@um II CPUs for 3 months. Example: sliding block puzzle Arrange +les in numerical order. • states? • successor funcon? • goal test? • step cost? Solving a search problem • state space: ( x,y) coordinates of ant • ac@ons: {‘ n’,’s’,’e’,’w’} – obstacles(state) • start: (0,0) • goal: (2,0) etc. Solving a search problem omnomnom path = [(0,0),(0,1),(0,2),(1,2),(1,3), (2,3),(3,3),(3,2),(3,1),(2,1),(2,0)] General search problem code def graph_search(start,goal):! visited = {}! “fringe” fringe = [start]! visited while True:! states !if fringe.empty():! goal !! !return False! !state = fringe.pop()! !if is_goal(node):! whole state space !! !return True! !if state not in visited:! !! !visited.append(state)! ! !fringe.append(get_successors(state))! fringe as stack: DFS def graph_search(start,goal):! next state visited = {}! state just fringe = [start]! to visit added while True:! !if fringe.empty():! !! !return False! !state = fringe.pop()! !if is_goal(node):! !! !return True! !if state not in visited:! STACK !! !visited.append(state)! ! !fringe.append(get_successors(state))! fringe as queue: BFS def graph_search(start,goal):! state visited = {}! just fringe = [start]! added while True:! !if fringe.empty():! !! !return False! next !state = fringe.pop()! state !if is_goal(node):! to visit !! !return True! !if state not in visited:! QUEUE !! !visited.append(state)! ! !fringe.append(get_successors(state))! Blundering around in the dark Ac@on @es are broken in order: N, E, S, W What does BFS do? DFS? start goal Proper@es of search • Completeness: does it find a solu+on? • Op@mality: does it find the best (least cost) soluon? • Is DFS complete? op@mal? • Is BFS complete? op@mal? Informed search Which way to try next? Heuris@cs Define: • h(n) = es@mated cost of path from node n to the goal node. • g(n) = cost to get from the start node to the node n. • f(n) = h(n) + g(n) = es@mated cost of a full path going through node n New strategy: Visit nodes in the fringe in order of f(n) Informed search: A* def a_star(start,goal):! visited = {}! g[start] = 0! fringe = PriorityQueue()! “fringe” fringe.put(priority=0,val=start)! visited while True:! states !if fringe.empty():! !! !return False! goal !state = fringe.pop_highest_priority()! !if is_goal(node):! !! !return True! whole state space !if state not in visited:! !! !visited.append(state)! ! !for succ in get_successors(state):! ! ! !g[succ] = g[state]+1! ! ! !h[succ] = heuristic_estimate(succ)! ! ! !f[succ] = g[succ]+h[succ]! ! ! !fringe.put(priority = f[succ], val = succ)! Example: path planning Find the shortest path from point A (ant) to point B (food) • states? ant • successor funcon? • goal test? • path cost? • heurisc? food A* behavior • What does A* do here with a straight-line heurisc? h(n) = dist(n,goal) • What does A* do here with h(n) = 0 for all n? • Ok, what about h(n) = 9e999 / dist(n,goal) ? (inversely propor@onal to goal, and so large that g(n) doesn’t maer) start goal A* proper@es • Complete? – well, depends on h(n) • Op@mal? – well, depends on h(n)… Admissability • h(n) should be an underes+mate of the cost to get to the goal • Why? h(here): sorry dude, you are really really far away from the goal right now ant: …ok... A* with an admissible heuris@c is op@mal Proof: • Let C* be the cost of the op@mal path to the goal G: f(Gbest-path) = C* • If goal appears in the fringe via a bad path, it must be that f(Gbad-path) > C*, just by defini@on of it being a bad path • For any node n on the best path, we know f(n) = g(n) + h(n) < C*, since h(n) is admissable. • Thus, nodes n on the best path will always be picked from the fringe before we reach G on a bad path. QED. f(Gbad-path) > C* node n G f(n) < C* f(Gbest-path) = C* start One catch goal ALREADY VISITED start Catch: fixes • Re-visit visited states – messy bookkeeping, but does the job • Ensure your heuris@c is admissible and monotonic (never increases as you get closer) – almost always the case for any admissible heuris@c you might dream up goal start Designing heuriscs • Relax constraints on the problem • Combine heuriscs – h(n) = max(h1(n),h2(n),…) is admissible – h(n) = w1*h1(n)+w2*h2(n)+… also admissible, if w1+w2+… = 1 Example: path planning Find the shortest path from point A (ant) to point B (food) • states? ant • successor funcon? • goal test? • path cost? • heurisc? food Example: sliding block puzzle Arrange +les in numerical order. • states? • successor funcon? • goal test? • path cost? • heurisc? Example: planning a course schedule Select a course plan to graduate: • Can take a limited number of courses at the same +me. • Most courses require pre- reqs first. • Most courses are not offered every semester. • states? • successor funcon? • goal test? • step cost? • heurisc? Homework • Due next Tuesday, midnight • More work than previous homework • 3 parts – Path planning, 1 ant – Coordinated path planning with mul@ple ants – Combining search and DFAs to win the compe@@on • Good luck! .