Search Agents

Ben Sapp 9/19/2011 A simple reﬂex agent

[Galvani, 1791]

Agent sensors Environment What the world is like now

Transion rules: What acon should I state -> acon do now?

actuators A model-based reﬂex agent

Agent sensors Environment What the world state is like now

Transion rules: What acon should I state -> acon do now?

actuators What’s missing?

Happiness this way

Life ✔ Liberty ✔ Pursuit of Happiness ✗ A model-based, goal-based agent

Agent sensors

What the world state is like now Environment

what my acons What it will be like do to the world if I do acon a

What acon should I goals do now?

actuators A simple goal-oriented agent input: current_state, goal, world, turn if turn == 0: acon_sequence = search(current_state, goal, world) else: do(acon_sequence[turn])

turn 0 turn > 0 rawr! hmm…ok, got it Problem-solving agent: assumpons

• Environment is stac • Environment is observable • States and acons are discrete or discrezable • Acons and environment are determinisc Well-deﬁned problems

• State space – How big is it? – Is it redundant? • Acons – Possibly indexed by state, a.k.a. a successor funcon get_successors(state) • Inial state • Goal state or goal test • Step cost: map (state,acon) pair to a real valued cost Example: path planning

Find the shortest path from point A (ant) to point B (food)

• states? ant • successor funcon? • goal test? • step cost?

food Example: tour planning

Find the shortest path vising every node.

• states? • successor funcon? • goal test? • step cost?

13,509 cies in the US with populaon > 500

Solved opmally in 2003 at Rice, using 44 Penum II CPUs for 3 months. Example: sliding block puzzle

Arrange les in numerical order.

• states? • successor funcon? • goal test? • step cost? Solving a search problem

• state space: ( x,y) coordinates of ant • acons: {‘ n’,’s’,’e’,’w’} – obstacles(state) • start: (0,0) • goal: (2,0)

etc. Solving a search problem

omnomnom

path = [(0,0),(0,1),(0,2),(1,2),(1,3), (2,3),(3,3),(3,2),(3,1),(2,1),(2,0)] General search problem code def graph_search(start,goal):! visited = {}! “fringe” fringe = [start]! visited while True:! states !if fringe.empty():! goal !! !return False! !state = fringe.pop()! !if is_goal(node):! whole state space !! !return True! !if state not in visited:! !! !visited.append(state)! ! !fringe.append(get_successors(state))! fringe as stack: DFS def graph_search(start,goal):! next state visited = {}! state just fringe = [start]! to visit added while True:! !if fringe.empty():! !! !return False! !state = fringe.pop()! !if is_goal(node):! !! !return True! !if state not in visited:! STACK !! !visited.append(state)! ! !fringe.append(get_successors(state))! fringe as queue: BFS def graph_search(start,goal):! state visited = {}! just fringe = [start]! added while True:! !if fringe.empty():! !! !return False! next !state = fringe.pop()! state !if is_goal(node):! to visit !! !return True! !if state not in visited:! QUEUE !! !visited.append(state)! ! !fringe.append(get_successors(state))! Blundering around in the dark

Acon es are broken in order: N, E, S, W What does BFS do? DFS?

start goal Properes of search

• Completeness: does it ﬁnd a soluon? • Opmality: does it ﬁnd the best (least cost) soluon?

• Is DFS complete? opmal? • Is BFS complete? opmal? Informed search

Which way to try next? Heuriscs Deﬁne: • h(n) = esmated cost of path from node n to the goal node. • g(n) = cost to get from the start node to the node n. • f(n) = h(n) + g(n) = esmated cost of a full path going through node n

New strategy: Visit nodes in the fringe in order of f(n) Informed search: A* def a_star(start,goal):! visited = {}! g[start] = 0! fringe = PriorityQueue()! “fringe” fringe.put(priority=0,val=start)! visited while True:! states !if fringe.empty():! !! !return False! goal !state = fringe.pop_highest_priority()! !if is_goal(node):! !! !return True! whole state space !if state not in visited:! !! !visited.append(state)! ! !for succ in get_successors(state):! ! ! !g[succ] = g[state]+1! ! ! !h[succ] = heuristic_estimate(succ)! ! ! !f[succ] = g[succ]+h[succ]! ! ! !fringe.put(priority = f[succ], val = succ)! Example: path planning

Find the shortest path from point A (ant) to point B (food)

• states? ant • successor funcon? • goal test? • path cost? • heurisc?

food A* behavior

• What does A* do here with a straight-line heurisc? h(n) = dist(n,goal) • What does A* do here with h(n) = 0 for all n? • Ok, what about h(n) = 9e999 / dist(n,goal) ? (inversely proporonal to goal, and so large that g(n) doesn’t maer)

start goal A* properes

• Complete? – well, depends on h(n)

• Opmal? – well, depends on h(n)… Admissability

• h(n) should be an underesmate of the cost to get to the goal

• Why?

h(here): sorry dude, you are really really far away from the goal right now

ant: …ok... A* with an admissible heurisc is opmal Proof:

• Let C* be the cost of the opmal path to the goal G: f(Gbest-path) = C* • If goal appears in the fringe via a bad path, it must be that f(Gbad-path) > C*, just by deﬁnion of it being a bad path • For any node n on the best path, we know f(n) = g(n) + h(n) < C*, since h(n) is admissable. • Thus, nodes n on the best path will always be picked from the fringe before we reach G on a bad path. QED.

f(Gbad-path) > C*

node n G

f(n) < C* f(Gbest-path) = C* start One catch

goal

ALREADY VISITED

start Catch: ﬁxes

• Re-visit visited states – messy bookkeeping, but does the job • Ensure your heurisc is admissible and monotonic (never increases as you get closer) – almost always the case for any admissible heurisc you might dream up

goal

start Designing heuriscs

• Relax constraints on the problem

• Combine heuriscs – h(n) = max(h1(n),h2(n),…) is admissible – h(n) = w1*h1(n)+w2*h2(n)+… also admissible, if w1+w2+… = 1 Example: path planning

Find the shortest path from point A (ant) to point B (food)

• states? ant • successor funcon? • goal test? • path cost? • heurisc?

food Example: sliding block puzzle

Arrange les in numerical order.

• states? • successor funcon? • goal test? • path cost? • heurisc? Example: planning a course schedule

Select a course plan to graduate: • Can take a limited number of courses at the same me. • Most courses require pre- reqs ﬁrst. • Most courses are not oﬀered every semester.

• states? • successor funcon? • goal test? • step cost? • heurisc? Homework

• Due next Tuesday, midnight • More work than previous homework • 3 parts – Path planning, 1 ant – Coordinated path planning with mulple ants – Combining search and DFAs to win the compeon • Good luck!