Principles of Ar Ficial Intelligence

Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory

Principles of Arﬁcial Intelligence

Vasant Honavar Arﬁcial Intelligence Research Laboratory College of Informaon Sciences and Technology Bioinformacs and Genomics Graduate Program The Huck Instutes of the Life Sciences Pennsylvania State University

[email protected] hp://vhonavar.ist.psu.edu hp://faculty.ist.psu.edu/vhonavar

• The agent seeks to achieve a speciﬁed goal • Aaining a goal may require a long sequence of acons • Needs a model (representaon) of the world

• Goal-based agents • Design of simple goal-based agents – Discrete, fully observable states – Discrete acons • Problem formulaon – Problem solving as search – State space search – Example problems • (Review of) Basic (Uninformed) Search Algorithms

• Formulate the goals – Explicit speciﬁcaon – Implicit speciﬁcaon (goal predicate) • Formulate the acons – Precondions (before) – Post-condions (aer) • Design a representaon that – Captures relevant aspects of the world – Abstracts away unimportant details

• States? – Posion of each le on the board • Inial state? – Any state can be inial • Acons? – {Le, Right, Up, Down} • Goal test? Check whether goal conﬁguraon is reached

Simplifying assumpons • Discrete, fully observable states – ‘in class’, ‘at home’ • Discrete acons – Mary executes acon ‘Go home’ in state ‘in class’ to reach the ‘at home’ state – In this setup, we can’t speak of Mary being on her way home • Passive environment – All state changes due to the agent’s acon – Mary can’t end up at home because her mom picked her up

A representaon • Maps each (physical) state of the external environment into the corresponding abstract state via sensors • Maps each (physical) acon on an environmental state into an abstract acon on the corresponding abstract state • Maps effects of an abstract acon on an abstract state into a corresponding effect on the corresponding environmental state via effectors The mapping from • environmental states and abstract states is many to one • abstract state to an environmental state is one to many

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Representaon The mapping from environmental states and abstract states is many to one abstract state to an environmental state is one to many A representaon induces a paron over environmental states

4 abstract states, acons may allow only lateral or vercal moves – not all environmental state transions can be modeled by the agent Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Representaon • Effects of abstract acons in the abstract state space may be fully determinisc and predictable… but… • The corresponding effects of the physical acons on the environmental state space are predictable only to the extent – allowed by the resoluon of the representaon and the fidelity of sensors and effectors – That the environment is indeed determinisc

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Representaon • Is a surrogate inside an agent’s ‘brain’ for enes that exist in the external world • Is not just a data structure – why? • Derives its semancs through semanc grounding (sensors, eﬀectors) – Correspondence between descripons and states of the world or acons that change the state of the world • Embodies a set of ontological commitments – assumpons about the enes, properes, relaonships, and acons that we care about • Choice of representaon maers!

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Example: Missionaries and Cannibals • Inial state: 3 missionaries, 3 cannibals, and the boat on the le bank of the river • Goal: all on the right bank • Constraints: – The boat which can carry at most 2 people at a me – If missionaries are outnumbered by cannibals, the cannibals will eat the missionaries • States: The posions of missionaries, cannibals, and the boat on either side of the river • Acons: Movement of the boat with its occupants from one side of the river to the other • Soluon: A sequence of boat trips across the river complete with their passenger lists

• On holiday in Romania; currently in Arad – Flight leaves tomorrow from Bucharest • Formulate goal – Be in Bucharest • Formulate problem – States: various cies – Acons: drive between cies • Find soluon – Sequence of cies; e.g. Arad, Sibiu, Fagaras, Bucharest, …

• A problem is deﬁned by: – An inial state, e.g. Arad – Successor funcon S(X)= set of acon-state pairs • e.g. S(Arad)={,…} – Goal test, can be • Explicit, e.g. x=‘at bucharest’ • Implicit, e.g. checkmate(x)

• Inial state + successor funcon deﬁnes a state space • A soluon is a sequence of acons from the inial to goal state

A state space search problem is specified by a 3-tuple (s, A, G) where • s is a start state – s ∈ S, the set of possible start states • O is the set of acons (operators) – Paral funcons that map a state into another • G the set of goal states – G may be explicitly enumerated or implicitly specified using a goal predicate goal (g) = True iff g ∈ G Soluon to a state space search problem is a sequence of acon applicaons leading from the start state s to a goal g ∈ G

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Problem formulaon – finding an opmal soluon • A problem is defined by: – An inial state, e.g. Arad – Successor funcon S(X)= set of acon-state pairs • e.g. S(Arad)={,…} inial state and the successor funcon together define the state space – Goal test • Explicit, e.g. x=‘at bucharest’ • Implicit, e.g. checkmate(x) – Path cost (addive) • e.g. sum of distances, number of acons executed, … • c(x,a,y) is the step cost, assumed to be ≥ 0 Opmal soluon has the lowest path cost

• All operator applicaons may not be equally expensive • Suppose we have a cost funcon c: S x O à ℜ+ • c (s,o,r) = cost of applying operator o in state q to reach state r • Path cost is typically assumed to be the sum of costs of operator applicaons along the path • An opmal soluon is one with the lowest cost path from the speciﬁed start state s to a goal g ∈ G

• Real world can be absurdly complex • State space representaon is an abstracon – (Abstract) state corresponds to a set of real world states – (Abstract) acon corresponds to a complex combinaon of real world acons – e.g. Arad → Zerind represents a complex set of possible routes, detours, rest stops, etc. – The abstracon is valid if the path between two states is reﬂected in the real world. • (Abstract) soluon = set of real paths that are soluons in the real world.

Scks and Squares Problem • 17 scks arranged in 6 squares • Goal remove 5 scks so we are le with exactly 3 squares (no extra scks) • What is the size of the state space?

Scks and Squares Problem • 17 scks arranged in 6 squares • Goal remove 5 scks so we are le with exactly 3 squares (no extra scks) • What is the size of the state space? – Depends on the representaon

! 17 $ ⎛6⎞ # & ⎜ ⎟ 5 ⎜ ⎟ " % ⎝3⎠

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Importance of representaon • Ontological commitment maers • Abstracon maers • Granularity of the representaon maers • Good representaons – preserve the relevant aspects of the problem – expose the relevant problem structure • Bad representaons – Lose potenally relevant informaon – obscure the relevant problem structure • How to automacally discover good representaons is a fundamental problem in AI • Millions of years of evoluon have given humans a head-start

• States? • Inial state? • Acons? • Goal test? • Path cost?

• States? two locaons with or without dirt, with or without the vacuum cleaner: 2 x 22=8 states. • Inial state? Any state can be inial • Acons? {Le, Right, Cleanup} • Goal test? Check whether both locaons are clean. • Path cost? Number of acons to reach goal Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Example: 8-puzzle

• States? • Inial state? • Acons? • Goal test? • Path cost?

• States? Integer locaon of each le • Inial state? Any state can be inial • Acons? {Le, Right, Up, Down} • Goal test? Check whether goal conﬁguraon is reached • Path cost? Number of acons to reach goal

Constraints: No two queens can share – A row – A column – A diagonal

• States? • Inial state? • Acons? • Goal test? • Path cost?

Problem formulaon • States? – Any arrangement of 0 to 8 queens on the board • Inial state? – Empty board (no queens) • Acons? – Add a queen in empty square • Goal test? – 8 queens on board and none under aack

Soluon 1

Any arrangement of 0 to 8 queens on the board • 64 squares, 8 queens – (64)(63)(62)(61)..(57) ≈ 3 × 1014 ≈ 1.2681 × 247 states!

Soluon 2

Any arrangement of 0 to 8 queens on the board • 8 rows – need to specify the column in which a queen is placed in each row – (8)(7)(6)(5)(4)(3)(2) ≈ 1.231 × 215 states! – Absorbed the `no two queens can share a row’ constraint into the representaon!

Soluon 3

Any arrangement of 0 to 8 queens on the board • States: n (0≤ n≤ 8) queens on the board, one per column in the n lemost columns with no queen aacking another • Acons: Add a queen to the lemost empty column so as not to aack the other queens • Number of states = 2057 Representaon maers!

Let L be a list of nodes yet to be expanded 1. Let L = (s) 2. If L is empty, return failure else pick a node n from L (which node?) 3. If n is a goal node, a. return path from s to n and stop. b. Otherwise i. Delete n from L ii. Expand n: Add to L all of n’s successors (where?) 4. Return to 2.

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory State space search • A state is an (internal representaon of) a physical conﬁguraon • A node is a data structure that is used to construct a search tree – A node has a parent, successors, and includes bookkeeping informaon e.g., depth, … – node = – Each arc corresponds to an operator applicaon – Each node in the search tree implicitly represents a candidate paral soluon

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Basic Search strategies A search strategy specifies a parcular order of node expansion Search strategies are evaluated in terms of: • Completeness: – Does it always find a soluon if one exists? • Admissibility – Russell and Norvig call this opmality – Does it always find an opmal soluon? • Time Complexity – Number of nodes generated or expanded • Space Complexity – memory needed to store L during search • Opmality – Opmal in its use of space, me, or both

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar

Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Analysis of Basic Search strategies • Time and space complexity are measured in terms of problem difficulty defined by: – b - maximum branching factor of the search tree – d - depth of the least-cost soluon – m - maximum depth of the state space (may be ∞) • Assumpons – Uniform, finite branching factor b – A single goal node exists at a finite depth d – Goal is uniformly distributed at depth d – Maximum depth of search space is m

• Use only informaon available in problem definion • Blind search strategies: – Breadth-first search – Uniform-cost search – Depth-first search – Depth-limited search – Iterave deepening search – Bidireconal search

• Expand shallowest unexpanded node • Implementaon: L is a FIFO queue – Nodes on L = (A)

• Expand shallowest unexpanded node • Implementaon: L is a FIFO queue

L = (B, C) A

B C

• Expand shallowest unexpanded node • Implementaon: L is a FIFO queue

L = (C, D, E) A

B C

D E

• Expand shallowest unexpanded node • Implementaon: L is a FIFO queue

L = (D, E, F, G)

B C

F G D E

• Is BFS complete? – Is BFS guaranteed to find a soluon if one exists? – Yes (b is finite, d is finite)

• Worst case space complexity – Every node at depth d must be on the list L before a soluon at depth d can be found – In the worst case, all successors of depth d non-goal nodes must be on the list before a soluon at depth d can be found

O(bd +1)

• Worst case me complexity - Number of nodes generated 1+b +b2 +b3 +... +bd + (bd +1 −b +1) = O(bd +1)

- Number of nodes expanded

1+ b + b2 + b3 +... + bd = O(bd +1)

- Average case me complexity - number of nodes expanded - Best case = 1+ b + b2 + b3 +... + bd −1 +1 - Worst case = 1+ b + b2 + b3 +... + bd −1 + bd - The goal is uniformly distributed at depth d - Expected case me complexity Best +Worst ⎛1+ bd ⎞ =1+ b + b2 + b3 +... + bd −1 + ⎜ ⎟ 2 ⎜ 2 ⎟ ⎝ ⎠ ⎛ bd −1⎞ ⎛1+ bd ⎞ = ⎜ ⎟ + ⎜ ⎟ ⎝ b −1 ⎠ ⎝ 2 ⎠

• Is BFS admissible? – Yes, if all operator costs are equal – Otherwise, in general, No • Is BFS opmal? – As we will see later, No – We can do signiﬁcantly beer than BFS in terms of space requirement

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory BFS Summary • Memory requirements are more problemac than execuon me • Uninformed search methods are infeasible for all but the smallest problem instances DEPTH NODES TIME MEMORY 2 1100 0.11 seconds 1 megabyte 4 111100 11 seconds 106 megabytes 6 107 19 minutes 10 gigabytes 8 109 31 hours 1 terabyte 10 1011 129 days 101 terabytes 12 1013 35 years 10 petabytes 14 1015 3523 years 1 exabyte b = 10, processing speed = 10,000 nodes / sec space = 1000 bytes per node Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Depth ﬁrst search (DFS)

• Expand deepest unexpanded node • Implementaon: L is a LIFO queue (stack) L = (A) A

• Expand deepest unexpanded node • Implementaon: L is a LIFO queue (stack)

A L = (B C) B C

• Expand deepest unexpanded node • Implementaon: L is a LIFO queue (=stack)

L = (D E C) A

B B C C

D E

• Expand the deepest unexpanded node • Implementaon: L is a LIFO queue (stack)

A B C

D E

H I I

• Expand deepest unexpanded node • Implementaon: L is a LIFO queue (=stack)

C B D E

H I

• Expand deepest unexpanded node • Implementaon: L is a LIFO queue (stack)

A B C D E

H I

• Expand deepest unexpanded node • Implementaon: L is a LIFO queue (stack)

B C

D E

H I J K

• Expand deepest unexpanded node • Implementaon: L is a LIFO queue (stack)

A B C

D E

H I J K

• Expand deepest unexpanded node • Implementaon: L is a LIFO queue (stack)

A B C

D E

H I J K

• Expand deepest unexpanded node • Implementaon: L is a LIFO queue (stack)

B C

D E F G

H I J K

• Expand deepest unexpanded node • Implementaon: L is a LIFO queue (stack)

A B C

D E F G

H I J K L M

• Completeness – Is DFS guaranteed to find a soluon if one exists? – No! • unless search space is finite and no loops are possible • Admissibility – Is DFS guaranteed to find an opmal soluon? • No!

b-1 1+(b−1)

b-1 1+ 2(b −1)

Space complexity O(1+ m(b−1))= O(mb) where m = maximum depth of the search space

• Unlike BFS, DFS has space complexity that grows linearly in b and m • Space complexity of DFS can be further improved to O(m) with backtracking search – Each parally expanded node remembers which successor to generate next – Avoids the need to put all successors on the list

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory DFS Summary m • Worst case me complexity: O(b ) • Space complexity O(mb) or even O(m) • Complete? No • Admissible? No • Opmal? No €

• BFS is complete, but has terrible space complexity • DFS has aracve space complexity, but is not complete • Can we get the best of both worlds?

Combines beneﬁts of DFS and BFS 1. Set depth limit l =1 2. Perform DFS with depth limit l 3. If soluon is found, return soluon 4. Otherwise, increment depth limit l and return to step 2 Space complexity O(bd) or O(d) if backtracking is used Time complexity d −1 d −1 l+1 ∑(1+ b +...bl+1 )= ∑∑bk l=0 l=0 k=0 = O(bd +1 )

• IDS is complete • IDS is admissible (if acons have equal cost) • IDS is an opmal blind search algorithm • Time complexity – Soluon located at depth d – Cannot avoid examining O( bd ) nodes àTime complexity of IDS is opmal • Space complexity – Run me O( bd ) – Must be able to count up to bd – Counter needs O(d) bits

• Limit=0

• Limit=1

• Limit=2

• Limit=3

bd / 2 + bd / 2 << bd

• Two simultaneous searches from start and goal states • Check whether a node belongs to the other fringe before expansion • The predecessor of a node must be easy to compute – helps if the acons are reversible • Complete and opmal if both searches are BF Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Summary of algorithms Criterion Breadth Depth- Iterative Bi- -First First deepening directional Complete? YES NO YES YES* Time bd+1 bm bd bd/2

Space bd+1 bm bd bd/2

Admissible? YES NO YES YES*

Optimal? NO NO YES NO

Assuming all arc costs are equal m – max depth of search d – depth of soluon, b – ﬁnite branching factor * Assuming forward and backward search are BFS

• Failure to detect repeated states can turn solvable problems into unsolvable ones

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Graph search algorithm • Use an open list to store the list of nodes to be expanded • Use a closed list to store previously expanded nodes • Maintaining closed list means space complexity of DFS and IDS can’t be linear • Cycles need to be dealt with

• Previous assumpon: – Environment is fully observable – Environment is determinisc – Agent knows the eﬀects of its acons

What if knowledge of states or acons is incomplete?

• Paral knowledge of states and acons: – Sensor-less or conformant problem • Agent may have no idea the state it is in • Agent knows the eﬀects of its acons – conngency problem • Percepts provide new informaon about current state – soluon is a tree or policy – interleave search and execuon • If uncertainty is caused by acons of another agent: adversarial problem – exploraon problem • When states and acons of the environment are unknown

• Start in {1,2,3,4,5,6,7,8} e.g Right goes to {2,4,6,8} Soluon? – [Right, Suck, Le,Suck] • When the world is not observable – reason about a set of states that might be reached – Acons correspond to transions between sets of states (belief states)

• Search space of belief states • Soluon = belief state with each member being a goal state • If the state space has S states then there are 2S belief states in the worst case • what if the environment is non determinisc?

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Conngency problems • Conngency, start in {1,3} • Murphy’s law, Cleanup can dirty a clean carpet • Local sensing: dirt, locaon only – Percept = [L,Dirty] ={1,3} – [Cleanup] = {5,7} – [Right] ={6,8} – [Cleanup] in {6}={8} (Success) – BUT [Cleanup] in {8} = failure

• Soluon? – [Cleanup, Right, if [R,dirty] then Cleanup] – It is hard to account for every possible conngency before acng – Select acons based on conngencies arising during execuon

• Can be viewed as an extreme case of conngency problems • Oen solved using reinforcement learning

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory General recipe for uninformed search function TREE-SEARCH(problem,open-list) return a solution or failure open-list ← INSERT(MAKE-NODE(INITIAL- STATE[problem]), open-list) loop do if EMPTY?(open-list) then return failure node ← REMOVE-FIRST(open-list) if GOAL-TEST[problem] applied to STATE[node] succeeds then return SOLUTION(node) open-list ← INSERT-ALL(EXPAND(node, problem), open-list) A strategy is defined by picking the order of node expansion

• Branch and bound search (BBS) with dynamic programming • Open list nodes sorted by cost g(n) of cheapest known paral paths to the respecve nodes • Terminate when a node picked from the open list happens to be a goal node • Quesons: – Is BBS complete? • Yes – Is BBS admissible? • Yes – Under the assumpon that each arc cost is bounded from below by some posive constant δ

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar