Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Delibera(ve Agents Knowledge Representa(on: Decision- Theorec Vasant Honavar Ar3ficial Intelligence Research Laboratory College of Informaon Sciences and Technology Bioinformacs and Genomics Graduate Program The Huck Ins3tutes of the Life Sciences Pennsylvania State University [email protected] hHp://vhonavar.ist.psu.edu hHp://faculty.ist.psu.edu/vhonavar Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Outline • Combining beliefs and preferences • Basics of U3lity Theory • U3lity Func3ons • Decision Networks • Value of Informaon Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Decision-Theore3c Agent • Different states have different u3lity to the agent • U3lity func3on of a simple decision theore3c agent maps each state onto a real number (u3lity) • Ac3ons are chosen based on the expected u3lity of the resul3ng state • More general seng involves making complex decisions involving sequences of ac3ons Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Making Simple Decisions - Example Decision 3me for Agent Joe Sixpack Study S1 Party S 0 S2 Sleep S3 Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Combining Beliefs Desires • Raonal Behavior – Based on beliefs about the world, in par3cular, consequences of one’s ac3ons in a given state – Must cope with uncertainty • There is no way to know for sure the outcome of an ac3on because of par3al ignorance or inherently stochas3c effects of ac3ons – Must provide a means of comparing alternaves using a common currency • Partying on a Thursday night versus geng an A Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Should Bob buy insurance? • Bob is contemplang whether he should take insurance on a shipment from Amsterdam to St. Petersburg. • If the ship does not encounter a storm, the shipment arrives on 3me in St. Petersburg, and Bob will earn 10,000 rubles • If the ship encounters a storm, the shipment will be delayed and Bob will earn only 8000 rubles. • The Amsterdam underwriters want Bob to pay 800 rubles for a full coverage insurance policy • Should Bob buy the policy? Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Should Bob buy insurance? • Poten3al loss to Bob in the event of delayed arrival of shipment = 2000 rubles • Cost of insurance policy = 1000 rubles • Should Bob buy the policy? Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Should Bob buy insurance? • Depends • On what? – Probability of storm-related delay – If the storm is highly unlikely (say probability of storm ≈ 0.2) at that 3me of the year, perhaps not – If the storm is likely (say probability of storm ≈ 0.8) then perhaps – Can we translate this intui3on into a precise prescrip3on for decision making under uncertainty? Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Should Bob buy insurance? • Suppose Bob believes that the probability of a storm related delay ≈ 0.2 • Bob’s expected earnings in the absence of insurance = (0.2)(8000) + 0.8 (10,000) = 1600 + 8000 = 9600 rubles • Bob’s expected earnings if he purchases insurance = 10,000 – 1000 = 9000 rubles • Bob is perhaps beHer off without insurance than with it. Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Should Bob buy insurance? • Suppose Bob believes that the probability of a storm related delay ≈ 0.8 • Bob’s expected earnings in the absence of insurance = (0.8)(8000) + 0.2 (10,000) = 6400 + 2000 = 8400 rubles • Bob’s expected earnings if he purchases insurance = 10,000 – 1000 = 9000 rubles • Bob is beHer off with insurance than without it. Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory St. Petersburg Paradox • From Nicolas Bernoulli’s leer • Consider the following game – Peter flips a fair coin repeatedly un3l a head shows up and will give Paul: • $2k if the first head shows up on the kth flip – How much should Paul pay Peter to play this game? Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory St. Petersburg Paradox (cont) k k=∞ k" 1% E(payoff ) = ∑ 2 $ ' =1+1+! = ∞ k=1 # 2& • The expected payoff is infinite • Does this mean it is raonal for Paul to pay Peter any € finite amount (say $1 million) to play this game? Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory St Petersburg Paradox • How much should Paul be willing to pay Peter for a chance to play the game? • Expected payoff = infinity • But.. There is a risk of loss – Suppose Paul pays $1,000,000 to pay the game – Suppose the first head shows up on the 2nd toss – Paul will receive $4 and lose $999996 • Is the gamble worth the risk? • Depends.. Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory St Petersburg paradox • Is Paul’s gamble worth the risk of losing almost $1,000,000? • Depends • On what? – How much money Paul has to start with – The risk might be unacceptable if Paul’s en3re life savings is $1,000,000 – The risk might be perfectly reasonable if Paul has billions in the bank – If Paul is poor, he may be jus3fied in paying no more than two dollars, the minimum possible pay-off of the game Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Making simple decisions • Can we turn the previous intui3on into a recipe for decision making under uncertainty? • Von Neumann – Morgenstern’s soluon – MEU principle • Choose ac3ons that maximize expected u)lity of outcome – As evident from the St. Petersburg paradox, for most people, the u)lity of money is not a linear func3on of the amount of money Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Making simple decisions • Notaon – U(S) : u3lity of state S – S : snapshot of the world – A : ac3on of the agent – Result : i (A) ith outcome of (state resul3ng from doing) A – E : available evidence – Do(A) : execung A in current state S Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Making simple decisions • U3lity func3on – assigns a single number to each outcome – models the desirability of the state to an agent – combined with probability of each outcome resul3ng from an ac3on yields expected u3lity for ac3on leading to each outcome S1 Action A S2 s0 S1 Action B S2 Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Making Decisions • Expected U3lity EU (A| E) P(Result (A) | E, Do(A))U(Result (A)) = ∑ i i i • Maximum Expected U3lity(MEU) – Choose an ac3on which maximizes agent’s expected u3lity • Compu3ng requires a probabilis3c P(Resulti (A) | E, Do(A)) model of the world (Bayes Network) • Compu3ng the u3lity of a state may require search U(Resulti (A)) because it can be hard to tell how good a state is un3l we know where it would lead us Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Decision Theore3c Agent Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Loeries LoHeries are used to model decision making scenarios LoHeries have a