The high availability of extreme events serves resource-rational decision-making Falk Lieder ([email protected]) Helen Wills Neuroscience Institute, University of California at Berkeley, CA, USA Ming Hsu ([email protected]) Haas School of Business, University of California at Berkeley, CA, USA Thomas L. Griffiths (tom griffi[email protected]) Department of Psychology, University of California at Berkeley, CA, USA Abstract Sampling as a Decision Strategy To evaluate a potential action a, decision makers should in- Extreme events come to mind very easily and people over- tegrate the probabilities P(ojA = a) of possible outcomes estimate their probability and overweight them in decision- making. In this paper we show that rational use of limited cog- o with their utilities u(o) into the action’s expected utility nitive resources can generate these ’availability biases.’ We hy- Ep(OjA=a)[u(O)] (Von Neumann & Morgenstern, 1944). In pothesize that availability helps people to quickly make good the real-world–unlike in the laboratory–each action has in- decisions in very risky situations. Our analysis shows that agents who decide by simulating a finite number of possible finitely many possible outcomes. As a consequence, the ex- outcomes (sampling) should over-sample outcomes with ex- pected utility of action a becomes an integral: treme utility. We derive a cognitive strategy with connections to fast-and-frugal heuristics, and we experimentally confirm Z its prediction that an event’s extremity increases the factor Ep(OjA=a)[u(O)] = p(oja) · u(o)do: (1) by which people overestimate its frequency. Our model also explains three context effects in decision-making under risk: framing effects, the Allais paradox, and preference reversals. In the general case, this integral is intractable to compute, but it can be approximated by sampling methods (Hammersley & Keywords: availability; Bayesian; bounded rationality; cog- nitive biases; heuristics; judgment and decision-making Handscomb, 1964). Mental simulations of a decision’s poten- tial consequences can be thought of as samples. In fact, there is neural evidence (Fiser, Berkes, Orban,´ & Lengyel, 2010) Introduction and behavioral evidence (Vul, Goodman, Griffiths, & Tenen- People overestimate the probability of extreme events such baum, 2014; Denison, Bonawitz, Gopnik, & Griffiths, 2013; as terrorism (Sunstein & Zeckhauser, 2011) and other threats Griffiths & Tenenbaum, 2006) that the brain handles uncer- (Lichtenstein, Slovic, Fischhoff, Layman, & Combs, 1978; tainty by sampling. For instance, people’s predictions of an Rothman, Klein, & Weinstein, 1996) and overreact accord- uncertain quantity X given partial information y are roughly ingly (Sunstein & Zeckhauser, 2011). This phenomenon has distributed according to the posterior distribution p(Xjy) as been explained by the availability bias (Tversky & Kahne- if they were sampled from it (Griffiths & Tenenbaum, 2006; man, 1973) according to which people overestimate the prob- Vul et al., 2014). These results suggest that people often ability of events that come to mind very easily. use only one or very few samples, and this is what rational agents with finite computational resources (bounded rational The availability bias appears irrational, but here we ar- agents) should do (Vul et al., 2014). The evidence for sam- gue that it reflects the rational use of finite time (resource- pling in human cognition raises the question which sampling rationality; Lieder, Griffiths, & Goodman, 2013, Griffiths, algorithm(s) are implemented in the brain. Lieder, & Goodman, in revision). In brief, we hypothesize that the high availability of extreme events helps decision- Importance sampling is a popular sampling algorithm in makers to allocate their finite time towards considering the computer science and statistics (Hammersley & Handscomb, most important consequences their decision might have. We 1964; Geweke, 1989), and it has been shown to have connec- model the strategy that might determine which events come tions to both neural network (Shi & Griffiths, 2009) and psy- to mind first and how they influence judgment and decision- chological process models (Shi, Griffiths, Feldman, & San- making. Our model explains not only why people overesti- born, 2010). Self-normalizing importance sampling estimates mate the probability of extreme events, but it also explains the expected value of a function by the weighted average of three context effects in decision-making under risk. the function’s values at points drawn from a distribution q: The plan of this paper is as follows: The first section intro- p(x j) duces the theoretical and empirical background. The second X1;··· ;Xs ∼ q; w j = (2) q(x j) section derives a rational model of decision-making in high- s risk situations. The third section presents an experiment test- E ˆ IS 1 p[ f (X)] ≈ Eq;s = s · ∑ w j · f (x j); (3) ing the model’s predictions for frequency judgment, and the ∑ j=1 w j j=1 fourth section applies the model to explain context-effects in th decision-making under risk. The final section discusses our where wi is the weight of the i sample. With finitely many results and their implications. samples, this estimate is generally biased. Following Zabaras (2010), we approximate its bias and variance by though the expected utility is about −1000. In conclusion, under high risk, representative sampling is insufficient for Z p(x)2 ˆ IS 1 E resource-bounded decision-making. Bias[Eq;s] ≈ · · ( p[ f (x)] − f (x))dx (4) s q(x) What is the problem with representative sampling and how 1 Z p(x)2 can it be overcome? Representative sampling fails when it Var[Eˆ IS ] ≈ · · ( f (x) − E [X])2 dx (5) q;s s q(x) p neglects crucial eventualities. Neglecting some eventuali- ties is necessary, but particular eventualities are more im- Importance sampling can be used to approximate the ex- portant than others. Intuitively, the importance of potential pected utility Ep(OjA=a)[u(O)] of taking action a and to es- outcome oi is determined by jp(oi) · u(oi)j because neglect- ? timate the optimal decision a = argmaxa Ep(OjA=a)[u(O)]: ing oi amounts to dropping the addend p(oi) · u(oi) from the expected-utility integral (Equation 1). Thus, intuitively, the ? IS IS aˆ = argmaxUˆ (a); Uˆ (a) ≈ Ep(Oja)[u(O)] (6) problem of representative sampling can be overcome by con- a q;s q;s sidering outcomes whose importance (jp(o ) · u(o )j) is high 1 s i i Uˆ IS (a) = w · u(o ); o ;··· ;o ∼ q: (7) and ignoring those whose importance is low. q;s s w ∑ j j 1 s ∑ j=1 j j=1 Formally, the agent’s goal is to maximize the expected util- ity of a decision made from only s samples. The utility fore- Note that importance sampling is a family of algorithms: each gone by choosing a sub-optimal action can be upper-bounded importance distribution q yields a different estimator, and two by the error in a rational agent’s utility estimate. Therefore estimators may recommend opposite decisions. We thus con- the agent should minimize the expected squared error of its sider which distribution q yields the best decisions. utility estimate, which is the sum of its squared bias and vari- E ˆ IS E 2 ˆ IS 2 ˆ IS Which distribution should we sample from? ance, i.e. (Uq;s − [U]) ] = Bias[Uq;s] + Var[Uq;s . As the number of samples s increases, the estimate’s squared bias Intuitively, the rational way to make a decision is to simu- decays much faster (O(s−2)) than its variance (O(s−1)); see late consequences o according to one’s best knowledge of the Equations 4-5. Therefore, as the number of samples s in- probability p that they will occur, i.e. q = p: creases, minimizing the estimator’s variance becomes a good s approximation to minimizing its expected squared error. RS 1 Uˆ (a) = u(oi); o1;··· ;os ∼ p(O): (8) According to variational calculus the variance (Equation 5) s s ∑ i=1 of the utility estimate in Equation 7 is minimized by This importance sampling algorithm, which we call represen- qvar(o) ∝ p(o) · ju(o) − E [U]j: (10) tative sampling, computes unbiased utility estimates. Yet– p surprisingly– representative sampling is insufficient for mak- Interestingly, the optimal sampling distribution overrepre- ing good decision with very few samples. Consider, for in- sents outcomes with large absolute utility, Thus, biased sam- stance, the choice to play Russian roulette with the popu- pling can lead to better decisions. Unfortunately, importance lar six-round NGant M1895 revolver. Playing the game will sampling with qvar is intractable, because it presupposes the 5 most likely, i.e. with probability p1 = 6 , reward you with a expected utility Ep[U] that importance sampling is supposed thrill and a gain in status (u(o1) = 1) but kill you otherwise to approximate. A priori the expected utility of a prospect 1 9 (p2 = 6 ; u(o2) = −10 ). Ensuring that representative sam- whose outcomes may be good or bad is equally likely to be pling declines a game of Russian roulette at least 99:99% of positive or negative. The same is true for choosing action a the time, would require 51 samples–potentially a very time- over action b or vice versa. Therefore, the most principled consuming computation. Many real-life decisions involve guess an agent can make for the expected utility Ep[U] in high risks and are even more challenging, because the prob- Equation 10–before computing it–is 0. Thus when the ex- 1 ability of disaster is orders of magnitude smaller than 6 but pected utility is not too far from zero, then the importance may or may not be large enough to warrant caution. Exam- distribution qvar is efficiently approximated by ples include risky driving, the stock market, and air travel.