Optimal decision making and matching are tied through diminishing returns Jan Kubaneka,1 aDepartment of Neurobiology, Stanford University School of Medicine, Stanford, CA 94305 Edited by Ranulfo Romo, Universidad Nacional Autonoma´ de Mexico,´ Mexico City, D.F., Mexico, and approved July 3, 2017 (received for review March 1, 2017) How individuals make decisions has been a matter of long- Which of these models is the more appropriate to capture and standing debate among economists and researchers in the life predict choice behavior has been a subject of substantial debate sciences. In economics, subjects are viewed as optimal decision (7, 8, 18–22). In some cases, subjects maximize and do not match makers who maximize their overall reward income. This frame- (23, 24), whereas in other cases subjects match even though max- work has been widely influential, but requires a complete knowl- imization would be a better strategy (3, 20, 25, 26). Psychologists edge of the reward contingencies associated with a given choice have compared matching and maximization in tasks that used spe- situation. Psychologists and ecologists have observed that indi- cific schedules of reinforcement (24, 27, 28), but whether the two viduals tend to use a simpler “matching” strategy, distributing models can be linked analytically at a general level has remained their behavior in proportion to relative rewards associated with elusive. The present article turns matching and maximization face their options. This article demonstrates that the two dominant to face and at a general level. By doing so, it identifies a connec- frameworks of choice behavior are linked through the law of tion between the economic and psychological frameworks, and diminishing returns. The relatively simple matching can in fact the nature of the connection provides an explanation for why provide maximal reward when the rewards associated with deci- humans and animals so often follow the matching strategy. sion makers’ options saturate with the invested effort. Such sat- urating relationships between reward and effort are hallmarks of Results the law of diminishing returns. Given the prevalence of diminish- Optimal Decision Making. An optimal decision maker distributes ing returns in nature and social settings, this finding can explain her effort across options such as to maximize the total harvested why humans and animals so commonly behave according to the reward. When effort Ei allocated at option i yields reward rate matching law. The article underscores the importance of the law Ri (Ei ), the total reward rate the decision maker obtains for a of diminishing returns in choice behavior. specific distribution of effort among her n options amounts to R1(E1) + R2(E2) + ::: + Rn (En ). The total effort that an indi- P choice behavior j rational choice theory j economic maximization j vidual can invest is limited, i Ei = Emax . Given that, reward neuroeconomics j matching law optimum is attained (Methods) when dR1(E1) = dR2(E2) = ::: = dE1 dE2 dRn (En ) . A decision maker can maximize her total reward by eople’s decisions define the future of individuals and social dEn equalizing marginal reward per effort dR(E) across her options. Pgroups. How to suitably model, understand, and predict indi- dE viduals’ choice behavior has therefore been a matter of intense R, the expected rate of reward, is in this article for simplicity research efforts involving multiple disciplines. referred to as “reward.” Economic models provide normative prescriptions of how Matching Behavior. Humans and animals often follow a matching individuals should make decisions. According to these models, strategy (3, 8–16), distributing their behavior Bi in proportion individuals make decisions in order to maximize their expected reward, utility, or income (1–3). For example, in prospect the- Significance ory (4), subjects maximize the expected utility of potential deci- sion outcomes. The expected utility is computed as the sum of the outcomes’ values weighted by the probabilities that the indi- Decisions critically affect the well-being of individuals and vidual outcomes will occur. Within such maximization models, societies. However, how to suitably model, understand, and Bayesian decision theories can be used to dictate how subjects predict people’s decisions has been a long-standing challenge. should optimally compute the individual probability terms (5). Economic models view individuals as optimal decision mak- Despite the normative appeal of such maximization models, it ers who maximize their overall reward income. Psychologists has been unclear whether organisms are capable of implement- and ecologists have observed that decision makers tend to ing and acting on the complex computations prescribed by these use a relatively simpler “matching” strategy, distributing their models (6–8). behavior in proportion to relative worth of their options. This Researchers in psychology, ecology, sociology, and neuro- article demonstrates that matching can be an optimal strat- science found evidence for a relatively simpler model for decision egy for decision makers when the rewards associated with making. It has been found that decision makers tend to distribute their options diminish with the invested effort, a relationship their behavior in proportion to relative rewards associated with known as the law of diminishing returns. Because diminishing their options (3, 8–16). The match between the behavioral and returns are prevalent in nature and social settings, the com- Bi Ri monly observed matching behavior is not only simple but also reward distributions, = , where Bi is B1+B2+:::+Bn R1+R2+:::+Rn efficient and rational. the rate of behavior allocated at option i, and Ri is the corre- sponding rate of the obtained reward, has come to be known Author contributions: J.K. designed research, performed research, contributed new as the “matching law” (8, 9, 11, 13, 14). Analogously to eco- reagents/analytic tools, analyzed data, and wrote the paper. nomic models, Ri can also represent utilities. This way, matching The author declares no conflict of interest. may capture a wide range of decision conditions while remaining This article is a PNAS Direct Submission. relatively compact (14, 17). However, matching has been crit- 1Email: [email protected]. icized for lacking a theoretical basis; matching is an empirical This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10. COGNITIVE SCIENCES phenomenon (17–19). 1073/pnas.1703440114/-/DCSupplemental. PSYCHOLOGICAL AND www.pnas.org/cgi/doi/10.1073/pnas.1703440114 PNAS j August 8, 2017 j vol. 114 j no. 32 j 8499–8504 Downloaded by guest on October 7, 2021 to relative rewards Ri associated with their options, Bi = Ri . This “matching” equation (9, 11) B1+B2+:::+Bn R1+R2+:::+Rn can be equivalently written as R1 = R2 = ::: = Rn . Further- B1 B2 Bn more, noting that behavior B embodies effort E, matching can be stated as R1 = R2 = ::: = Rn . It becomes apparent that when E1 E2 En R subjects match, they equalize average reward per effort E across their options. When the value of an option i, Vi , is defined as Ri the ratio of the obtained reward per the invested effort, Vi = , Ei matching equalizes the values Vi = V across the choice options. Maximization–Matching Relation. Given these observations, max- imization and matching align when marginal reward per effort associated with an option is a strictly monotonic function g of dR R average reward per effort associated with the option, dE = g E (Methods). The marginal and average quantities for an example relationship between reward and effort are illustrated as dotted and dashed lines in Fig. 1, respectively. This formulation empowers us to identify the relationships Fig. 2. Relationships between reward and effort for which matching is (henceforth referred to as contingencies) between reward and optimal exhibit diminishing returns. Choice behavior has often been stud- effort R(E) for which the two approaches provide an equal ied using the variable-interval schedule task. In this task, a single reward amount of reward. The contingencies emerge simply by defin- is scheduled at an option at a specific rate, such as, on average, one per ing a particular form of the strictly monotonic function g. There minute here. Once scheduled, the reward is available until a subject chooses are only two requirements (Methods) on g to yield contingencies the option. A simulation of the reward harvested in this task (black curve) for which matching delivers maximal reward: (i) g(V ) must have shows that when a subject chooses an option at the same rate as the sched- ule rate (once per minute), reward is delivered on average in 50% of the its derivative greater than zero and (ii) it must be g(V ) < V . cases, as expected. Furthermore, as expected, investing more effort leads to Simple forms of g (Methods) yield contingencies including more reward. Nonetheless, at a certain point, there are diminishing returns. a R(E) = aR ln(E) + c with a > 0, R(E) = RE with 0 < a < 1, The diminishing-returns character of the data can be well captured with the E derived reward–effort contingencies, exemplified by three particular func- and R(E) = a+cE with a; c > 0, where R is the reward associ- ated with an option, E is the invested effort, and a and c are tions (colored curves). A logarithmic function (R(E) = aR ln(E) + c) explains 97:8% of variance in the data, a Cobb–Douglas function (R(E) = REa) 95:9% constants. E of variance, and a hyperbola (R(E) = a+cE ) 99:7% of variance. Similarly tight These derived contingencies provide tight fits to the relation- fits are observed also for other reward scheduling rates. ship between reward and effort in a dominant task used to study choice behavior (8, 9, 13). In particular, in the variable- interval schedule task (Fig. 2), an increase in a subject’s effort profile of the variable-interval schedule task, exhibit diminishing generally leads to an increase in the harvested reward.