Behavioral The official journal of the ISBE Ecology International Society for Behavioral Ecology Behavioral Ecology (2018), 29(1), 193–203. doi:10.1093/beheco/arx145 Original Article Rate maximization and hyperbolic discounting in human experiential intertemporal decision making Maayke Suzanne Seinstra, Manuela Sellitto, and Tobias Kalenscher Comparative Psychology, Institute of Experimental Psychology, Heinrich-Heine University Düsseldorf, Universitätsstraße 1, 40225 Düsseldorf, Germany Received 25 March 2017; revised 2 October 2017; editorial decision 11 October 2017; accepted 17 October 2017; Advance Access publication 16 November 2017. Decisions between differently timed outcomes are a well-studied topic in as diverse academic disciplines as economics, psychol- ogy, and behavioral ecology. Humans and other animals have been shown to make these intertemporal choices by hyperbolically devaluing rewards as a function of their delays (“delay discounting”), thus often deemed to behave myopically. In behavioral ecology, however, intertemporal choices are assumed to meet optimization principles, that is, the maximization of energy or reward rate. Thus far, it is unclear how different approaches assuming these 2 currencies, reward devaluation and reward rate maximization, could be reconciled. Here, we investigated the degree at which humans (N = 81) discount reward value and maximize reward rate when making intertemporal decisions. We found that both hyperbolic discounting and rate maximization well approximated the choices made in a range of different intertemporal choice design conditions. Notably, rate maximization rules provided even better fits to the choice data than hyperbolic discounting models in all conditions. Interestingly, in contrast to previous findings, rate maximization was universally observed in all choice frames, and not confined to foraging settings. Moreover, rate maximization correlated with the degree of hyper- bolic discounting in all conditions. This finding is in line with the possibility that evolution has favored hyperbolic discounting because it subserves reward rate maximization by allowing for flexible adjustment of preference for smaller, sooner or larger, later rewards. Thus, rate maximization may be a universal principle that has shaped intertemporal decision making in general and across a wide range of choice problems. Key words: hyperbolic discounting, intertemporal choice, preference reversal, reward rate maximization. INTRODUCTION choice and impulsive decision—the approaches in these different In our daily life, we make countless decisions between delayed disciplines came up with different accounts. consequences. These intertemporal decisions shape important Economics and psychology literature addressed great atten- aspects of our life, such as education, housing, diet, and financial tion to intertemporal decision making because the myopic, short- well-being. Intertemporal decision making is well studied in both sighted choice patterns of humans and other animals represent humans and nonhuman animals (Kalenscher et al. 2005; Rosati violations of the efficiency assumptions of utility maximization and et al. 2007; Kalenscher and Pennartz 2008; Sellitto et al. 2011) time preference in economics (Kalenscher and Pennartz 2008). In by as diverse academic disciplines as economics, psychology, and behavioral economics and psychology, intertemporal choice behav- behavioral ecology. All these fields share their interest in the typical ior is typically expressed as delay discounting (Samuelson 1937; behavior, common to humans and other animals, of overweight- Kalenscher and Pennartz 2008; Hayden 2016), according to which ing short-term outcomes or underweighting long-term outcomes the subjective value of a delayed reward decreases with increasing and, by consequence, of making impulsive decisions (Kalenscher delay of its receipt (Frederick et al. 2002; Sellitto et al. 2011). et al. 2005, 2008; Namboodiri and Hussain Shuler 2016). However, In both humans and other animals, delay discounting is best although trying to explain the same phenomenon—intertemporal described by hyperbolic discounting models, which reflect a decrease in the subjective value of a reward with a nonconstant decay rate, characterized by a steep decline in subjective value at initial delays, and flatter decline at longer delays (Mazur 1984; Address correspondence to M. Sellitto. E-mail: [email protected]. M.S.S. and M.S. contributed equally to this work. Green and Myerson 1996; Kalenscher and Pennartz 2008) Due © The Author(s) 2017. Published by Oxford University Press on behalf of the International Society for Behavioral Ecology. All rights reserved. For permissions, please e-mail: [email protected] Downloaded from https://academic.oup.com/beheco/article-abstract/29/1/193/4633897 by Universitaets- und Landesbibliothek, [email protected] on 17 January 2018 194 Behavioral Ecology to this property, hyperbolic discounting well explains the so-called binary self-control problems. This has been indeed shown in ani- “preference reversals”, which previous exponential discounting mals (Stephens and Anderson 2001; Stephens and McLinn 2003) models failed to account for (e.g., Frederick et al. 2002; Sellitto and more recently also in humans (e.g., Schweighofer et al. 2006; et al. 2011). When choosing between smaller-sooner and larger- Bixter and Luhman 2013; Zarr et al. 2014; Carter et al. 2015). later rewards, humans and nonhuman animals often reverse their A striking illustration of how organisms may implement short- preference when front-end delays are added or subtracted from sighted rules in order to achieve LTR maximization lies again in a choice set (Green et al. 1994; Kirby and Herrnstein 1995). For preference reversals—which, as said before, seem to indicate that example, even though an individual may prefer (A) €10 today over individuals overweight short-term outcomes (Thaler 1981; Benzion (B) €20 in 6 months, she may prefer (B’) €20 in 1 year over (A’) et al. 1989) and, from a normative economic perspective, that they €10 in 6 months (Frederick et al. 2002), despite the discounted util- act against their own future interest. If we go back to the previous ity theory (DUT) in economics (Samuelson 1937) prescribes that a example and consider preference reversals from the perspective of rational agent should meet the stationarity axiom and choose option reward rate maximization, choosing option A (€10 today) would A’ over option B’ since she preferred option A before (Fishburn and yield a reward rate of €10 per day, and choosing option B (€20 in Rubinstein 1982). 6 months) would yield a reward rate of €0.10 per day. The rate max- Next to the economics approach, humans and nonhuman ani- imization principle would prescribe choosing option A over option B mals’ myopic choices have also drawn the attention of the optimal because of its higher reward rate. However, if both outcomes were foraging theory in behavioral ecology (Stephens and Krebs 1986; then shifted in time by a front-end delay of 6 months, the alterna- Bateson and Kachelnik 1996; see also Hayden 2016 for review). tives would now be option A’: €10 in 6 months—which yields a rate Inspired by evolution theory, optimal foraging theory prescribes of €0.05 per day—and option B’: €20 in 1 year—which yields a that a Darwinian-fitness-maximizing organism should maximize rate of €0.11 per day. While, as said before, the DUT in econom- energy intake over time—principle of energy rate maximiza- ics (Samuelson 1937) prescribes that a rational agent should meet tion—when foraging for food (Pyke et al. 1977). However, impul- the stationarity axiom and choose option A’ since she preferred option sive decisions that, as mentioned before, do not meet the efficiency A before (Fishburn and Rubinstein 1982), option B’ yields a higher assumptions of utility maximization and time preference in eco- rate in the new pair, therefore a reward-rate maximizing agent nomics (Kalenscher and Pennartz 2008), also apparently fail to should reverse her preference, and choose B’ over A’. Hence, rate maximize long-term energy rate (Mcdiarmid and Rilling 1965; maximization could only be achieved by a decision rule allowing for Kalenscher et al. 2005; Kalenscher and Pennartz 2008). To recon- time-inconsistent preference reversals. Because rate maximization cile these findings with the assumption in optimal foraging theory models have been developed to account for nonhuman animals’ for- that evolution should have shaped optimal intertemporal decision aging behavior, the logic of our example may be better understood making, Stephens and colleagues (Stephens and Anderson 2001; when replacing financial rewards with food rewards. Consider an Stephens et al. 2004) argued that short-sighted, present-biased deci- animal that chooses between option A: 2 food-items in 2 s (rate: 1 sions can result in energy rate maximization, but only in natural for- item/s) and option B: 4 items in 8 s (rate: 0.5 items/s). The rate aging contexts to which animals’ decision systems are adapted to. maximization principle would prescribe choosing option A because Natural foraging contexts are characterized by sequential back- of its higher energy rate. If both outcomes were then shifted in time ground-foreground problems (Stephens 2008; Rosati and Stevens by 10 s, the alternatives would now yield A’: 2 food-items in 10 + 2 s 2009) in which one alternative is the background to all other alterna- (rate: