Foraging for Foundations in Decision Neuroscience: Insights from Ethology

PerSpeCTIveS

neurophysiological, optogenetic and OPINION neuroimaging paradigms are regularly integrated in decision neuroscience. Foraging for foundations in decision Central features of most work in decision neuroscience are carefully controlled and structured environments neuroscience: insights from ethology that rigorously specify the costs and benefits of behaviours — that is, what Dean Mobbs, Pete C. Trimmer, Daniel T. Blumstein and Peter Dayan needs to be optimized and under what constraints. This has led to many functional Abstract | Modern decision neuroscience offers a powerful and broad account of MRI (fMRI) studies making use of only human behaviour using computational techniques that link psychological and a relatively modest collection of tasks neuroscientific approaches to the ways that individuals can generate near-optimal that reflect aspects of the structural and choices in complex controlled environments. However, until recently , relatively functional implementation of choice. little attention has been paid to the extent to which the structure of experimental However, whether these tasks are environments relates to natural scenarios, and the survival problems that individuals ethologically credible and comprehensive is not always clear. These two questions have evolved to solve. This situation not only risks leaving decision-theoretic are of particular importance given the accounts ungrounded but also makes various aspects of the solutions, such as influential observation in decision theory hard-wired or Pavlovian policies, difficult to interpret in the natural world. Here, we that completely optimal behaviour is suggest importing concepts, paradigms and approaches from the fields of ethology uncommon because of its computational and behavioural ecology , which concentrate on the contextual and functional and mnestic complexity, implying that approximations must be ubiquitous. Such correlates of decisions made about foraging and escape and address these lacunae. approximations are presumably tailored to work well in common and relevant It begins to be difficult, and even in some cases These theories fall into the global environments at the expense of rare ones; impossible, to say where ethology stops and frameworks of Huxley, Mayr and Tinbergen thus, it is vital to consider problems that neurophysiology begins. Tinbergen, 1963 (Box 1), which seek to answer wider are ethologically well founded and thus questions of the proximate (for example, functionally relevant. Ethology, the scientific study of animal physiological) and ultimate (for example, In this Opinion article, we lay out behaviour under natural conditions, and adaptive or Darwinian fitness-increasing) a blueprint for a closer integration of its offspring, behavioural ecology, which repercussions of an animal’s decisions. ethological and behavioural ecological studies how behaviour may vary adaptively By contrast, decision neuroscience has approaches with work on the neural in response to environmental variation, built accounts of the proximate causes of basis of decision-making. The objective have provided valuable insights into animal behaviour that tie optimizing theories such is to build biological realism into decision-making in the natural world. as dynamic programming and Bayesian decision neuroscience with a focus on The paradigms used by ethologists and decision theory to their psychological and translating the extensive work that has behavioural ecologists examine decisions neurobiological substrates. Such optimizing been conducted on non-human animals relating to the exploitation of resources and theories are shared with ethology and to the study of humans. We concentrate optimal foraging, the ways that predators behavioural ecology, along with other on foraging, which is an early harbinger can be avoided and deterred, the reduction fields such as economics, operations of this approach13–17. We consider three of competition and the maximization of research and control theory. However, the constraints on foraging that are of interest reproductive success1. Such decisions are bulk of the work in decision neuroscience for human decision neuroscience: first, often complex because individuals need to focuses on building and understanding energy-based decisions in which metabolic balance energy costs with other survival process models, such as models of needs, including energy consumption needs; a central lesson from behavioural inference (for example, diffusion-to- and fuelling, need to be balanced; ecology is that trade-offs are ubiquitous. In bound decision-making8, realized in second, competitive foraging; and, third, addition, the computations that characterize neural substrates such as the parietal and foraging under the risk of predation, these decisions are described in formal prefrontal cortex9) and learning models when the requirement is to be strategic mathematical accounts — such as optimal (for example, rules based on prediction about averting threats and facilitating escape theory2, foraging theories3 including error, such as the Rescorla–Wagner rule10, the possibility of escape. Along with the marginal value theorem (MVT)4, or temporal-difference learning11, which these examples, we discuss formal tools ideal free distribution (IFD) theory5 and is realized in substrates such as the phasic developed by behavioural ecologists to state-dependent valuation6 — using methods activity of dopamine neurons12). Data from examine various decisions that are made by such as stochastic dynamic programming7. a wealth of anatomical, pharmacological, animals. Taken together, these constraints

and tools provide new insights and help neuroscience is the bandit modelling example, trust games23) and intertemporal integrate human decision-making into approach, which involves binary (or choice24,25. This relatively limited in number the guiding frameworks (Box 1). Indeed, multiple) choice static or restless ‘bandits’ set of paradigms, together with their we note that this form of integration is that deliver rewards and punishments. This offshoots, has provided us with our current already becoming popular in comparative approach is used to measure value-based set of potentially oversimplified tests of how neuroscience13,18,19. and explore-or-exploit decision processes20. the brain computes decisions. Crucially, Other paradigms have been used to measure most realizations of the aforementioned Decision neuroscience other aspects of the decision processes, paradigms leave substantial gaps in our Human decision neuroscience has drawn including working memory (for example, understanding of the ways that decisions upon a set of experimental decision the AX-continuous performance task21), are made in the real world, or indeed of paradigms from the fields of economics sequential information integration and solutions to “The problems that the brain and psychology. These approaches have diffusion-to-bound decision-making (for evolved to solve” (ref.26). been highly successful because they example, motion discrimination using More broadly, ecologically important, carefully control the variables of interest random dot kinematograms9), retrospective fitness-determining decisions relate to and constrain computational models that and prospective planning (for example, the core problems that all animals face. These fit the data. One stalwart of human decision two-step task22), social decision-making (for decisions include rules underlying fighting, foraging, fleeing and reproduction. Such rules are under strong selection pressures Box 1 | The overlapping frameworks of Huxley, Tinbergen, Mayr and Marr and probably underpin all decision it has long been recognized that there are two fundamentally different classes of questions that processes26. Although it is widely understood biologists can ask: why a behaviour or neural structure takes the form it does and how this comes to that the brain contains a set of heuristic happen, with subtle and complex nuances within each class. Many different authors in cognitive mechanisms that evolved to make fast and 18,108 107,110,111 science and ethology have proposed versions of these questions. One of the first to offer accurate decisions with as little effort as guidelines for biobehavioural questioning was Julian Huxley, who proposed that one should attempt possible, there has been less work in decision to address three issues. First, what are the behavioural and neurobiological mechanisms that facilitate ecological decision-making (the mechanistic-physiological question)? second, what is the neuroscience paying due heed to the functional or survival value of such decisions (the adaptive-function question)? Last, what is the selection pressures or the bounds on their evolutionary course of the physiology or behaviour that, for example, supports decision-making use (when one task paradigm transitions to processes (the historic (palaeontological) question)? these three questions were extended and another). Ethologically inspired theorists further refined by tinbergen107, who proposed that biologists should also answer questions of the have begun to consider the evolutionary ontogeny of the behaviour and biology (how does decision-making change over an individual’s roots of human decision-making to lifespan?). Mayr combined these four questions to form two core questions. First, what are the explain how we systematically deviate from proximate causes of the behaviour (short-term factors such as physiological mechanisms or rational-choice models27 and the extent to development during a lifetime)? second, what are the ultimate causes of the behaviour which these biases are uniquely human. (long-term functional trade-offs in terms of natural selection or how phylogeny has developed as a However, such ethological thinking remains series of small changes over ‘evolutionary time’)110? rather rare in human decision neuroscience, Perhaps the most influential framework in decision neuroscience is that of Marr, which focuses particularly on information processing and the different levels at which such processing can be which, in most cases, has yet to embrace viewed. His computational level of investigation has a rough parallel with the adaptive-function fully the synthesis that ethological and question — except with a focus on the information processing aspects of the task and the logic of behavioural ecological approaches offer the (computational) strategy used to solve a problem. Marr then split the mechanistic-physiological between theory, mathematical modelling question into two separate algorithmic and implementational levels of investigation that aim to and natural observation. answer questions concerning how problems are solved. there are, in principle, many algorithms that Some contemporary researchers have can realize a particular computation and many implementations in computational (biophysical) started to exploit ethological insights hardware that can run any particular algorithm. Biologists are typically aiming to understand the when designing and interpreting more particular implementation at hand and therefore lack such freedom. there are various Marrian conventional decision-making problems. 112 approaches to aspects of ontogeny , although this framework does not appear to have been used For instance, binary decision tasks to characterize evolutionary change (see ref.18 for a detailed discussion of the relationship between the theories of Huxley, tinbergen, Mayr and Marr). the figure shows how the questions of Huxley, are being adapted to address foraging tinbergen, Mayr and Marr map onto each other. aHypothesized levels of Marr’s theory as they apply concerns such as the average value of to tinbergen’s theory. the environment or the cost of changing choices14–17. Equally, impulsivity, in the Ontogeny (development) Mechanism (causation) form of a preference for smaller rewards How does decision-making change over What are the behavioural and neurobiological sooner rather than larger ones later, can development? mechanisms that facilitate decision-making? be studied in the light of changing natural Mayr’s proximate How Marr’s levels of Mayr’s proximate Marr’s algorithmic and environments or threats27. In particular, question analysis alter over question implementation levels of developmenta analysis because it may be computationally difficult to make the right decisions, adopting an ethological perspective will tell us the types Phylogeny (evolution) Function (adaptation) of environment to which the necessary How does decision-making change over Why does the animal make the decisions heuristics will have been tailored28. Here, the evolutionary course of the species? it makes? we argue that a more comprehensive Mayr’s ultimate How Marr’s levels of Mayr’s ultimate Marr’s computational synthesis of the two traditions, marrying the question analysis alter over question level of analysis approaches used by decision neuroscientists evolutiona (but see ref.29) with the naturalistic

importance of ethology and behavioural (and how to handle it) and the choice of a the possibility of predation. We discuss ecology, is an important but, as yet, foraging location. Given a choice between aspects of this later in the article. incompletely realized goal. We also offer different food types, an individual should some prospects for the future. choose the one with the most energy The marginal value theorem. Another (if everything else is equal; for example, important issue for rate maximization Foraging see ref.36) and, when the energy content concerns a property of many environments Behavioural ecologists and ethologists have of the food options is equal, they should — that is, resources deplete over time as used value-based decision-making to study choose the option that requires the least they are exploited. The MVT identifies an foraging decisions and the management of energy to acquire it, taking time into optimal foraging strategy in this context4,49. food resources (and of other contributors appropriate account. For example, classic Consider a notional apple-picking task to homeostatic well-being), raising at least work with starlings has shown that they (Fig. 1). When an agent reaches a new apple two sets of unanswered questions in human attempt to maximize the rate of net tree, the number of apples that can be decision neuroscience. One of these sets energy gain for themselves as well as their acquired every minute is high; however, concerns the external environment: the (energetically expensive) begging offspring over time, this depletes as the easy-to-pick, spatiotemporal arrangement of different when hunting for leatherjackets37; it is low-hanging fruits become scarce. The agent sorts of resources; the explorative, energetic also known that gulls and crows modify must repeatedly choose between two options: and other costs required to find and the heights at which they drop crabs and remaining at the same tree and trying to extract them; and the risk of predation walnuts, respectively, to minimize energetic obtain more apples or spending time (and/or while doing so. There have been numerous costs and avoid kleptoparasitism38. energy) travelling to another tree with its studies focusing on how variation in In decision neuroscience, there are a own low-hanging fruits. The MVT calculates the foraging environment and temporal wealth of experiments manipulating the the policy that maximizes the rate of energy variation in predation risk influence amount of work that subjects have to do intake using these two options and includes decisions30,31. The second set of questions to get outcomes, showing, for instance, a an opportunity cost for the passage of time concerns the subjects themselves — notably complex and crucial role for dopamine in to help make the comparison. The MVT state dependence, which arises from the overcoming subjective effort costs39. It has is an important theory because it extends frequent propinquity of catastrophic been suggested that the dorsal anterior classic cost–benefit behavioural economic boundaries whereby animals run out of cingulate cortex (dACC) is involved in the tools and makes testable predictions about reserves and starve to death32. This scenario cost–benefit analysis of control-demanding the effect on this choice of the time needed to creates an inevitable trade-off between situations and that it is engaged in energizing transit between patches and the dynamics of dying from lack of resources and death behaviour40. This proposed function would resource depletion. from predation6,30,31,33. account for the dACC’s entanglement Of course, it is possible to mimic Behavioural neuroscientists such as in decisions associated with energy MVT-relevant conditions in controlled LeDoux34 have also proposed that energy consumption. Indeed, human imaging work studies on humans (for examples, see management exerts a critical influence over has begun to examine the role of physical41 Refs50,51). For instance, in one study, survival-based decision-making. Indeed, and cognitive42 effort in decision-making the authors designed a version of the adaptive behaviour system theory outlines processes. Several groups have shown that apple-picking task mentioned above and how foraging may be triggered by neural anticipation of high-effort investment is administered it to humans16. They analysed systems that monitor energy reserves (for associated with activity in the dACC43,44. leaving decisions by comparing the classic example, fat)35. This theory is proposed to Supporting these claims, comparative MVT with temporal-difference learning result in animals attempting to maximize research has shown that when the dACC models, which focus on the learning of values net energy intake rate (a proposal we qualify region is damaged, it impairs effort-based for different options. The authors performed later) by measuring various key quantities: decisions45,46. In the healthy human brain, two ‘apple-tree-picking’ experiments the average success in acquiring prey, the such impairments in decision-making in which they varied opportunity cost cost of time travel between foraging patches are often mitigated by a set of systems across blocks, thereby making the subject and the time since the last capture of the that increase vigour when the payoff is adjust their behaviour to the changing prey. A related model characterizing the high41, yet persistent effort can result in profitabilities in their environment (Fig. 1). optimal diet also includes consideration of decision impairments47. By comparing the predictions made by the energy or caloric value of the prey, the Together, these ethologically inspired MVT and temporal-difference learning, time that is required to prepare and consume studies are consistent with human lesion they found that the MVT was a better model it and its ease of discovery3. studies showing that damage to the dACC when subjects adjusted their decisions on can result in impairments in motivation, the basis of environmental profitability. Net rate maximization. If we temporarily whereas electrical stimulation of this region This study16 and another investigation15 ignore predation risk and state dependence, elicits perseverance and vigour48. Foraging were among the first to compare models then the obvious approach to understanding tasks show that anticipation and exertion of used in decision neuroscience directly with foraging is provided by the principle of effort are key components of representation, those used in ethology and behavioural rate maximization, which optimizes the valuation and action selection. These ecology, and they show how these models difference between the benefits and the components will allow us to probe the may be fruitfully adopted for use in human costs per unit time. In this context, the dACC and its functional role further; this decision-making studies. elements influencing decisions include can be done by testing for the effects of Given the popularity of MVT in energy, various forms of opportunity how depleting resources, time and effort ethology, it is perhaps surprising that it has costs and time. Rate maximization has influence engagement of this region. A very rarely been used in neurophysiological ramifications for what food type to choose different sort of potential cost comes from experiments. One important exception

choice of when to switch was consistent with MVT-like processes55. Gains drop off as patch Exploration versus exploitation in foraging. is depleted The analysis in the MVT is based on a

gy gain or simplifying assumption about the regularity food intake

Ener of finding a new patch (or tree). In a study on humans, the authors15 designed a task Time Time spent in patch that made the trade-off between exploration travel (search) and exploitation (encounter) more explicit, as in an optimal diet model56. They used simple binary decision-making tasks in which participants were presented with two stimuli that represented the encounter values; above these two stimuli, there were six additional shapes that represented the ‘search value’, drawn from a set of 12 objects that the subjects had learned about on a previous session. Each individual’s goal was to decide whether to engage Fig. 1 | example of classic apple-picking task. Subjects forage in an unlimited orchard and have to with (and ultimately to choose between) decide whether to continue picking from their current tree or to switch to another one to maximize the encounter options or search for an their harvest16. Switching is associated with a time penalty ; however, if the subject decides to stay and alternative. Searching resulted in a cost pick apples from the tree, the rate of return will decrease. Thus, at some point, the subject should (or loss of money); note further that, unlike switch to a new tree. To maximize the long-term harvest rate of apples, the subject repeatedly decides our previous discussion of exploration, between choosing to stay at a tree and switching to a new one. searching did not result in the acquisition of information that could be useful over is a study of the neural basis of MVT in signals work in harmony to regulate patch the long run and instead merely led to a non-human primates (Macaca mulatta)13. leaving. This finding supports a large potentially better alternative. The results In this investigation, monkeys were body of work from the field of behavioural showed that the dACC encoded both the presented with two options, with one ecology. For example, one study showed average value of the foraging environment being a stay option in which individuals both experimentally and theoretically that and the costs of foraging. Furthermore, the were given juice at a specific patch. With myopic ‘short-term’ rules can be favoured authors integrated their results with those time, the amount of juice would reduce, over feeding because they have adaptive from ref.13 and proposed that the dACC is thereby leading to less and less reward. The ‘long-term’ consequences52. also involved in searching for alternatives, other option was to leave, which provided Recent work suggests that the MVT whereas the ventromedial prefrontal no reward and a delay. However, once an foraging framework can be generalized cortex (vmPFC) is involved in well-defined individual switched, the juice would be to intangible resources beyond money ‘economic’ values of options (but see the replenished. Therefore, monkeys had to and other primary-value domains that subsequent debate elewhere57,58). balance the trade-off of the immediate are central to traditional decision-making reward against the long-term benefits of research. Indeed, one study used foraging State dependence. Despite the power of switching to a new patch. models to examine how humans allocate rate maximization in explaining some The results showed that monkeys limited attention53, and in a different study, foraging choices, it does not address made near-optimal decisions in all the MVT framework was applied to rhesus certain important factors. In particular, experimental conditions, leading the macaques’ quests for social information54. when reserves are tight and homeostatic authors to conclude that the context of The latter involved an environment in challenges become critical, the variance foraging reduces biases in time preferences, which social information (pictures of other of outcomes is important3. Subjects can a finding that is inconsistent with classic monkeys) was available for up to a fixed time become more risk-seeking in terms of patch intertemporal choice findings13. While following a choice (in an abstraction of a choice and predation risk30. With a few individuals performed the task, the authors ‘patch’). However, having looked at a picture, important exceptions (for examples, see also recorded neuronal activity in the the subjects had to wait until a waiting time Refs59,60), most behavioural neuroscience dACC. The dACC was chosen because, had elapsed before making another choice decision-making studies have taken the as discussed above, it is a region that is (abstracting the travel cost between patches). view, typically implicitly, that individuals are thought to integrate reward and cost signals Monkeys duly spent more time ‘foraging’ for not constrained by energy or homeostatic with action selection. The results showed social information when the travel cost was challenges. Indeed, it is rare for subjects that as the patch gradually depleted in greater54. Equally, in a study of the recall of even to be able to run out of experimentally juice, neurons in the dACC would increase items from semantic memory by humans, provided money. Given the ubiquity of their firing rates to the point where, at a individuals generated a whole collection energetic constraints in nature, this seems specific threshold, monkeys would switch of answers from one set of overlapping an unfortunate assumption when the brain patches. The dACC neuronal signals subcategories (the informational equivalent may often process the acquisition of money rose more slowly when the travel time of a patch) and then moved to another set as (an exogenous resource) in a similar manner increased, suggesting that multiple control recall of items in the first one ‘dried up’. The to the acquisition of food resources (that is,

primary reinforcers that influence one’s the amount of bread being dropped on refuge to reach food patches, a phenomenon endogenous abilities61,62). Furthermore, either side of the lake. The study showed that referred to as central-place foraging72. Once research shows that social rewards also mallards generally followed the assumptions a squirrel reaches the food patch, it must modulate activity in overlapping rewards of the IFD, but it also found that some make a decision to eat in the patch or to circuits63. Perhaps some of the debates individuals exhibited despotic behaviour use the energy to take the food back to the about neural structures such as the dACC, (dominance), resulting in some mallards safety of its refuge. Going back to the refuge the involvement of which in conventional receiving unequal amounts of food. These makes sense only if there is a cost to staying decision-theoretic tasks is complex and findings suggest that the behaviour of these in the food patch (for example, predation confusing57, might be resolved by adopting birds depends on two types of information: risk). Empirical observations confirm the a more naturalistic stance in which crucial the supply rate in each patch and the predictions of mathematical models as to resources are limiting. competitive value of the other birds69. what happens: the squirrels acquire more In one application of the IFD in decision food if it is close and therefore can be taken Competitive foraging neuroscience, fMRI was used to examine the back to the refuge cheaply. Similar trade-offs When the best choice by (and payoff human brain during the sort of competitive affect the portion sizes they pick. Together, to) one individual is influenced by the foraging that underlies the IFD14. The these behaviours reduce the chance of being actions of another, the appropriate authors created a foraging task in which the eaten by a predator. modelling framework is game theory64,65. available reward and competition varied In nature, the context of such decisions across two patches. The competition was Common currency. To understand, at a may occur when making territorial manipulated by telling the participants functional level, how decisions should decisions to avoid despotic (for example, that they were performing the task live trade off different benefits (or risks), such dominant) individuals, or when studying online with real competitors (represented as the worth of foraging, versus avoiding parasitic behaviours as captured in the as red dots), allowing the experimenters to predation risk or attempting to court a producer–scrounger game model66. One manipulate the number of competitors in mate, a common currency is required73. goal of game theory, particularly when each patch. The subject’s goal was to acquire By comparing the expected future repro it is applied to non-human animals, is to as many tokens as possible by pressing a ductive success of an individual (often predict behaviours that, in conjunction response key as soon as a token appeared, referred to as ‘reproductive value’ (ref.74)) with one another type of behaviour, are before any of their competitors. They could under different strategies, the choice that evolutionarily stable strategies (a particular switch between two habitats to minimize maximizes reproductive value can be type of Nash equilibrium67). competition and compete for tokens in the identified as the normative expectation patch with the highest supply rate. At a small under natural selection. How the value Ideal free distribution. One useful concept cost of time, subjects could switch patches of decisions alters with environmental is captured by the theory of IFD, which as the rewards and competition increased conditions and an individual’s internal proposes that foragers geographically or decreased in each patch. The increased state has been studied extensively from a distribute themselves in relation to the drive to switch patches resulted in increased theoretical perspective29,75. We suggest that proportion of food available and to the activity in several regions, including the these ethologically inspired insights about density of competition5. The IFD assumes dACC, the supplementary motor area and optimal behaviour can help to ground that the forager has perfect information the insula; the authors speculated that all expectations about conditions under which about patch quality and the density of three regions might be involved in the urge particular trade-off paradigms should competition (the ideal aspect of IFD) and to switch habitats. By contrast, staying put come into play under the guiding principle can move freely among patches without in an advantageous habitat was associated that mental processes are likely to produce time delay (the free aspect of IFD). In a very with activity in the reward-related areas — outcomes that approximate the behaviours simple form, this distribution is expressed by namely, the striatum and medial prefrontal that maximize the reproductive value. the input-matching rule in which, if habitat cortex. Moreover, the level of amygdala Consequently, this approach may help to A contains more food and less competition activity steered individual preferences in define and understand the boundaries of than habitat B, habitat A will be the preferred competition avoidance (for example, high particular experimental paradigms and to location. Although despotic distributions amygdala activity was associated with high understand the apparent ‘boundaries’ (or have also been modelled (for example, levels of avoidance). This study suggests framing effects) of different mental processes. dominance and territoriality), theorists that input-matching decisions are made As a very simple example, if an individual is have suggested that the input-matching rule through the medium of a distributed set of close to starvation, then their reproductive is an evolutionarily stable strategy67, and neural circuits. value will approach zero if they do not pioneering work from behavioural ecology soon find food. Therefore, the normative has shown it to be pervasive across various Foraging under predation risk perspective suggests that their mental species from invertebrates to humans5,68. Behavioural ecologists have shown processes should take relatively less account One notable study relating to the IFD the importance of predation risk in of predation risk when their reserves are low. examined foraging behaviours in mallard understanding foraging decisions3,70. ducks (Anas platyrhynchos)69 — specifically, The near-ubiquitous importance of Escape theory. Ydenberg and Dill76 theorized how mallards distributed themselves when predation risk suggests a deficiency in that when a predator approaches, prey make researchers dropped bread into a lake at decision-making paradigms that ignore economic decisions in terms of the cost– fixed locations approximately 20 m apart. it. An early example examined how risk of benefit ratio of remaining compared with This approach mimicked a patch foraging predation affected rate maximization in fleeing. Studies of flight initiation distance task, and the researcher could keep track of grey squirrels (Sciurus carolinensis)71. Like in hundreds of species of birds, mammals, the movement of the mallards while altering other animals, squirrels leave their safety lizards and other taxa have shown that prey

Distance to safety Distance to predator

Refuge (for example, herd) Prey Predator

b cd

d* d*NH d*CH

d*CL Cost of fleeing Cost of fleeing Cost of fleeing d*NL Cost of not fleeing Cost of not fleeing Cost of not fleeing

Distance between prey and predator Distance between prey and predator Distance between prey and predator Fig. 2 | example of foraging and escape choice, and the cost-of-fleeing proposed by Ydenberg and Dill76. As the distance between the prey and the and cost-of-staying curves. a | The solitary prey (for example, a zebra) must predator decreases, the cost of fleeing decreases (red line), whereas the cost make a decision about whether to flee to safety when a predator is of not fleeing increases (blue line). d* represents the optimal distance approaching (for example, a lion). The flight initiation distance (FID) is the between the prey and the predator at which the prey should flee. c | In situ- measurable distance at which the prey makes the decision to flee from an ations in which there are multiple benefits of remaining at a particular patch approaching threat2. The FID is influenced by several factors, including the but a single cost of fleeing, d* is longer for the higher of the two degree of threat posed by the predator (for example, whether it is a cost-of-not-fleeing curves. d | Conversely , d* is longer for the lower of the fast-moving or slow-moving predator) and the value of its current location two cost-of-fleeing curves when there is a single cost-of-not-fleeing curve2.

(for example, there is an abundance of food or mates). b | The schematic d*CH, high cost; d*CL, low cost; d*NH, not fleeing, high cost; d*NL, not fleeing, represents the cost of fleeing versus remaining and is based on the model low cost. Adapted with permission from ref.2,Cambridge University Press. are remarkably adept at escape, making the vmPFC81, and that the hippocampus and midcingulate cortex (MCC) were decisions on the basis of prior experience is activated when individuals engage in activated. Conversely, the close-attacking with the predator and the predator’s threat engagement, a finding supported by virtual predators with larger buffer zones relative position and bearing, lethality and investigations of patients with hippocampal82 elicited the activation of brain regions that velocity2,77. This well-understood natural or amygdala lesions83 as well as animal and are presumed to be involved in cognitive paradigm offers substantial opportunities for theoretical models79,84–87. Together, these fear and strategic planning, including decision neuroscience (Fig. 2). findings support a network of survival the vmPFC, hippocampus, posterior circuits that are involved in ecologically cingulate cortex and retrosplenial cortex. Flight initiation distance and the neural defined threats. The authors then investigated how close basis of escape. In humans, fMRI has More recently, a flight initiation each subject was to adopting the optimal been employed during ‘active escape’ distance task was developed to investigate escape strategy by applying a Bayesian paradigms in which the goal was to escape how survival circuits facilitate escape decision-theory model. The model from a virtual predator that possessed the decisions when subjects encounter revealed the MCC as a candidate region capacity to chase, capture and shock the fast-attacking or slow-attacking for optimal escape from fast-attacking participant. In this experiment, the vmPFC threats88. Fast-attacking predators were predators, consistent with the known role was preferentially active when the virtual characterized by the virtual predator for this area in adaptive motor learning89. predator was distant, but this activity quickly switching from a slow-approach Hippocampal activity was associated with switched to the midbrain periaqueductal to a fast-attack velocity, thereby requiring optimal escape from slower-attacking grey (PAG) as the threat moved closer78. the subject to make quick escape threats, supporting a role for this region This finding was replicated79 and extended decisions. By contrast, slow-attacking in escape only when the subject is not by showing that panic-related motor errors predators slowly approached for longer under time pressure. This study provides correlated with activity in the midbrain time periods, resulting in larger buffer one example of how models from the field encompassing the PAG and the dorsal raphe zones and more time to strategize escape. of ethology and behavioural ecology can nucleus80. Other researchers have shown that When subjects were faced with quick be applied to decision neuroscience and evading virtual predators is also associated decisions to flee virtual predators with far demonstrates a previously unknown link with greater connectivity between the attack distances, regions associated with between defensive survival circuits and amygdala, the anterior cingulate cortex and reactive fear including the midbrain PAG their role in adaptive escape decisions.

Future directions with changing proximity to threat allow be analysed and understood at several The development of a new decision researchers to examine the dynamic different levels” (ref.108). Although decision neuroscience framework that is based on relationship between anxiety and fear neuroscience largely abides by Marr and ethology and behavioural ecology provides circuits in individuals with affective Poggio’s framework, we suggest that it is a new set of paradigms that can be adapted disorders. Studies are also possible into in some danger of isolating itself by, as to understand (human) decision-making. how decisions associated with effort Tinbergen put it, “Losing touch with the Such a framework links the guiding are impaired in mood disorders such as natural phenomena” (ref.107). We have frameworks of evolutionary biology (Box 1) depression, in which lethargy and fatigue attempted to address this problem by with the formal tools used by ethologists are rampant99. discussing three examples from the foraging and behavioural ecologists (Fig. 2). Moreover, Finally, the approach we advocate is literature that are central to studying it specifies experimental protocols and not a one-way street: it can be reciprocally decision-making in the natural environment mathematical models that characterize illuminating. Ethological thinking by and that bring together disparate fields to the mechanistic and dynamic signals that decision neuroscientists will in turn have generate a multilevel approach to human support moment-to-moment decisions and an impact on ethologists and behavioural decision-making. highlights the role of state dependence in ecologists. The key issues of ethology are A small, but growing, band of researchers decision-making. Furthermore, it provides often based around ‘why’ questions; as are adopting new approaches to task design information about the environments in Stephens and Krebs point out, “Asking and analysis that are inspired by ethology which priors and heuristics, which are what a machine is for helps the engineer and behavioural ecology. These approaches necessary to cope with the complexity of understand how it works” (ref.31). Conversely, have given rise to new information natural decision-making tasks, evolved understanding relevant brain mechanisms about how the brain computes decisions and developed. might help ethologists understand what under various conditions. A more critical Neuroscience has made considerable specific condition the mechanisms outcome of these approaches is that they progress in identifying the function evolved for and will certainly help identify move the laboratory closer to nature109, of particular brain regions in relation important constraints on information thereby allowing better understanding to particular tasks and, in some cases, processing. There is evidence that decision and prediction of how we make decisions generalization across tasks (for example, science is having an impact on ethology, in the real world. Crucially, the functional reward functions of the basal ganglia90). including the use of classic paradigms approach of ethology (and behavioural Arguably, the next challenge is to understand from behavioural economics such as risk ecology) identifies fitness trade-offs that the effect of trade-offs across tasks — how aversion, delayed discounting and framing help us to understand how one set of a particular mechanism is adapted for use effect paradigms100–102. The question of the approximately optimal rules (such as rate on multiple tasks (thus being suboptimal on calculational complexity of decision-making, maximization when foraging) can, under any one) and the more immediately tractable which informs much thinking about different circumstances, conflict with question of how coordination and control process models and heuristics28,103 in rules that are approximately optimal in take place among different functions and decision neuroscience, could offer relevant another task (such as predator avoidance). the brain regions that support them (for constraints for ethological paradigms. By understanding such trade-offs, we example, in the case of self-control91–93). Following on from Tinbergen’s work, which may better understand how to design Behavioural ecology may be particularly identified different classes of biological neuroscientific studies to identify the helpful in identifying relevant trade-off questions (Box 1), McNamara and Houston29 (bounds on) conditional use of particular situations. have pushed for us to understand how neural structures. Adopting this framework This more biologically relevant function and mechanism have co-evolved; will thus enhance our understanding framework, along with evolutionary such ‘evo-mecho’ approaches are necessary of natural diversity and likely improve approaches to medicine94, also sits well for a comprehensive understanding of applications for human health and with the Research Domain Criteria (RDoC) decision-making because it is not tenable well-being. framework advocated by the US National to consider the evolution of just function Dean Mobbs1*, Pete C. Trimmer2, Daniel T. Blumstein3 Institute of Mental Health, which aims to or mechanism and to pretend that the and Peter Dayan 4 integrate disparate levels of information and other remained fixed. Gaining a deeper 1Department of Humanities and Social Sciences and provide a more foundational understanding understanding of cognitive processes Computation and Neural Systems Program at the of the neural and psychological dimensions can help enable this integration and, California Institute of Technology, Pasadena, CA, USA. that underpin healthy and abnormal brain in turn, benefit the fields of ‘cognitive 2Department of Environmental Science and Policy, function95. Homologous studies in humans ecology’ (ref.104) and the study of social University of California, Davis, Davis, CA, USA. can create dynamic virtual ecologies that behaviour105,106. 3Department of Ecology and Evolutionary Biology, University of California, Los Angeles, Los Angeles, provide new dimensions to investigate CA, USA. adaptive and affective decision-making in Concluding remarks 4Gatsby Computational Neuroscience Unit, University 107 healthy humans and across a broad range of Like Mayr and Huxley, Tinbergen College London, London, UK. psychiatric disorders. For example, a recent suggested that we must take a multilevel *e-mail: [email protected] study showed that subjects who gambled approach to scientific questions in biology, https://doi.org/10.1038/s41583-018-0010-7 more showed increased exploration on an writing, “If we do not continue to give Published online xx xx xxxx explore–exploit foraging task and earned thought to the problem of our overall 96,97 1. Krebs, J. R. & Davies, N. B. An Introduction To less money on a patchy foraging task . aims, our field [ethology] will be in danger Behavioural Ecology (Blackwell Scientific Publications, Moreover, other researchers have linked of becoming an isolated ‘ism’”. Likewise, 1993). 98 2. Cooper, W. E. Jr & Blumstein, D. T. Escaping From foraging behaviour to depression . Marr and Poggio opined that “Complex Predators: An integrative View Of Escape Decisions Shifts in neural activity that are associated systems, like a nervous system … must (Cambridge University Press, 2015).

3. Ydenberg, R. C., Brown, J. S. & Stephens, D. W. 33. Houston, A. I. & McNamara, J. M. Models Of 59. Wright, N. D. et al. Human responses to unfairness Foraging: Behavior And Ecology (The University of Adaptive Behaviour, An Approach Based On State with primary rewards and their biological limits. Sci. Chicago Press, 2007). (Cambridge University Press, 1999). Rep. 2, 593 (2012). 4. Charnov, E. L. Optimal foraging, the marginal value 34. LeDoux, J. Rethinking the emotional brain. Neuron 60. Williams, E. F., Pizarro, D., Ariely, D. & Weinberg, J. D. theorem. Theor. Popul. Biol. 9, 129–136 (1976). 73, 653–676 (2012). The Valjean effect: visceral states and cheating. 5. Fretwell, S. D. & Lucas, H. L. Jr On territorial behavior 35. White, D. W., Dill, L. M. & Crawford, C. B. A. Common, Emotion 16, 897–902 (2016). and other factors influencing habitat distribution in conceptual framework for behavioral ecology and 61. Chib, V. S., Rangel, A., Shimojo, S., & O’Doherty, J. P. birds. Acta Biotheor. 19, 16–36 (1970). evolutionary psychology. Evol. Psychol. 5, 275–288 Evidence for a common representation of decision 6. Pompilio, L., Kacelnik, A. & Behmer, S. T. (2007). values for dissimilar goods in human ventromedial State-dependent learned valuation drives choice in an 36. Elner, R. W. & Hughes, R. N. Energy maximization in prefrontal cortex. J. Neurosci. 39, 12315–12320 invertebrate. Science 311, 1613–1615 (2006). the diet of the shore crab, carcinus maenas. J. Anim. (2009). 7. Krebs, J. R., Kacelnik, A. & Taylor, P. Test of optimal Ecol. 47, 103 (1978). 62. D’Ardenne, K., McClure, S. M., Nystrom, L. E. & sampling by foraging great tits. Nature 275, 27–31 37. Kacelnik, A. Central place foraging in starlings Cohen, J. D. BOLD responses reflecting dopaminergic (1978). (Sturnus vulgaris). I. Patch residence time. J. Anim. signals in the human ventral tegmental area. Science 8. Ratcliff, R., Smith, P. L., Brown, S. D. & McKoon, G. Ecol. 53, 283 (1984). 1264–1267 (2008). Diffusion decision model: current issues and history. 38. Switzer, P. V. & Cristol, D. A. Avian prey-dropping 63. Izuma, K., Saito, D. N. & Sadato, N. Processing of Trends Cogn. Sci. 20, 260–281 (2016). behavior I: effects of prey characteristics and prey loss. social and monetary rewards in the human striatum. 9. Gold, J. I. & Shadlen, M. N. The neural basis of Behav. Ecol. 10, 213–219 (1999). Neuron 58, 284–294 (2008). decision making. Annu. Rev. Neurosci. 30, 535–574 39. Walton, M. E., Croxson, P. L., Rushworth, M. F. S. & 64. Nash, J. F. Non-cooperative games. Ann. Math. 54, (2007). Bannerman, D. M. The mesocortical dopamine 286–295 (1951). 10. Rescorla, R. A. & Wagner, A. R. in Classical projection to anterior cingulate cortex plays no role in 65. Von Neumann, J. & Morgenstern, O. Theory Of Games Conditioning II: Current Theory and Research guiding effort-related decisions. Behav. Neurosci. 119, And Economic Behavior (Princeton University Press, (eds Black, A. H. & Prokasy, W. F.) 64–99 323–328 (2005). 2007). (Appleton-Century-Crofts, 1972). 40. Kennerley, S. W., Behrens, T. E. J. & Wallis, J. D. 66. Barnard, C. J. & Sibly, R. M. Producers and 11. Sutton, R. S. & Barto, A. Reinforcement Learning: Double dissociation of value computations in scroungers: a general model and its application to An Introduction (MIT Press, 1998). orbitofrontal and anterior cingulate neurons. captive flocks of house sparrows. Anim. Behav. 29, 12. Montague, P. R., Dayan, P. & Sejnowski, T. J. A Nat. Neurosci. 14, 1581–1589 (2011). 543–550 (1981). framework for mesencephalic dopamine systems 41. Pessiglione, M. et al. How the brain translates money 67. Maynard Smith, J. Evolution And The Theory of Games based on predictive Hebbian learning. J. Neurosci. into force: a neuroimaging study of subliminal (Cambridge University Press, 1982). 16, 1936–1947 (1996). motivation. Science 316, 904–906 (2007). 68. Kennedy, M. & Gray, R. D. Can ecological theory 13. Hayden, B. Y., Pearson, J. M. & Platt, M. L. Neuronal 42. Kool, W. & Botvinick, M. A labor/leisure tradeoff in predict the distribution of foraging animals? A critical basis of sequential foraging decisions in a patchy cognitive control. J. Exp. Psychol. Gen. 143, 131–141 analysis of experiments on the ideal free distribution. environment. Nat. Neurosci. 14, 933–939 (2011). (2014). Oikos 68, 158 (1993). 14. Mobbs, D. et al. Foraging under competition: the 43. Croxson, P. L., Walton, M. E., O’Reilly, J. X., Behrens, 69. Harper, D. G. C. Competitive foraging in mallards: neural basis of input matching in humans. J. Neurosci. T. E. J. & Rushworth, M. F. S. Effort-based cost-benefit ‘ideal free’ ducks. Anim. Behav. 30, 575–584 (1982). 33, 9866–9872 (2013). valuation and the human brain. J. Neurosci. 29, 70. Lima, S. L. & Dill, L. M. Behavioral decisions made 15. Kolling, N., Behrens, T. E. J., Mars, R. B. & Rushworth, 4531–4541 (2009). under the risk of predation: a review and prospectus. M. F. S. Neural mechanisms of foraging. Science 336, 44. Kurniawan, I. T., Guitart-Masip, M., Dayan, P. & Dolan, Can. J. Zool. 68, 619–640 (1990). 95–98 (2012). R. J. Effort and valuation in the brain: the effects of 71. Abrahams, M. V., Dill, L. M., Abrahams, M. V. & 16. Constantino, S. M. & Daw, N. D. Learning the anticipation and execution. J. Neurosci. 33, Dill, L. M. A determination of the energetic opportunity cost of time in a patch-foraging task. 6160–6169 (2013). equivalence of the risk of predation. Ecology 70, Cogn. Affect. Behav. Neurosci. 15, 837–853 (2015). 45. Walton, M. E., Bannerman, D. M., Alterescu, K. & 999–1007 (1989). 17. Shenhav, A., Straccia, M. A., Cohen, J. D. & Botvinick, Rushworth, M. F. S. Functional specialization within 72. Lima, S. L. & Valone, T. J. Influence of predation risk M. M. Anterior cingulate engagement in a foraging medial frontal cortex of the anterior cingulate for on diet selection: a simple example in the grey context reflects choice difficulty, not foraging value. evaluating effort-related decisions. J. Neurosci. 23, squirrel. Anim. Behav. 34, 536–544 (1986). Nat. Neurosci. 17, 1249–1254 (2014). 6475–6479 (2003). 73. McNamara, J. M. & Houston, A. I. A common currency 18. Brase, G. L. Behavioral science integration: a practical 46. Rudebeck, P. H., Walton, M. E., Smyth, A. N., for behavioral decisions. Am. Nat. 127, 358–378 framework of multi-level converging evidence for Bannerman, D. M. & Rushworth, M. F. S. Separate (1986). behavioral science theories. New Ideas Psychol. 33, neural pathways process different decision costs. 74. Fisher, R. A. The Genetical Theory Of Natural 8–20. Nat. Neurosci. 9, 1161–1168 (2006). Selection (Dover, 1958). 19. Anderson, D. & Perona, P. Toward a science of 47. Blain, B., Hollard, G. & Pessiglione, M. Neural 75. Mangel, M. & Clark, C. W. Dynamic Modeling In computational ethology. Neuron 84, 18–31 (2014). mechanisms underlying the impact of daylong Behavioural Ecology (Princeton University Press, 20. Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B. & cognitive work on economic decisions. Proc. Natl 1988). Dolan, R. J. Cortical substrates for exploratory Acad. Sci. USA 113, 6967–6972 (2016). 76. Ydenberg, R. C. & Dill, L. M. Advances In The Study Of decisions in humans. Nature 441, 876–879 (2006). 48. Parvizi, J., Rangarajan, V., Shirer, W. R., Desai, N. & Behavior (eds Rosenblatt, J. S., Beer, C., Busnelz, M. 21. Barch, D. M. et al. Dissociating working memory from Greicius, M. D. The will to persevere induced by C. & Slater, P. J. B.) 229–249 (Elsevier, 1986). task difficulty in human prefrontal cortex. electrical stimulation of the human cingulate gyrus. 77. Stankowich, T. & Blumstein, D. T. Fear in animals: a Neuropsychologia 35, 1373–1380 (1997). Neuron 80, 1359–1367 (2013). meta-analysis and review of risk assessment. Proc. 22. Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P. & 49. Kolling, N. & Akam, T. (Reinforcement?) Learning to Biol. Sci. 272, 2627–2634 (2005). Dolan, R. J. Model-based influences on humans’ forage optimally. Curr. Opin. Neurobiol. 46, 162–169 78. Mobbs, D. et al. When fear is near: threat imminence choices and striatal prediction errors. Neuron 69, (2017). elicits prefrontal - periaqueductal grey shifts in 1204–1215 (2011). 50. Hutchinson, J. M. C., Wilke, A. & Todd, P. M. Patch humans. Science 317, 1079–1083 (2007). 23. King-Casas, B. et al. Getting to know you: reputation leaving in humans: can a generalist adapt its rules to 79. Meyer, C. T., Padmala, S. & Pessoa, L. Tracking and trust in a two-person economic exchange. Science dispersal of items across patches? Anim. Behav. 75, dynamic threat imminence. bioRxiv https://doi. 308, 78–83 (2005). 1331–1349 (2008). org/10.1101/183798 (2017). 24. Frederick, S., Loewenstein, G. & O’donoghue, T. Time 51. Wilke, A., Hutchinson, J. M. C., Todd, P. M. & 80. Mobbs, D. et al. From threat to fear: the neural discounting and time preference: a critical review. Czienskowski, U. Fishing for the right words: decision organization of defensive fear systems in humans. J. Econ. Lit. 40, 351–401 (2002). rules for human foraging behavior in internal search J. Neurosci. 29, 12236–12243 (2009). 25. Ainslie, G. & Haslam, N. in Choice Over Time tasks. Cogn. Sci. 33, 497–529 (2009). 81. Gold, A. L., Morey, R. A. & McCarthy, G. (eds Loewenstein, G. & Elster, J.) 57–92 (Russell Sage 52. Stephens, D. W. & Anderson, D. The adaptive value of Amygdala-prefrontal cortex functional connectivity Foundation, 1992). preference for immediacy: when shortsighted rules during threat-induced anxiety and goal distraction. 26. Pearson, J. M., Watson, K. K. & Platt, M. L. Decision have farsighted consequences. Behav. Ecol. 12, Biol. Psychiatry 77, 394–403 (2015). making: the neuroethological turn. Neuron 82, 330–339 (2000). 82. Bach, D. R. et al. Human hippocampus arbitrates 950–965 (2014). 53. Pirolli, P. & Card, S. Information foraging. Psychol. approach-avoidance conflict. Curr. Biol. 24, 1435 27. Santos, L. R. & Rosati, A. G. The evolutionary roots of Rev. 106, 643–675 (1999). (2014). human decision making. Annu. Rev. Psychol. 66, 54. Turrin, C., Fagan, N. A., Dal Monte, O. & Chang, S. W. C. 83. Korn, C. W. et al. Amygdala lesions reduce anxiety-like 321–347 (2015). Social resource foraging is guided by the principles of behavior in a human benzodiazepine-sensitive 28. Volz, K. G. & Gigerenzer, G. Cognitive processes in the marginal value theorem. Sci. Rep. 7, 11274 approach-avoidance conflict test. Biol. Psychiatry 82, decisions under risk are not the same as in decisions (2017). 522–531 (2017). under uncertainty. Front. Neurosci. 6, 105 (2012). 55. Hills, T. T., Jones, M. N. & Todd, P. M. Optimal foraging 84. McNaughton, N. & Corr, P. J. A two-dimensional 29. McNamara, J. M. & Houston, A. I. Integrating function in semantic memory. Psychol. Rev. 119, 431–440 neuropsychology of defense: fear/anxiety and and mechanism. Trends Ecol. Evol. 24, 670–675 (2012). defensive distance. Neurosci. Biobehav. Rev. 28, (2009). 56. Krebs, J. R., Erichsen, J. T., Webber, M. I. & 285–305 (2004). 30. Lima, S. L. & Bednekoff, P. A. Temporal variation in Charnov, E. L. Optimal prey selection in the great 85. Yilmaz, M. & Meister, M. Rapid innate defensive danger drives antipredator behavior: the predation tit (Parus major). Anim. Behav. 25, 30–38 responses of mice to looming visual stimuli. Curr. Biol. risk allocation hypothesis. Am. Nat. 153, 649–659 (1977). 23, 2011–2015 (2013). (1999). 57. Shenhav, A., Cohen, J. D. & Botvinick, M. M. Dorsal 86. Vale, R., Evans, D. A. & Branco, T. Rapid spatial 31. Stephens, D. W. & Krebs, J. R. Foraging Theory anterior cingulate cortex and the value of control. learning controls instinctive defensive behavior in (Princeton University Press, 1986). Nat. Neurosci. 19, 1286–1291 (2016). mice. Curr. Biol. 27, 1342–1349 (2017). 32. Caraco, T., Martindale, S. & Whittam, T. S. An 58. Kolling, N. et al. Value, search, persistence and model 87. Blanchard, D. C. Translating dynamic defense patterns empirical demonstration of risk-sensitive foraging updating in anterior cingulate cortex. Nat. Neurosci. from rodents to people. Neurosci. Biobehav. Rev. 76, preferences. Anim. Behav. 28, 820–830 (1980). 19, 1280–1285 (2016). 22–28 (2017).

88. Qi, S. et al. How cognitive and reactive fear circuits implications for understanding depression. Evol. Med. 109. Camerer, C. & Mobbs, D. Comparing cognitive optimize escape decisions in humans. Proc. Natl Acad. Public Health 2015, 123–135 (2015). and neural processes during hypothetical and Sci. USA 115, 3186–3191 (2018). 99. Huys, Q. J. M., Daw, N. D. & Dayan, P. Depression: a real choices. Trends Cogn. Sci. 21, 46–56 89. Shackman, A. J. et al. The integration of negative decision-theoretic analysis. Annu. Rev. Neurosci. 38, (2017). affect, pain and cognitive control in the cingulate 1–23 (2015). 110. Mayr, E. Cause and effect in biology: kinds of causes, cortex. Nat. Rev. Neurosci. 12, 154–167 (2011). 100. Marsh, B. & Kacelnik, A. Framing effects and risky predictability, and teleology are viewed by a practicing 90. Mikhael, J. G. & Bogacz, R. Learning reward decisions in starlings. Proc. Natl Acad. Sci. USA 99, biologist. Science 134, 1501–1506 (1961). uncertainty in the basal ganglia. PLoS Comput. Biol. 3352–3355 (2002). 111. Huxley, J. Evolution: The Modern Synthesis (MIT 12, e1005062 (2016). 101. Monteiro, T., Vasconcelos, M. & Kacelnik, A. Starlings Press, 2010). 91. Botvinick, M. & Braver, T. Motivation and cognitive uphold principles of economic rationality for delay and 112. Price, D. J. & Willshaw, D. J. Mechanisms Of Cortical control: from behavior to neural mechanism. Annu. probability of reward. Proc. Biol. Sci. 280, 20122386 Development (Oxford Univ. Press, 2000). Rev. Psychol. 66, 83–113 (2015). (2013). 92. Boureau, Y.-L., Sokol-Hessner, P. & Daw, N. D. 102. Kalenscher, T. & Pennartz, C. M. A. Is a bird in the Acknowledgements Deciding how to decide: self-control and meta-decision hand worth two in the future? The neuroeconomics of The authors thank T. Akam and the reviewers for their very making. Trends Cogn. Sci. 19, 700–710 (2015). intertemporal decision-making. Prog. Neurobiol. 84, stimulating comments. This work was supported by US 93. Reynolds, J. J. & McCrea, S. M. Environmental 284–315 (2008). National Institute of Mental Health grant 2P50MH094258 constraints on the functionality of inhibitory 103. Kahneman, D. & Tversky, A. Prospect theory: an and a Chen Institute Award (P2026052) (support to D.M.), self-control: sometimes you should eat the donut. Self analysis of decision under risk. Econometrica 47, 263 the Gatsby Charitable Foundation (support to P.D.) and the and Identity https://doi.org/10.1080/15298868.201 (1979). US National Science Foundation (support to D.T.B. and P.C.T.; 7.1354066 (2017). 104. Stamp Dawkins, M. Cognitive ecology: the P.C.T. was supported by an Integrative Organismal System 94. Nesse, R. M. An evolutionary perspective on evolutionary ecology of information processing and grant (1456724) to A. Sih). The content is solely the respon- psychiatry. Compr. Psychiatry 25, 575–580 (1984). decision making. Trends Cogn. Sci. 2, 304 (1998). sibility of the authors and does not necessarily represent the 95. Perusini, J. N. & Fanselow, M. S. Neurobehavioral 105. Blumstein, D. T. et al. Toward an integrative official views of the authors’ funders. perspectives on the distinction between fear and understanding of social behavior: new models and anxiety. Learn. Mem. 22, 417–425 (2015). new opportunities. Front. Behav. Neurosci. 4, 34 Author contributions 96. Addicott, M. A., Pearson, J. M., Sweitzer, M. M., (2010). D.M., P.C.T., D.T.B. and P.D. researched data for the article, Barack, D. L. & Platt, M. L. A. Primer on foraging and 106. Taborsky, M. et al. Taxon matters: promoting made substantial contributions to discussions of the content, the explore/exploit trade-off for psychiatry research. integrative studies of social behavior: NESCent wrote the article and reviewed and/or edited the manuscript Neuropsychopharmacology 42, 1931–1939 (2017). working group on integrative models of vertebrate before submission. 97. Addicott, M. A., Pearson, J. M., Kaiser, N., Platt, M. L. sociality: evolution, mechanisms, and emergent & McClernon, F. J. Suboptimal foraging behavior: a properties. Trends Neurosci. 38, 189–191 (2015). Competing interests new perspective on gambling. Behav. Neurosci. 129, 107. Tinbergen, N. On aims and methods in ethology. Z. The authors declare no competing interests. 656–665 (2015). Tierpsychol. 20, 410–433 (1963). 98. Trimmer, P. C., Higginson, A. D., Fawcett, T. W., 108. Marr, D. & Poggio, T. A computational theory of Publisher’s note McNamara, J. M. & Houston, A. I. Adaptive learning human stereo vision. Proc R Soc B Biol Sci 204, Springer Nature remains neutral with regard to jurisdictional can result in a failure to profit from good conditions: 301–328 (1979). claims in published maps and institutional affiliations.