BEHAVIORAL AND BRAIN SCIENCES (2008) 31, 415–487 Printed in the United States of America doi:10.1017/S0140525X0800472X A unified framework for addiction: Vulnerabilities in the decision process A. David Redish Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455 [email protected] http://umn.edu/~redish/ Steve Jensen Graduate Program in Computer Science, University of Minnesota, Minneapolis, MN 55455 [email protected] Adam Johnson Graduate Program in Neuroscience and Center for Cognitive Sciences, University of Minnesota, Minneapolis, MN 55455 [email protected] Abstract: The understanding of decision-making systems has come together in recent years to form a unified theory of decision-making in the mammalian brain as arising from multiple, interacting systems (a planning system, a habit system, and a situation-recognition system). This unified decision-making system has multiple potential access points through which it can be driven to make maladaptive choices, particularly choices that entail seeking of certain drugs or behaviors. We identify 10 key vulnerabilities in the system: (1) moving away from homeostasis, (2) changing allostatic set points, (3) euphorigenic “reward-like” signals, (4) overvaluation in the planning system, (5) incorrect search of situation-action-outcome relationships, (6) misclassification of situations, (7) overvaluation in the habit system, (8) a mismatch in the balance of the two decision systems, (9) over-fast discounting processes, and (10) changed learning rates. These vulnerabilities provide a taxonomy of potential problems with decision-making systems. Although each vulnerability can drive an agent to return to the addictive choice, each vulnerability also implies a characteristic symptomology. Different drugs, different behaviors, and different individuals are likely to access different vulnerabilities. This has implications for an individual’s susceptibility to addiction and the transition to addiction, for the potential for relapse, and for the potential for treatment. Keywords: Addiction; decision making; dopamine; frontal cortex; gambling; hippocampus; striatum 1. Introduction Dickerson & O’Connor 2006; Dowling et al. 2005; Parke & Griffiths 2004; Redish et al. 2007; Wagenaar 1988). Addiction can be operationally defined as the continued However, how those interactions drive maladaptive making of maladaptive choices, even in the face of the decision-making remains a key, unanswered question. explicitly stated desire to make a different choice (see Over the last 30 years, a number of theories have been the Diagnostic and Statistical Manual of Mental Disorders proposed attempting to explain why an agent might con- [DSM-IV-TR], American Psychiatric Association 2000; tinue to seek a drug or maladaptive behavior. These the- International Classification of Diseases [ICD-10], World ories can be grouped into the following primary Health Organization 1992). In particular, addicts continue categories: (1) opponent processes, based on changes in to pursue drugs or other maladaptive behaviors despite homeostatic and allostatic levels that change the needs terrible consequences (Altman et al. 1996; Goldstein of the agent (Becker & Murphy 1988; Koob & Le Moal 2000; Koob & Le Moal 2006; Lowinson et al. 1997). Addic- 1997; 2001; 2005; 2006; Solomon & Corbit 1973; 1974); tive drugs have been hypothesized to drive maladaptive (2) reward-based processes and hedonic components, decision-making through pharmacological interactions based on pharmacological access to hedonically positive with neurophysiological mechanisms evolved for normal signals in the brain (Kalivas & Volkow 2005; Volkow learning systems (Berke 2003; Everitt et al. 2001; et al. 2003; 2004; Wise 2004); (3) incentive salience, Hyman 2005; Kelley 2004a; Lowinson et al. 1997; based on a sensitization of motivational signals in the Redish 2004). Addictive behaviors have been hypoth- brain (Berridge & Robinson 1998; 2003; Robinson & Ber- esized to drive maladaptive decision-making through ridge 1993; 2001; 2003; 2004); (4) non-compensable dopa- interactions between normal learning systems and the mine, based on a role of dopamine as signaling an error in reward distribution of certain behaviors (Custer 1984; the prediction of the value of taking an action, leading to # 2008 Cambridge University Press 0140-525X/08 $40.00 415 Redish et al.: A unified framework for addiction an overvaluation of drug-seeking (Bernheim & Rangel decision-making arising from multiple interacting systems 2004; Di Chiara 1999; Redish 2004); (5) impulsivity,in (Cohen & Squire 1980; Daw et al. 2005; Dickinson 1980; which users make rash choices, without taking into 1985; Nadel 1994; O’Keefe & Nadel 1978; Packard & account later costs (Ainslie 1992; 2001; Ainslie & Monter- McGaugh 1996; Redish 1999; Squire 1987). Briefly, a osso 2004; Bickel & Marsch 2001; Giordano et al. 2002; decision can arise from a flexible planning system capable Odum et al. 2002); (6) situation recognition and categoriz- of the consideration of consequences or from a less flexible ation, based on a misclassification of situations that habit system in which actions are associated with situations produce both gains and losses (Custer 1984; Griffiths (Daw et al. 2005; Redish & Johnson 2007). Behavioral 1994; Langer & Roth 1975; Redish et al. 2007; Wagenaar control can be transferred from one system to the other 1988); and (7) deficiencies in the balance between executive depending on the statistics of behavioral training (Balleine and habit systems, in which it becomes particularly & Dickinson 1998; Colwill & Rescorla 1990; Killcross & difficult to break habits through cognitive mechanisms Coutureau 2003; Packard & McGaugh 1996). Both either through over-performance of the habit system systems also require a recognition of the situation in (Robbins & Everitt 1999; Tiffany 1990) or under-perform- which the agent finds itself (Daw et al. 2006; Redish et al. ance of flexible, executive, inhibitory systems (Gray & 2007; Redish & Johnson 2007). These processes provide McNaughton 2000; Jentsch & Taylor 1999; Lubman multiple access points and vulnerabilities through which et al. 2004) or a change in the balance between them the decision process can be driven to make maladaptive (Bechara 2005; Bickel et al. 2007; Everitt et al. 2001; choices. Everitt & Wolf 2002). (See Table 1.) Although each of these theories has been attacked as incomplete and unable to explain all of the addiction 2. Scope of the work data, the theories are not incompatible with each other. We argue, instead, that each theory explains a different Addiction is a complex phenomenon, with causes that can vulnerability in the decision-process system, capable of be identified from many perspectives (Volkow & Li 2005a; driving the agent to make an addictive choice. Thus, the West 2001), including social (Davis & Tunks 1991), set of theories provides a constellation of potential environmental (DeFeudis 1978; Dickerson & O’Connor causes for addictive choice behavior. Each different drug 2006; Maddahian et al. 1986; Morgan et al. 2002), legal of abuse or maladaptive behavior is likely to access a (Dickerson & O’Connor 2006; Kleber et al., 1997; subset of that constellation of potential dysfunction. Indi- MacCoun 1993), as well as psychological and neurobiolo- vidual differences are likely to define the importance of gical (Goldman et al. 1987; 1999; Heyman 1996; 2000; each vulnerability for an individual’s dysfunction. Success- Koob & Le Moal 2006; Redish 2004; Robinson 2004; ful treatment depends on treating those vulnerabilities Robinson & Berridge 2003; Tiffany 1990), economic that are driving the individual’s choice. The identification (Ainslie 1992; 2001; Becker & Murphy 1988; Bernheim of addiction as vulnerabilities in the biological decision- & Rangel 2004; Hursh 1991; Hursh et al. 2005), and making system means that understanding addiction will genetic (Crabbe 2002; Goldman et al. 2005; Hiroi & Agat- require an understanding of how animals (including suma 2005) perspectives. All of these perspectives have humans) make decisions. explanatory power as to the causes of addiction, and all The understanding of decision processes has come of them provide suggested methods of treatment of addic- together in recent years to form a unified theory of tion. However, a thorough treatment of addiction from all of these perspectives is beyond the scope of a paper such as this one. In this target article, we address an exp- lanation for addictive decisions based on animal learning theory, the neuroscience of learning and memory, human A. DAVID REDISH is Associate Professor of Neuro- decision-making, and neuroeconomics, which we argue science at the University of Minnesota. Dr. Redish has published computational and theoretical papers have converged on a unified theory of decision-making on neural mechanisms of navigation, memory, as arising from an interaction between two learning decision-making, and addiction, as well as experimental systems (a quickly learned, flexible, but computationally papers on neural information processing patterns in expensive-to-execute planning system and a slowly learned, hippocampus and striatum during behavioral tasks. inflexible, but computationally inexpensive-to-execute He is the author of Beyond the Cognitive Map: From habit system). Place Cells to Episodic Memory (MIT Press, 1999). STEVEN L. JENSEN is a graduate student in Computer 2.1. Our goals Science and Neuroscience at the University of Minne- sota, and Principal Scientist