Separate Mesolimbic Dopaminergic Pathways Mediate the Opposing Motivational Effects of Acute Caffeine

Home , Caffeine dependence, Mesolimbic pathway

Separate mesolimbic dopaminergic pathways mediate the opposing motivational effects of acute caffeine

Mandy Yee

A thesis submitted in conformity with the requirements for the degree of Master of Science

Institute of Medical Science University of Toronto

Separate mesolimbic dopaminergic pathways mediate the opposing motivational effects of acute caffeine

Mandy Yee Master of Science Institute of Medical Science University of Toronto 2018

Abstract

Caffeine is the most commonly consumed psychoactive drug in the world, yet little is known about the neural substrates that underlie its rewarding and aversive properties. Using male

Wistar rats in a place conditioning procedure, we showed that systemic caffeine at a low intraperitoneal dose of 2 mg/kg (or 100µM injected directly into the rostral, but not caudal, ventral tegmental area) induced reward. By contrast, high doses of systemic caffeine (10 and 30 mg/kg) produced aversions that were not recapitulated by a caffeine analog restricted to the periphery. We demonstrated that pharmacological blockade of dopamine receptors using α-flupenthixol injected into the nucleus accumbens shell, but not core, blocked caffeine reward. Conversely, α- flupenthixol injected into the nucleus accumbens core, but not shell, blocked caffeine aversions.

Thus, our findings reveal two dopamine-dependent and functionally dissociable mechanisms for processing caffeine motivation that are segregated between nucleus accumbens subregions.

Acknowledgments

I would like to acknowledge my friends and family for supporting me through all the ups and downs of research. First and foremost, I would like to thank my parents for their loving support. I would also like to thank my siblings – Lance, Angela, Holly, and Kingly – for their support.

A great big thanks to my graduate supervisor and mentor, Dr. Derek van der Kooy, whose combination of excellent guidance, thoughtful critiques, and vast knowledge inspired me to grow and learn as a scientist. Thank you to my program advisory committee members, Drs. Cindi Morshead and Paul Fletcher, for their insights, advice, and patience.

I would also like to thank my lab mentors, Taryn and Geith, who patiently taught me everything I needed to know for my project. A huge thanks to our lab staff extraordinaire, Brenda and Monica, for being incredibly kind and helpful. I also cannot stress enough the importance of my amazing fellow lab members that I have worked with over the years. I want to especially thank Isabel, Michael, and the other undergraduates in the motivation group for keeping me company in the necropsy and surgery room those long days and nights. Additionally, I would like to thank Justin, Daniel, Ken, Sylvia, and the other lab members for always being in the lab even at the increasingly odd hours that I came in to do lab work. Even though we were not working on the same projects, you guys are excellent company and made the evenings far more interesting. I would also like to thank the staff at the Division of Comparative Medicine for taking excellent care of the animals, and the Zandstra lab members who patiently helped me with my secondary project.

A special thank you to Marco for putting up with all my complaining – and there was a lot of it! I am especially grateful for the time you spent giving me the push I needed to finish this thesis.

I dedicate this thesis to my family and friends who have supported me every step of the way.

iii

Statement of Contributions

Mandy Yee (author) solely prepared this thesis. All aspects of this body of work were performed by the author in whole or in part.

The following contributions by other individuals is formally acknowledged below:

Dr. Derek van der Kooy (primary supervisor) - Mentorship, including guidance in planning experiments and interpreting results - Provision of laboratory materials and equipment - Preparation of manuscript and thesis

Dr. Cindi Morshead (program advisory committee member) - Mentorship, including guidance in planning experiments and interpreting results

Dr. Paul Fletcher (program advisory committee member) - Mentorship, including guidance in planning experiments and interpreting results

Dr. Taryn Grieder - Mentorship, including guidance in planning experiments and interpreting results

Geith Maal-Bared - Mentorship, including guidance in planning experiments and interpreting results - Assistance in executing experiments

Dr. Ryan Ting-A-Kee - Assistance in executing experiments and laying foundation of project

Michal Chwalek - Assistance in executing experiments and laying foundation of project

Isabel MacKay-Clackett - Assistance in executing experiments

Michael Bergamini - Assistance in executing experiments

Table of Contents

Abstract ...... ii

Acknowledgments ...... iii

Statement of Contributions ...... iv

Table of Contents ...... v

List of Abbreviations...... viii

List of Figures and Tables ...... xi

Chapter 1 - Literature Review ...... 1

1.1 Introduction to motivation ...... 2

1.2 Introduction to drug addiction ...... 4

1.3 Measurements of motivation ...... 7

Operant conditioning (self-administration)...... 8

Classical conditioning (place conditioning) ...... 11

1.4 Models and theories of drug motivation ...... 16

The mesolimbic dopamine reward hypothesis ...... 16

The error prediction model ...... 19

The incentive-sensitization theory ...... 22

The non-deprived/deprived hypothesis...... 23

The opponent process theory ...... 26

1.5 Neuroanatomy of motivation ...... 32

Anatomy of the ventral tegmental area ...... 32

Anatomy of the nucleus accumbens ...... 38

1.6 Neurobiology of caffeine motivation ...... 41

Introduction to caffeine consumption ...... 41

1.7 Research hypotheses and objectives ...... 43

Chapter 2 - Materials and Methods ...... 46

2.1 Animal subjects and drugs ...... 47

2.2 Experimental procedures ...... 49

Surgical cannulation ...... 49

Place conditioning ...... 49

2.3 Histology ...... 51

Verification of cannula placements ...... 51

2.4 Statistical methods ...... 52

Chapter 3 - Results ...... 53

3.1 Systemic caffeine dose-response ...... 54

Systemic caffeine elicits reward at low doses and aversion at higher doses ...... 54

3.2 Intra-rVTA caffeine reward ...... 59

Caffeine injections into the rostral VTA elicit reward ...... 59

Intra-rVTA caffeine reward is mediated through the NAc shell, but not core ...... 65

3.3 High dose systemic caffeine aversion ...... 69

Systemic caffeine aversion is mediated centrally through the NAc core, not shell ...... 69

Chapter 4 - General Discussion ...... 72

4.1 Overview...... 73

Neuroanatomy of caffeine aversion...... 73

Neuroanatomy of caffeine reward ...... 75

Caffeine and other drugs of abuse ...... 77

Caffeine and the role of adenosine receptors ...... 78

Caffeine pharmacokinetics and metabolism ...... 79

Caffeine use by people ...... 80

4.2 Conclusions ...... 83

4.3 Future Directions ...... 84

References ...... 89

vii

List of Abbreviations

6-OHDA 6-hydroxydopamine

8-SPT 8-(p-sulfophenyl)theophylline

α-flu α-flupenthixol (also known as α-flupentixol)

A1R adenosine receptor subtype-1

A2AR adenosine receptor subtype-2A

ACC anterior cingulate cortex

ANOVA analysis of variance

CeA central amygdala cm centimeter

CNS central nervous system

CPA conditioned place aversion

CPP conditioned place preference

CR conditioned response

CS conditioned stimulus cVTA caudal ventral tegmental area

D1R dopamine receptor subtype-1

D2R dopamine receptor subtype-2

DA dopamine

DR dorsal raphe

DStr dorsal striatum

DSM-5 diagnostic and statistical manual of mental disorders-fifth edition

EEG electroencephalogram

viii

GABA γ-aminobutyric acid

GAD glutamic acid decarboxylase

GP globus pallidus

Glu glutamate

HPLC high-pressure liquid chromatography

Hz hertz i.c. intracranial

ICD-10 international classification of diseases and related health problems-tenth revision i.p. intraperitoneal

KO knockout

LH lateral hypothalamus

LHb lateral habenula

MFB medial forebrain bundle mg/kg milligrams per kilogram mm millimeter mPFC medial prefrontal cortex ms millisecond

MSN medium spiny neuron mV millivolt

NAc nucleus accumbens

NAc core nucleus accumbens core

NAc shell nucleus accumbens shell

NMDA N-methyl-D-aspartate

NS neutral stimulus

PAG periaqueductal grey

PBS phosphate-buffered saline

PFA paraformaldehyde

PPTg pedunculopontine tegmental nucleus rVTA rostral ventral tegmental area s second s.c. subcutaneous

SEM standard error of the mean

TH tyrosine hydroxylase

TPP tegmental pedunculopontine nucleus tVTA tail of the ventral tegmental area

UR unconditioned response

US unconditioned stimulus

VP ventral pallidum

VTA ventral tegmental area

VTAR ventral tegmental area-rostral subregion

WT wild-type

List of Figures

Figure 1.1 - Schematic of the place conditioning paradigm (Page 12)

Figure 1.2 - Schematic of the mesolimbic dopamine system in the rat brain (Page 17)

Figure 1.3 - Schematic illustrating the opponent process theory of motivation (Page 27)

Figure 1.4 - Selected input and output projections of the VTA (Page 37)

Figure 1.5 - Schematic of the nucleus accumbens shell and core (Page 39)

Figure 3.1 - Systemic caffeine produces DA-dependent rewarding effects at low doses, and DA- dependent aversive effects at high doses that are centrally mediated (Page 56)

Figure 3.2 - Caffeine reward is mediated through the rostral, not caudal, VTA (Page 61)

Figure 3.3 - Verification of bilateral cannula placements (Page 66)

Figure 3.4 - Intra-rVTA caffeine reward is DA-dependent and mediated centrally through the

NAc shell, but not the core (Page 67)

Figure 3.5 - Caffeine aversion is mediated centrally through the NAc core, but not the shell (Page

70)

List of Tables

Table 1 - Estimated percentage of cell types in the VTA (Page 34)

Table 2 - Projections to and from the VTA (Page 36)

Chapter 1 - Literature Review

1 Literature Review

1.1 Introduction to motivation

In the evolutionary struggle for survival and reproduction, a group of organisms are successful if they can contribute to the gene pool of the next generation. To influence evolutionary

(phylogenetic) success of a species, certain key features must be obtained over the developmental

(ontogenetic) history of each organism. One of these ontogenetic features is motivation, which can be broadly defined as the process in which an organism responds to stimuli to maintain goal- oriented behaviours. Generally, motivation encompasses physiological, psychological, and social processes that can manifest as a set of approach and avoidance behaviours. For example, organisms can learn to approach the necessities of life such as food and water to reduce hunger and thirst, while avoiding potential harm or dangers such as poisons and predators. Over time, organisms can learn to associate previously neutral cues or environments with either positive (rewarding or beneficial to the organism in some way), negative (aversive or detrimental), or neutral (neither rewarding or aversive) outcomes. Repeated associations result in reinforcement of behaviours that ensure a selection of optimal behaviours for the well-being and survival of the organism.

Therefore, the study of motivation and its underlying mechanisms is crucial for understanding behaviour.

Integral to the study of motivation is what happens when motivational processes go awry and become maladaptive. In the extreme case where there is a complete absence of motivation, organisms would be unlikely to meet their needs, and may perish early. However, a more intriguing example of maladaptive motivation is drug addiction, in which there is a large range in severity.

An individual that actively and compulsively engages in drug-seeking and drug-taking behaviour despite the negative consequences is colloquially termed a “drug addict.” Individuals may consume one or many types of drugs of abuse – including nicotine, cocaine, ethanol, cocaine, and amphetamine – that produce subjective feelings of pleasure or reward. As more of the drug is consumed, the pleasure of taking the drug is positively reinforced and leads to continued consumption and drug dependence (also known as addiction). Upon halting consumption of a drug in a dependent/addicted individual, the aversive feelings of withdrawal typically lead to further drug-seeking behaviour to alleviate the aversive feelings. The studies described within this thesis focus on the neurobiological substrates and possible pathways that mediate the rewarding and aversive responses to acute caffeine given to drug-naïve rats.

1.2 Introduction to drug addiction

According to the Diagnostic and Statistical Manual of Mental Disorders – Fifth Edition

(DSM-5), a substance use disorder or addiction can be described as a pattern of drug use that results in impairment or distress in daily life. It is a brain disease characterized by compulsive drug- seeking behaviour, despite adverse consequences (American Psychiatric Association, 2013). The

International Classification of Diseases and Related Health Problems (ICD-10) by the World

Health Organization (10th revision, 1992) provides a classification of mental and behavioural disorders with a similar definition. Along with addiction, three other common terms that are often used in the literature (reviewed in Nestler, 1992) to describe different states of substance use disorder, include:

1) Tolerance, defined as “a reduced effect upon repeated exposure to a drug at constant

dose, or the need for an increased dose to maintain the same effect;”

2) Dependence, defined as “the need for continued exposure to a drug so as to avoid a

withdrawal syndrome,” and;

3) Withdrawal, defined as “physical or psychological disturbances when the drug is

withdrawn.”

The DSM-5 defines several diagnostic criteria for substance-related disorder that fall into four major categories, including:

1) Impaired control, described as use for longer than intended periods of time or in larger

amounts than intended;

2) Social impairment, evidenced by continued use of the drug, causing problems with

work, school, or other social obligations;

3) Risky use, demonstrated by drug use in conjunction with dangerous situations (e.g.

drinking and driving) or continued use despite knowing the harmful physiological side

effects, and;

4) Pharmacological indicators, which include tolerance of the drug such that increased

amounts of the drug are required to achieve the same pleasurable effect, and withdrawal

where cessation of the drug leads to unpleasant symptoms.

An individual that meets at least two of these criteria can be diagnosed with substance use disorder, of which the severity is dependent on how many of the criteria are met.

Caffeine dependence and withdrawal did not meet these criteria in the previous editions of the DSM, but numerous preclinical and clinical studies have supported the addition of a caffeine use disorder category (Dingle et al., 2008; Silverman et al., 1992; Kendler & Prescott, 1999;

Ogawa & Ueki, 2007; reviewed in Juliano & Griffiths, 2004). The DSM criteria that would be most applicable to caffeine dependence are the categories of social impairment and pharmacological indicators. While short-term caffeine use is associated with enhance alertness and mood, prolonged use of caffeinated products has been associated with stress, anxiety, and depression (Pettit & DeBarr, 2011; Richards & Smith, 2015). While these relationships are not causal, caffeine may indirectly impair social activities or obligations by placing mental or emotional strain on the consumer. A more direct effect of caffeine is the development of tolerance, a prominent pharmacological indicator listed in the DSM-5. Caffeine tolerance is well-documented in rodents and humans (Chou et al., 1985; Finn & Holtzman, 1986; Evans & Griffiths, 1992).

However, despite the support and recommendation for the addition of caffeine dependence and withdrawal disorder into the DSM-5, caffeine will not be listed as a substance use disorder on par with the other established disorders, including the use of alcohol, cannabis, nicotine, etc., without further research (Hasin et al., 2013).

Though the further investigation of caffeine use and other licit drugs, including nicotine and alcohol, can be performed in humans, the pervasive use of caffeine worldwide means that it is difficult to find a naïve population for studying caffeine’s acute motivational effects. Further, as addiction is primarily mediated by neural mechanisms, research can be more easily performed in rodent models. Therefore, this present thesis uses a rodent model to study the acute motivational effects of caffeine as a foundation for future studies on the interdependent effects of caffeine with other drugs of abuse.

1.3 Measurements of motivation

Over the past century, the use of rodent models in neuroscience has increased dramatically.

The two most common types of laboratory rodents – mice (Mus musculus) and rat (Rattus norvegicus) – share many advantages: they are purposefully bred to reduce biological variation, their relatively short life cycles and prolific breeding ensure that they remain affordable, they are well-characterized in numerous studies as bridges between in vitro and in vivo experiments, and they share homologous traits with humans. Though the mouse is a powerful transgenic model, the use of rats is more advantageous in neurobiological experiments requiring targeting of specific brain regions due to their larger brain sizes. Therefore, rats are especially useful to model mental disorders or diseases common to humans, including the studies of drug addiction presented here.

As there are no complete animal models of drug addiction and studies in people are heavily constrained by the number of available samples to manipulate and investigate, rodents that model different aspects of drug addiction remain the best alternative for unraveling its complexities.

A fundamental task of the mammalian brain is to assign motivational valence to stimuli to ensure an organism’s survival and well-being. Though this process can be affected by any number of psychological, physiological, or social factors, motivation can nonetheless be quantified with behavioural paradigms that measure an organism’s response to specific stimuli. Studies focusing on the neurophysiological aspect of motivation, as in this present thesis, generally require preclinical models. The standard preclinical behavioural models used to study the motivational effects of drugs can be largely divided into two major categories with a basis in either operant or classical conditioning.

Operant conditioning (self-administration paradigm)

Operant conditioning (also known as instrumental conditioning) is a learning process that is characterized by the modification of behaviour through reinforcement or punishment. An action

(or operant) is initially instinctual or spontaneous (e.g. accidentally pressing on a bar when the animal is freely roaming in an enclosure), but the consequence of that action reinforces or inhibits the likelihood of the action’s reoccurrence. Early observations by Edward Thorndike (1898) serve as a basis for our current understanding of this type of animal learning. He measured how long it took for hungry cats to perform simple actions (including lever pressing, turning buttons, and pulling on string) to escape from enclosures for a food reward (Thorndike, 1898). He found that the time it took for the animal to perform the action shortened after numerous trials and generally, actions that bore fruitful reward were strengthened (reinforced) while those that did not were weakened. Burrhus Frederic (B.F.) Skinner (often referred to as the father of operant conditioning) rejected the notion of intangible mental states referenced by his predecessor, Thorndike, in favour of only observable behaviours and its subsequent observable consequences (reviewed in Skinner,

1950). Skinner’s experiments where he carefully controlled what stimuli the animals were exposed to and limiting his response criterion to simple repeatable responses, laid the groundwork for the notion of modifying behaviour. This “behaviour shaping” by delivering rewards and/or punishments would spur an animal to behave in a desired, and more complex, way (reviewed in

Skinner, 1950).

When an animal performs an observable action and receives a rewarding stimulus, the action is positively reinforced such that it is more likely to be repeated. The rate or quantity of response at which the animal performs the action is inferred to be a measurement of how rewarding

8 the stimulus is. This action can be varied widely across experiments and could have animal subjects pressing a lever or nose-poking for a reward. These rewards can come in many forms – ranging from access to food or water, pleasurable electrical stimulation to the brain, or administration of a pleasurable drug. The effects of experimental manipulations can then be performed to determine its impact on the subjects’ response. These manipulations can include brain lesions, pharmacological pretreatment with chemical agonists or antagonists, optogenetic activation or silencing of specific neurons, and many other types of manipulations. Reinforcement schedules also can be manipulated (Machado, 1989). The reward can simply be provided on a continuous schedule such that one unit of action (e.g. a nose poke) delivers one unit of reward or be provided after a constant amount of time (fixed interval) or amount of actions (fixed ratio). The reward can also be provided on a variable interval or variable ratio schedule such that the amount of time or actions, respectively, is reinforced at random (Yukl et al., 1972).

The advantage of operant conditioning is that it more closely mirrors the reward-seeking and reward-using behaviours of people who compulsively take drugs. This self-administering aspect of operant conditioning makes it suitable for studying the rewarding properties of drugs, however, it does not come without its limitations. Most troubling, both an increase or decrease in the rate of self-administration can be interpreted as rewarding. An increase in rate of response could indicate that the animal finds the drug pleasurable and therefore, increases responding for more of the drug. A decrease in rate of response could be interpreted as the animal finding each subsequent drug administration more rewarding and thus, they do not have to respond as often to reach a similar state. Further, this dependency on the animal engaging in a motor response must be considered when studying drugs that are known to potentially affect locomotion. For example, a drug that increases locomotion like caffeine can overestimate the number of positive responses 9 in bar-pressing or nose-poking tests (Kuzmin et al., 2000). Dopamine receptor antagonists that decrease animal locomotion to near paralysis also would have obvious confounding ramifications

(Vezina & Stewart, 1989). If the subject cannot respond at high dosages of rewarding yet motor- depressing drugs, then the lack of response could be mistakenly interpreted as unrewarding or even misconstrued as aversive. Additionally, standard self-administration paradigms generally lack the ability to sensitively measure aversive responses, as animals tend to avoid performing an action if it leads to an aversive consequence no matter the severity of the aversion. This can be overcome by negatively reinforcing a behaviour (i.e. animals learn to perform an action to remove an aversive consequence) though interpretations of the results from these procedures can be difficult to generalize, as positive and negative reinforcement may be mediated by different neural substrates

(Namburi et al., 2015). Most important, the reinforcing properties of caffeine are inconsistent in the self-administration paradigm. Griffiths and Woodson (1988) compared the results of six studies that attempted to demonstrate caffeine self-administration in non-human primates and found that three failed to observe caffeine self-administration, while the other three observed erratic patterns of self-administration.

Therefore, though operant conditioning (and the self-administration paradigm) has been a great tool for modeling and measuring rewarding properties of drugs, the nature of caffeine and the paradigm itself make it difficult to study the aversive properties of drugs or drugs that affect locomotion. These limitations can be circumvented to an extent using classical conditioning procedures.

Classical conditioning (place conditioning paradigm)

Classical conditioning is a learning process that is characterized by the acquisition of a new behaviour via association between a cue (stimulus) and an innate bodily reflex or instinct. Ivan

Petrovich Pavlov first developed this theory of learning using an audio cue (neutral stimulus or

NS) paired with presentation of food, which is an unconditioned stimulus (US) because it would elicit a natural and reflexive unconditioned response (UR) of salivation in dogs (Pavlov, 1927).

During conditioning, as the NS and US are repeatedly paired together, the dog learns to associate the two such that the NS becomes a conditioned stimulus (CS). After conditioning, the presentation of the CS alone can produce a new conditioned response (CR) of salivation (Pavlov, 1927).

According to Pavlov (1927), “It is pretty evident that under natural conditions that the normal animal must respond not only to stimuli which themselves bring immediate benefit or harm, but also to other physical or chemical agencies – waves of sound, light, and the like – which in themselves only signal the approach of these stimuli…” Therefore, learning by association has applications in many aspects of an animal’s survival and well-being, including motivated behaviour in response to drugs. Based on the principles of classical Pavlovian conditioning, the place conditioning paradigm relies on repeated exposure of a biologically motivating US (e.g. food or drug) paired with a neutral environment (acting as the NS), such that the animal learns to associate the previously neutral environment with the subjective effects of the motivating stimulus, as described in more detail in Figure 1.1.

1) Habituation 2) Conditioning 3) Testing

Drug

Drug is rewarding or

Vehicle

Drug is aversive

Figure 1.1 - Schematic of the place conditioning paradigm

In the place conditioning paradigm, animals are first habituated to a neutral environment to reduce novelty effects before conditioning. During conditioning, animals passively receive an injection of the drug and then are immediately placed in a chamber with distinct features for a fixed amount of time. For example, in the schematic, the animal is conditioned to associate a black- walled and smooth-floored chamber with the experience of the drug. On alternate conditioning sessions, the animal is given an injection of vehicle solution in a different chamber. In the schematic, the animal is given a vehicle injection in a chamber with white walls and a wire grid floor. The animal receives the drug- and vehicle-pairing over a fixed number of conditioning cycles, such that there is equal exposure to both drug- and vehicle-pairings. After which, the animals are given a period of rest to allow for testing in a drug-free state. During the testing phase,

12 the animal is given free access to roam in a testing apparatus that has the distinct features of both conditioning chambers on either end. The amount of time that the animal spends on each side of the testing apparatus is recorded. If the animal spends relatively more time in the previously drug- paired side, then the animal exhibits a conditioned place preference (CPP) and we can infer that the animal finds the drug pleasurable or rewarding. If the animal spends more time in the vehicle- paired side and avoids the previously drug-paired side, we can infer that the animal finds the drug aversive in what is termed a conditioned place avoidance (CPA). If the animal spends approximately equal amounts of time in both sides, then the drug is inferred to be motivationally neutral (neither rewarding nor aversive).

The advantages of place conditioning are numerous. In comparison to the self- administration paradigm, the major advantage of the place conditioning paradigm is that it allows for the investigation of the rewarding, aversive, or motivationally neutral properties of drugs in one testing phase. Because of this, the inference of motivational valence can be compared across drugs more easily and with good sensitivity. For example, Mucha and colleagues (1982) were able to use the place conditioning paradigm in rats to compare the preferences or aversions produced by many drugs, including morphine (and other opiate agonists), naloxone, cocaine, and lithium chloride. In addition, since the animals are tested in the absence of the drug, the consequences of the drugs on motor systems can be circumvented. Rather, the impact on locomotion can be investigated during conditioning (with the appropriate sensory equipment) as an additional property of the drug of interest.

However, the place conditioning paradigm is not without its criticisms. One criticism is that the passive administration of drug means that it does not closely mirror the reward-oriented behaviours in people who seek out drugs intentionally. However, the place conditioning paradigm has elements of both classical and operant conditioning (reviewed in Huston et al., 2013).

Inadvertent reinforcement of behaviours that are spontaneous in nature, which then increase in frequency are hallmarks of operant conditioning but could also occur in place conditioning. For instance, an animal that is again confronted with a place/context (and associated environmental cues) where it experienced a pleasurable drug may re-engage in behaviours that encourage the animal to seek out or stay in that particular place. This approach behaviour is analogous to positively reinforced bar-pressing behaviour in that it is an increased rate of motor response. Along the same vein, people who abuse drugs may be conditioned to associate certain environmental cues

14 with the experience of drugs, leading to an increase in drug use and abuse. Thus, the place conditioning paradigm has great value as a tool to investigate the mechanisms behind drug-seeking behaviour as well as drug relapse that can occur when an individual finds themselves in the place where they had previously experienced the rewarding effects of drugs. Other criticisms target the protocol of the paradigm. The requirement of pairing a drug or neutral experience to distinct environmental contexts means that the animal may have an innate baseline preference for one environment over the other. As in the present thesis, this concern can be addressed by testing a separate group of animals in the same cohort as the experimental groups. Scenting the preferred side with varying percentages of acetic acid can help to balance the two environments.

Additionally, rats can be handled prior to experimentation to reduce stress and novelty effects from experimenter manipulation.

Therefore, although the place conditioning paradigm has its own advantages and drawbacks, it can provide a simpler measure of both the rewarding and aversive properties of drugs. The behavioural studies presented in this thesis utilized fully unbiased and counterbalanced place conditioning procedures with caffeine as the main drug of interest in rats.

1.4 Models and theories of drug motivation

Early demonstrations of rats self-administering electrical stimulation in the brain by James

Olds and Peter Milner (1954) revealed that there was a rewarding circuit in the brain that could be identified anatomically. Since then, many theories and rodent models of drug addiction have arisen to investigate the neural mechanisms underlying the motivationally rewarding effects of different stimuli, including natural rewards (e.g. food and mating) or drug rewards.

The numerous studies on drug addiction gave rise to different theories of drug motivation, including the mesolimbic reward hypothesis, the error prediction model, the incentive- sensitization theory, the non-deprived/deprived hypothesis, and the opponent process theory, which will be discussed below.

The mesolimbic dopamine reward hypothesis

An early proposal for drug addiction that sought to provide a unified theory underlying the mechanism of how drugs worked to produce their interoceptive rewarding effects was the

“dopamine hypothesis” (reviewed in Wise, 1987; Di Chiara & Imperato, 1988). The dopamine

(DA) hypothesis, elegant in its simplicity, suggested that drugs that led to an increase in the release of the neurotransmitter, dopamine, would lead to a pleasurable/hedonic effect as experienced by an individual, thus compelling the user to seek out more of the same drug. Many studies suggested that a key area responsible for drug and brain stimulation reward was in the medial forebrain bundle (MFB), which is a neural pathway containing fibres connecting the limbic forebrain and

16 the mesencephalon/midbrain regions (Millhouse, 1969; Gallistel et al., 1985). The MFB also contains fibres from the mesolimbic dopaminergic (DAergic) brain reward circuit, which mainly projects from the ventral tegmental area (VTA) DAergic cell bodies in the midbrain to the nucleus accumbens (NAc) in the ventral striatum. Studies showed that rewarding activation of this mesolimbic DAergic circuit by drugs of abuse (including ethanol, nicotine, morphine) could also be disrupted by lesions or pharmacological DAergic antagonism to the same area (Gessa et al.,

1985; Corrigall et al., 1994; Roberts & Koob, 1982).

DA VTA NAc

Figure 1.2 - Schematic of the mesolimbic dopamine system in the rat brain

Schematic illustrating the approximate location of the mesolimbic dopaminergic (DA) reward pathway in a sagittal section of the rat brain. The mesolimbic pathway is comprised of

DAergic neurons in the midbrain VTA that project to and terminate in the NAc and olfactory tubercle of the ventral striatum (only the NAc is shown in the schematic for simplicity). One of the earlier versions of the DA hypothesis posits that these structures compose the final common pathway for drug reward, although we now know that there are other DA-independent reward pathways in the brain.

Although the hypothesis that an increase in dopamine release is responsible for the rewarding effects of drugs may be appealing, it cannot account for the complete story of drug addiction as there is a compelling body of evidence suggesting that DA-independent reward can also be elicited in rodents (Sturgess et al., 2010; Laviolette et al., 2002; reviewed in Fibiger, 1978).

For example, brain regions that do not have known DAergic innervation (e.g. the tegmental pedunculopontine nucleus or TPP) were found to mediate the motivationally rewarding effects of both morphine and amphetamine (Bechara & van der Kooy, 1989).

Tolerance (reduced response to drugs upon repeated exposure) poses another problem to the DA hypothesis. For instance, the increased presentations of drug lead to decreased DAergic neuron activation, but drug-seeking may persist. However, as this present thesis focuses on the acute, and not chronic, effects of caffeine, the mesolimbic DA hypothesis is, nonetheless, the most relevant theory to this thesis.

In contrast to the simplest version of the mesolimbic DA hypothesis, later work has revealed that DA is important for mediating signals other than reward. For example, DA mediates aversive responses to foot-shocks in rats, as evidenced by an increase in DA release following foot–shock measured using microdialysis by Young and colleagues (1993). DA also has been found to be an error prediction signal – a model developed and refined by Schultz and colleagues.

The error prediction model

Common to the mesolimbic DA reward hypothesis, Wolfram Schultz and colleagues proposed that an increase in DA release plays a key role in reward. However, in this proposed error prediction model, phasic DA is initially released in response to any potentially rewarding stimulus

(even if they turn out to be neutral or aversive) and then, evolves to be an established signal for a rewarding value (Schultz et al., 1993; reviewed in Schultz, 2016). This response could be split into two temporally dissociable components: the initial detection of a stimulus, followed by the main valuation component.

Initial detection component

In the initial detection component, a brief activation of DA neurons (demonstrated by an increase in neuronal impulse frequency) occurs in response to a broad range of stimuli perceived by any of the sensory modalities that have potential to predict an outcome that may be rewarding, neutral, or aversive in nature. However, because there exist innumerable stimuli of varying intensities, not all stimuli are strong enough to evoke this initial activation. Though this activation appears to be unselective and prone to errors during reward prediction, the probability of activation to these stimuli can be enhanced by several factors that are related to potential reward availability.

For instance, pairing its presentation with a reward or reducing the number of stimulus presentations (as the response of DA neurons decrease with repetition) can increase its salience.

Stronger reactions to presentations of novel stimuli prior to interaction also mean that the chances of missing a potential reward are lower, as any novel stimulus could be a potential reward until its value is established. For example, in Schultz and colleagues’ experiments with monkeys, they

19 found that unpredicted rewards or conditioned, reward-predicting stimuli were sufficient to induce

DA neuron activation (reviewed in Schultz, 2016). They found that a liquid reward by itself, or a cue predicting a food reward could induce an increase in DA neuron firing, which is consistent with the mesolimbic DA hypothesis (Ljungberg et al. 1992; Schultz et al.1993; Mirenowicz &

Schultz, 1996).

Valuation component

The main valuation component temporally evolves from the initial detection of effective stimuli (reviewed in Schultz, 2016). Conceptually, the transition process from initial detection to valuation sharpens the unselective detection of a stimulus to more specific identification of the stimulus and its corresponding value. While the initial detection causes a transient spike in neuronal activation, the subjective valuation component persists during the animal’s behaviour until the reward is received. It is difficult to directly measure the subjective value of a stimulus.

Instead, one can offer a choice of two relatively equal rewards that differ by a single factor to find the relative rewarding value (e.g. an animal choosing orange juice over the same amount of apple juice can be assumed to subjectively find orange juice more rewarding). The neuronal DAergic response to a correctly predicted outcome of reward or no reward results in a return to baseline activity, which indicates that no prediction error had occurred. If a prediction error does occur, then the second response component reflects the error. For instance, a predicted reward that is unrewarded, results in depression of DA neuron activity while conversely, an unpredicted reward that is rewarded results in a spike of DA neuron activation. The importance of this subjective valuation component is its role in modifying subsequent behaviour. As this main valuation component occurs within 10 ms, Schultz (2016) proposed that stimulant drugs, which are known

20 to increase DA concentrations, may be prolonging the initial detection component such that DA surges overlap with the second valuation component, resulting in a false reward value. After persistent consumption of stimulant drugs, neuronal adaption could result in the erroneous attribution of a reward value to unrewarded stimuli.

The error prediction model is conceptually appealing – correctly predicted outcomes are signaled by the lack of DA neuron activation, which indicate that no modification of behaviour is required, while incorrectly predicted outcomes elicit DA neuron activation or depression that can modify behaviour via downstream signaling to postsynaptic neurons. However, a direct criticism of this theory, similar to the mesolimbic DAergic hypothesis, is the evidence for aversive stimulus- induced DA activation. It can be argued that the removal of an aversive stimulus could be experienced as rewarding, which would then be signaled by DA neuron activation. However, in this study, where high intraperitoneal (i.p.) doses of caffeine (eliciting aversion) is experienced by the animals over a 40-minute conditioning period every other day, it is difficult to determine the moment when animals would be experiencing ‘rewarding’ relief from caffeine aversion. Another disadvantage of this theory is that it does not make strong mechanistic predictions about how acute drug reward experiences can evolve to drug addiction in a subset of individuals. The incentive- sensitization theory, to be discussed below, partially addresses this concern.

The incentive-sensitization theory

The incentive-sensitization theory was first proposed by Berridge, Robinson, and colleagues (1989, 1993). The primary difference between this theory and the others is that this theory dissociates the incentive salience or attractiveness of a drug (inducing ‘wanting’) and its subjective pleasurable/hedonic effects (inducing ‘liking’). Subjective ‘liking’ can be measured by the taste reactivity test that had been developed by Grill and Norgren (1978) in which they recorded and assessed videoframes of rat orofacial expressions denoting pleasure (e.g. rhythmic mouth movements, rhythmic tongue protrusions, and lateral tongue movements) and aversion (e.g. gaping, chin rubbing, lateral head shake, face washing, paw pushing). In an early experiment,

Berridge and colleagues (1989) lesioned the mesostriatal DA system with 6-hydroxydopamine (6-

OHDA) in rats that had produced hedonic orofacial expressions to sugar reward and found that taste reactivity did not change. These results suggested that DA mediated the ‘wanting’ component of reward by imbuing the drug and its associated cues with incentive salience but did not mediate the ‘liking’ component. Unlike the other theories that assumed that drugs are ‘wanted’ by individuals because they ‘like’ the pleasurable sensations produced by rewarding drugs, this theory argues that ‘liking’ a drug does not explain why ‘wanting’ of a drug increases while less pleasure is derived from repeated drug use (reviewed in Berridge & Robinson, 2016).

The incentive-sensitization theory is useful in that it addresses three fundamental questions regarding drug craving: why an addict may crave drugs, what could be the underlying mechanisms of craving drugs, and why do cravings persist after prolonged abstinence. The three items posited by the incentive-sensitization theory help to address these questions:

1) Addictive drugs increase DA neuron activity in the mesotelencephalic (mesostriatum,

mesolimbic, and mesocortical) regions of the brain (Hefco et al., 2003)

2) One consequence of increased DA neuron activity is the attribution of ‘incentive

salience’ (imbuing attractiveness or ‘wanting’) to events that are associated with DA

neuron activation

3) ‘Wanting’ turns into craving due to incremental neuroadaptation, leading to

hypersensitivity of the DAergic neural systems to drugs and its associated stimuli.

However, despite these strong predictions, this theory was criticized for its stance against the traditional models of drug reward, including the mesolimbic DA hypothesis, because it suggested that DA was not necessary for a drug’s hedonic effects despite the abundant evidence suggesting otherwise (Roberts & Koob, 1982; reviewed in Wise, 1987; Di Chiara & Imperato,

1988). However, we now know that DA is not necessary for expression of every type of drug- seeking behaviour and may play a key role in aversion.

The non-deprived/deprived hypothesis

The non-deprived/deprived hypothesis, developed by van der Kooy and colleagues, proposes that the brain substrates and mechanisms underlying motivation are dependent on the motivational state of the animal for both natural (e.g. food) and drug (e.g. opiates) rewards

(Bechara & van der Kooy, 1992; Nader et al., 1994; Nader & van der Kooy, 1997; Laviolette et al. 2002; Ting-A-Kee R et al., 2009). This hypothesis makes strong predictions on the neural systems responsible for drug-seeking behaviour, as a function of the animal’s history of drug exposure.

1) Non-deprived animals are in one of three states: drug-naïve, drug-dependent and not in

withdrawal, or recovered from a state of dependence. Upon drug exposure, the expression

of drug-seeking behaviour was found to be dependent on a brainstem region called the

TPP, which is also known as the pedunculopontine tegmental nucleus (PPTg) (Bechara et

al, 1992; Nader & van der Kooy, 1997; Olmstead et al. 1998). For example, in the place

conditioning paradigm, animals that find a drug rewarding would spend more time in the

previously drug-paired compartment. While TPP lesions could reduce the rewarding

effects of the drugs, antagonism of DA had no effect on this drug-seeking behaviour.

2) Deprived animals are drug-dependent and withdrawn. In other words, they have a history

of drug use that renders them addicted to continued drug consumption, such that abstinence

from the drug induces a state of aversive withdrawal. For example, in the place

conditioning paradigm, animals that are drug-dependent and withdrawn tend to avoid the

compartment that was previously paired with the absence of drug (withdrawal). In direct

contrast to non-deprived animals, this behaviour can be blocked by a widespread DA

receptor antagonism but is unaffected by TPP lesions (Bechara et al, 1992).

Derived from the double dissociation outlined above (between the motivational state of the animals (non-deprived or deprived) and its underlying mechanisms (TPP- or DA-dependent), this hypothesis suggests that drug history is crucial for our understanding of how the brain processes motivation. Nader and van der Kooy (1997) showed that this double dissociation could be reproduced via microinjections of morphine directly into the VTA eliciting reward (indicated by

CPP), which could be blocked by TPP lesions or DA antagonism depending on the motivational state of the animal.

In contrast to the other theories of motivation in which the brains of individuals are sensitized (the incentive-sensitization theory) or go through incremental and erroneous neuroadaptation (the error prediction model), the work supporting the non-deprived/deprived hypothesis demonstrates that drug motivational states are more transient, meaning that switching between the two motivational states can be accomplished in rodent models (Bechara & van der

Kooy, 1992; reviewed in Bechara et al, 1998). This hypothesis would suggest that clinical application would have to account for drug history and arguably presents a more complete picture than other theories of motivation regarding how an individual that has become addicted to drugs can revert to a state of non-dependence.

Despite the advantages of this hypothesis for explaining opiate motivation, this hypothesis does not appear to hold true for caffeine motivation. Sturgess and colleagues (2010) lesioned the

TPP of rats that were non-deprived and found that there was no effect on caffeine motivation.

However, it is possible that DA-independent reward exists elsewhere in the brain. Additionally, this hypothesis does not account for the DAergic rewarding effects of cocaine or amphetamine in drug-naïve rodents (Hurd et al., 1989; Ventura et al., 2003; Sellings & Clarke, 2003). Further, a study by Stuber and colleagues (2002) demonstrated that amphetamine reward can be sensitized after food deprivation in animals that were not drug-deprived. This suggests that the motivational states of deprived and non-deprived are not limited to the drug being tested, and that any change in motivational state (e.g. deprived from food) can affect which neural substrates mediate the drug

25 reward. Another limitation of the non-deprived/deprived hypothesis is that the different motivational states may affect drugs of abuse differently. The opponent process theory, described next, proposes that motivation can be looked at more holistically as a homeostatic process.

The opponent process theory

First proposed by Solomon and Corbit (1973, 1974), the opponent process theory of motivation is a counter-adaptive process, wherein an affective (rewarding or aversive) stimulus generates a corresponding sharp perturbation to the homeostatic state of an animal (the ‘a- process’), followed by a gradual recruitment of opposing systems that allow for restoration of homeostasis (the ‘b-process’).

Characteristics of the a-process

The a-process is thought to be a simple, short-acting response that closely follows exposure to the drug stimulus. An a-process could apply to both reinforcing or aversive stimuli. For example, the intense pleasure (‘rush’ or ‘high’) following consumption of an opiate drug would reflect a hedonic a-process, as would the aversion (indicated by CPA) seen in drug-naïve rodents after nicotine administration (Koob et al., 1989; Grieder et al., 2010).

Characteristics of the b-process

In contrast, b-processes have a longer latency in the opposing direction of the a-process.

The b-process is also slow to build up and slow to decay (Solomon & Corbit, 1974). For rewarding drugs, this means that the aversive effects of withdrawal (provided that the rewarding drug is not

26 taken continuously without break) would persist even after the rewarding effects of the drug wear off. The neurobiological mechanisms of the b-process are thought to be the manifestation of a combination of two factors: the decrease in function of the brain’s reward system and the recruitment of the brain’s stress or anti-reward system (Koob & Le Moal, 2005; Koob & Le Moal,

2008a; Koob & Le Moal, 2008b).

Response to a rewarding stimulus Response to an aversive stimulus

Reward (+) a-process Anti-aversion (+) b-process

Time → Time → c Anti-reward (-) c b-process Aversion (-) a-process

Adapted from Grieder et al., 2010

Figure 1.3 - Schematic illustrating the opponent process theory of motivation

The opponent process theory proposes that any affective (rewarding or aversive) stimulus would generate a corresponding a-process of fast onset and short duration. It would then be followed by a counter-adaptive b-process (anti-rewarding or anti-aversive) of later onset and longer duration to gradually restore the homeostatic state of the animal. This figure was adapted from Grieder et al., 2010.

Building upon the work of Solomon and Corbit (1973, 1974), Koob, Le Moal, and colleagues (reviewed in Koob & Bloom, 1988; Koob & Le Moal, 2001; Koob & Le Moal, 2005;

Koob & Le Moal, 2008a; Koob & Le Moal, 2008b) expanded the opponent process theory to an allostatic model of the brain motivational system. Koob and Le Moal (2001) hypothesized that infrequent drug use would allow for sufficient time for individuals to experience the hedonic a- process, followed by the counter-adaptive b-process to appropriately restore a homeostatic state.

Upon repeated drug use, the b-process is unable to restore homeostasis before each repeated drug- taking session, leaving the individual in an allostatic state of drug dependence. Therefore, Koob &

Le Moal (2008a) had conceptualized drug addiction as a cycle of impulsive (early stage) and compulsive (late stage) drug-taking, comprised of three stages:

1) Binge/intoxication, in which the pleasant effects of the drug lead to a positively

reinforced increase in drug intake and tolerance;

2) Withdrawal/negative affect, characterized by the emergence of an aversive emotional

state following the loss of drug euphoria, leading to negatively reinforced drug intake

to reduce the aversive state;

3) Preoccupation/anticipation (craving), defined as “the memory of the rewarding

effects of a drug superimposed upon a negative state,” which is hypothesized to be the

key factor in chronic relapse and reversion to the binge/intoxication stage of drug

addiction.

Our lab has also refined and applied the opponent process theory to rodent models of acute and chronic nicotine substance use disorders (Grieder et al., 2010; Grieder et al., 2014). Notably,

Grieder and colleagues (2014) demonstrated a direct link between the VTA reward system and the brain’s stress system by identifying a population of VTA DAergic neurons in both rodents and humans that express corticotropin-releasing factor, a peptide hormone involved in the stress response.

Although the opponent process theory accounts for both the hedonic/aversive effects of drugs and subsequent restoration to baseline, it is not without its criticisms. For instance, the b- process is unlikely to require months to build up and decay, yet individuals who have ceased taking drugs for months, or even years, may relapse (Brown et al., 1989; Xie et al., 2005).

To briefly summarize the theories discussed above,

1) The mesolimbic DA hypothesis proposes that increased DA signals hedonic reward and its

acquisition (e.g. reviewed in Wise, 1987; Di Chiara & Imperato, 1988; Roberts & Koob,

1982).

2) The error prediction model proposes that DA signals are split into two components, in

which the initial detection of unselective stimuli evolve to a subjective valuation

component. Overlapping DA activity may falsely signal reward, leading to erroneous

neuroadaptation (proposed by Schultz et al. 1993).

3) The incentive-sensitization theory proposes that increased DA signals the subjective

reward attractiveness (or ‘wanting’). Subsequent neuroadaptation to drugs and associated

stimuli could lead to hypersensitivity to drugs transforming the ‘wanting’ into a drug

craving (proposed by Robinson & Berridge, 1993)

4) The non-deprived/deprived hypothesis proposes that the neurobiological mechanisms

underlying drug addiction and its development is dependent on the motivational state of

the animal (proposed by van der Kooy and colleagues. e.g. Bechara & van der Kooy, 1992).

5) The opponent process theory proposes that addictive drugs perturb the homeostatic state of

the animal, followed by a more gradual counter-adaptive process that restores homeostasis.

Drug dependence occurs as a cycle of impulsive and compulsive drug-taking such that an

allostatic state persists (proposed by Solomon & Corbit, 1973, and later refined by Koob

& Le Moal, 2001).

While none of these theories can account for the complete picture of drug addiction, each of these theories target different facets of drug addiction and can be thought of as complementary.

For example, as individuals experience drugs for the first time (i.e. in a non-deprived state), both

DA-dependent and DA-independent mechanisms (e.g. through the TPP) could simultaneously signal to any number of separate downstream pathways, indicating the physical acquisition of the reward, the attractiveness of the reward, the context in which the reward was received, the reward’s subjective value, etc. This perturbation of the motivational systems, having also widely activated other subsequent pathways, could lead to a counter-adaptive negative feedback system that restores homeostasis. As individuals persist in their consumption of addictive drugs and become dependent, incremental neuroadaptation could conceivably occur such that the DAergic neural system becomes hypersensitive to drugs and associated stimuli. Other brain regions could also be recruited to mediate the rewarding or aversive experience of drugs. The neural systems mediating a dependent and withdrawn (deprived) state would, then, be different than those that mediate a non- deprived motivational state.

1.5 Neuroanatomy of motivation

Although the theories described previously have many contradictions, the DAergic neurons have a significant role that cannot be discounted. As the VTA contains cell group A10 (the largest group of DAergic neurons in rodents and primates), it has been heavily implicated as a key reward processing centre in the brain. As the results of caffeine reward in this thesis project aligns closely with the mesolimbic DAergic hypothesis, a description of the anatomy of the VTA and NAc will follow.

Anatomy of the ventral tegmental area

The VTA, also known as the VTA of Tsai, is a component of the medial forebrain bundle, that passes through the lateral hypothalamus and the basal forebrain (Veening et al., 1982).

Anatomically, the VTA is a group of neurons located near the midline on the floor of the midbrain.

It is bordered rostrally by the mammillary bodies and the posterior hypothalamus, and caudally by the pons and hindbrain (reviewed in Oades & Halliday, 1987). The VTA contains a large population of DA and γ-aminobutyric acid (GABA) neurons in the brain, with emerging evidence suggesting that there are glutamatergic neurons as well (Yamaguchi et al., 2007). DAergic and

GABAergic cells have been traditionally identified through various methods (of which some are listed below in Table 1), including immunohistochemistry, electrophysiology, and pharmacology.

For example, Steffensen and colleagues (1998) found that GABA neurons in the VTA can be distinguished from DAergic neurons by their rapid-firing, non-bursting activity (~19 Hz), short duration action potentials (~310 µsec), and small action potentials (~68 mV). However, identifying

DA neurons alone through electrophysiological means has proven more difficult. Margolis and

32 colleagues (2006) measured widely accepted electrophysiological criteria for DA neuron activity, including spike duration and spontaneous firing rate, and concluded that these criteria are not reliable. Generally, DAergic and GABAergic neurons are distinguished in the VTA based on immunohistochemical staining. DAergic neurons can be stained for tyrosine hydroxylase (TH), which is the rate-limiting enzyme of catecholamine biosynthesis (Olson & Nestler, 2007). Some receptor types located on the VTA DAergic neurons are D1, D2, GABAA, GABAB, N-methyl-D- aspartate (NMDA), and cholinergic (Westerink et al., 1996). GABAergic neurons can be stained for glutamic acid decarboxylase (GAD), which is the rate-limiting enzyme in the conversion of glutamate (Glu) to GABA (Olson & Nestler, 2007).

Table 1 - Estimated percentage of cell types in the VTA

DA GABA Glu Co-labelled Species Method Source

VGluT2 radioactive Yamaguchi - - ~5% - Rat antisense riboprobes et al., 2007

Immunohistochemistry Nair- ~0.58% GABA (TH) and in situ ~66% ~35% ~2-3% Rat Roberts et (GAD) /DA (TH) hybridization (GAD and al., 2008 VGlut2 mRNA)

~39 Immunohistochemistry Margolis et to - - - Rat (TH) al., 2006 ~72%

The VTA is a highly heterogenous region of the midbrain proposed to have four or five subdivisions (reviewed in Ikemoto, 2007 and Sanchez-Catalan et al., 2014). In an early cytoarchitectural definition, the lateral subdivisions contained two nuclei – the paranigral nucleus and the parabrachial pigmented nucleus (also known as the parabrachial pigmentosus nucleus).

The medial subdivisions included three nuclei – the interfascicular nucleus, the rostral linear nucleus, and caudal linear nucleus. However, as the three medial subdivisions have been proposed to belong to the raphe nuclei instead of the VTA, Ikemoto (2007) proposed that the VTA has only four lateral subdivisions: the paranigral nucleus, the parabrachial pigmented nucleus, the parafascicular retroflexus area, and the tail of the VTA. Additionally, there is rostrocaudal heterogeneity in the VTA (reviewed in Sanchez-Catalan et al., 2014), however this will be discussed further in the thesis discussion in conjunction with the experimental results.

Because of the heterogeneity of the VTA, the input and output projections of the VTA have been investigated by many research groups (Beckstead et al., 1979; Swanson, 1982; Geisler &

Zahm, 2005; Poller et al., 2013; Beier et al., 2015). Selected projections to and from the VTA are outlined in Table 2 and Figure 1.4).

Table 2 - Projections to and from the VTA

Afferent (inputs) Efferent (outputs)

Glutamatergic Telencephalon • Lateral Hypothalamus° • Nucleus accumbens*,# • Lateral Habenula- • Anterior limbic cortex# • Prefrontal cortex@ • Lateral septal nucleus*,# • Medial forebrain bundle& • Amygdala*,# • Fasciculus retroflexus& • Entorhinal Area* • Hippocampal formation#,~ GABAergic • Cingulate gyrus^ • Nucleus accumbens$ • Prefrontal cortex^ • Ventral pallidum$ • Dorsal striatum+ • Diagonal band of Broca$ • Rostromedial tegmental Diencephalon nucleus/Tegmental • Lateral Habenula*,#,° pedunculopontine nucleus! • Thalamus*

Brain stem • Parabrachial nucleus# • Locus coeruleus*,#

+ Balleine et al., 2007 * Beckstead et al., 1979 - Beier et al., 2015 @ Carr & Sesack, 2000 ~ Gasbarri et al, 1994 & Geisler & Zahm, 2005 ! Jhou et al., 2009a; Jhou et al., 2009b $ Kalivas et al., 1993 ^ Loughlin & Fallon, 1984 ° Poller et al., 2013 # Swanson, 1982

Figure 1.4 - Selected input and output projections of the VTA

Schematic illustrating selected input and output projections to and from the VTA based on information listed in Table 1 and Table 2. Intra-VTA circuitry shown in more detail with three cell type populations (DA, GABA, and Glu). Anterior cingulate cortex (ACC), dorsal striatum (DStr), ventral pallidum (VP), globus pallidus (GP), lateral hypothalamus (LH), lateral habenula (LHb), dorsal raphe (DR), medial prefrontal cortex (mPFC), central amygdala (CeA), tegmental pedunculopontine nucleus (TPP), and nucleus accumbens (NAc).

Anatomy of the nucleus accumbens

The NAc is a brain region that is comprised of >90% GABAergic medium spiny neurons

(MSNs) (reviewed in Kourrich et al., 2015). The NAc receives a major input from the DAergic projections of the VTA with mainly non-overlapping medial-lateral topography (Phillipson &

Griffiths, 1985). Aside from the DAergic inputs from the VTA, the NAc also receives excitatory glutamatergic inputs from the prefrontal cortex (PFC), hippocampus, and amygdala (reviewed in

Kourrich et al., 2015). The NAc projects to the globus pallidus, dorsal striatum, bed nucleus of the stria terminalis, septum, preoptic region, thalamus, lateral habenula, and ventral pallidum (Nauta et al., 1978).

Neuroanatomically, the NAc can be divided into three sub-regions: the lateral NAc shell, the medial NAc shell, and the lateral NAc core. For the purposes of this thesis project, the NAc shell notation refers to the medial NAc shell only (as demonstrated by the verification of cannula placements shown in Figure 3.3). The NAc shell and core also have differential inputs and outputs: the NAc shell shares afferents with the extended amygdala but has stronger outputs towards the cortex, while the NAc core is related more closely to the caudate-putamen (nigrostriatal system) and is thought to be more involved in locomotion (Zahm, 1999; Deutch & Cameron, 1992). More specifically, although both NAc subregions are reciprocally connected to the VTA, the shell strongly projects to the ventromedial part of the subcommissural VP and the pre-optic area/lateral hypothalamus, while the core projects to the dorsolateral VP and the medial substantia nigra (Zahm

& Heimer, 1990; Heimer et al., 1991; Zahm & Heimer, 1993; Zahm, 1999). Anterograde tracing performed by Zahm and colleagues (1993) also showed that the rostral pole of the NAc has differing connectivity: the lateral NAc core-like area projects to the GP, VP, lateral VTA, dorsal

38 pars compacta, entopeduncular nucleus, LH, lateral mesencephalic tegmentum, and central grey, while the NAc shell-like area projects to the VP, the lateral preoptic regions, LH, VTA, dorsal- most pars compacta, retrorubral field, and central grey. To distinguish the NAc shell and core, different methods can be used. For example, baseline concentrations of DA are greater in the NAc shell than the NAc core (Deutch & Cameron, 1992). Immunoreactivity to substance P demonstrates NAc core, while calbindin 28 kD demonstrates NAc shell (Zahm & Heimer, 1993).

Lateral core Medial shell

Lateral shell

Figure 1.5 - Schematic of the nucleus accumbens shell and core

Schematic illustrating the approximate location of the NAc core and shell in a coronal section of the rat brain. The NAc core is bordered by the medial and lateral components of the

NAc shell.

Although the NAc shell and core sub-regions have been established as separate subdivisions for neuroanatomical reasons, these sub-regions have received increased attention due to their functional differences in mediating the motivational properties of drug stimuli, including alcohol, cocaine, morphine and, amphetamine (Pontieri et al., 1995; Ito et al. 2004; Chaudhri et al., 2010). For example, intravenous cocaine, morphine, and amphetamine preferentially increased

DA in the NAc shell over the NAc core in the rat (Pontieri et al., 1995). Additionally, 6-OHDA lesions of the NAc shell reduced nicotine conditioned place preferences, while lesions of the NAc core abolished nicotine conditioned taste aversions (Sellings et al., 2008). The present thesis also finds separable roles in the two sub-regions mediating the opposing motivational effects of caffeine.

1.6 Neurobiology of caffeine motivation

Introduction to caffeine consumption

The history of caffeine use has firm roots in mythology. From Islamic mythology, it is said that the archangel Gabriel offered the prophet Muhammad a “heavenly brew” to help him overcome sleepiness. This “brew” was a cup of coffee so potent that just one sip would enable him to “unhorse 40 men and make 40 women happy.” Though some of the myths are rather outlandish, they all share a common feature where caffeine was found to have stimulating properties

(Fredholm, 2011). Today, caffeine is the most extensively consumed psychoactive drug in the world. It is readily available in food, drugs, dietary supplements, and beverages (Fredholm et al.,

1999; Persad, 2011).

As a central nervous system (CNS) stimulant, caffeine has a broad range of effects on the brain including increased arousal, improved mood, and heightened attention (Lazarus et al., 2011;

Smith, 2009; Brice & Smith, 2002; Lorist et al., 1996). The bulk of these effects have been attributed to its role as a competitive adenosine receptor antagonist (Lazarus et al., 2011; Higgins et al., 2007, Fredholm et al.,1999). Previous work has further suggested that caffeine’s motivationally rewarding or aversive effects also could be acting through antagonism of the adenosine receptors in the brain, more specifically the A1 receptors (A1Rs) located throughout the brain and A2A receptors (A2ARs) located mainly in the ventral striatum (Fredholm et al., 1999).

However, our group has demonstrated that the motivational effects of caffeine demonstrated in a place conditioning paradigm is spared in mouse knockouts (KOs) of A1Rs and/or A2ARs (Sturgess et al., 2010). A1R KO, A2AR KO, and double KO mice showed similar aversion to wild-type (WT) 41

C57BL/6 mice after caffeine (10 mg/kg i.p.) administration in a place conditioning paradigm. This indicates that neither the two adenosine receptors individually nor in tandem are necessary for the motivational effects of caffeine under those conditions. Rather, the DAergic receptors are more likely to be critical for the motivated response to caffeine as pharmacological blockade of DAergic receptors was sufficient to block or reduce caffeine motivation (Sturgess et al., 2010). Of note, a radioligand competitive binding assay conducted by Watanabe and Uramoto (1986) demonstrated that caffeine does not bind directly to DA receptors, yet their measures of DA stimulation showed that caffeine was able to mimic the activity of a DA agonist.

The mechanisms and pathways involved in the midbrain reward circuitry have mainly been investigated through its response after administration of strong stimulants, including amphetamine and cocaine (Ashok et al., 2017; Ito & Hayen, 2011). Though less than 2% of the general population are direct consumers, the motivational effects of these highly addictive drugs are predictable upon consumption (Ashok et al., 2017). For example, acute and short-term administration of cocaine is generally self-reported to be followed by euphoria, increased confidence, and mental alertness (Walsh et al., 2009; van der Poel et al., 2009). Conversely, approximately 90% of people consume caffeine with a large variance in global consumption practices, making it more suitable for investigating the opposing rewarding and aversive behaviours possibly mediated by the mesolimbic pathway (Fredholm et al., 1999).

1.7 Research hypotheses and objectives

The mesolimbic DAergic projection – from the VTA to the NAc – has received great attention as the reward centre of the mammalian brain with mounting evidence for its involvement in processing aversive stimuli (Sellings et al., 2008; Lammel et al., 2012; Kim et al., 2004;

Carlezon & Thomas, 2009). How the same area can elicit opposing approach and avoidance behaviours has been poorly described for caffeine. The downstream NAc is composed of the anatomically and functionally dissociable medial shell and lateral core subregions; these subregions provide one conceivable way by which information encoding approach and avoidance behavior can be separated.

Therefore, it was hypothesized that the separation of the downstream NAc into the shell and core subregions could provide a way for information encoding caffeine reward and aversion as these two contending behaviours can be elicited by other drugs, including nicotine and amphetamine (Sellings et al., 2008; Ito & Hayen, 2011).

Therefore, the overarching objectives of this project were to:

1) Characterize a reliable rodent model of caffeine reward and aversion using the conditioned

place preference paradigm as a basis

2) Target the mesolimbic DAergic pathway with DA receptor antagonists to see which brain

regions were necessary for expression of caffeine reward and aversion

Objective 1: Characterizing a rodent model of caffeine reward and aversion

In this study, I generated a dose-response curve for systemic caffeine ranging from 1 mg/kg to 60 mg/kg i.p. in rats using the place conditioning paradigm. Systemic caffeine was found to elicit reward (indicated by CPP) at a dosage of 2 mg/kg i.p., which could be blocked by prior systemic treatment of α-flupenthixol (α-flu) i.p., suggesting that the behavioural expression of caffeine reward was dependent on DA transmission. Of note, injections of caffeine (100 µM) directly into the rostral VTA was sufficient to induce a comparable CPP magnitude. Systemic caffeine also was found to elicit aversion (indicated by CPA) at tested doses equal or greater than

10 mg/kg i.p, but less than 60 mg/kg i.p. Caffeine aversion could also be blocked by prior systemic treatment of α-flu i.p., suggesting that the behavioural expression of caffeine aversion was also dependent on DA transmission. Of note, injections of caffeine (1000 µM) into the caudal VTA produced aversion, which did not reach statistical significance. However, combined with the aversion seen in the high dose systemic caffeine aversion, it suggests that testing higher concentrations in future studies could potentially yield an expression of aversion of comparable magnitude to systemic caffeine aversion.

Objective 2: Investigating the brain regions responsible for caffeine reward and aversion

Given the essential role that the VTA plays in both drug reward and aversion, I sought to investigate the effects of targeting the VTA with a broad-spectrum DA receptor antagonist by applying it to the rodent model of caffeine and aversion characterized under the first objective. I found that direct injections to the downstream targets of the VTA projections – the NAc shell and core – could separately block caffeine reward and aversion, respectively. Notably, DA antagonism

44 of the NAc shell had no effect on caffeine aversion and DA antagonism of the NAc core had no effect on caffeine reward. This suggests that the two caffeine pathways are separately mediated by the two sub-regions of the NAc.

Chapter 2 - Materials and Methods

2.1 Animal subjects and drugs

Subjects

All animals were handled in accordance with the regulations and guidelines of the

University of Toronto Animal Care Committee, the Animals for Research Act in Ontario, and the

Canadian Council on Animal Care. Adult male Wistar rats (weighing 250-350 g) were purchased from Charles River (Montreal, Canada) and maintained in the animal facilities of the University of Toronto Division of Comparative Medicine. Female rats were not used due to sex differences

(e.g. of the gonadal hormones) that have been demonstrated to impact drug-seeking behaviors in the place conditioning paradigm (Russo et al., 2003). The rats were double-housed with environmental enrichments (plastic tunnel, wooden block, and chew toy) in clear Plexiglas cages at a constant temperature of 22°C on a controlled 12-hour light/dark cycle (lights on at 7am). Rats were given ad libitum access to food and water. Rats did not receive any drugs prior to the first day of conditioning.

Drugs

Caffeine (C0750, Sigma-Aldrich) was dissolved in warm saline (pH adjusted to 7.4). A dose-response curve for the motivational effects of i.p. caffeine injections was generated in the present study with doses ranging from 1 mg/kg to 60 mg/kg. Bilateral intracranial (i.c.) injections of caffeine (100 µM) were performed through surgically implanted cannulas (Plastics One) targeting the VTA at a total volume of 0.5 µl over 1 minute. The injector tip was kept in place for an additional minute to ensure complete diffusion of the drug from the tip of the injector. The

47 caffeine analog, 8-(p-sulfophenyl)theophylline (8-SPT, sc-217511, Santa Cruz), was dissolved in warm saline and administered via i.p injections at a dose of 10 mg/kg only. The DA receptor antagonist, α-flupenthixol (α-flu, F114, Sigma-Aldrich), was dissolved in saline and administered as a pretreatment via i.p. injections (0.8 mg/kg, 2.5 hours prior to conditioning) or bilateral i.c. injections (3 µg/0.5 µl per hemisphere in the NAc, 15 minutes prior to conditioning). The doses and timings of injection were selected based on previous studies showing that α-flu could antagonize postsynaptic DA (both D1 and D2 subtypes) receptors (Creese et al., 1976), but without producing any motivational effects of its own (Laviolette & van der Kooy, 2003).

2.2 Experimental procedures

Surgical cannulation

To target specific brain regions, rats were surgically cannulated under anesthesia (2-5% isoflurane) and injected with analgesic subcutaneously (ketoprofen, 5 mg/kg s.c.). Rats were shaved and then placed in a stereotaxic frame for craniotomy. Small burr holes were drilled into the skull based on coordinates (in mm from Bregma) for the rostral VTA (AP: -5.3, ML: ± 0.5, and DV: -7.6), NAc shell: +1.3, ML: ±1.0, and DV:-7.4) or NAc core (AP:+1.3, ML:± 2.4, and

DV:- 7.4) (Olson et al., 2005; Paxinos & Watson, 2005; Olson & Nestler, 2007). Stainless steel guide cannulas (C313G(2)-G11/SP, Plastics One) were fixed permanently into position using dental acrylic and jeweler’s screws (0-80 X 3/32, Plastics One) implanted in the skull, with the tips positioned 1 mm above the target brain region. Removable dummy cannulas (C313DC/1/SPC,

Plastics One) were inserted into the guide cannulas to prevent occlusion over the course of surgical recovery and behavioral experiments. Rats were given at least one week to recover from their surgeries before further experimentation. Following which, removable internal cannulas

(C313I/SPC, Plastics One) were inserted into the guide cannulas with the tips of the internal cannulas extending 1 mm ventral to the guide cannula tips during conditioning.

Place conditioning

Prior to place conditioning, the rats were habituated for 20 minutes in a neutral, grey chamber (41 cm x 41 cm x 38 cm box) to reduce novelty effects. Rats that had received surgical cannulations received i.c. injections of saline vehicle on the following day to habituate the rats to 49 i.c. injections. The rats then underwent an unbiased place conditioning procedure, which was counterbalanced in terms of drug-place pairing and day of first drug exposure. Conditioning alternated between two place conditioning boxes (41 cm x 41 cm x 38 cm boxes) that were distinct in colour, floor texture, and scent. One box had black walls, a smooth black Plexiglas floor, and was scented with ~0.3 mL of glacial acetic acid (AX0073, EMD) solution. The other box had white walls, a metal grid wire floor over a smooth metallic surface, and remained unscented. The amount of glacial acetic acid was determined by testing a separate group of rats from the same cohort to ensure that there was no initial baseline preference. During the conditioning period, rats were pretreated with α-flu or vehicle solution (administered i.p. or i.c. where appropriate). Following pretreatment, caffeine was administered to the rats i.p. or i.c. immediately before exposure to a conditioning box for 40 minutes, which is a sufficient period to capture peak caffeine levels in the brain following i.p. injections (Latini et al., 1978; Lukas et al., 1971). The drug and vehicle solutions were administered on alternate days over a period of 8 days for a total of 4 drug-place pairings and 4 vehicle-place pairings. At least 48 hours following the final conditioning session, rats were tested in a rectangular testing box consisting of two compartments with features identical to the conditioning boxes separated by a neutral, grey zone. On testing day, rats were placed on the centre grey zone and allowed to roam freely in the testing box for 10 minutes. All rats were tested in a drug-free state. The amount of time spent on each side of the testing box was recorded.

A greater amount of time spent in the previously drug-paired compartment indicates a conditioned place preference, while a greater amount of time spent in the previously vehicle-paired side indicates a conditioned place aversion.

2.3 Histology

Verification of cannula placements

Following testing, rats were deeply anesthetized with 54 mg/kg i.p. of sodium pentobarbital

(Ceva Santé Animale). Rats were perfused with ~100ml of phosphate-buffered saline (PBS), followed by ~100ml of 4% paraformaldehyde (PFA, P6148, Sigma-Aldrich) dissolved in PBS.

Brains were removed and preserved in 4% PFA solution overnight, and then transferred to a 30% sucrose (BioShop) solution stored at 4°C. Coronal sections of flash-frozen brains were cut at a thickness of 40 µM with a freezing microtome (Cryostat model Hm525 NX, Thermo Scientific).

The sectioned slices were mounted on slides and stained with Cresyl violet (C5042, Sigma-

Aldrich). The slides were viewed under a light microscope to verify the bilateral cannula placements in the NAc and VTA.

2.4 Statistical Methods

Statistical analysis

For the systemic caffeine dose-response curve, a two-way analysis of variance (ANOVA) followed by post hoc Bonferroni-Dunn multiple comparison tests were conducted using GraphPad

Prism. Comparisons of the mean difference scores to zero were conducted with one-sample t-tests with p-values adjusted using the Bonferroni correction. Results are displayed as mean difference scores (time spent in caffeine-paired minus vehicle paired environment) ± the standard error of the means (SEMs). A two-way ANOVA followed by post hoc Bonferroni-Dunn multiple comparison tests were conducted for the experiment comparing the effects of 8-SPT and caffeine, with results displayed as the mean amount of time that animals spent on each side of the testing apparatus ±

SEM. For the experiments comparing intra-VTA injections with 100 µM and 1000 µM of caffeine, three-way ANOVAs (Type III tests) were conducted using R statistical software, followed by post hoc Bonferroni-Dunn multiple comparison tests. Results are displayed as the mean amount of time that animals spent on each side of the testing apparatus ± SEM. Two-way ANOVAs followed by post hoc Bonferroni-Dunn multiple comparison tests also were conducted using GraphPad Prism for the experiments comparing the effects of vehicle and α-flu in the NAc core and in the NAc shell, with results displayed as the mean amount of time that animals spent on each side of the testing apparatus ± SEM. Animals with mistargeted cannula placements (e.g. mistargeted in one hemisphere and complete misses, n = 8 in total) were excluded from statistical analyses.

Differences were considered significant if the p-value < 0.05.

Chapter 3 - Results

3.1 Systemic caffeine dose-response curve

Systemic caffeine elicits reward at low doses and centrally-mediated aversion at higher doses

To examine the behavioral response to systemic caffeine in male rats, a place conditioning paradigm was used (Figure 3.1a). First, a separate group of adult male rats were tested for baseline bias towards either conditioning box type to ensure that changes in preference were due only to drug-place conditioning (Figure 3.1b). Following this, a dose-response curve for systemic i.p. injections of caffeine using doses ranging from 1 mg/kg to 60 mg/kg was generated (Figure 3.1c).

A two-way ANOVA indicated that there was an interaction between caffeine dose and pretreatment, confirming that α-flu pretreatment can disrupt the motivational effects of caffeine

(Figure 3.1c). A low dose of caffeine (2 mg/kg i.p.) produced conditioned place preferences, while caffeine doses of 10 and 30 mg/kg i.p. produced conditioned place aversions (Figure 3.1c). The difference scores between the time that the animals spent in the caffeine-paired side minus the saline-paired side at doses of 10 and 30 mg/kg i.p. were not significantly different (Figure 3.1c).

Therefore, the dose of 10 mg/kg i.p. was used for the remainder of this study when examining the aversive effects of caffeine.

We hypothesized that caffeine aversions would diminish in response to DA receptor antagonism based on our previous work showing this to be the case in mice (Sturgess et al., 2010).

Comparable to the results seen in mice, a DA receptor antagonist (α-flu, 0.8 mg/kg i.p.) blocked the conditioned place aversions produced by 10 and 30 mg/kg i.p of caffeine in rats (Figure 3.1c).

The pretreatment with α-flu also blocked the conditioned place preferences when a low dose of 2

54 mg/kg i.p. of caffeine was administered (Figure 3.1c). Thus, acute caffeine reward and aversion in rats as elicited by systemic administrations of low and high doses, respectively, are dependent on

DAergic transmission.

Next, we asked whether the aversive effects of high doses of systemic caffeine were solely mediated by the CNS or if caffeine-responsive adenosine receptors in the peripheral nervous system also contributed. A quaternary caffeine analog (8-SPT) that is unable to cross the blood- brain barrier was administered at 10 mg/kg i.p. in an identical place conditioning paradigm as the systemic caffeine experiment described above. Despite 8-SPT having greater binding affinity to adenosine receptors than theophylline, a methylxanthine with similar binding properties to caffeine (Tao & Abdel-Rahman, 1993), there was no significant difference in the time that the rats spent in the vehicle- or 8-SPT-paired chambers at testing, suggesting that peripheral activation of adenosine receptors alone at 10 mg/kg i.p. does not contribute to the behavioral expression of caffeine aversions (Figure 3.1d).

C) *

* *

Figure 3.1 - Systemic caffeine produces DA-dependent rewarding effects at low doses, and

DA-dependent aversive effects at high doses that are centrally mediated

a) Schematic of place conditioning paradigm and timeline.

b) Initial preference for each side of the testing apparatus was found to be similar in a separately tested group of animals (t-test, p = 0.7795, n = 12). These animals only received saline injections during the conditioning phase. Bars represent mean amount of time animals spent on each side of the testing apparatus ± SEM.

c) In the systemic caffeine dose-response curve, a two-way ANOVA revealed a significant dose by pretreatment interaction, confirming that α-flu pretreatment (gray bars) disrupted the motivational effects of caffeine (F4,109 = 7.990, p = 0.0001). At a dose of 2 mg/kg i.p of caffeine, multiple comparison tests (Bonferroni) of the mean difference scores to zero indicated that the non-pretreated animals (black bars) experienced a significant caffeine reward (p = 0.001, n = 13),

57 which was blocked by α-flu pretreatment (p = 0.9999, n = 13). At 10 mg/kg i.p. (p = 0.001, n =

24) and 30 mg/kg i.p. (p = 0.001, n = 11), caffeine produced significant aversions, which were not significantly different from each other (p = 0.9999). These aversions could also be blocked by α- flu pretreatment (10mg/kg: p = 0.001, n = 24; 30mg/kg: p = 0.001, n = 11). Bars represent difference scores between the times animals spent in the caffeine-paired side minus vehicle-paired side in the testing apparatus ± SEMs. * indicates p < 0.05 for comparisons of mean difference scores compared to zero.

d) Animals treated with caffeine (10mg/kg i.p.) showed aversions that were not recapitulated by the administration of 8-SPT (10 mg/kg i.p.), as indicated by an interaction between the type of drug administered (caffeine or 8-SPT) and drug-pairing to each side of the testing apparatus (two- way ANOVA, F1,26 = 20.1, p = 0.0001). In the caffeine group, animals spent more time in the vehicle-paired side compared to the drug-paired side (p = 0.0002, n = 8), indicating an aversion to caffeine. In the 8-SPT group, the amount of time that the animals spent in each side was not significantly different (p = 0.8485, n = 7). Bars represent mean amount of time animals spent on each side of the testing apparatus ± SEMs. * indicates a significant difference between the time spent in the vehicle-paired and drug-paired sides of the testing apparatus (p < 0.05)

3.2 Intra-rVTA caffeine reward mediated through NAc shell

Caffeine injections into the rostral VTA elicit reward

To determine where caffeine motivation may be mediated within the brain, the VTA was chosen for its well-established role as part of a drug motivation circuit (Nader & van der Kooy,

1997; Laviolette & van der Kooy, 2003; Nestler, 2005). Caffeine was directly injected into the rat

VTA immediately preceding place conditioning. Preliminary results suggested that the rostral, but not caudal VTA, produced reward so these regions were targeted separately. Rostral VTA (rVTA) and caudal VTA (cVTA) were defined as in Olson and colleagues’ study (2005) showing that there is an inflection point between -5.5mm and -5.6 mm Bregma where cocaine is found to be rewarding rostrally and aversive caudally. This rVTA area is defined to include the anatomical VTA subregions, composed of the rostral subregion and the rostral portion of the parabrachialis pigmentosus (Paxinos & Watson, 2005). Indeed, we found that intra-VTA caffeine injections produced conditioned place preferences when specifically targeted to the rVTA (Figure 3.2a).

Given that the VTA is comprised of a majority of DAergic neurons and that systemic caffeine at 2 mg/kg i.p. was able to produce DA-dependent reward, we injected rats with α-flu (0.8 mg/kg i.p) to determine if DA activity was also required for intra-rVTA caffeine reward. α-flu blocked the rewarding effects of caffeine produced by intra-rVTA injections, suggesting that rVTA caffeine at a 100 µM concentration is behaviorally and mechanistically equivalent in the place conditioning paradigm to 2 mg/kg i.p. of caffeine. It seems unlikely that this blockade is due to non-specific effects of α-flu, as this same dose of α-flu does not block the rewarding effects of

59 opiates or food in previously drug-naïve or food-sated animals, respectively (Laviolette et al.,

2002; Bechara et al., 1992).

Unlike the intra-rVTA caffeine reward, injections of 100 µM caffeine directly into the cVTA had no effect on motivation with or without pretreatment of α-flu i.p. (Figure 3.2a).

However, injections of a higher concentration of caffeine (1000 µM) into the cVTA produced a trend towards caffeine aversion (Figure 3.2b). Though this result was not significant, it is consistent with the mounting evidence for the role of the cVTA in mediating aversive motivational responses (Balcita-Pedicino et al., 2011; Bourdy & Barrot, 2012; Sanchez-Catalan et al., 2017).

Figure 3.2 - Caffeine reward is mediated through the rostral, not caudal, VTA a) Schematic of place conditioning paradigm and timeline.

b) Animals given caffeine at three concentrations of 10, 100, and 1000 µM nonspecifically in the

VTA did not result in any rewarding or aversive effects as indicated by the lack of CPP and CPA expression, respectively (p > 0.05).

c) A significant interaction (three-way ANOVA, F1,68 = 6.0274, p = 0.01665) of pretreatment

(vehicle i.p. vs. α-flu i.p.) by drug pairing (vehicle- vs. caffeine-paired) by VTA subregion (rVTA vs. cVTA) was revealed when caffeine was injected at a concentration of 100 µM. Multiple comparison tests indicated that caffeine (100 µM) directly injected into the rVTA only could elicit a significant reward (p = 0.0009, n = 10) that could be disrupted by α-flu pretreatment (p = 0.9999, n = 12). In the cVTA groups, animals spent similar amounts of time in both sides of the testing apparatus without (p = 0.9999, n = 9) or with (p = 0.9999, n = 7) α-flu pretreatment. Bars represent mean amount of time animals spent on each side of the testing apparatus ± SEMs. * indicates a significant difference between time spent in the vehicle-paired and caffeine-paired sides of the testing apparatus (p < 0.05).

d) Conversely, when caffeine was injected at a higher dose of 1000 µM in the VTA, there was no interaction between pretreatment, drug pairing and VTA subregion (three-way ANOVA, F1,62 =

2.0414, p = 0.1581). Multiple comparison tests indicated that the time that animals spent in each side of the testing apparatus was similar in the rVTA groups without (p = 0.9999, n = 13) or with

(p = 0.9999, n = 6) α-flu i.p. pretreatment. In the cVTA groups, animals that had received caffeine

63 without α-flu i.p. pretreatment showed a nonsignificant aversion to caffeine (p = 0.1510, n = 7), whereas those with α-flu i.p. pretreatment spent similar amounts of time in both the vehicle- and caffeine-paired sides (p = 0.9999, n = 9). Bars represent mean amount of time animals spent on each side of the testing apparatus ± SEMs.

Intra-rVTA caffeine reward is mediated through the NAc shell, but not core

Having determined that rVTA DA neurotransmission is necessary for caffeine reward, the roles of the downstream NAc subregions were investigated. While increased DA levels in the NAc shell are associated with increased caffeine locomotion and arousal (Solinas et al., 2002; Lazarus et al., 2011), the role of both NAc subregions in the DA-mediated motivational effects of caffeine is unknown. Cannulas were implanted in the rVTA and NAc, targeting either the shell or core

(placements shown in Figure 3.3). To determine which NAc subregion is necessary for the manifestation of caffeine reward, we injected α-flu into either the shell or core, followed by an injection of caffeine into the rVTA. Intra-NAc shell injections of α-flu blocked the intra-rVTA caffeine reward (Figure 3.4a). Conversely, this effect was not observed when α-flu was injected into the NAc core (Figure 3.4b). This suggests that the rVTA-NAc shell pathway is necessary for

DA-dependent caffeine reward.

NAc rVTA

Figure 3.3 - Verification of bilateral cannula placements

Bilateral cannula tips targeted at the medial NAc shell (●), lateral NAc core (○), and rostral

VTA for data shown in Figure 4. The outlines shown in the rVTA slices represent the anatomical

VTA subregions, composed of the rostral VTA (VTAR) and the rostral portions of the parabrachialis pigmentosus. For Figure 3.4a) and b), caffeine was injected in the rVTA following

α-flu or vehicle pretreatment in the intra-NAc shell or intra-NAc core. For Figure 3.4c) and d), caffeine was administered i.p. following α-flu or vehicle pretreatment in the NAc shell or NAc core. Shown in Figure 3.3 are coronal slice reconstructions based on the stereotaxic atlas of

Paxinos and Watson (2005) with reference to Bregma. Numbers indicate the approximate distance from Bregma in millimetres (mm).

Figure 3.4. Intra-rVTA caffeine reward is DA-dependent and mediated centrally through the NAc shell, but not the core

a) A two-way ANOVA indicated that animals treated with intra-rVTA caffeine (100µM) experienced caffeine reward that could be blocked by α-flu pretreatment within the NAc shell

(NAcSh in graph), as indicated by an interaction of pretreatment and drug pairing (F1,26 = 5.31, p

= 0.0294). Multiple comparison tests revealed that animals that had received caffeine in the rVTA without α-flu pretreatment spent more time in the caffeine-paired side than the vehicle-paired side

(p = 0.0051, n = 8). In the intra-NAc shell α-flu pretreatment group, animals spent similar amounts of time in both sides of the testing apparatus (p = 0.9999, n = 7). Bars represent mean amount of time animals spent on each side of the testing apparatus ± SEMs. * indicates a significant difference between time spent in the vehicle-paired and caffeine-paired sides of the testing apparatus (p < 0.05).

b) In separate groups of animals treated with intra-rVTA (100µM) caffeine, a two-way ANOVA did not reveal an interaction between pretreatment and drug pairing (F1,26 = 0.4572, p = 0.5049).

Rather, animals that had received intra-rVTA caffeine exhibited preference for caffeine regardless of α-flu pretreatment within the NAc core (NAcC in graph), as indicated by a main effect of caffeine administration (F1,26 = 70.30, p = 0.0001). Multiple comparison tests confirmed that animals spent more time in the caffeine-paired than the vehicle-paired side in the vehicle pretreatment group (p = 0.0001, n = 7) and the α-flu pretreatment group (p = 0.0001, n = 8). Bars represent mean amount of time animals spent on each side of the testing apparatus ± SEMs. * indicates a significant difference between time spent in the vehicle-paired and caffeine-paired sides of the testing apparatus (p < 0.05).

3.3 High dose-systemic caffeine aversion mediated by NAc core

Systemic caffeine aversion is mediated centrally through the NAc core, but not shell

Given the evidence above showing that the NAc core is not necessary for caffeine reward in the conditioned place preference paradigm but is implicated in nicotine taste aversions (Sellings et al., 2008), we hypothesized that the core would mediate caffeine aversions. As higher doses of systemic caffeine produced conditioned place aversions that were blocked by systemic α-flu, rats were pretreated with injections of α-flu targeting the NAc core prior to place conditioning. α-flu blocked the aversions produced by 10mg/kg i.p. of caffeine, indicating that the NAc core specifically mediates caffeine aversion in a DA-dependent manner (Figure 3.4d). Additionally, we tested whether the rVTA-NAc shell reward pathway influenced the NAc core-mediated aversive pathway. We did not observe a block of conditioned place aversions to systemic caffeine in rats pretreated with α-flu targeted to the NAc shell (Figure 3.4c). As our earlier findings indicated that caffeine reward may be mediated by the rVTA-NAc shell pathway, and not the NAc core, the data in this section serve to double dissociate functionally segregated caffeine reward and aversion pathways in the NAc shell and core, respectively.

Figure 3.5. Caffeine aversion is mediated centrally through the NAc core, but not the shell

a) For animals treated with systemic caffeine (10 mg/kg i.p.), a two-way ANOVA did not reveal an interaction between intra-NAc shell α-flu pretreatment and drug pairing (F1,26 = 0.02237, p =

0.8822). Rather, animals that were given systemic caffeine experienced significant aversions to caffeine regardless of pretreatment condition, as indicated by a main effect of caffeine administration (F1,26 = 45.37, p = 0.0001). Multiple comparison tests confirmed that animals spent more time in the vehicle-paired side than the caffeine-paired side in the vehicle pretreatment group

(p = 0.0004, n = 7) and α-flu pretreatment group (p = 0.0003, n = 8). Bars represent mean amount of time animals spent on each side of the testing apparatus ± SEMs. * indicates a significant difference between time spent in the vehicle-paired and caffeine-paired sides of the testing apparatus (p < 0.05).

b) Animals treated with caffeine i.p. (10mg/kg) exhibited caffeine aversions that could be blocked by pretreatment with intra-NAc core α-flu, as indicated by an interaction between pretreatment and drug-pairing to each side of the testing apparatus (two-way ANOVA, F1,28 = 15.21, p =

0.0005). Multiple comparisons tests revealed that the animals that had received systemic caffeine in the intra-NAc core vehicle pretreatment group spent significantly less time in the caffeine-paired side than the vehicle-paired side (p = 0.0001, n = 6). In the intra-NAc core α-flu pretreatment group, animals did not spend significantly different amounts of time in the two sides of the testing apparatus (p = 0.9999, n = 10). Bars represent mean amount of time animals spent on each side of the testing apparatus ± SEMs. * indicates a significant difference between time spent in the vehicle- paired and caffeine-paired sides of the testing apparatus (p < 0.05).

Chapter 4 - General Discussion

4.1 Overview

A fundamental task for the mammalian brain is to assign motivational valence to stimuli to determine whether to approach a reward or avoid an aversive stimulus. Although VTA DA neurons are known to play a key role in processing various motivational stimuli, the roles of mesolimbic DA regarding acute caffeine’s motivational effects are not yet understood. We have separately elicited conditioned caffeine place preferences through low dose systemic injections and intra-rVTA caffeine, or conditioned caffeine place aversions through higher dose systemic injections by performing in vivo administration of caffeine and pharmacological blockade of DA receptors in a place conditioning paradigm. These two motivational effects were blocked by α-flu in the NAc shell or core, suggesting that there are two separate and double dissociated DA- dependent pathways that mediate caffeine reward and aversion, respectively.

Neuroanatomy of caffeine aversion

Previous work in mice demonstrated that doses of 10 and 30 mg/kg i.p. of caffeine produced high aversions (Sturgess et al., 2010), which was recapitulated in rats in the present study. The present study demonstrates that caffeine aversion is mediated within the CNS, as a quaternary caffeine analogue restricted to the peripheral nervous system did not have any motivational effect despite its greater binding affinity to adenosine receptors (Tao & Abdel-

Rahman, 1993). Further, this aversion is mediated by DA activity in the NAc core, as both systemic and direct intra-NAc core injections of α-flu were sufficient to block the aversions.

Given that the lipid solubility of caffeine allows it to pass through membranes easily and that systemic caffeine should reach both subregions of the NAc, it is unclear why the NAc core- mediated aversion is the effect that prevails at higher caffeine doses. It may be that the 10 and 30 mg/kg i.p. doses of caffeine reach higher concentrations in the VTA than does 100 µM of caffeine injected directly into the VTA and thus, overshadow the rewarding effects of rVTA activation, resulting in an overall aversion. This could be through a cVTA-mediated mechanism, which includes the tail of the VTA (also known as the rostromedial tegmental nucleus). The tail of the

VTA (tVTA) is composed of a larger proportion of GABA neurons than DA neurons and is reported to act as a brake to the midbrain DA systems, which is important for avoidance behaviors elicited by the LHb (Balcita-Pedicino et al., 2011; Bourdy & Barrot, 2012; Lammel et al., 2012;

Sanchez-Catalan et al., 2017). Higher doses of caffeine could preferentially activate the tVTA, leading to a brake on the DA-mediated caffeine reward elicited at lower doses in the proposed rVTA-NAc shell reward pathway (Balcita-Pedicino et al., 2011; Bourdy & Barrot, 2012).

Consistent with this notion, Kaufling and colleagues (2010) quantified the number of

FosB/ΔFosB-positive nuclei (early gene indicating cell activation) following caffeine exposure and found that doses equal or higher than 10 mg/kg i.p. induced a significant increase in recruited

FosB/ΔFosB-positive nuclei in the tVTA as compared to saline exposure. Given that the GABA- rich neurons in the tail of the VTA provide inhibitory input to VTA DA cells (Matsui & Williams,

2011), this would also predict a lower activation of FosB/ΔFosB in the rVTA. This suggests that caffeine reward elicited through the rVTA-NAc shell reward pathway would be inhibited when caffeine is administered at high doses. Given that aversions (though not statistically significant) were seen with caffeine injections into the cVTA, it also is possible that caffeine may work in

74 other areas than the VTA; areas which then project to the NAc core to interact with DA mechanisms there.

However, at the highest dose of 60 mg/kg i.p. of caffeine, rats did not experience reward or aversion. A secondary rewarding process could be counteracting the expected higher aversion, but a more likely explanation is that higher caffeine doses may be more stressful to the animal. A high dose of caffeine is known to cause the body temperature of an animal to rise, which indicates physiological stress (Pechlivanova et al., 2010). In response to stress, animals become anhedonic, reduce locomotor activity, and reduce exploratory activity (Rygula et al., 2005). It is conceivable that these behavioral changes that represent a loss of interest may interfere with learning about the drug-place cues during conditioning.

Neuroanatomy of caffeine reward

In contrast to the acute caffeine aversions that were elicited systemically by a wide range of high doses, caffeine reward was only seen at a restricted low dose range, as reported by others

(Brockwell et al, 1991). Further, caffeine reward is dependent on activity localized to a specific subregion of the VTA. There is ample evidence for heterogeneity along the rostrocaudal axis of the VTA from anatomical and behavioural studies (Olson et al., 2005; Ikemoto, 2007; Olson &

Nestler; 2007; Yamaguchi et al., 2007). Our findings that caffeine can elicit reward only in the rVTA align closely with the work of Olson and colleagues (2005) who administered cocaine to the rostrocaudal subregions of the VTA and found that only the rVTA could elicit cocaine reward

75 in the place conditioning paradigm. As caffeine reward was blocked by α-flu in the NAc shell, the rVTA DA neurons are the most likely candidates for mediating reward.

The functional distinction along the rostrocaudal axis could be explained if the rVTA DA neurons project more to the NAc shell and the cVTA neurons project more to the NAc core.

However, it remains unclear whether this is the case as anatomical labeling studies often tend to examine the NAc or VTA, without specifically distinguishing subregions in both at the same time.

Brog and colleagues (1993) did find that the VTA projects to the NAc with mediolateral topography, evidenced by retrograde Fluoro-Gold injections in the medial NAc shell resulting in dense labeling of the more medial portions of the VTA subnuclei. However, a more recent retrograde labeling study conducted by Rodriguez-Lopez and colleagues (2017) distinguished both

VTA and NAc subregions more clearly. Of the retrograde tracer injections that targeted the NAc shell or core without overlap, they found that more cells with initial injections in the NAc shell than the NAc core were positive for TH (marker for DA) in the rostral VTA (VTAR) – an anatomical VTA subregion that, along with the rostral parabrachialis pigmentosus, comprises the rVTA defined in the present study. However, further studies will be required to anatomically determine more quantitatively the strength of the specific connections from the rVTA to NAc shell versus NAc core.

The functional heterogeneity of the accumbens is arguably more well-defined than that of the VTA (Pontieri et al., 1995; Jones et al., 1996; Ikemoto & Panksepp,1999; Di Chiara, 2002;

Solinas et al., 2002; Carlezon & Thomas, 2009). For example, the rewarding and aversive effects of nicotine have been found to also be segregated in the NAc shell and core, respectively (Sellings

76 et al., 2008). Intravenous cocaine self-administration also preferentially increased levels of DA in the NAc shell over the NAc core (Pontieri et al., 1995). Although these functional studies support our findings for caffeine, it remains to be seen whether these functional distinctions arise due to dissimilar patterns of connectivity between the rVTA and the cVTA.

Caffeine and other drugs of abuse

Caffeine consumption is further complicated by its co-occurrence with other licit substances with addictive properties, including tobacco and alcohol (Istvan & Matarazzo, 1984).

Istvan and Matarazzo (1984) found that the use of both caffeine and nicotine (the active ingredient in tobacco) is more strongly correlated than caffeine paired with alcohol. There is an abundance of evidence implicating the mesolimbic DA neurons in nicotine use with DA loss-of-function manipulations (via lesions, pharmacological blockade, or genetic knockouts) disrupting expression of motivated behaviors (Clarke & Pert, 1985; Corrigall et al., 1992; Picciotto et al., 1998;

Laviolette & van der Kooy, 2003; Sellings et al., 2008). More specifically, Picciotto and colleagues

(1998) found that activation of the β2 subunit-containing nicotinic acetylcholine receptors is necessary for the reinforcing properties of nicotine. However, as caffeine has a low binding affinity to nicotinic receptors (Reavill et al., 1988), caffeine’s motivational effects are unlikely to be produced through the same mechanism.

In contrast to nicotine, acute ethanol reward in rodents is more similar to acute caffeine reward in that both are mediated by mesolimbic DA neurons in a place conditioning paradigm

(Ting-A-Kee et al., 2013). However, for ethanol, the state of drug dependence must also be

77 considered. When rodents become dependent on ethanol and are then deprived, rodents given ethanol are thought to experience a rewarding effect due to alleviation from the negative/aversive feelings of deprivation (Ting-A-Kee et al., 2013). The acute ethanol reward is thought to be mediated through the mesolimbic DA pathway before ‘switching’ to a DA-independent mechanism mediated by the TPP when rodents experience ethanol reward in a deprived state

(Ting-A-Kee et al., 2013). Though Sturgess and colleagues (2010) found that the TPP is not involved in acute caffeine reward in mice, this does not rule out the possibility that the TPP may be necessary for the reward experienced by caffeine-deprived rodents. Further investigation into the shared mechanisms between caffeine, nicotine, and alcohol are required to shed light on whether these mechanistic similarities confer additive addictive properties that could help to explain their prevalent use in contemporary society.

Caffeine and the role of adenosine receptors

In addition to the functional heterogeneity within the VTA and NAc, the specific role of receptors and mechanisms for caffeine motivation remains unknown. Caffeine can act through a variety of biochemical mechanisms (reviewed in Fredholm et al., 1999). Low millimolar concentrations of caffeine causes direct release of intracellular calcium and influences 5’- nucleotidases and alkaline phosphatases, while higher concentrations (greater than that achieved during human consumption) inhibit cyclic phosphodiesterases (McPherson et al. 1991; Smellie et al., 1979; reviewed in Nehlig & Debry, 1994). At the levels generally assumed to be reasonable for human consumption patterns of caffeine, caffeine likely acts as an adenosine receptor antagonist (reviewed in Nehlig & Debry, 1994). For example, caffeine’s widespread arousal effects can be attributed to its role as an adenosine receptor antagonist. However, the adenosine

78 receptors are unlikely to be directly involved in caffeine motivation as A1R KO, A2AR KO, and double KO mice behave similarly to WT mice in an analogous caffeine place conditioning paradigm (Sturgess et al., 2010). Adenosine receptors form heterodimers with DA receptors and directly inhibit DA receptors in the striatum (Chen et al., 2001). Therefore, the loss of adenosine receptor inhibition likely results in greater DA receptor activation and upregulation of DA receptors in the ventral striatum (Chen et al., 2001; Volkow et al., 2015). As caffeine does not directly bind to DA receptors (Watanabe & Uramoto, 1986), caffeine may act through a downstream mechanism that affects DA transmission. The present results also provide evidence for the specific involvement of the DA mesolimbic pathway in caffeine motivation. However, the present results do not rule out the possibility of an unknown receptor mediating caffeine motivation.

Caffeine pharmacokinetics and metabolism

In humans and animals, the hydrophobic caffeine molecules can freely bypass through all biological membranes (reviewed in Fredholm, 1999). It is rapidly absorbed from the gastrointestinal tract with peak levels at around 30-60 minutes in plasma (Marks & Kelly, 1973;

Kaplan et al., 1997; reviewed in Fredholm, 1999). Caffeine (also known as 1,3,7- trimethylxanthine) is metabolized by the liver to form dimethylxanthines (e.g. theophylline, theobromine, and paraxanthine), monomethylxanthines, and other derivatives with lower pharmacological activity (Kaplan et al., 1997; reviewed in Fredhom, 1999). Salivary and plasma levels of caffeine can be measured using high-pressure liquid chromatography (HPLC) or through a radioimmunoassay procedure (Cook et al., 1976). In HPLC, the sample mixture containing

79 caffeine is pumped through a column that is filled with a known solid adsorbent material. Each component in the sample mixture interacts differently with the solid material, leading to separation of components based on varying flow rates. Cook and colleagues (1976) found that both procedures were comparable with caffeine being equally distributed between saliva and plasma.

Additionally, they found that caffeine had a half-life of about 4 hours (Cook et al., 1976).

Subjective effects of caffeine can also be quantified using electroencephalogram (EEG), as doses of 250 mg and 500 mg were able to produce reductions in alpha waves, beta waves, theta waves, and total wave amplitudes (Kaplan et al., 1997). However, these measures are more variable than

HPLC and radioimmunoassay. It is known that lower doses of caffeine in humans have pleasant stimulant effects and higher doses produce aversive anxiogenic effects (Kaplan et al., 1997). This also is seen with the rats in the present study, so it would be of interest to use these methods to measure caffeine in a comparative study between animals and humans.

Caffeine use by people

The human consumption patterns of caffeine, for which caffeine reward is likely heightened by its general pairing with other rewarding substances like sugar, social environments, tobacco, or alcohol (Istvan & Matarazzo, 1984; Keast et al., 2015), might explain some of the human versus non-human species differences found in substance abuse studies. Direct comparisons of caffeine consumption between humans and rodents ought to be made with caution and with consideration of the higher metabolic function in rodents. Nonetheless, Fredholm and colleagues (1999) estimated that a dose of “10 mg/kg in a rat represents about 250 mg of caffeine in a human weighing 70 kg (3.5 mg/kg), and that this could correspond to about 2 to 3 cups of

80 coffee.” Lao-Peregrin and colleagues (2016) had a similar estimate of 5 to 7 mg/kg of caffeine representing 1 energy drink with high caffeine content or 3 cups of coffee. Extrapolating from these two estimates would suggest that 1 cup of coffee is roughly equivalent to 2.3 mg/kg to 5 mg/kg in rodents. Although the lower end of the estimate is higher than the 2 mg/kg seen to produce reward in rats and many people certainly consume greater amounts than 1 cup of coffee, the duration in which the caffeine is administered also must be considered. While caffeine was administered to rats in a single injection, people generally consume their caffeine over a period, which would arguably result in people experiencing a lower dose of caffeine (over a greater number of consumption sessions). Therefore, in humans experiencing caffeine reinforcement, caffeine consumption may be tightly regulated with minor variation in intake amount such that

100µM intra-rVTA caffeine or a 2 mg/kg i.p dosage in rats may be comparable to the level of caffeine, which is rewarding in humans (Nehlig, 1999; Lawson et al., 2004).

In 1976, Cook and colleagues conservatively estimated that people in the United States consumed at least 10 million kilograms of caffeine from coffee. In 1999, Fredholm and colleagues found that over 90% of people consumed caffeine. The prevalence of caffeine consumption appears to have remained stable as one recent study showed that an astonishing 89% of adults in the United States still consume caffeine regularly (Fulgoni et al., 2015). Despite this, very little is known about the neurobiological mechanisms that lead to caffeine consumption. Our data suggest that caffeine’s motivational effects are mediated by mesolimbic dopaminergic pathways, as is seen with other psychoactive drugs (Ikemoto & Panksepp, 1999; Laviolette & van der Kooy, 2003;

Grieder et al., 2010, Wise & Morales, 2010; Lammel et al., 2011; Ting-A-Kee et al., 2013; Ashok et al., 2017). In fact, given the early evidence that caffeine consumption is correlated with tobacco

81 and nicotine use (Friedman, 1974; Istvan & Matarazzo, 1984), a question that warrants further study is whether caffeine’s reliance on the same neural systems as these drugs of abuse can explain the extraordinary prevalence of its use.

4.2 Conclusions

Due to its ready availability in dietary sources, caffeine is the most commonly consumed psychoactive drug. Though considerable effort has been made to investigate the mechanisms underlying caffeine’s arousal and mood-enhancing properties, far less is known about the neural substrates mediating its motivational properties. Using a rodent place conditioning model, we demonstrate that caffeine’s rewarding and aversive properties are mediated downstream of the

VTA by two functionally dissociable pathways. Specifically, caffeine reward is contingent upon

DA activity in the NAc shell, whereas caffeine aversion is contingent upon DA activity in the NAc core.

4.3 Future Directions

What could account for the differences seen between mice and rats?

Though 10 and 30 mg/kg i.p. doses of caffeine reliably produced caffeine aversion in mice and rats, caffeine reward was only seen in rats at a low dose of 2 mg/kg i.p. This is consistent with other rat studies showing that caffeine produces biphasic low dose reward and high dose aversion

(Patkina et al., 1998; Brockwell et al., 1991). However, in the mouse study conducted by Sturgess and colleagues (2010), they did not observe caffeine reward at lower doses. The discrepancy between mice and rats could be due to stress or anxiety caused by the lack of mouse handling prior to experimental procedures and the shorter conditioning time (40 minutes in present rat study versus 15 minutes in mice) used by Sturgess and colleagues (2010). Prior handling of animals has been demonstrated to promote reward in animals that do not show reward at the same dosage. For example, nicotine place conditioning at low doses in mice was rewarding only after the mice were handled by the experimenter prior to experimentation (Grabus et al., 2006). A shorter conditioning time would also allow animals less time to overcome the anxiety or stress from injections, which could be masking the expression of the weak rewarding effects in mice at low doses. This is consistent with Vautrin and colleagues’ (2005) work, demonstrating that caffeine preferences appear later in anxious mice than in non-anxious mice. Pechlivanova and colleagues (2010) also examined the effect of caffeine dose in a chronic unpredictable stress model of depression and found that caffeine dose-dependently modulated the behaviour of rats in a variety of tests, including the open field test and the foot shock test. Investigating the effects of handling and

84 conditioning time in rats versus mice could help to resolve the discrepancies between the two species.

If these experiments were repeated in female rats, would the results be similar?

As systemic caffeine self-administration in rodents is inconsistent (Griffiths & Woodson,

1988) and it is known that there are sex differences related to both caffeine consumption

(Noschang et al., 2009) and place conditioning (Russo et al., 2003), female rats were excluded from these studies. Nonetheless, the use of female rats would be both generally informative and could improve on the time course of the place conditioning paradigm. For example, female rats could acquire a preference or aversion to caffeine with fewer conditioning sessions, as is seen with cocaine place conditioning (Russo et al., 2003). Female rats also appear to be more vulnerable to drugs of abuse, as a comparative study of male and female rats showed that a greater percentage of female rats acquired cocaine and heroin self-administration behavior (Lynch & Carroll, 1999).

Could we elicit caffeine aversion through higher doses of caffeine in the cVTA?

As mentioned previously, caffeine can easily bypass all biological membranes. As such, systemic caffeine should affect both the NAc core and shell. Therefore, it is unclear why the NAc core-mediated aversion is the effect that prevails at 10 and 30 mg/kg i.p. doses of caffeine. Though it was speculated previously that higher systemic doses may correspond to higher than 100 µM of caffeine in the VTA and could obscure the effects of rVTA activation through a cVTA-mediated mechanism, this has yet to be determined. Though we have presented data in this thesis using

85 bilateral intra-cVTA injections as high as 1000 µM of caffeine, it appears that even higher doses may be required to elicit comparable caffeine aversion of that seen using 10 or 30 mg/kg i.p. doses of caffeine. Additionally, though it was not observed in these current experiments, somatic signs of caffeine aversion may also be produced with higher doses of caffeine. General somatic signs of aversion for other drugs of abuse (e.g. amphetamine) and aversive substances (e.g. lithium chloride) include suppression of grooming, less scratching, and increased time that rats spend lying on their bellies (Parker et al., 1984; Tuerke et al., 2012).

Is 8-SPT an appropriate proxy as a peripherally-restricted analog of caffeine?

In the present experiments, 8-SPT was injected at a dose of 10 mg/kg i.p. and was found to produce neither aversion nor reward. While this is consistent with what was seen in mice

(Sturgess et al., 2010), it is possible that 8-SPT may be able to produce rewarding or aversive effects if given at a different dose. Nonetheless, the interpretation of the present results suggested that the location of caffeine action was mediated by the CNS. Though it was determined that caffeine reward and aversion are indeed mediated centrally in the NAc shell and core, respectively, a more thorough approach would be to repeat some of the experiments previously conducted with caffeine using 8-SPT instead. For example, if 8-SPT directly injected in the rVTA could elicit reward that was DA-dependent, then it would suggest that 8-SPT acts in a comparable manner to caffeine.

What other neural substrates mediate DA-independent caffeine reward and aversion?

If activating the tail of the VTA (located in the cVTA) alone is sufficient to produce caffeine aversion at higher concentrations than 100µM, the LHb (upstream of the tail of the VTA) would be a good target candidate as Balcita-Pedicino and colleagues (2011) found that the tail of the VTA is likely an intermediate region between the LHb and the VTA DA neurons. Additionally, the surrounding regions of the VTA were not definitively ruled out to be involved in caffeine motivation. To ensure that the targeted intracranial injections into the VTA did not diffuse to surrounding tissues, small volumes of caffeine were used, and the tissue was examined for discoloration during histology. However, it is still possible that caffeine was not fully restricted to the VTA. One way to rule out this possibility is to target the surrounding VTA tissue to see if there are any behavioral changes. Also, as DA antagonism revealed a DA-independent reward in mice, it follows that caffeine motivation could also be mediated by other neural substrates and neurotransmitters. Sturgess and colleagues (2010) showed that lesioning the TPP did not affect caffeine aversions or DA blockade-induced reward in mice, however, considering other drugs of abuse could provide valuable insight on which areas are good potential candidates. For example, injections of morphine into the periaqueductal grey (PAG) has been shown to produce reward (van der Kooy et al., 1982).

Are the same brain regions mediating acute and chronic caffeine motivation?

As mentioned earlier, the non-deprived/deprived hypothesis developed by van der Kooy and colleagues suggests that there is a switch of brain regions mediating the effects of opiates

87 depending on the motivational state of the animal. As the work presented in this thesis is based on the effects of acute caffeine, a future aim could be to test if animals that have become dependent and withdrawn from caffeine still experience the reward of caffeine through NAc shell mediation or experience the aversion of caffeine withdrawal if animals learn to associate caffeine withdrawal with a particular environment. These place conditioning experiments could be repeated in rodents exposed to caffeine chronically either through daily i.p. injections or simply by administering caffeine in their drinking water over a long period. After which, blockade of DA in the NAc shell or NAc core could help determine if these areas are important for mediating caffeine’s motivational effects in caffeine dependent and withdrawn animals.

References

1) American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

2) Ashok AH, Mizuno Y, Volkow ND, Howes OD (2017) Association of stimulants with dopaminergic alterations in users of cocaine, amphetamine, and methamphetamine: a systematic review and meta-analysis. Jama Psychiat 74:511-519.

3) Balleine BW, Delgado MR, Hikosaka O (2007) The role of the dorsal striatum in reward and decision-making. J Neurosci 27:8161-8165.

4) Balcita-Pedicino JJ, Omelchenko N, Bell R, Sesack SR (2011) The inhibitory influence of the lateral habenula on midbrain dopamine cells: ultrastructural evidence for indirect mediation via the rostromedial mesopontine tegmental nucleus. J Comp Neurol 519:1143- 1164.

5) Bechara A, Nader K, Harrington F, Nader K, van der Kooy D (1992) Neurobiology of motivation: Double dissociation of two motivational mechanisms mediating opiate reward in drug-naïve versus drug-dependent animals. Behav Neurosci 106:798-807.

6) Bechara A, Nader K, van der Kooy D (1998) A two-separate-motivational-systems hypothesis of opioid addiction. Pharmacol Biochem Behav 59:1-17.

7) Bechara A, van der Kooy D (1989) The tegmental pedunculopontine nucleus: A brain-stem output of the limbic system critical for the conditioned place preferences produced by morphine and amphetamine. J Neurosci 9:3400-3409.

8) Bechara A, van der Kooy D (1992) A single brain stem substrate mediates the motivational effects of both opiates and food in nondeprived rats but not in deprived rats. Behav Neurosci 106:351-363.

9) Beckstead RM, Domesick VB, Nauta WJH (1979) Efferent connections of the substantia nigra and ventral tegmental area in the rat. Brain Res 175:191-217.

10) Beier KT, Steinberg EE, DeLoach KE, Xie S, Miyamichi K, Schwarz L, Gao XJ, Kremer EJ, Malenka RC, Luo L (2015) Circuit architecture of VTA dopamine neurons revealed by systematic input-output mapping. Cell 162:622-634.

11) Berridge KC, Robinson TE (2016) Liking, wanting and the incentive-sensitization theory of addiction. Am Psychol 71:670-679.

12) Berridge KC, Venier IL, Robinson TE (1989) Taste reactivity analysis of 6- hydroxydopamine-induced aphagia: Implications for arousal and anhedonia hypotheses of dopamine function. Behav Neurosci 103:36-45.

13) Bourdy R, Barrot M (2012) A new control center for dopaminergic systems: pulling the VTA by the tail. Trends Neurosci 35:681-690.

14) Brice CF, Smith AP (2002) Effects of caffeine on mood and performance: a study of realistic consumption. Psychopharmacology 164:188-192.

15) Brockwell NT, Eikelboom R, Beninger RJ (1991) Caffeine-induced place and taste conditioning: Production of dose-dependent preference and aversion. Pharmacol Biochem Be 38:513-517.

16) Brog JS, Salyapongse AS, Deutch AY, Zahm DS (1993) The patterns of afferent innervation of the core and shell in the “accumbens” part of the rat ventral striatum: Immunohistochemical detection of retrogradely transported Fluoro-Gold. J Comp Neurol 338:255-278.

17) Brown SA, Vik PW, Creamer VA (1989) Characteristics of relapse following adolescent substance abuse treatment. Addict Behav 14:291-300.

18) Carlezon WA, Thomas MJ (2009) Biological substrates of reward and aversion: A nucleus accumbens activity hypothesis. Neuropharmacology 56:122-132.

19) Carr DB, Sesack SR (2000) Projections from the rat prefrontal cortex to the ventral tegmental area: Target specificity in the synaptic associations with mesoaccumbens and mesocortical neurons. J Neurosci 20:3864-3873.

20) Chaudhri N, Sahuque LL, Schairer WW, Janak PH (2010) Separable roles of the nucleus accumbens core and shell in context- and cue-induced alcohol-seeking. Neuropsychopharmacol 35:783-791.

21) Chen JF, Moratalla R, Impagnatiello F, Grandy DK, Cuellar B, Rubinstein M, Beilstein MA,

Hackett E, Fink JS, Low MJ, Ongini E, Schwarzchild MA (2001) The role of the D2

dopamine receptor (D2R) in A2A adenosine receptor (A2AR)-mediated behavioral and cellular

responses as revealed by A2A and D2 receptor knockout mice. P Natl Acad Sci USA 98:1970- 1975.

22) Chou DT, Khan S, Forde J, Hirsh KR (1985) Caffeine tolerance: Behavioral, electrophysiological and neurochemical evidence. Life Sci 36:2347-2358.

23) Clarke PBS, Pert A (1985) Autoradiographic evidence for nicotine receptors on nigrostriatal and mesolimbic dopaminergic neurons. Brain Res 348:355-358.

24) Cook CE, Tallent CR, Amerson EW, Myers MW, Kepler JA, Taylor GF, Christensen HD (1976) Caffeine in plasma and saliva by a radioimmunoassay procedure. J Pharmacol Exp Ther 199:679-686.

25) Corrigall WA, Coen KM, Adamson KL (1994) Self-administered nicotine activates the mesolimbic dopamine system through the ventral tegmental area. Brain Res 653:278-284.

26) Corrigall WA, Franklin BJ, Coen KM, Clarke PBS (1992) The mesolimbic dopaminergic system is implicated in the reinforcing effects of nicotine. Psychopharmacology 107:285- 289.

27) Creese I, Burt DR, Snyder SH (1976) Dopamine binding predicts clinical and pharmacological potencies of antischizophrenic drugs. Science 192:481-483.

28) Deutch AY, Cameron DS (1992) Pharmacological characterization of dopamine systems in the nucleus accumbens core and shell. Neuroscience 46:49-56.

29) Di Chiara G (2002) Nucleus accumbens shell and core dopamine: differential role in behaviour and addiction. Behav Brain Res 137:75-114.

30) Di Chiara G, Imperato A (1988) Drugs abused by humans preferentially increase synaptic dopamine concentrations in the mesolimbic system of freely moving rats. Proc Natl Acad Sci USA 85:5274-5278.

31) Dingle RN, Dreumont-Boudreau SE, Lolordo VM (2008) Caffeine dependence in rats: effects of exposure duration and concentration. Physiol Behav 95:252-257.

32) Evans SM, Griffiths RR (1992) Caffeine tolerance and choice in humans. Psychopharmacology 108:51-59.

33) Fibiger HC (1978) Drugs and reinforcement mechanisms: A critical review of the catecholamine theory. Ann Rev Pharmacol Toxicol 18:37-56.

34) Finn IB, Holtzman SG (1986) Tolerance to caffeine-induced stimulation of locomotor activity in rats. J Pharmacol Exp Ther 238:542-546.

35) Fredholm BB (2011) Notes on the history of caffeine use. Handb Exp Pharmacol 200:1-10.

36) Fredholm BB, Bättig K, Holmén A, Zvartau EE (1999) Actions of caffeine in the brain with special reference to factors that contribute to its widespread use. Pharmacol Rev 51:83-133.

37) Friedman GD (1974) Cigarettes, alcohol, coffee and peptic ulcer. N Engl J Med 290:469- 473.

38) Fulgoni VL, Keast DR, Lieberman HR (2015) Trends in intake and sources of caffeine in the diets of US adults: 2001-2010. Am J Clin Nutr 101:1081-1087.

39) Gallistel CR, Gomita Y, Yadin E, Campbell KA (1985) Forebrain origins and terminations of the medial forebrain bundle metabolically activated by rewarding stimulation or by reward-blocking doses of pimozide. J Neurosci 5:1246-1261.

40) Gasbarri A, Packard MG, Campana E, Pacitti C (1994) Anterograde and retrograde tracing of projections from the ventral tegmental area to the hippocampal formation in the rat. Brain Res Bull 33:445-452.

41) Geisler S, Zahm DS (2005) Afferents of the ventral tegmental area in the rat-anatomical substratum for integrative functions. J Comp Neurol 490:270-294.

42) Gessa GL, Muntoni F, Collu M, Vargiu L, Mereu G (1985) Low doses of ethanol activate dopaminergic neurons in the ventral tegmental area. Brain Res 348:201-203.

43) Grabus SD, Martin BR, Brown SE, Damaj MI (2006) Nicotine place preference in the mouse: influences of prior handling, dose and strain and attenuation by nicotinic receptor antagonists. Psychopharmacology 184:456-463.

44) Grieder TE, Herman MA, Contet C, Tan LA, Vargas-Perez H, Cohen A, Chwalek M, Maal- Bared G, Freiling J, Schlosburg JE, Clarke L, Crawford E, Koebel P, Canonigo V, Sanna P, Tapper A, Roberto M, Kieffer BL, Sawchenko PE, Koob GF, van der Kooy D, George O (2014) CRF neurons in the ventral tegmental area control the aversive effects of nicotine withdrawal and promote escalation of nicotine intake. Nat Neurosci 17:1751-1758.

45) Grieder TE, Sellings L, Vargas-Perez H, Ting-A-Kee R, Siu EC, Tyndale RF, van der Kooy D (2010) Dopaminergic signaling mediates the motivational response underlying the opponent process to chronic but not acute nicotine. Neuropsychopharmacol 35: 943-954.

46) Griffiths RR, Woodson PP (1988) Reinforcing properties of caffeine studies in humans and laboratory animals. Pharmacol Biochem Be 29:419-427.

47) Grill HJ, Norgren R (1978) The taste reactivity test. I. Mimetic responses to gustatory stimuli in neurologically normal rats. Brain Res 143:263-279.

48) Hasin DS, O’Brien CP, Auriacombe M, Borges G, Bucholz K, Budney A, Compton WM, Crowley T, Ling W, Petry NM, Schuckit M, Grant BF (2013) DSM-5 criteria for substance use disorders: recommendations and rationale. Am J Psychiat 170:834-851.

49) Hefco V, Yamada K, Hefco A, Hritcu L, Tiron A, Nabeshima T (2003) Role of the mesotelencephalic dopamine system in learning and memory processes in the rat. Eur J Pharmacol 475:55-60.

50) Heimer L, Zahm DS, Churchill L, Kalivas PW, Wohltmann C (1991) Specificity in the projection patterns of accumbal core and shell in the rat. Neuroscience 41:89-125.

51) Higgins GA, Grzelak ME, Pond AJ, Cohen-Williams ME, Hodgson RA, Varty GB (2007) The effect of caffeine to increase reaction time in the rat during a test of attention is mediated

through antagonism of adenosine A2A receptors. Behav Brain Res 185:32-42.

52) Hurd YL, Weiss F, Koob GF, And N-E, Ungerstedt U (1989) Cocaine reinforcement and extracellular dopamine overflow in rat nucleus accumbens: an in vivo microdialysis study. Brain Res 498:199-203.

53) Huston JP, de Sousa Silva MA, Topic B, Müller CP (2013) What’s conditioned in conditioned place preference? Trends Pharmacol Sci 34:162-166.

54) Ikemoto S (2007) Dopamine reward circuitry: Two projection systems from the ventral midbrain to the nucleus accumbens-olfactory tubercle complex. Brain Res Rev 56:27-78.

55) Ikemoto S, Panksepp J (1999) The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res Rev 31:6-41.

56) Istvan J, Matarazzo JD (1984) Tobacco, alcohol, and caffeine use: A review of their interrelationships. Psychol Bull 95:301-326.

57) Ito R, Hayen A (2011) Opposing roles of nucleus accumbens core and shell dopamine in the modulation of limbic information processing. J Neurosci 31:6001-6007.

58) Ito R, Robbins TW, Everitt BJ (2004) Differential control over cocaine-seeking behavior by nucleus accumbens core and shell. Nat Neurosci 7:389-397.

59) Jhou TC, Fields HL, Baxter MG, Saper CB, Holland PC (2009a) The rostromedial tegmental nucleus (RMTg), a GABAergic afferent to midbrain dopamine neurons, encodes aversive stimuli and inhibits motor responses. Neuron 61:786-800.

60) Jhou TC, Geisler S, Marinelli M, DeGarmo BA, Zahm DS (2009b) The mesopontine rostrotegmental nucleus: a structure targeted by the lateral habenula that projects to the ventral tegmental area of Tsai and substantia nigra compacta. J Comp Neurol 513:566-596.

61) Jones SR, O'Dell SJ, Marshall JF, Wightman RM (1996) Functional and anatomical evidence for different dopamine dynamics in the core and shell of the nucleus accumbens in the slices of rat brain. Synapse 23:224-231.

62) Juliano LM, Griffiths RR (2004) A critical review of caffeine withdrawal: empirical validation of symptoms and signs, incidence, severity, and associated features. Psychopharmacol 176:1-29.

63) Kalivas PW, Churchill L, Klitenick MA (1993) GABA and enkephalin projection from the nucleus accumbens and ventral pallidum to the ventral tegmental area. Neuroscience 57:1047-1060.

64) Kaplan GB, Greenblatt DJ, Ehrenberg BL, Goddard JE, Cotreau MM, Harmatz JS, Shader RI (1997) Dose-dependent pharmacokinetics and psychomotor effects of caffeine in humans. J Clin Pharmacol 37:693-703.

65) Kaufling J, Waltisperger E, Bourdy R, Valera A, Veinante P, Freund-Mercier M-J, Barrot Michel (2010) Pharmacological recruitment of the GABAergic tail of the ventral tegmental area by acute drug exposure. Brit J Phamacol 161:1677-1691.

66) Keast RSJ, Swinburn BA, Sayompark D, Whitelock S, Riddell (2015) Caffeine increases sugar-sweetened beverage consumption in a free-living population: a randomized controlled trial. Brit J Nutr 113:366-371.

67) Kendler KS, Prescott CA (1999) Caffeine intake, tolerance, and withdrawal in women: a population-based twin study. Am J Psychiat 156:223-228.

68) Kim JA, Pollak KA, Hjelmstad GO, Fields HL (2004) A single cocaine exposure enhances both opioid reward and aversion through a ventral tegmental area-dependent mechanism. P Natl Acad Sci USA 101:5664-5669.

69) Koob GF, Bloom FE (1988) Cellular and molecular mechanisms of drug dependence. Science 242:715-723.

70) Koob GF, Le Moal (2001) Drug addiction, dysregulation of reward, and allostasis. Neuropsychopharmacol 24:97-129.

71) Koob GF, Le Moal (2005) Plasticity of reward neurocircuitry and the ‘dark side’ of drug addiction. Nature Neurosci 8:1442-1444.

72) Koob GF, Le Moal (2008a) Addiction and the brain antireward system. Annu Rev Psychol 59:29-53.

73) Koob GF, Le Moal (2008b) Neurobiological mechanisms for opponent motivational processes in addiction. Philos Trans R Soc Lond B Biol Sci 363:3113-3123.

74) Koob GF, Stinus L, Le Moal M, Bloom FE (1989) Opponent process theory of motivation: Neurobiological evidence from studies of opiate dependence. Neurosci Biobehav Rev 13:135-140.

75) Kourrich S, Calu DJ, Bonci A (2015) Intrinsic plasticity: An emerging player in addiction. Nat Rev Neurosci 16:173-184.

76) Kuzmin A, Johansson B, Fredholm BB, Örgren, S (2000) Genetic evidence that cocaine and caffeine stimulate locomotion in mice via different mechanisms. Life Sci 66:PL113-118.

77) Lammel S, Ion DI, Roeper J, Malenka RC (2011) Projection-specific modulation of dopamine neuron synapses by aversive and rewarding stimuli. Neuron 70:855-862.

78) Lammel S, Hetzel A, Hackel O, Jones I, Liss B, Roeper J (2008) Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system. Neuron 57:760- 773.

79) Lammel S, Lim BK, Ran C, Huang KW, Betley MJ, Tye K, Deisseroth K, Malenka RC (2012) Input-specific control of reward and aversion in the ventral tegmental area. Nature 491:212-217.

80) Lao-Peregrin C, Ballesteros JJ, Fernandez M, Zamora-Moratalla A, Saavedra A, Lazaro MG, Perez-Navarro E, Burks D, Martin ED (2016) Caffeine-mediated BDNF release regulates long-term synaptic plasticity through activation of IRS2 signaling. Addict Biol 22:1706- 1718.

81) Latini R, Bonati M, Castelli D, Garattini S (1978) Dose-dependent kinetics of caffeine in rats. Toxicol Lett 2:267-270.

82) Laviolette SR, Nader K, van der Kooy D (2002) Motivational state determines the functional role of the mesolimbic dopamine system in the mediation of opiate reward processes. Behav Brain Res 129:17-29.

83) Laviolette SR, van der Kooy D (2003) Blockade of mesolimbic dopamine transmission dramatically increases sensitivity to the rewarding effects of nicotine in the ventral tegmental area. Mol Psychiatr 8:50-59.

84) Lawson CC, LeMasters GK, Wilson KA (2004) Changes in caffeine consumption as a signal of pregnancy Reprod Toxicol 18:625-633.

85) Lazarus M, Shen HY, Cherasse Y, Qu WM, Huang ZL, Bass CE, Winsky-Sommerer R, Semba K, Fredholme BB, Bolson D, Hayaishi O, Urade Y, Chen JF (2011) Arousal effect of

caffeine depends on adenosine A2A receptors in the shell of the nucleus accumbens. J Neurosci 31:10067-10075.

86) Ljungberg T, Apicella P, Schultz W (1992) Responses of monkey dopamine neurons during learning of behavioural reactions. J Neurophysiol 67:145-163.

87) Lorist MM, Snel J, Kok A, Mulder G (1996) Acute effects of caffeine on selective attention and visual search processes. Psychophysiology 33:354-361.

88) Loughlin SE, Fallon JH (1984) Substantia nigra and ventral tegmental area projections to cortex: Topography and collaterization. Neuroscience 11:425-435.

89) Lukas G, Brindle SD, Greengard P (1971) The route of absorption of intraperitoneally administered compounds. J Pharmacol Exp Ther 178:562-566.

90) Lynch WJ, Carroll ME (1999) Sex differences in the acquisition of intravenously self- administered cocaine and heroin in rats. Psychopharmacology 144:77-82.

91) Machado, A (1989) Operant conditioning of behavioural variability using a percentile reinforcement schedule. J Exp Anal Behav 52:155-166.

92) Margolis EB, Lock H, Hjelmstad GO, Fields HL (2006) The ventral tegmental area revisited: Is there an electrophysiological marker for dopaminergic neurons. J Physiol 577:907-924.

93) Marks V, Kelly JF (1973) Absorption of caffeine from tea, coffee, and coca cola. Lancet 1:827.

94) Matsui A, Williams JT (2011) Opioid-sensitive GABA inputs from rostromedial tegmental nucleus synapse onto midbrain dopamine neurons. J Neurosci 31:17729-17735.

95) McPherson PS, Kim Y-K, Valdivia H, Knudson CM, Takekura H, Franzini-Armstron C, Coronadot R, Campbell KP (1991) The brain ryanodine receptor: A caffeine-sensitive calcium release channel. Neuron 7:17-25.

96) Millhouse OE (1969) A Golgi study of the descending medial forebrain bundle. Brain Res 15:341-363.

97) Mirenowicz, J, Schultz W (1996) Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379:449-451.

98) Mucha RF, van der Kooy D, O’Shaughnessy M, Bucenieks P (1982) Drug reinforcement studied by the use of place conditioning in rat. Brain Res 243:91-105.

99) Nader K, Bechara A, Roberts DCS, van der Kooy D (1994) Neuroleptics block high- but not low-dose heroin place preferences: Further evidence for a two-system model of motivation. Behav Neurosci 108:1128-1138.

100) Nader K, van der Kooy D (1997) Deprivation state switched the neurobiological substrates mediating opiate reward in the ventral tegmental area. J Neurosci 17: 383-390.

101) Nair-Roberts RG, Chatelain-Badie SD, Benson E, White-Cooper H, Bolam JP, Ungless MA (2008) Stereological estimates of dopaminergic, GABAergic and glutamatergic neurons in the ventral tegmental area, substantia nigra and retrorubral field in the rat. Neuroscience 152:1024-1031.

102) Namburi P, Beyeler A, Yorozu S, Calhoon GG, Halbert SA, Wichmann R, Holden SS, Mertens KL, Anahtar M, Felix-Ortiz AC, Wickersham IR, Gray JM, and Tye KM (2015) A circuit mechanism for differentiating positive and negative associations. Nature 520:675- 678.

103) Nauta WJH, Smith GP, Faull RLM, Domesick VB (1978) Efferent connections and nigral afferents of the nucleus accumbens septi in the rat. Neuroscience 3:385-401.

104) Nehlig A (1999) Are we dependent upon coffee and caffeine? A review on human and animal data. Neurosci Biobehav Rev 23:563-576.

105) Nehlig A, Debry G (1994) Caffeine and sports activity: A review. Int J Sports Med 15:215- 223.

106) Nestler EJ (1992) Molecular mechanisms of drug addiction. J Neurosci 12:2439-2450.

107) Nestler EJ (2005) Is there a common molecular pathway for addiction? Nature Neurosci 8:1445-1449.

108) Noschang CG, Pettenuzzo LF, von Pozzer Toigo E, Andreazza AC, Krolow R, Fachin A, Avila MC, Arcego D, Crema LM, Diehl LA, Goncalvez CA, Vendite D, Dalmaz C (2009) Sex-specific differences on caffeine consumption and chronic stress-induced anxiety-like behavior and DNA breaks in the hippocampus. Pharamcol Biochem Be 94:63-69.

100

109) Oades RD, Halliday GM (1987) Ventral tegmental (A10) system: Neurobiology. 1. Anatomy and connectivity. Brain Res Rev 12:117-165.

110) Ogawa N, Ueki H (2007) Clinical importance of caffeine dependence and abuse. Psychiat Clin Neuros 61:263-268.

111) Olds J, Milner P (1954) Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psych 47:419-427.

112) Olmstead MC, Munn EM, Franklin KBJ, Wise RA (1998) Effects of pedunculopontine tegmental nucleus lesions on responding for intravenous heroin under different schedules of reinforcement. J Neurosci 18:5035-5044.

113) Olson VG, Nestler EJ (2007) Topographical organization of GABAergic neurons within the ventral tegmental area of the rat. Synapse 61:87-95.

114) Olson VG, Zabetian CP, Bolanos CA, Edwards S, Barrot M, Eisch AJ, Hughes T, Self DW, Neve RL, Nestler EJ (2005) Regulation of drug reward by cAMP response element-binding protein: Evidence for two functionally distinct subregions of the ventral tegmental area. J Neurosci 25:5553-5562.

115) Parker LA, Hills K, Jensen K (1984) Behavioral CRs elicited by a lithium- or an amphetamine-paired contextual test chamber. Anim Learn Behav 12:307-315.

116) Patkina NA, Zvartau EE (1998) Caffeine place conditioning in rats: comparison with cocaine and ethanol. Eur Neuropsychopharmacol 8:287-291.

117) Pavlov IP (1927) Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex. London: Oxford University Press.

118) Paxinos G, Watson C (2005) The rat brain in stereotaxic coordinates, Ed 5. San Diego: Academic Press.

101

119) Pechlivanova D, Tchekalarova J, Nikolov R, Yakimova K (2010) Dose-dependent effects of caffeine on behavior and thermoregulation in a chronic unpredictable stress model of depression in rats. Behav Brain Res 209:205-211.

120) Persad LAB (2011) Energy drinks and the neurophysiological impact of caffeine. Front Neurosci 5:1-8.

121) Pettit ML, DeBarr KA (2011) Perceived stress, energy drink consumption, and academi performance among college students. J Am Coll Health 59:335-341.

122) Phillipson OT, Griffiths AC (1985) The topographic order of inputs to nucleus accumbens in the rat. Neuroscience 16:275-296.

123) Picciotto MR, Zoli M, Rimondini R, Lena C, Marubio LM, Pich EM, Fuxe K, Changeux J- P (1998) Acetylcholine receptors containing the β2 subunit are involved in the reinforcing properties of nicotine. Nature 391:173-177.

124) Poller WC, Madai VI, Bernaud R, Laube G, Veh RW (2013) A glutamatergic projection from the lateral hypothalamus targets VTA-projecting neurons in the lateral habenula of the rat. Brain Res 1507:45-60.

125) Pontieri FE, Tanda G, Di Chiara G (1995) Intravenous cocaine, morphine, and amphetamine preferentially increase extracellular dopamine in the "shell" as compared to the "core" of the rat nucleus accumbens. P Natl A Sci USA 92:12304-12308.

126) Reavill C, Jenner P, Kumar R, Stolerman IP (1988) High affinity binding of [3H] (-)-nicotine to rat brain membranes and its inhibition by analogues of nicotine. Neuropharmacology 27:235-241.

127) Richards G, Smith A (2015) Caffeine consumption and self-assessed stress, anxiety, and depression in secondary school children. J Psychopharmacol 29:1236-1247.

102

128) Roberts DCS, Koob GF (1982) Disruption of cocaine self-administration following 6- hydroxydopamine lesions of the ventral tegmental area in rats. Pharmacol Biochem Be 17:901-904.

129) Robinson TE, Berridge KC (1993) The neural basis of drug-craving: an incentive- sensitization theory of addiction. Brain Res Rev 18:247-291.

130) Rodriguez-Lopez C, Clasca F, Prensa L (2017) The mesoaccumbens pathway: a retrograde labeling and single-cell axon tracing analysis in the mouse. Front Neuroanat 11:1-15.

131) Russo SJ, Festa ED, Fabian SJ, Gazi FM, Kraish M, Jenab S, Quinones-Jenab V (2003) Gonadal hormones differentially modulate cocaine-induced conditioned place preference in male and female rats. Neuroscience 120:523-533.

132) Rygula R, Abumaria Nm Flugge G, Fuchs E, Ruther E, Havemann-Reinecke U (2005) Anhedonia and motivational deficits in rats: Impact of chronic social stress. Behav Brain Res 162:127-134.

133) Sanchez-Catalan MJ, Faivre F, Yalcin I, Muller MA, Massotte D, Majchrzak M, Barrot M (2017) Response of the tail of the ventral tegmental area to aversive stimuli. Neuropsychopharmacol 42:638-648.

134) Sanchez-Catalan MJ, Kaufling J, Georges F, Veinante P, Barrot M (2014) The antero- posterior heterogeneity of the ventral tegmental area. Neuroscience 282:198-216.

135) Schultz W (2016) Dopamine reward prediction-error signaling: a two-component response. Nat Rev Neurosci 17:183-195.

136) Schultz W, Apicella P, Ljungberg T (1993) Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of leaning a delayed response task. J Neurosci 13:900-913.

103

137) Sellings LHL, Baharnouri G, McQuade LE, Clarke PBS (2008) Rewarding and aversive effects of nicotine are segregated within the nucleus accumbens. Eur J Neurosci 28:342-352.

138) Sellings LHL, Clarke PBS (2003) Segregation of amphetamine reward and locomotor stimulation between nucleus accumbens medial shell and core. J Neurosci 23:6295-6303.

139) Silverman K, Evans SM, Strain EC, Griffiths RR (1992) Withdrawal syndrome after the double-blind cessation of caffeine consumption. New Engl J Med 327:1109-1114.

140) Skinner BF (1950) Are theories of learning necessary? Psychol Rev 57:193-216.

141) Smellie FW, Davis CW, Daly JW, Wells JN (1979) Alkylxanthines: Inhibition of adenosine- elicited accumulation of cyclic AMP in brain slices and of brain phosphodiesterase activity. Life Sci 24:2475-2481.

142) Smith AP (2009) Effects of caffeine in chewing gum on mood and attention. Hum Psychopharmacol Clin Exp 24:239-247.

143) Solinas M, Ferré S, You ZB, Karcz-Kubicha M, Popoli P, Goldberg SR (2002) Caffeine induces dopamine and glutamate release in the shell of the nucleus accumbens. J Neurosci 22:6321-6324.

144) Solomon RL, Corbit JD (1973) An opponent-process theory of motivation: II. Cigarette addiction. J Abnorm Psychol 81:158-171.

145) Solomon RL, Corbit JD (1974) An opponent-process theory of motivation: I. Temporal dynamics of affect. Psychol Rev 81:119-145.

146) Steffensen SC, Svingos AL, Pickel VM, Henriksen SJ (1998) Electrophysiological characterization of GABAergic neurons in the ventral tegmental area. J Neurosci 18:8003- 8015.

104

147) Stuber GD, Evans SB, Higgins MS, Pu Y, Figlewicz DP (2002) Food restriction modulates amphetamine-conditioned place preference and nucleus accumbens dopamine release in the rat. Synapse 46:83-90.

148) Sturgess JE, Ting-A-Kee RA, Podbielski D, Sellings LHL, Chen JF, van der Kooy D (2010)

Adenosine A1 and A2A receptors are not upstream of caffeine's dopamine D2 receptor- dependent aversive effects and dopamine-independent rewarding effects. Eur J Neurosci 32:143-154.

149) Swanson LW (1982) The projections of the ventral tegmental area and adjacent regions: A combined fluorescent retrograde tracer and immunofluorescence study in the rat. Brain Res Bull 9:321-353.

150) Tao S, Abdel-Rahman AA (1993) Neuronal and cardiovascular responses to adenosine microinjection into the nucleus tractus solitarius. Brain Res Bull 32:407-417.

151) Ting-A-Kee R, Dockstader C, Heinmiller A, Grieder T, van der Kooy D (2009) GABAA receptors mediate the opposing roles of dopamine and the tegmental pedunculopontine nucleus in the motivational effects of ethanol. Eur J Neurosci 29:1235-1244.

152) Ting-A-Kee R, Vargas-Perez H, Bufalino M-R, Bahi A, Dreyer J-L, Tyndale RF, van der Kooy D (2013) Infusion of brain-derived neurotrophic factor into the ventral tegmental area switches the substrates mediating ethanol motivation. Eur J Neurosci 37:996-1003.

153) Thorndike EL (1898) Some experiments on animal intelligence. Science 7:818-824.

154) Tuerke KJ, Winters BD, Parker LA (2012) Ondansetron interferes with unconditioned lying- on belly and acquisition of conditioned gaping induced by LiCl as models of nausea-induced behaviors in rats. Physiol Behav 105:856-860.

105

155) van der Kooy D, Mucha RF, O’Shaughnessy M, Bucenieks P (1982) Reinforcing effects of brain microinjections of morphine revealed by conditioned place preference. Brain Res 243:107-117.

156) van der Poel A, Rodenburg G, Dijkstra M, Stoele M, van de Mheen D (2009) Trends, motivations and settings of recreational cocaine use by adolescents and young adults in the Netherlands. Int J Drug Policy 20:143-151.

157) Vautrin S, Pelloux Y, Costentin J (2005) Preference for caffeine appears earlier in non- anxious than in anxious mice. Neurosci Lett 386:94-98.

158) Veening JG, Swanson LW, Cowan WM, Nieuwenhuys R, Geeraedts LMG (1982) The medial forebrain bundle of the rat. II. An autoradiographic study of the topography of the major descending and ascending components. J Comp Neurol 206:82-108.

159) Ventura R, Cabib S, Alcaro A, Orsini C, Puglisi-Allegra S (2003) Norepinephrine in the prefrontal cortex is critical for amphetamine-induced reward and mesoaccumbens dopamine release. J Neurosci 23:1879-1885.

160) Vezina P, Stewart J (1989) The effect of dopamine receptor blockade on the development of sensitization to the locomotor activating effects of amphetamine and morphine. Brain Res 499:108-120.

161) Volkow ND, Wang GJ, Logan J, Alexoff D, Fowler JS, Thanos PK, Wong C, Casado V,

Ferre S, Tomasi D (2015) Caffeine increases striatal dopamine D2/D3 receptor availability in the human brain. Transl Psychiat 5:e549-554.

162) Watanabe H, Uramoto H (1986) Caffeine mimics dopamine receptor agonists without stimulation of dopamine receptors. Neuropharmacology 25:577-581.

106

163) Walsh SL, Stoops WW, Moody DE, Lin SN, Bigelow GE (2009) Repeated dosing with oral cocaine in humans: Assessment of direct effects, withdrawal and pharmacokinetics. Exp Clin Psychopharmacol 17:205-216.

164) Westerink BHC, Kwint HF, deVries JB (1996) The pharmacology of mesolimbic dopamine neurons: a dual-probe microdialysis study in the ventral tegmental area and nucleus accumbens of the rat brain. J Neurosci 16:2605-2611.

165) Wise RA (1987) The role of reward pathways in the development of drug dependence. Pharmacol Therapeut 35: 227-263.

166) Wise RA, Morales M (2010) A ventral tegmental CRF-glutamate-dopamine interaction in addiction. Brain Res 1314:38-43.

167) World Health Organization (1992) The ICD-10 classification of mental and behavioural disorders: clinical descriptions and diagnostic guidelines. Geneva: World Health Organization.

168) Xie H, McHugo GJ, Fox MB, Drake RE (2005) Substance abuse relapse in a ten-year prospective follow-up of clients with mental and substance use disorders. Psychiat Serv 56:1282-1287.

169) Yamaguchi T, Sheen W, Morales M (2007) Glutamatergic neurons are present in the rat ventral tegmental area. Eur J Neurosci 25:106-118.

170) Young AMJ, Joseph MH, Gray JA (1993) Latent inhibition of conditioned dopamine release in rat nucleus accumbens. Neuroscience 54:5-9.

171) Yukl G, Wexley KN, Seymore JD (1972) Effectiveness of pay incentives under variable ratio and continuous reinforcement schedules. J Appl Psychol 56:19-23.

107

172) Zahm DS (1999) Functional-anatomical implications of the nucleus accumbens core and shell subterritories. Ann NY Acad Sci 877:113-128.

173) Zahm DS, Heimer L (1990) Two transpallidal pathways originating in the rat nucleus accumbens. J Comp Neurol 302:437-446.

174) Zahm DS, Heimer L (1993) Specificity in the efferent projections of the nucleus accumbens in the rat: Comparison of the rostral pole projection patterns with those of the core and shell. J Comp Neurol 372: 220-232.

108