Susanne Koot MAKE up YOUR MIND

Susanne Koot MAKE UP YOUR MIND

how stress and sex affect decision-making Cover design Proefschrift-aio.nl Lay-out and design Proefschrift-aio.nl Printed by DPP

ISBN 978-90-393-6034-7

BESLISSEN Proefschrift Hoe stress en sekse keuzegedrag beïnvloeden ter verkrijging van de graad van doctor (met een samenvatting in aan de Universiteit Utrecht op gezag van het Nederlands) de rector magnificus, prof.dr. G.J. van der Zwaan, ingevolge het besluit van het college voor promoties in het openbaar te verdedigen op woensdag 16 oktober 2013 des ochtends te 10.30 uur

door Susanne Koot geboren op 12 juni 1983 te Nijmegen Promotoren Prof. dr. M. Joëls Prof. dr. L.J.M.J. Vanderschuren

Co-promotor Dr. R. van den Bos

Printing of this thesis was financially supported by PRS Italia, Rome, Italy. TABLE OF CONTENTS

CH. 1 General Introduction 7

CH. 2 Gender differences in delay-discounting under mild food 23 restriction

CH. 3 Home cage testing of delay-discounting in rats 43

CH. 4 Compromised decision-making and increased gambling 57 proneness following dietary serotonin depletion in rats

CH. 5 Time-dependent effects of corticosterone on reward-based 81 decision-making in a rodent model of the Iowa Gambling Task

CH. 6 Corticosterone and decision-making in male Wistar rats: 99 the effect of corticosterone application in the infralimbic and orbitofrontal cortex

CH. 7 Temporal domains in social decision-making following stress 113 in women

CH. 8 General Discussion 131

References 143 Nederlandse samenvatting 173 Curriculum vitae 181 List of publications 183 Dankwoord | Acknowledgements 185 1

6 CHAPTER 1

GENERAL INTRODUCTION

1 What is decision-making? 8

2 Modelling decision-making 9

2.1 Delay-discounting and gambling (risk-proneness) 9

2.2 Iowa Gambling Task 12

2.3 Social decision-making 14

3 Neural substrates 14

4 Research themes of this thesis 17

4.1 Stress and decision-making 17

4.2 Gender differences and decision-making 18

5 Outline of the thesis 19

7 1 What is decision-making?

Both humans and other animals make decisions throughout life. These may be big decisions about how to organise your professional and personal life, but also more 1 trivial daily choices like what to eat. Let’s take this example here: foraging behaviour. Suppose you are in the supermarket for your daily groceries and you are thinking about what to have for dinner. Do you fancy something healthy, or rather something fast and comforting? How much does each meal cost? Perhaps it is worth to go to another supermarket for something in discount. But what if you were a rodent? Imagine you are a rat living in the suburbs of a city like Rome. You are hungry, so you have to go out to find some food. You can stay comfortable and close by in the sewing system, finding some low energetic and rotten vegetables of the daily market. Or you could travel a bit further towards the city centre, taking some more effort and risk when going above ground. There you can scatter in a restaurant’s trash bin for the leftovers of a good smelling pasta carbonara, prepared by a famous Italian cook. However, apart from the fact that it takes a lot of energy to get there, there is also the chance that you encounter a big rat from nearby. He might be higher up in hierarchy, munching on that pasta carbonara already, ready to kick you out of the trash bin. Or, even worse, you could run accidentally into the restaurant’s cat, and already his smell stresses you out. Such stories, for rats or humans, can be described for many behavioural domains, whether related to food, shelter, sexual partners or social environment. And, in addition, in humans for job-related or private contexts. However, in essence they all boil down to the same thing: decision-making is related to assessing costs and benefits of competing actions with either a known outcome or an uncertain result. Decision-making is comprised of several abilities, such as flexibility in behaviour, inhibition of responses to immediate rewards in order to weigh long and short term losses and benefits, and continuous evaluation of the risks and rewards of various options. Impairments in these abilities, necessary for optimal decision-making, can have tremendous influence on daily life. This is particularly apparent in psychiatric disorders, such as pathological gambling, attention deficit hyperactivity disorder and drug addiction, in which impulsivity and decision-making are disrupted. To be able to treat these disorders, but also to understand the impact of e.g. stress and other neuromodulators like serotonin on these processes, unravelling the underlying mechanisms is demanded. Here, I will focus on two aspects associated with decision- making, which recently have gained more and more interest: the effects of stress and gender differences. Decisions are often made under stressful conditions, but unlike domains as memory, the effects of stress on a complex process like decision-making performance are still not fully understood. Likewise, the neurotransmitter serotonin is an important candidate for the modulation of decision-making, but clear evidence and how it relates to e.g. pathological gambling is still sparse. Furthermore, only recently gender differences in decision-making, and particularly the effects of stress thereon, were acknowledged, but it is still not understood very well where these differences originate from and what they imply. With this thesis I will contribute to the knowledge of these processes in these fields. In this Introduction, I will first outline how decision-making can be tested in a laboratory setting in humans and other animals, focussing on the tasks usedin

8 general introduction

this thesis. Subsequently, the neural substrates that underlie decision-making will be indicated briefly. Finally, the effects of stress on choice behaviour and gender differences in decision-making, the central themes of this thesis, will be introduced. This introduction ends with an outline of the experimental chapters and the general discussion.

2 Modelling decision-making

Decision-making in humans and animals has been modelled in many different ways (for reviews see e.g. Brand et al. 2006; De Visser et al. 2011c; Floresco et al. 2008; Weber and Johnson, 2009). Basically all these tasks involve a cost-benefit analysis of competing actions that differ in immediate and/or long-term gains and losses. In humans benefits are often framed as monetary wins, while costs are often represented as monetary losses or delays before gratification. In animals, benefits are often framed as palatable food pellets, while costs are often represented as delays before gratification, physical exercise (lever presses or barriers) or punishments (shock, unpalatable food). Here only those tasks are described which are used in the studies presented in this thesis: delay-discounting and probability-related tasks, the Iowa Gambling Task and social decision-making tasks. Human decision-making is largely mirrored in animal choice behaviour (Van Wingerden and Kalenscher, 2012), making it relevant to use animal models for the investigation of the mechanisms underlying decision-making. One important reason to use animal models of decision-making behaviour is the wide range of techniques available in animal modelling, which makes it possible to resolve causal contributions of brain regions, and to perform pharmacological manipulations which are technically or ethically impossible in human subjects (Kalenscher and Van Wingerden, 2011). Another reason for using animal models of decision-making is the study of the evolutionary roots of human decision-making, sharing neural substrates underlying choice behaviour with animals (Kalenscher and Van Wingerden, 2011).

2.1 Delay-discounting and gambling (risk-proneness) Many paradigms, involving a choice with delayed and/or uncertain reinforcers, have been developed in order to test the self-control abilities and/or impulsive decision- making (Evenden, 1999). Of the three dimensions of impulsivity (motor, attentional and non-planning; Moeller et al. 2001; Patton et al. 1995), here I focus on non- planning or impulsive decision-making, reflecting a lack of regard for the future (Winstanley, 2011). Temporal or delay discounting is a reduction in the subjective value of a delayed outcome (Hamilton and Potenza, 2012). The delay-discounting task measures impulsive choice, in which subjects have to choose between a small immediate reward and a large delayed reward. When testing humans or monkeys, the subjects can choose between a small immediate reward and a larger delayed reward, often both presented on a screen, by pushing the corresponding response button (Evenden, 1999; Homberg, 2012; Monterosso and Ainslie, 1999; Reynolds and Schiffbauer, 2004). Similarly, rodents can choose between an immediately available food pellet and a higher number of food pellets after a certain delay, by responding

9 to either of two operant levers in a Skinner box (figure 1). Despite the similarity between the two test situations, there are also obvious differences (see also below). For instance, in animals, before the testing phase with delayed rewards starts, they are first trained to prefer a large reward over a small one; human subjects instead 1 are given written instructions that they can choose between a small reward now and a large reward within a couple of months. Note that animals are offered a real reward during testing, while human subjects are provided a hypothetical reward, possibly becoming real after the completion of the test. Subjects are expected to shift their preference from the large/delayed to the small/immediate reinforcer as a consequence of late reward delivery (Adriani et al. 2004; Evenden and Ryan, 1996). Impulsive subjects in particular are intolerant to situations in which reward is delayed: smaller, immediate reinforcers are preferred to larger rewards that are available only after a delay (Evenden and Ryan, 1996; Evenden, 1999). In other words, elevated rates of delay discounting are a manifestation of impulsive choice, which has been associated with psychopathology (Swann et al. 2002; Winstanley et al. 2004a). Another proposed way to measure self-control is to exploit reward uncertainty, i.e. by means of investigating the (in)tolerance of an individual to an anticipated risk in rewards, instead of delay. Waiting for a delay means certainly receiving a reward eventually, while waiting for a probabilistic reward involves uncertainty of receiving a reward. In the Probabilistic-Delivery (PD) task (Adriani and Laviola, 2006; Green and Myerson, 2004; Mobini et al. 2000b) subjects are offered a choice between a smaller and certain reward and a larger but probabilistic one (figure 1). Specifically, in this task the large reinforcement is occasionally and randomly withheld, so that the experimental subject faces a lack of gain. The progressive accumulating lack of gains over time clearly have consequences for long-term payoff. This task also provides information reflecting an (in)ability to cope with irregularly delivered reinforcement (Adriani et al. 2012; Ho et al. 1999). In the PD task for rats, animals first learn to prefer a large reward over a small reward. Subsequently, the probability of occurrence of the large reward decreases progressively to very low levels, turning it into a risky choice. Control rats normally change their preference towards the certain (small and sure) reward, which is a safer option beyond probabilities of e.g. 33% (depending on reward size and task duration) for the large reward. Gambling proneness in rats and humans is associated with a maintained preference for the highly uncertain and eventually sub-optimal option (Adriani and Laviola, 2006; Mobini et al. 2002); that is, even beyond the point that shifting towards the small and sure option is more profitable, gambling-prone rats maintain their preference for the large but more risky choice. These two operant-behaviour tasks have, besides technical features like the apparatus used, some conceptual features in common. In both tasks a choice between two options is given, and subjects need to assess the cost-benefit ratios of both options and to possess self-control abilities in order to perform in an optimal way (Adriani and Laviola, 2006; Cardinal, 2006). The detection of reward value across the delayed or uncertain delivery requires a common neural substrate, e.g. an intact nucleus accumbens (Cardinal and Cheung, 2005; Cardinal and Howes, 2005). Nevertheless, time- and probability-induced discounting are different and dissociable processes (Green et al. 1999; Green and Myerson, 2004). The delay-discounting task reflects impulsive behaviour, while the PD task generates patterns of choice that

10 general introduction

rather resemble the features of gambling (Adriani and Laviola, 2006). Subjects might perform similarly on both tasks, but this depends on their sensitivity to risk and loss, flexibility and temptation to gamble (Adriani et al. 2012). Although both tasks seem to be quite straightforward in testing important features of impaired decision- making like impulsiveness and risk-proneness in animals, it should be noted that these tasks also measure other aspects of behaviour related to the fact that animals are supposed to change their choice behaviour. Thus, when animals do not shift to the small immediate reward or small certain reward, this may also be due to a lack of flexibility per se, to habit-formation related to prolonged training or an insensitivity to detect changes. Both human and animal subjects have been tested in the delay-discounting task and PD task. When interpreting the results across species some differences in task features should be taken into account. First, of the two used rewards, food is a primary reward, eliciting a strong direct hedonic response, while money is a strong secondary reinforcer, only rewarding in association with primary reinforcers and presented abstractly within the task (Kalenscher and Van Wingerden, 2011, cf. Petry 2003). Second, in delay-discounting tasks, the presented delays are in the range of months for human subjects (abstract reward) and in the range of seconds for animal subjects (directly eatable). Third, human subjects are generally instructed on the probabilities of a gain or loss, often in single trials (no repetitions), while animals have to learn probabilistic contingencies in multi-trial settings due to the impossibility of verbal instructions (see above). These differences, however, do not seem to negate the suitability of animal models for human studies: human subjects show the same behavioural patterns with both primary and secondary reinforcers (Hayden et al. 2007; McClure et al. 2007) and the same neural networks are involved in making intertemporal decisions with delays of seconds compared to delays of months (McClure et al. 2004, 2007). Therefore, the procedural differences in type of reward and mental processes involved may be less significant than feared (see Kalenscher and Van Wingerden, 2011). Since parallels in behavioural patterns and brain functioning cannot always be drawn directly between humans and animals in general, more comparative research is needed to provide a more thorough insight in mechanisms and brain structures involved in such tasks.

Figure 1 Schematic representation of the delay-discounting task and probabilistic delivery (PD) task, including training and testing phase.

11 2.2 Iowa Gambling Task A more complex and challenging decision-making task is the Iowa Gambling Task (IGT), which was originally developed to assess specific cognitive impairments of patients with damage of the ventromedial prefrontal cortex (Bechara et al. 1994). Later on, 1 impaired performance was also observed in patients with damage in brain regions like the amygdala, insular cortex and the dorsolateral prefrontal cortex (Bechara et al. 1998, 1999; Clark et al. 2003; Fellows and Farrah, 2005; Manes et al. 2002). Furthermore, the IGT is being used to understand the neural mechanisms underlying complex decision-making more thoroughly, as well as to determine decision-making deficits in psychiatric populations (see e.g., Brogan et al. 2010; Cavedini et al. 2002; Ernst and Paulus, 2005; Goudriaan et al. 2005; Sevy et al. 2007). At a more fundamental level, the IGT may mimic the complexity of daily life choices, as the IGT combines the unpredictability of the consequences of a choice, the need to weigh short- and long- term gains and losses, and the necessity to exert behavioural control in order to make the best long-term choices (Bechara et al. 1999). Flexibility in planning to account for various outcomes, the ability to inhibit choosing only immediately rewarding options and to monitor incoming information constantly including the evaluation of the risks and rewards of various options are important in order to perform successfully. In the IGT subjects choose cards, one by one from four different decks, to earn money. Two decks provide immediate moderate monetary gains but also moderate or low losses, and are advantageous in the long run. The other two decks are disadvantageous in the long run since, even though immediate gains are higher, the unpredictable losses are also higher. As such, a conflict is induced between immediate high rewards and long-term gains. Participants are instructed to gain as much as possible, but do not have prior knowledge about the contingencies. Because interest in the use of rodent models for decision-making is growing, several animal models of the Iowa Gambling Task have been developed over the last years (Pais-Vieira et al. 2007; Rivalan et al. 2009; Van den Bos et al. 2006a; Zeeb et al. 2009). The common factor in these models is that choice behaviour is positively reinforced by a food reward, but means of punishment, task duration, prior knowledge of contingencies and pre-training procedure differ between the different models. While they all measure IGT-like decision-making, each of these models has its own specific strengths and weaknesses. For a comprehensive review, see De Visser et al. (2011c). In three studies described in this thesis and involving the IGT, the animal model by Van den Bos et al. (2006a) was used. In Box 1 the task is explained. In order to reduce the amount of animal handling and the number of experimental sessions/days, there was a need for a home cage version of this model. This home-cage version is described in chapter 4 of this thesis.

12 general introduction

BOX 1 rodent Iowa Gambling Task

In the rodent version of the Iowa Gambling Task as developed by Van den Bos et al. (2006) rats have to make choices between four goal arms, two of them baited with either food rewards (sugar pellets) or punishments (unpalatable, but not uneatable, quinine-treated sugar pellets), two of them being empty. In one baited arm, rats receive frequent small rewards (one sugar pellet in 8 out of 10 trials) and infrequent punishments (one quinine-treated sugar pellet in 2 out of 10 trials); this is the advantageous arm. In the other baited arm, the disadvantageous arm, rats receive occasional big rewards (three sugar pellets in 1 out of 10 trials) among frequent punishments (three quinine-treated sugar pellets in 9 out of 10 trials). Thus, uncertainty is provided by varying the sequence of sugar or quinine pellet presentation. The other two arms are empty in order to control for non-specific exploration. The week before testing, rats are habituated to the sugar pellets and allowed to explore the apparatus for 10 min. The apparatus (figure 2) consists of a maze divided in a start box, a choice area and the four different arms. A trial starts when the slide door from the start box opens, allowing the rat to freely explore the choice area. A choice for one of the arms is made when the rat has entered into the arm for at least one-third of the length of the arm. Once a choice is made, the arm is closed to prevent the rat from returning without investigating the reward cup at the end of the arm. Testing occurs over nine days: 10 trials per session on the first six days, increasing to 20 trials per session on the last three days. Rats continuing to eat the quinine- treated sugar pellets are excluded from further testing.

Figure 2 Schematic representation of the rodent Iowa Gambling Task, maze model.

Besides the IGT, there are some other complex decision-making tasks, each focussing on a slightly different aspect of decision-making. The Game of Dice Task examines decision-making in a gambling situation with explicit rules (Brand et al. 2005). In the Balloon Analogue Risk Task (BART, Lejuez et al. 2002) also risk seeking is assessed, but under conditions of fully unknown contingencies. In order to assess decision-making and risk-taking behaviour outside of a learning context, the Cambridge Gamble Task was developed (Rogers et al. 1999b). However, as these tasks have not been used in the studies described in this thesis, I will not go further into detail here.

13 2.3 Social decision-making The majority of research on decision-making has examined individual decisions in which the subject has to consider exclusively his / her own values and preferences in order to select an option. However, in reality, decisions are often made in a context 1 of social interactions (Fehr and Fischbacher, 2003). These social decisions canbe defined as decisions that affect others as well as oneself and are therefore typically informed by both self and other-regarding preferences (Fehr and Camerer, 2007). To study social decision-making in the laboratory, tasks originating from experimental economics have been used to investigate the underlying neural systems (Rilling and Sanfey, 2011). These tasks, with compelling social scenarios, being easy for participants to understand, and easily adaptable to neuroscientific studies, have been used to study several aspects of social decision-making, primarily reciprocal exchange, responses to fairness and equity, and altruism and punishment (Rilling and Sanfey, 2011). In all of these tasks a sum of money is asked to be donated or divided between oneself and another subject (see Box 2 and Rilling and Sanfey, 2011 for detailed explanations).

3 Neural substrates

The output of the decision-making process, i.e. the action to be taken, is determined by an interaction between impulsive or emotionally based systems on the one hand, responding to immediate (potential) rewards as well as losses or threats, and reflective or cognitive control systems controlling long-term perspective on the other hand (Bechara, 2005; Bechara et al. 2000, Mobini et al. 2002). Thus, healthy decision-making requires the activity and functional connectivity between different prefrontal and subcortical areas. As was shown in studies using imaging techniques and in brain-lesioned patients, the main structures involved are the amygdala, the insular cortex, the striatum, and several frontal cortical regions, including the ventromedial prefrontal cortex (vmPFC), orbitofrontal cortex (OFC), the anterior cingulate cortex (ACC), and dorsolateral prefrontal cortex (dlPFC) (Bechara et al. 1999; Bolla et al. 2004; Brand et al. 2007; Fellows and Farah, 2005; Hsu et al. 2005; Manes et al. 2002; Tucker et al. 2004). Table 1 presents an overview of the role of different brain areas in decision-making. These areas and the connections between them underlie a complex pattern of different aspects of decision-making. Roughly speaking, it has been suggested that the output of decision-making processes, i.e. which action is taken ultimately, is determined by an interaction between two different forebrain loops: a limbic (affective/motivational) loop, encompassing the orbitofrontal cortex, the amygdala and ventral striatum, and a cognitive (executive/ motor) loop, encompassing the dorsolateral prefrontal cortex, anterior cingulate cortex and dorsal striatum (Bechara, 2005; De Visser et al. 2011a; 2011b; 2011c; Doya, 2008; McClure et al. 2004; Rivalan et al. 2011; Tanaka et al. 2004; 2007; Yin and Knowlton, 2006; Yin et al. 2008; Zeeb and Winstanley, 2011). While the limbic loop is involved in immediate responding to (potential) rewards, losses or threats as well as in emotional control, the cognitive loop is more involved in long-term or future perspectives, i.e. in cognitive control (Bechara, 2005; Doya, 2008; Gläscher et al. 2012; McClure et al. 2004; Tanaka et al. 2004, 2007).

14 general introduction

BOX 2 testing social decision-making

In the following social decision-making tasks a sum of money is asked to be donated or divided between oneself and another subject (figure 3).

The Dictator Game tests altruism: the second player is a passive recipient of the proposer’s offer and therefore cannot reject it.

Reciprocal exchange is tested in the Trust Game: a player, the investor, decides how much of a sum of money to invest with a trustee. Once transferred, this money is multiplied by some factor and then the trustee has the opportunity to return money to the investor but, importantly, does not need to return anything. If the trustee honours trust and returns money, both players end up with a higher monetary payoff than the original endowment, but if the trustee abuses trust and keeps the entire amount, the investor takes a loss.

Fairness and inequity are tested in the Ultimatum Game: the proposer divides a sum of money. The responder has the option of accepting or rejecting this offer. If the offer is accepted, the sum is divided as proposed, but if rejected, neither player receives anything.

Figure 3 Schematic representation of the Dictator Game, Trust Game and Ultimatum Game.

Decision-making at the level of the frontal cortex is among other factors regulated by monoaminergic neurotransmission (Floresco and Jentsch, 2011; Robbins and Arnsten, 2009). Especially the dopaminergic and serotonergic systems are known to facilitate functional connectivity between the limbic and cortical regions (see table 1), and are therefore important in decision-making. It is beyond the scope and aim of this thesis to discuss the details regarding decision-making and neuromodulators. For recent reviews, see Cools et al. (2011), Homberg (2012) and Rogers (2011).

15 Table 1 Role of different brain areas and neuromodulators in decision-making (NB: the mPFC in rats has been suggested to share an anatomical and functional homology to the ACC and dlPFC in humans, and particularly the infralimbic and the prelimbic region of the mPFC in rats might be similar to the vmPFC and ACC resp. in humans (Brown and Bowman, 2002; Uylings et al. 2003)). 1 brain area function in decision-making reference

vmPFC role in performance monitoring, behavioural flexibility, Birrell and Brown, 2000; Dias et al. 1996; strategy shifting, goal-directed learning behaviour, Floresco et al. 2008; Fuster, 2001; Miller, maintenance of a long-term perspective by withholding to 2000; Ragozzino and Kesner, 2001 respond to immediate rewards or losses

dlPFC role in suppressing risk-taking behaviour, choosing future large Fecteau et al. 2007a, 2007b; Knoch et rewards over immediate small rewards or while incurring small al. 2006; McClure et al. 2004; Tanaka et immediate losses al. 2004

OFC signalling the (emotional) valence of stimuli and expected Elliott et al. 2000b; Frank and Claus, 2006; outcomes, and adjust behaviour (suppressing previously Kable and Glimcher, 2009; Kringelbach rewarded behaviour) when contingencies change 2005; Mar et al. 2011; McClure et al. 2004; Noonan et al. 2010; O’Doherty et al. 2001; Rushworth et al. 2007, 2011; Sescousse et al. 2010; Stalnaker et al. 2009; Sul et al. 2010; Windmann et al. 2006

ACC converging area for cognitive and motor commands: monitors Kennerley et al. 2006; Magno et al. 2006; error-likelihood, the presence of conflicts related to actions, Oliveira et al. 2007; Paus et al. 1993; cost/benefit ratio of alternative actions; key role in choosing/ Quilodran et al. 2008; Rudebeck et al. 2006; updating actions in an uncertain environment Sallet et al. 2007

amygdala signals the appetitive or aversive (emotional) valence of Faure et al. 2010; Gupta et al. 2011; Hayes environmental stimuli; updates value representations; triggers et al. 2011; Morrison and Salzman, 2010; emotional responses to immediate outcomes Pessoa, 2010; Pessoa and Adolphs, 2010

ventral striatum a limbic-motor interface, important in response inhibition Christakou et al. 2004; Clarke et al. 2008; and cognitive flexibility; signals dynamic changes over time; Divac et al. 1967; Floresco et al. 2006; is a motivational engine for the continuation of behaviour; Heekeren et al. 2007; Mogenson et al. 1980; generates emotional responses Schoenbaum and Setlow, 2003; Stern and Passingham, 1995

insular cortex signals incentive or reward value and risky aversive stimuli; Clark et al. 2008; Contreras et al. 2007; integrates representations of the bodily state Kesner and Gilbert, 2007; Naqvi and Bechara, 2009; Preuschoff et al. 2008 ; Weller et al. 2009 ; Xue et al. 2010

neuromodulators role in decision-making reference

serotonin inhibition of impulsive behavioural responses, through Branchi 2011; Cools et al. 2008, 2011; Hayes mediating the interplay between the limbic and cognitive and Greenshaw, 2011; Rogers 2011 loops; attention to environmental stimuli; reward and punishment sensitivity; behavioural flexibility

dopamine signalling within the reward system; associative learning; time Cools et al. 2011; Di Chiara and Bassareo, perception; resistance to short-term reward, despite long-term 2007; Homberg 2012; Meck 2006; Rogers negative consequences 2011; Schultz 2002; Sevy et al. 2006

16 general introduction

4 Research themes of this thesis

4.1 Stress and decision-making In the modern, currently high-demanding (Western) society, people often make decisions under stressful conditions. Especially in stressful jobs, like in the police service, the army, and medical and financial sectors (e.g. Coates and Herbert, 2008) making decisions under stress is part of the daily work. Recently, more and more studies have appeared trying to elucidate the effects of stress on decision-making (review: Starcke and Brand, 2012). In decision-making tasks stressed subjects consider less options and are faster in making their choices (Chajut and Algom, 2003; Keinan, 1987), are less likely to consider possible contingencies (Johnston et al. 1997), need more time to learn cost-benefit contingencies of different choice options (Preston et al. 2007), have impaired cognitive flexibility (Alexander et al. 2007) and adjust insufficiently from automatic responses (Kassam et al. 2009; Porcelli and Delgado, 2009). In general men seem to be more risk seeking when stressed, while women become more risk averse or task-focussed (Lighthall et al. 2009; Van den Bos et al. 2009; Preston et al. 2007). The effect of stress on decision-making is mediated by transmitters and hormones released in response to the potentially threatening situation. One of these stress mediators is (nor)adrenaline which is quickly released after activation of the sympathetic nervous system. Corticosteroids (cortisol in humans, corticosterone in rodents) are released slightly later as the end product of the hypothalamic–adrenal– pituitary (HPA) axis, and are known to cross the blood–brain barrier easily after which they affect brain processing (McEwen 1979). Corticosteroids ensure sufficient energy supply to challenged tissues and control the excitability of neuronal networks, and as such support and regulate the stress response (de Kloet et al. 1999). Cortisol exerts its actions upon binding to the mineralocorticoid (MR) and particularly the glucocorticoid receptor (GR), which is fully activated after stress and is abundantly expressed in the brain (de Kloet 1991; Reul and de Kloet 1985; Sapolsky et al. 1983). Both receptors act as transcription factors and thus slowly but long-lastingly change neuronal function. However, as has become evident recently, corticosteroids can also affect neuronal cell function rapidly in a non-genomic manner via membrane- bound receptors (Joëls et al. 2011). The rapid actions of corticosteroids have been suggested to synergise with and amplify the effects of catecholamines (Roozendaal et al. 2006; Joëls and Baram, 2009), to optimise rapid adaptive behaviour by relocating neural resources away from the higher-order cognitive processing prefrontal regions to the limbic structures (Diamond et al. 2007). The slower genomic cascade seems to be responsible for the regulation of the stress response and the restoration of homeostasis in the aftermath of stress, thus contributing to the normalisation of attentional processing in the aftermath of stress (de Kloet et al. 2005; Henckens et al. 2010, 2011). In general, high levels of cortisol – acting via genomic GR-dependent pathways – are thought to be associated with a loss of top-down control of prefrontal over subcortical areas, at least in male subjects (Arnsten, 2009; Dias-Ferreira et al. 2009; Erickson et al. 2003; Goldstein et al. 2010; Kern et al. 2008; Piazza and Le Moal, 1997; Stark et al. 2006; Wang et al. 2007) promoting e.g. risk-taking behaviour. Interestingly, decision-making of individuals responding with high cortisol levels to

17 the stressful situation is affected while this is not the case in low-responders (Leder et al. 2013, Lighthall et al. 2009, Van den Bos et al. 2009, but see Lighthall et al. 2012). To delineate the effect of stress on decision-making, careful design of the 1 experimental conditions is required. Often the task situation itself is arousing, or even stressful. Especially in animal testing, some arousal is often inevitable, for example when handling the animals or using food deprivation as a method to motivate the animals to engage in the task. This arousal may influence test results. Several efforts have been made to minimise arousal, like a period of rest for human subjects prior to testing, and in the case of animals habituation to the environment and procedures, less severe food deprivation schedules, and even home cage testing to minimise handling prior to or during testing (De Visser et al. 2006; Knapska et al. 2006). While many studies have reported effects of stress on decision-making in humans (Starcke and Brand, 2012), systematic data in animals were lacking at the time that I started my thesis project. Furthermore, in humans data on the effects of stress on social decision-making tasks are scarce. In this thesis, therefore, the effects of stress on decision-making will be explored in order to enhance our understanding of how stress affects this complex behaviour and the underlying prefrontal-subcortical circuits. More specifically, the role of experimental conditions on decision-making in mice and rats will be studied using a delay-discounting paradigm and the rIGT, as well as the effects of corticosterone on decision-making using the rIGT in rats. Furthermore, I report on the effects of stress on social decision-making in humans as well (see section 5 for details).

4.2 Gender differences and decision-making In the majority of decision-making studies only the behaviour of male subjects or animals has been investigated. However, recently gender differences in decision- making performance (Van den Bos et al. 2013) have been found in both humans (Bolla et al. 2004; Overman et al. 2004, 2006; Reavis and Overman, 2001; Van den Bos et al. 2007, 2013) and animals (Van den Bos et al. 2006a, 2007, 2012b). In these studies, using the IGT, male subjects gradually chose the more advantageous long- term options, while female subjects continued choosing both advantageous and disadvantageous options. Gender differences in risk-adjustment (Deakin et al. 2004; Lee et al. 2009; Sapienza et al. 2009; Schubert et al. 1999; Van den Bos et al. 2012b), sensitivity and focus on punishments and rewards ( Eriksson and Simpson, 2010; Lee et al. 2009; Powell and Ansic, 1997), and cognitive control over affective processes in general (Mak et al. 2009) may contribute to differences in decision-making outcomes. It has been suggested that gender differences in the OFC, dlPFC and/or differential hemispheric activation may underlie these behavioural differences (Bolla et al. 2004; Tranel et al. 2005; Van Hasselt et al. 2012; Wager et al. 2003). Often outcomes of gender differences are interpreted as one gender performing better or worse than the other. However, male and female subjects may process information differently rather than performing simply better or worse: in general male subjects acquire information more globally, while female subjects acquire information in a more detailed manner (see review Van den Bos et al. 2013). Whether the outcome is good or bad depends on the context. Factors inherent to the context in which a task is presented, e.g. reward forms (food, monetary, social contact) or

18 general introduction

motivational aspects (in animals food deprivation, in humans economic status), can greatly influence the outcome of a task, possibly resulting in gender differences. Furthermore, the majority of decision-making tasks has been shaped according to decision-making in individual contexts. However, as mentioned above, decisions are often made in social contexts. For example, under demanding conditions like stress, men appear to punish altruistically differently than under relaxed conditions (Vinkers et al. 2013), i.e. stress lowers generosity in men, and 75 min after stress men had a lower propensity for altruistic punishment. As it is unclear how women react for example under stress in such complex social situations, it is important to test this experimentally. Despite recent progress in studies on gender-differences in decision-making (Van den Bos et al. 2013), gender-differences in decision-making, including the effects of stress hereon, are still poorly studied. In this thesis therefore, the effects of gender-differences in decision-making will be studied. More specifically, the role of experimental conditions on decision-making in mice using a delay-discounting paradigm, as well as the interaction of stress and gender on social decision-making in humans will be investigated (see section 5 for details).

5 Outline of the thesis

The overall aim of this thesis is to enhance our understanding of the relevance of gender and stressful conditions on decision-making, in rodents and in humans. In chapter 2 one aspect of decision-making, i.e. impulsive choice, is examined by testing mice in the delay-discounting task. As there are only a few studies on differences between males and females in this task, in this chapter both sexes have been tested. Besides factors like gender and internal state, also the amount of handling the animals receive prior to testing could affect the outcome of more delicate or complex tasks. Therefore, home cage testing has become increasingly popular. To work on a less invasive or interfering method of testing rodent decision- making, in chapter 3 a pilot study on home cage testing of delay-discounting in rats is described. As this study resulted in behavioural patterns similar to those found in traditional Skinner box testing, the way was paved to continue exploring the possibilities of testing rodents on decision-making in a home cage setting. This was done in chapter 4, where a new protocol to execute the rodent Iowa Gambling Task (rIGT; Van den Bos et al. 2006a) in a home-cage setting is presented. In this chapter it is assessed whether manipulation of brain serotonin levels in rats affects their performance in operant-based tasks on both decision-making (rIGT) and gambling proneness (probabilistic-delivery task, PD). After exploring the possibilities to reduce the amount of arousal due to testing itself, the focus of the remainder of this thesis is shifted towards the effect of deliberately induced stress on decision-making. In chapter 5 the early (putatively non-genomic) versus late (putatively genomic) effects of stress on decision-making are addressed by treating rats with corticosterone (the rodent equivalent of cortisol) at two time-points (30 or 180 min) prior to being tested in the classical rIGT in the maze. In chapter 6 the importance of two prefrontal brain areas for decision-making under stressful conditions is further examined. In this study,

19 corticosterone is applied directly in the infralimbic cortex or the lateral orbitofrontal cortex. In chapter 7 the time-dependent effects of stress on social decision-making are addressed in humans. This thesis is ended with a general discussion, placing the results in perspective based on the current literature in the field of decision-making. 1

20 general introduction

21 2

22 CHAPTER 2

GENDER DIFFERENCES IN DELAY-DISCOUNTING UNDER MILD FOOD RESTRICTION

S. Koot a,b a Ethology and Welfare, Faculty R. van den Bos a of Veterinary Medicine, Utrecht W. Adriani b University, The Netherlands G. Laviola b b Behavioural Neuroscience Section, Department of Cell Biology & Neurosciences, Istituto Superiore di Sanità, Roma, Italy

Behavioural Brain Research (2009) 200: 134-143.

23 Abstract

Impulsivity, a core symptom of attention-deficit hyperactivity disorder (ADHD), is tested in animal models by delay- discounting tasks. So far, mainly male subjects have been used in this paradigm at severe levels of food restriction. 2 Here we studied the impulsive behaviour of CD-1 adult male and female mice at mild levels of food restriction. Mice maintained at 90±5% of ad libitum bodyweight, were tested in operant chambers provided with nose-poking holes. Nose poking in one hole resulted in the immediate delivery of one food pellet (small-soon, SS), whereas nose poking in the other hole delivered five food pellets after a delay (large-late, LL), which was increased progressively each day (0–150 s). Two subgroups emerged: individuals that shifted at short delays (“steep”) and individuals that did not shift, even at the highest delays (“flat”). Analysis showed that “steep” females shifted at shorter delays than “steep” males, while no difference existed between males and females within the “flat” sub-population. In homecage circadian activity as well as in a novelty-seeking test, females were more active than males. It can be concluded from these results that female mice are more impulsive than male mice under mild food restriction. This is in contrast with findings in earlier studies with more severe food restriction. Therefore, an alternative explanation is that females are more explorative, and that different features might be tested in delay-discounting paradigms, depending on restriction levels.

24 Gender differences in delay-discounting under mild food restriction

1 Introduction

An important symptom of several psychiatric disorders is the lack of self-control, like, for example, in attention-deficit hyperactivity disorder (ADHD) (Swanson et al. 1998). Impulsive decision-making has recently received more and more attention (Bechara 2004; Krawczyk 2002). Several aspects of impulsivity have been studied with operant-behaviour paradigms in animals. Impulsiveness can be defined in several ways, including (a) the failure to resist an impulse, drive, or temptation, (b) responding without consideration of alternatives and/or consequences, or (c) behaving in a way which is inadequate to the environmental contingency (Adriani et al. 2003a). Impulsivity may be caused by an altered function of the serotonergic (5- HT) system as this modulates premature and impulsive responding (Bizot et al. 1999; Linnoila et al. 1983; Soubrié 1986; Thiébot et al. 1985). A widely adopted paradigm to study impulsivity in laboratory settings assumes that impulsive subjects are intolerant to situations when reward is delayed (Solanto et al. 2001; Sonuga-Barke et al. 1992). This paradigm involves a delay-discounting task in which subjects are given a choice between a small, immediate (small-soon, SS) reinforcer and a larger, delayed (large-late, LL) one: subjects are expected to shift their preference from the LL to the SS reinforcer as a consequence of late delivery well before a theoretical economic indifference point is reached (Adriani et al. 2004; Evenden and Ryan, 1996). This task provides an indication about processing the value of a reward with delayed delivery. Temporal discounting is known to involve serotonergic projections to the prefrontal cortex (PFC) (Dalley et al. 2002; Harrison et al. 1997a) and intact connections between the PFC and the striatum (STR) Cardinal et al. 2001, 2004; Christakou et al. 2001, 2004). Furthermore, the indirect dopamine (DA) agonist d-amphetamine can decrease impulsivity in the delay-discounting paradigm, potentially through interactions between the 5-HT and DA systems (Winstanley et al. 2005). 5-HT/DA interactions within the nucleus accumbens may play a role in the control of impulsivity, and interacting alterations of 5-HT and/or DA activity may produce the actual behavioural phenotype (Oades 2002), but the exact mechanism is yet unknown. In animal models of the delay-discounting task, mainly males have been used. However, gender differences have been found in both humans and animals in other tasks concerning impulsivity and decision-making. For example, the Iowa Gambling Task (IGT) – involving uncertainty of outcomes and conflicting immediate and long- term payoffs – is used to study decision-making processes and long-term efficiency of behaviour (Van den Bos et al. 2006a). After the initial exploratory trials in the IGT men choose gradually the more advantageous long-term options, while women continue choosing both advantageous and disadvantageous options (Bolla et al. 2004; Overman et al. 2004, 2006; Reavis and Overman, 2001; Van den Bos et al. 2007). This gender difference is also observed in mice and rats in an animal analogue of the IGT (Van den Bos et al. 2006a, 2007). It was concluded from cross-species findings on this task that males tend to focus on long-term goals only, while females tend to balance these with short-term interests as well. In other words, it was suggested that males shift from exploration to exploitation, while females remain exploratory. These behavioural differences may be due to changes in 5-HT and DA activity, as

25 an increase of brain 5-HT levels in female rats led to a suppression of exploratory behaviour (Laruelle 2000) and lowered DA activity resulted in a weaker performance in the IGT (Sevy et al. 2006). The menstrual cycle appeared not to be a decisive factor in gender differences in the IGT (Van den Bos et al. 2007). Generally, mice and rats are food-restricted in operant schedules, to increase their motivation to work for a food reward. Animals were maintained at 80–85% 2 of their initial bodyweight throughout the training and testing phase of the delay- discounting task (Adriani and Laviola, 2003; Adriani et al. 2003a, 2003b). However, food restriction may be perceived as a stressor (Roper and Nieto, 1979; Wayner and Rondeau, 1976). Indeed, a food shortage may increase the release of glucocorticoid hormones and sensitize mesencephalic dopaminergic projections to the nucleus accumbens, influencing behavioural responses (Cabib et al. 1997). In other words, food restriction up to 80–85% of ad libitum bodyweight in combination with solitary housing promotes behavioural sensitisation (Cabib et al. 2000; Piazza and Le Moal, 1998; Rougé-Pont, 1995), a phenomenon used in laboratory settings to model psychotic symptoms (Laruelle 2000; Lyon 1991; Robinson and Becker, 1986). Accordingly, in response to food restriction (84% of ad libitum bodyweight), mice show very high rates of cage-cover climbing, a stereotyped behaviour which can also be induced by administration of DA agonists (Costall et al. 1980; Marcais et al. 1978; Martres et al. 1977). Therefore, it is suggested that food restriction has a modulating effect on forebrain 5-HT/DA interactions, and consequently on operant behaviour. Hence, we propose that psychiatric symptoms, rather than the baseline motivational processes, are possibly modelled by conducting this task with heavily food-restricted mice and rats. As a matter of fact, since food restriction can be perceived as a stressor, decision-making processes may be much better evaluated at mild restriction levels. Gender differences in the delay-discounting task have hardly been addressed. However, using food restriction levels of 80–85% (compared to the ad libitum bodyweight), it was found that males were more impulsive than females in both rats (Adriani et al. 2003b) and mice (Laviola and Adriani, unpublished). Conversely, when comparing different food restriction levels, female rats appeared to show more impulsive-choice behaviour at the 90% level than at the 80% level (Bradshaw and Szabadi, 1992; Ho et al. 1997; Wogar et al. 1992). In otherwords, levels of food restriction might affect the outcome, and indeed the interpretation of data, originating from the delay-discounting task. High levels of locomotor activity, exploration and impulsivity tend to be associated (Swanson et al. 1998). Gender differences have been observed in both activity levels (Beatty 1979; Broida and Svare, 1984; Palanza et al. 2001) and exploratory activity, such as measured in a novelty-seeking task (Palanza et al. 2001). Therefore, we also measured activity levels as well as novelty-seeking behaviour in male and female mice. Furthermore these data indicate whether our mice show normal base-line behaviour. In short, the aim of the present study was to investigate gender differences in the delay-discounting task but under a mild food restriction level. Gender differences in home-cage circadian activity and behaviour in the novelty-seeking task were studied as well.

26 Gender differences in delay-discounting under mild food restriction

2 Materials and methods

This study is parallel part of a project contributing to the development of a new model in mice for the aetiology of addiction (Adriani et al. 2008). For this purpose, mice received vehicle treatment for immunisation (see Capone et al. 2008) and blood samples were drawn, with no interference however on validity of behavioural testing. All experimental procedures were approved by Institutional Animal Survey Board, on the behalf of Italian Ministry of Health, and were in close agreement with European Communities Council Directive (86/609/EEC) and Italian law. All efforts were made to minimize animal suffering, to reduce the number of animals used, and to use alternatives to in vivo testing.

2.1 Subjects Male and female mice of the outbred CD-1 strain (Charles River, Italy) were used. Animals were 21 days old (PND 21) upon arrival and were housed in Plexiglas cages (Macrolon type II cages, 33cm × 13 cm × 14 cm) with sawdust bedding. They were kept in pairs according to sex in an air-conditioned room (temperature 21±1 °C, relative humidity 60±10%) under a reversed 12 h light–dark cycle (lights off at 09.00 AM). Water and standard food (Altromin-R, A. Rieper S.p.A., Vandoies, Italy) were available ad libitum. After 1 week of acclimatisation, mice of both genders (n = 12 each) entered into the testing protocol (see Figure 1 for an overview of the entire experimental design).

2.2 Vehicle treatment Animals were injected intraperitoneally (i.p.) with an emulsion of 0.125ml sterile saline plus 0.125ml complete Freund’s adjuvant (CAF; at PND 34) or incomplete Freund’s adjuvant (IFA; at PND 46) (Capone et al. 2008; Levite et al. 1999).

2.3 Blood samples Blood samples were collected by lateral tail vein incision on PND 32 and 68.

Figure 1 Experimental design.

27 2.4 Home-cage circadian activity monitoring At three time windows, mice were continuously monitored for spontaneous home- cage activity (Adriani et al. 2003a; Capone et al. 2008; Dell’Omo et al. 2002). This was done in three different periods for three subsequent days (PND 29–31; PND 42– 44; PND 51–53). An automatic device was used, with small passive infrared sensors positioned on the top of each cage (Activiscope system; http://www.newbehavior. 2 com). These sensors detect any movement of the mice (sampling rate 20 events/ second). Data were recorded by a computer with dedicated software. Scores were obtained during 60-min intervals and were expressed as counts/minute (cpm). The 24 h profile of circadian activity was obtained by averaging 3 consecutive days of continuous registration. The access of authorised personnel to the animal room was not restricted and followed the routine daily schedule.

2.5 Novelty seeking 2.5.1 Apparatus The experimental apparatus consisted of an opaque Plexiglas rectangular box with smooth walls and floors, which was subdivided into two differing compartments (20cm × 14cm × 27cm each). The opening between the two compartments could be closed with a temporary partition. Visual cues were associated with each compartment: one compartment had a white floor, one white wall and three black walls, while the other compartment had a black floor, one black wall and three white walls. Each compartment was provided with four pairs of infrared photobeams, placed on the wall at a few cm above the floor, spaced 5.5cm apart. Beam interruptions were recorded by a computer with custom-made software. The following measures were obtained automatically: (1) time spent in each compartment, (2) activity rate in each compartment (number of beam interruptions/second), and (3) frequency of crossings between the two compartments (number of crossings/minute). The whole session was subdivided into 5-min intervals. Two boxes placed in a soundproof test room with dim illumination were used; each gender was always tested in the same box. The floor of the test apparatus was cleaned after each animal.

2.5.2 Procedure The whole experimental schedule (Adriani et al. 1998) took a total of 4 days, each subject being tested between 13.00 and 17.00 h. The experiment was performed either at PND 54–57 (one cage mate) or at PND 60–63 (the other cage mate). Testing of different experimental groups was counterbalanced across time. During testing, the remaining cage mate stayed in the animal room to prevent visual, auditory or olfactory communication between cage mates. On the first 3 days, mice were placed in the black floor/white wall compartment of the apparatus for 25 min, in order to familiarise them with the procedure and apparatus, while the other compartment was left unknown. On the fourth day, animals were placed in the familiar compartment as usual. After 20 min, the partition separating the two compartments of the apparatus was opened and mice were allowed to explore both compartments of the apparatus freely for 5 min.

28 Gender differences in delay-discounting under mild food restriction

2.6 Delay-discounting task 2.6.1 Apparatus Computer-controlled operant chambers (LabLink, Coulbourn Instruments, Allentown, PA, USA) were used as experimental apparatus. The chambers, constructed of aluminium and Plexiglas with grid floor, were provided with a chamber light, two nose-poking holes, two feeder devices, two magazines placed above the nose- poking holes where pellets (20 mg, BioServ, Frenchtown, NJ, USA) were dropped, two magazine lights, and an aluminium platform with climbing ramps, designed (PRS Italia, Rome, Italy) to make the food magazines accessible to mice. Nose poking into the holes was detected by a photocell and was recorded by a computer, which also controlled food delivery. Four chambers were used; each mouse was tested in the same chamber. The grid and walls of the operant chamber were cleaned after each animal.

2.6.2 Procedure Mice were tested at adulthood (PND > 75). Before the schedule started, animals were familiarised with BioServ pellets in the home-cage on 3 subsequent days. Subsequently, each animal was placed in the operant chamber for 20 min daily. The first day was just for familiarisation with the novel environment (no pellets delivered). For the 2 subsequent days, pellets were automatically dropped in the magazine with a 60 s RT interval. During these 3 days of chamber familiarisation and magazine training, nose-poking holes were closed by covering aluminium plates. After the 20 min session, mice were returned to their home-cages, where they were given some standard food. Mice received approximately 65% of their ad libitum food intake, in order to deprive the animals up to 90±5% of their free feeding bodyweight. This procedure resembles the mild level of food restriction in other studies (Homberg et al. 2008; Van den Bos et al. 2006a, 2006c) on decision-making in rats and mice. Food restriction was applied to increase the animal’s motivation to work for food delivery during the training and testing phase. After these 3 days, the regular training procedure started (8 days). Again each animal was placed in the operant chamber for 20min daily (Adriani et al. 2003b). Nose poking in one of the two holes [termed small and soon (SS) hole] resulted in the delivery of one pellet of food in one magazine, whereas nose poking in the other hole [termed large and late (LL) hole] resulted in the delivery of five pellets in the other magazine. After nose poking and before food delivery, the chamber light was switched on for 1 s. After food delivery, the particular magazine light was switched on for 25 s during which additional nose poking was without any scheduled consequences (timeout). By considering an average trial made of 25 s timeout plus 10 s of spontaneous waiting, the estimated indifferent point would be 140 s (Adriani and Laviola, 2009). At this point, by definition, animals would gain the same quantity of food irrespective of their decision, whereas LL-preference would be the optimal strategy before this point. Any decrease of LL-preference before this point is classically taken as an index of intolerance to delay (Evenden and Ryan, 1996, 1999). During the testing phase (9 days), a delay was inserted between nose poking in the LL hole and the delivery of the five-pellet reward. The chamber light was switched on during the full length of this delay. Any additional nose pokes during this interval

29 were not reinforced, i.e., ineffective nose pokes or “inadequate responding” (Sagvolden, 2000; Sagvolden and Sergeant, 1998). The delay was kept fixed for each daily session and was progressively increased across subsequent days (0, 5, 10, 17, 30, 42, 60, 90, 120, 150 s). The whole experimental schedule took 20 days; subjects were tested between 10.00 and 17.00 h. Bodyweight was measured at PND 32, 68 and 76 and daily during the training and testing phase of the impulsivity experiment 2 (PND 83–99).

2.7 Data analysis 2.7.1 Bodyweight To detect gender differences in bodyweight both preceding as well as throughout the delay-discounting task, repeated measures ANOVAs were performed (within subjects factor: time or delay; between-subjects factor: gender).

2.7.2 Home-cage circadian activity To detect gender differences in home-cage activity repeated measures ANOVAs were performed (within-subjects factor: hour; between-subjects factor: gender).

2.7.3 Novelty seeking Data were analysed by ANOVAs (between-subjects factor: gender) to detect gender differences in time spent in novel compartment as well as the number of crossings. Repeated measures ANOVAs (within-subjects factor: interval; between subjects factor: gender) were performed to analyse general activity rate within sessions.

2.7.4 Delay-discounting task The dependent variable was the choice (%) for the large reinforcer, i.e., the percentage of preference for LL over total choices during the test phase. Data were analysed by repeated measures ANOVAs (within-subjects factor: delay; between-subjects factor: gender). Separate analyses were also performed when allowed. Another parameter was the slope of the preference-delay curve. For each individual mouse, the slope value was calculated using the Microsoft Excel “slope” function, with LL-preference as the y-axis data and log(delay + 1) as the x-axis data (Evenden and Ryan, 1999). Then, each gender group was divided into two subgroups on the basis of the distribution of slope values and median values (Adriani et al. 2003a). Within each gender group, half of the mice (whose slope value was above the median) were assigned to the subgroup “flat” and the other half (whose slope value was below the median) were assigned to the other subgroup “steep”. Such a procedure has already been validated in other animal studies (Adriani et al. 2003a, 2007). Thus, the design for these repeated measures ANOVAs was gender × subgroup × delay (0–150 s). Delay was the within-subjects factor, whereas gender and subgroup were the between-subjects factors. Multiple comparisons within significant interactions were performed with the one-sample T-test. For the ANOVAs and T test the statistical package SPSS 14.0 was used. Statistical significance level was set at p≤0.05 (two-tailed).

30 Gender differences in delay-discounting under mild food restriction

3 Results

3.1 Bodyweight Males were in general heavier than females and gained more bodyweight than females with time. An ANOVA yielded a main effect of gender (F(1,22) =53.040, p < 0.001), as well as a gender × time interaction (F(2,44) = 19.157, p <0.001; see Table 1).

Table 1 Mean (±SEM) bodyweight (g).

Post-natal day

32 68 76

Male mice 29.40 ± 0.53 39.54 ± 0.66 39.57 ± 0.81

Female mice 24.69 ± 0.53 31.19 ± 0.92 31.57 ± 0.84

3.2 Home-cage circadian rhythm The circadian activity patterns differed according to gender: females were more active than males in all three periods. An ANOVA showed a strong trend for gender during the first period (baseline, PND 29–31: F(1,10) = 4.653, p = 0.056), and yielded a significant gender effect during the second (half-term, PND 42–44: F(1,10) = 13.011, p = 0.005) and third (final, PND 51–53: F(1,10) = 18.859, p = 0.002) period. Gender × time interactions were found in the half-term period (baseline: F(23,230) = 1.374, p = 0.125; half-term: F(23,230) = 1.916, p = 0.009; final: F(23,230) = 1.241, p = 0.211). Figure 2 shows the time-course of the 3-day averaged values of the mean activity scores/hour during the final period.

Figure 2 Circadian rhythm of home-cage activity in male and female mice, housed in pairs. Activity scores were obtained as counts/minute expressed during 1 h points (n = 6 for each group). The 24 h profile was obtained by averaging 3 days of continuous registration during the final observation period (PND 51–53) and shown as mean per group (SEM values were omitted for sake of clarity).

31 3.3 Novelty seeking 3.3.1 Familiarisation period Females were overall significantly more active than males on each familiarisation day (day 1: F(1,22) = 6.733, p = 0.017; day 2: F(1,22) = 6.047, p = 0.022, day 3: F(1,22) = 9.547, p = 0.005). No gender × phase interaction was found (day 1: F(4,88) = 1.843, p = 0.128; day 2: F(4,88) = 0.420, p = 0.794, day 3: F(4,88) = 0.310, p = 0.871) indicating 2 that the time-course of activity (habituation profile during familiarisation) did not differ between sexes (see Figure 3).

Figure 3 Mean (±SEM) locomotor activity, measured as number of photobeam interruptions per interval during a 25 min session/day, shown by male and female mice on familiarisation days (day 1, day 2 and day 3; see panel A) and testing day (day 4; see panel B). The partition opening occurred on testing day after 20 min, giving free access to both the familiar and the novel compartments for 5 min.

32 Gender differences in delay-discounting under mild food restriction

3.3.2 Testing day For the final test we compared activity during the 5 min before vs. the 5min after partition opening. Mice were on the whole more active after (1246.2±52.95 photobeam interruptions) the opening of the partition than before (464.6±39.96; phase (F(1,20) = 226.180, p < 0.001). During the exploration of the novel compartment, females were more active than males, with a significant gender effect being found (F(1,20) = 6.216, p = 0.022). Mice showed many forth-and-back crossings between familiar and novel compartment after the partition was removed. Females (18.80±2.08) showed more crossings than males (12.75±1.32; F(1.18) = 7.288, p = 0.015). As for novelty seeking, both males (239.00±3.27 s) and females (241.90±8.27 s) significantly preferred the novel compartment, but they spent the same amount of time into it, during the 5 min (300 s) period after removing the partition (F(1.20) = 0.121, p = 0.731).

3.4 Delay-discounting task As expected, both male and female mice developed a clear preference for LL with training: males 74.27±4.62%, females 67.09±3.83% (see Figure 4, panel A, delay 0 s). As delays became longer the preference for the large reward appeared to decrease similarly in male and female mice (delay, F(9,198) = 5.769, p < 0.001; gender × delay, F(9,198) = 1.624, p = 0.111; gender, F(1,22) = 1.872, p = 0.185). Overall no preference for SS was found. However, when the data were analysed with respect to the “steep” or “flat” subgroups, the “steep” subgroup showed a stronger decrease in preference for LL as the delays increased than the “flat” group (subgroup × delay, F(9,180) = 4.462, p < 0.001). This was apparent in both males (F(9,90) = 2.848, p = 0.005) and females (F(9,90) = 2.546, p = 0.012; see Figure 4, panels B and C). Closer inspection of Figure 4, panels B and C shows that only in “steep” female mice a preference shift from LL towards SS emerged with longer delays. Analysis of the delays 30–150 s showed that “steep” female mice had a lower preference for LL than “steep” male mice (F(1,10) = 9.791, p = 0.011). Overall no significant difference in relative decrease in body weight was observed between males and females during the testing period of the impulsivity task (gender, F(1,20) = 0.007, p = 0.933; gender x delay, F(9,180) = 0.992, p = 0.448; see Figure 5), nor between “steep” males and “steep” females (gender, F(1,10) = 0.657, p = 0.436; gender × delay, F(9,90) = 0.868, p = 0.557). The total intake of pellets per session decreased across delays (delay, F(9,90) = 14.036, p < 0.001). The pellet intake during sessions tended to be higher in females than males as delays became longer (overall males and females: gender, F(1,20) = 1.741, p = 0.202; gender × delay, F(9,180) = 1.690, p = 0.094; “steep” males and “steep” females: gender, F(1,10) = 0.989, p = 0.344; gender × delay, F(9,90) = 1.809, p = 0.077) (see Figure 6).

33 Figure 4 Mean (±SEM) choice (%) of the large reinforcer, demanded by nose poking at the LL hole shown by male and female mice during the delay-discounting task. Two subgroups (n = 6 each) were formed on the basis of the median value in the slope values. Both within the male 2 (panel A) and female (panel B) groups distinct subpopulations were evident, characterised by either a “flat” or a “steep” preference delay curve. Within the “steep” population, female mice had a lower preference for LL than male mice with increasing delays (30–150 s) (panel C). *p < 0.05; **p < 0.005 in multiple comparisons between gender and/or subgroup.

34 Gender differences in delay-discounting under mild food restriction

Figure 5 Mean (±SEM) bodyweight of male and female mice (n = 12 per group), measured daily as the percentage of ad libitum body weight during the testing phase of the delay- discounting task. No significant difference between males and females was observed, nor between the “steep” and “flat” subpopulations.increasing delays (30–150 s) (panel C). *p < 0.05; **p < 0.005 in multiple comparisons between gender and/or subgroup.

Figure 6 Mean (±SEM) number of pellets received per session by males and females within the “steep” sub-population (n = 6 per group). No significant difference between males and females overall was observed, nor between the males and females of the “steep” sub-population. *p < 0.05 in multiple comparisons between gender and/ or subgroup.

4 Discussion

4.1 General In the present study we investigated gender differences in home-cage activity, in a novelty-seeking task, as well as in a delay-discounting task under mild restriction conditions (90±5% of ad libitum bodyweight). In the delay-discounting task, adult female mice showed a stronger shift than males from a large and late (LL) reward towards a small and soon (SS) reward as delays became longer, under a mild food restriction level. Furthermore, females were more active than male mice both in their home-cage and when experiencing the novel compartment during the novelty- seeking task.

4.2 Gender difference in activity and novelty seeking In agreement with earlier reports (Beatty, 1979; Broida and Svare, 1984; Palanza et al. 2001), female mice were more active in their home-cage than males across the age span that we measured (PND 29–53). However, as proposed by others (Broida, 1984) for photobeam interruptions, passive infrared interruptions might only provide an indication of level of movement, not the type of movement being performed. For example, it was found (Russell, 1973; Woods, 1962) that in the home-cage females spent more time on movements measured as highly active, such as rearing, ambulating, and sniffing, than males, which spent more time on grooming, eating,

35 and drinking. Such differences may explain the sex differences in activity measured by the number of passive infrared interruptions observed in the present study. In the novelty-seeking task, females were more active during the familiarisation period as well as on the testing day, both before and after opening the partition (cf. Palanza et al. 2001). Furthermore, females performed more crossings between the familiar and novel environment than males. Unlike previous reports on CD-1 2 mice (Palanza et al. 2001), but similarly to results with rats (Adriani et al. 2003b), no gender difference was found in the novelty preference: male and female mice spent comparable amounts of time in exploration of the novel compartment, which is taken as an indication of similar novelty-induced motivation. However, some considerations may be put forward. Firstly, in the current study, mice were tested nearly at adulthood (PND 54–63), while in the previous study (Palanza et al. 2001) animals were tested throughout adolescence (PND 33–43). When compared to adults, periadolescent mice (particularly males) express elevated basal levels of behavioural activation (Palanza et al. 2001) and elevated levels of novelty seeking (Adriani et al. 1998; Laviola et al. 1999). The gender difference in novelty- (and risk-) seeking throughout adolescence is classically explained by the fact that most male mammals reaching puberty usually emigrate from their natal areas, whereas females are typically philopatric. Thus, novelty seeking may be adaptively elevated in adolescent males, and motivation for novelty may then decrease as subjects grow towards adulthood (Laviola et al. 2003). Thus, the gender differences could disappear likewise. Secondly, the ontogenetic pubertal period is characterised by a prominent rise of gonadal hormones, with the consequent sexual maturation of each individual’s physiological and behavioural patterns (Ojeda and Urbanski, 1994). In the previous study (Palanza et al. 2001), animals were also monitored for enduring consequences of the “hormonal milieu” due to the intrauterine position. The gender difference in novelty-seeking drive, found by Palanza and colleagues (2001), was strongly dependent on the prenatal hormones: adolescent males which were surrounded in utero by two other males were the most novelty seekers. Conversely, when surrounded in utero by two females, males were indistinguishable from females in spite of their age (see also Laviola et al. 2003). Thus, gender differences in the novelty-seeking drive are not always detectable as a function of age and hormonal status. Furthermore, the period after partition opening lasted only 5 min in the present study, whereas Palanza and colleagues (2001) observed a gender difference during 20 min after partition opening. Visual inspection of their novelty preference data on the first 5 min interval does not show a clear gender difference either, suggesting that we could have found gender differences likewise if we had used a 20-min period as well. It is thus difficult to draw any direct comparison between these two studies, due to “hormonal” and “age” factors and/or to “methodological” differences. Thus, it may be concluded that our mice show normal baseline behaviour, at least regarding activity levels.

36 Gender differences in delay-discounting under mild food restriction

4.3 Delay-discounting task 4.3.1 Subgroup and gender differences Although in both male and female mice the preference for LL appeared to decrease as the length of the delay increased, no clear shift in preference from LL to SS was observed overall. Only when we analysed the data with respect to the individual profiles (“steep” vs. “flat”), clear-cut differences emerged. Such approach has been validated in an earlier study using the delay-discounting paradigm (Adriani et al. 2003a), where comparable subgroup differences (“steep” vs. “flat”) were found in adolescent male subjects of the spontaneously hypertensive rat (SHR) strain, a model for ADHD. With respect to subgroups, the “steep” animals show a strong decrease in preference for LL, whereas the “flat” ones remain to prefer the LL even when exposed to long delays. Compared to the “flat” subgroup, the “steep” subgroup showed reduced noradrenalin (NA) levels in the cingulated cortex (CC) and medial- frontal cortex (MFC), as well as a lower 5-HT turnover in the MFC (Adriani et al. 2003a). The authors concluded from these findings that the lowered 5-HT activity as well as reduced levels of NA contributed to the occurrence of impulsivity. Indeed, a reduced central 5-HT activity predisposes a subject to impulsive tendencies in tasks of sustained attention (Harrison et al. 1997a, 1997b) and in a delay-discounting paradigm (Mobini et al. 2000a, 2000b; Wogar et al. 1993). However, also an elevated serotonergic function in the prefrontal cortex (PFC) of the rat has been associated with a deficit in impulse-control (Dalley et al. 2002). These apparently contradictory findings regarding the effect of 5-HT activity on impulsivity ledto the argument that opposite deviations from an optimal level of physiological regulation can produce similar effects on impulsivity (Adriani et al. 2003a). Furthermore, dopaminergic (DA) dysfunction, such as in the frontal cortex region, is suggested to be another independent marker of impulsivity (Dalley et al. 2002; Puumala and Sirvio, 1998; Van Gaalen et al. 2006). The behaviour of the “flat” subgroup (Adriani et al. 2003a) was explained by their high levels of NA in the MFC and CC, causing a highly selective focus of attention, especially in stressful conditions (Carli et al. 1983; Robbins 2002). As a consequence, this subgroup might have been unable to pay attention to changes in the environment, and thus unable to shift from the LL to the SS hole (Adriani et al. 2003a). In other words, the behaviour of the “flat” subgroup may be explained in the domain of attention rather than self-control. With respect to present gender differences, “steep” females showed a shift towards the SS, starting a slow but steady decrease from a delay of about 10s and crossing the 50% chance-level between 42 and 60 s, i.e., when such a shift is still clearly suboptimal. Conversely, “steep” males hardly did so, starting toshift only from a delay of about 60 s and never really crossing the chance-level of 50% LL/50% SS. Interestingly, the opposite was found in a previous study (Laviola and Adriani, unpublished), in which feeding restriction was however stronger (80–85% of ad libitum bodyweight). In that condition, the shift in preference from LL to SS was quicker for males than for females, i.e., at delays of 30 s and 45 s, respectively. It was suggested that males were more intolerant to delays in delivery of a given reinforcement than females, at least under a great urge to seek for food. In classic rat models and other delay-discounting studies, a delay-intolerant profile is taken

37 as an index of elevated impulsivity (Adriani and Laviola, 2003, 2006; Adriani et al. 2003a, 2004, 2007; Bizot et al. 1999; Evenden and Ryan, 1996, 1999; Laviola et al. 2003; Logue 1988; Thiébot et al. 1985). Thus, one would conclude from the present results that female mice are more impulsive than male mice, at least under mild food restriction, when the urge to seek for food has a lesser impact. This would be in line with conclusions from previous studies on gender differences 2 in impulsivity. Perry and colleagues (Perry et al. 2007) showed that in a subgroup of rats, selectively bred for low saccharin intake, females were more impulsive than males in a similar delay-discounting paradigm. Furthermore, it was found in adult NMRI mice (Schiller et al. 2006b) that females tend to display higher 5-HT1A receptor binding compared to males: mainly in cortical regions females had higher [3H]8-OH- DPAT-specific binding compared to males. Earlier it was shown in individually housed male mice that high 5-HT1A receptor binding seems to be related to high locomotor activity, exploration and impulsivity (Schiller et al. 2006a). Moreover, females are more active (present study) (Beatty, 1979; Broida and Svare, 1984; Palanza etal. 2001) and show lower anxiety compared to male rodents (Broida and Svare, 1984; Fernandes et al. 1999; Johnston and File, 1991). From these considerations,

the functional relevance of high forebrain 5-HT1A receptor binding is pointed out, especially in the cortex, for the activity-driven and novelty-preferring behavioural phenotype seen in females (Schiller et al. 2006b). These neurobiological facts suggest that females are likely to be more impulsive than males, as the present data from the “steep” subgroups show as well. An alternative explanation could be that females are more flexible in their behaviour or more exploratory, in agreement with previous reports of increased reactivity to environmental changes in this sex (Palanza et al. 2001; Renner et al. 1992). It should be noted that females finally tended to earn more pellets than males at the longest delay (150 s, which is slightly beyond the indifferent point under present conditions). If delays were increased further beyond the indifferent point, namely when SS preference becomes optimal, then behaviour of “steep” females would be the most adaptive. On the contrary, “steep” males seemed to be regressed towards indifference, and got stick to chance-level (Evenden and Ryan, 1996, 1999). This may perhaps suggest that the strategy adopted by females, when facing the task’s requirements, could lead to a better adaptation in case of a continued increase to very long delays, better than males. In an experiment using another decision-making task, i.e., in a rodent version of the Iowa Gambling Task (IGT; Van den Bos et al. 2006a), a similar conclusion was reached with respect to exploratory tendencies for male and female rats and mice. The IGT is a model of the development of everyday life long-term profitable strategies (Bechara et al. 1994), using similar mild food restriction levels in the rodent IGT as in the present study. In this task, males shifted from exploration to exploitation across trials, while females remained largely exploratory (Van den Bos et al. 2006a). In other words, once males figured out what the best long-term advantageous option was after initial exploration, they stuck to that choice, while females continued to explore. Thus, males tend to focus exclusively on exploitation, being very goal-directed, and females combine exploration and exploitation. Additionally, it maybe suggested that the higher levels of activity of females compared to males are also an expression of this enhanced exploratory drive.

38 Gender differences in delay-discounting under mild food restriction

4.3.2 Level of food restriction One of the main differences in experimental set-up between the current and previous studies (Adriani and Laviola, 2003, 2006; Adriani et al. 2003a, 2003b, 2007; Laviola and Adriani, unpublished) is the level of food restriction. Generally animals are food deprived up to 80–85% of their ad libitum bodyweight, while in the present study mice received standard chow to maintain them on 90±5% of their ad libitum bodyweight. Food restriction is applied to increase the animal’s motivation to work for food delivery in operant paradigms like the delay- discounting task. However, the condition of food restriction up to 80–85% of initial bodyweight in combination with solitary housing has been shown to promote behavioural sensitisation (Cabib et al. 2000; Piazza and Le Moal, 1998; Rougé- Pont et al. 1995), which is a phenomenon used in laboratory settings to model psychotic symptoms (Laruelle 2000; Perry et al. 2007; Robinson and Becker, 1986). Furthermore, food shortage increases the activation of glucocorticoid hormones and the mesencephalic dopaminergic projection to the nucleus accumbens, influencing behavioural responses (Cabib et al. 1997). Clearly food restriction on this level induces stress both metabolically as well as neurophysiologically. Therefore, it is reasonable to suggest that the level of food restriction influences the behavioural response during operant tasks. An important dimension of impulsivity is a deficit in the inhibitory processes, which can be triggered by stress (Poulos et al. 1995). One may speak of stress-induced impulsivity. In other words, as food shortage induces stress, the outcome of the delay-discounting task therefore may be influenced bythe food restriction method. Thus, the level of food restriction determines to a certain extent the outcome of the delay-discounting task. This might be an explanation for the contradictory findings on gender differences between the current study and the previous study by Laviola and Adriani (unpublished). In fact, the model of Herrnstein (1981) suggested that an increase in the severity of restriction should give rise to an increase in the preference for a smaller, more immediate reinforcer over a larger, more delayed reinforcer (Bradshaw and Szabadi, 1992), thus resulting in increased impulsivity. Indeed, in human studies it has been indicated that when female subjects were more deprived (Kirk and Logue, 1997) or reported that they were currently dieting (Logue and King, 1991) they showed less self-control. Interestingly, in studies which tested Herrnstein’s model (1981), by exposing animals to series of discrete-trial schedules, the following was found: when comparing food restriction levels of 80% vs. 90% of ad libitum bodyweight, it was shown that more severe food restriction decreased impulsive-choice behaviour (Bradshaw and Szabadi, 1992; Ho et al. 1997; Wogar et al. 1993), at least in individually housed female rats. Female rats are more vulnerable to chronic mild stress exposure (i.e., a schedule of subsequently food or water deprivation, stroboscopic illumination, intermittent illumination, temporary paired housing, cage titling, and soiled cage) than males, showing increased corticosterone levels and decreased 5-HT activity inthe hippocampus and hypothalamus (Dalla et al. 2005). However, in response to an additional stressor (forced swim test), these females exhibited higher 5-HT activity in comparison with corresponding male rats. Thus, after a second stressful experience

39 females show a stronger response and by this are able to cope better with stress. Recall that Laviola and Adriani (unpublished) found at the food restriction level of 80–85% of ad libitum bodyweight that female mice were less impulsive than male mice, while in the present study at the milder food restriction level of 90±5% of ad libitum bodyweight females were more impulsive than males. The outcome of female mice could be in line with the mentioned earlier studies on 2 female rats (Bradshaw and Szabadi, 1992; Ho et al. 1997; Wogar et al. 1993). In other words, it might be that the female mice are more impulsive at the restriction level of 90±5% of ad libitum bodyweight (present study), while they are less impulsive at the restriction level of 80–85% (Laviola and Adriani, unpublished). The finding in male mice instead, could well be explained with Herrnstein’s model (1981), in that more severe food restriction will result in increased impulsivity (Laviola and Adriani, unpublished) whilst mild restriction induced little or no delay intolerance (present study). The foregoing points to the suggestion that in males more severe food restriction will result in increased impulsivity, while females cope better with additional stressors, resulting in decreased impulsivity. Apart from the severity, the duration of the (deprivation) stressor might influence the behavioural outcome as well. For example, short lasting food shortage or hunger might result in simple impatience, while a longer period of food deprivation induces the more complicated behavioural outcome of what is defined as impulsivity. The difference between male and female subjects might lie in the sensitivity to food deprivation and other involved systems affected by this. Future research should focus on testing this gender difference hypothesis using a delay-discounting paradigm with different food restriction levels and stressors. Additionally, different paradigms should be used to see whether overall gender differences in decision-making occur under the influence of stress.

4.3.3 Methodological remarks Although the testing phase was similar in the present and previous studies, the training phase in the current study differed slightly from the method used in earlier studies on mice (Adriani and Laviola, 2003; Laviola and Adriani, unpublished) and rats (Adriani and Laviola, 2006; Adriani et al. 2003a, 2003b, 2007). In the present study a pretraining phase of among others magazine training (see Section 2) preceded the regular training phase. This was done because a pilot study using the mild food restriction level of 90±5% of the ad libitum bodyweight showed learning difficulties (i.e., mice showed a significant preference for LL only after 2 weeks of training) with a lot of individual variation (data not shown). The implementation of such pre-training procedure prior to the regular training phase minimised these learning difficulties and variation between individuals. Furthermore, the operant chambers used in the previous mice studies (Adriani and Laviola, 2003; Laviola and Adriani, unpublished), containing one pellet magazine in the middle between the two nose-poking holes, also induced learning difficulties in mice (Adriani, unpublished observations). Therefore, in an attempt to simplify the learning process of linking nose-poking behaviour to pellet delivery in a specific magazine, a second pellet magazine was added to the chambers in the current study, so that each food magazine (LL vs. SS) was placed above the corresponding nose-poking hole. A platform was positioned in the operant cages to keep these higher placed magazines

40 Gender differences in delay-discounting under mild food restriction

accessible for the mice. However, it was observed that this platform did not interfere with task learning and with food collection after nose poking, although it induced an extra dimension to the operant chamber, invoking other behaviours (e.g., heightened locomotor activity, hiding underneath the platform). Further studies are warranted to investigate the cost/benefit ratio between ameliorated task learning and potential distraction of mice from the actual task.

4.4 Delay-discounting task to investigate gender differences in impulsivity? In conclusion, in the present study female mice showed a stronger tendency than males to decrease their initial preference for LL as delays became longer under mild levels of food restriction. Classically, one would interpret this result as female mice being more impulsive than males. However, a possibility is that males react differently than females to mild vs. strong food restriction. It is proposed that, as far as the level of food restriction is increased, males become more and more impulsive, whereas females cope better with this situation, apparently resulting in a decreased impulsivity. The alternative explanation for our results is that females are more explorative than males under mild levels of food restriction. This finding may be supported by the higher activity levels of female mice compared to male mice. Thus, depending on food restriction levels, different features of impulse-control and decision-making might be tested in a delay-discounting task. These considerations should be taken into account when interpreting results and for assessing the validity of this task to measure impulsivity.

Acknowledgements

Supported by the bilateral Italy–USA Program on Rare Diseases “X-linkedor autosomal rare mental retardation syndromes” (7NR1), the ERARE-EuroRETT Network (ERAR/6); by a Research Contract with Sigma-Tau SpA, Pomezia, Italy (to GL); by the European Mind & Metabolism Association (EMMA). The authors would like to thank Alessandra Berry for practical support, Sandra Columba-Cabezas and Harry Blom for theoretical advice, as well as Giacomo Dell’Omo and Pietro Serenellini for their precious technical support. Susanne Koot was supported by the Lifelong Learning Programme, providing her with an Erasmus grant.

41 3

42 CHAPTER 3

HOME CAGE TESTING OF DELAY-DISCOUNTING IN RATS

S. Koot a,b a Ethology and Welfare, Faculty W. Adriani b of Veterinary Medicine, Utrecht L. Saso c University, The Netherlands R. van den Bos a b Behavioural Neuroscience Section, G. Laviola b Department of Cell Biology & Neurosciences, Istituto Superiore di Sanità, Roma, Italy c Department of Physiology and Pharmacology “Vittorio Erspamer”, Sapienza University, Rome, Italy

Behavior Research Methods (2009) 41: 1169-1176.

43 Abstract

Testing rodents in their home cages has become increasingly popular. Since human intervention, handling, and transport are minimized, behavior can be recorded undisturbed and continuously. Currently existing home cage systems are too complex if only relatively simple operant-learning tests are to be carried out in rats. For that purpose, a new low-cost 3 computer-controlled operant panel was designed, which can be placed inside the home cage. A pilot study was carried out, using an intolerance-to-delay protocol, classically developed for testing behavioural impulsivity. Male adult rats were tested in their home cages, containing the operant panel provided with nose poking holes. Nose poking in one hole resulted in the immediate delivery of one food pellet (small– soon, SS), whereas nose poking in the other hole delivered five food pellets after a delay (large–late), which was increased progressively each day (0–150 sec). The two daily sessions, spaced 8 h apart, lasted 1 h each, and the time-out after food delivery was 90 sec. A clear-cut shift toward preference for SS, which is considered a classical index of cognitive impulsivity, was shown at the longest delay. It is noteworthy that rats shifted when the delay interval was longer than the mean intertrial interval—that is, when they experienced more than one delay-equivalent odds against discounting (see Adriani and Laviola, 2006). The shortened training (2 days) and testing (5 days) phases, as allowed by prolonged and multiple daily sessions, can be advantageous in testing rodents during selected shortphases of development. Current research is focusing on further validation of this and similar protocols.

44 Home cage testing of delay-discounting in rats

1 Introduction

Gene–brain–environment interactions are widely studied in animal models, among others, to advance the understanding of how organisms interact with and adapt to their environment and to model human psychiatric disorders. A broad range of test procedures is used for the description and analysis of the behavioral expression of an individual subject—that is, for behavioral phenotyping (De Visser 2008). However, as far as the study of behavioral phenotyping is concerned, crucial limitations of current approaches are also widely acknowledged (De Visser et al. 2006; Knapska et al. 2006). Most behavioral tests are restricted to a novel environment and are short-lasting. Furthermore, differences in laboratory environments and human interventions—for example, handling and transport—may compromise the reliability and reproducibility of behavioral data across laboratories (Crabbe et al. 1999; Wahlsten et al. 2003). Recently, much effort has been devoted to developing new and refined methods for behavioral phenotyping. Especially, testing animals in their home cages has become increasingly popular. Since human intervention, handling, and transport are minimized, behavior can be recorded undisturbed and continuously, for prolonged periods of time, as well as in circadian cycles (De Visser et al. 2006; Knapska et al. 2006). Different types of automated home cage systems have been developed, each having its specific methodology suitable for answering the specific questions of users. For example, the PhenoTyper is a video-based observation system for automation of behavioral tests with singly housed rats or mice, often used, for example, for studying (loco)motor patterns (De Visser 2008; De Visser et al. 2006). Another system, the IntelliCage, allows automated cognitive and behavioral screening of mice living in social groups—for example, implementing learning tasks (Knapska et al. 2006). Both systems are highly versatile and detailed in monitoring and testing behavior. However, besides being rather costly, these systems are too complex if only relatively simple operant-learning tests are to be carried out (IntelliCage) or if they are currently not able to carry out such operant procedures (PhenoTyper). For that purpose, a new low-cost computer-controlled home cage operant panel (HOP) was developed by joint effort of the Istituto Superiore di Sanità, Rome, and the small company PRS Italia, Rome. This apparatus can be placed inside a standard-size (Macrolon III) cage, enabling the rodent to operate it 24 h/day. In order to test the functionality of this panel, a pilot study was carried out, focusing on testing impulsivity. Lack of self-control or impulsive decision making may be an important symptom of psychiatric disorders, such as attention-deficit/ hyperactivity disorder (Bechara 2004; Strubbe and Woods, 2004). Animal models are crucial in studying the underlying neurobiology of such disorders. In the intolerance-to-delay (ID) task (Evenden 1999; Evenden and Ryan, 1999), subjects may choose between a large–late (LL) and a small–soon (SS) reward. Impulsive subjects are intolerant to the forced waiting for the large reward (Evenden and Ryan, 1996, 1999). Classically, the rats’ performance on the ID task is investigated by placing the animals in individual operant chambers for a short period daily (Evenden and Ryan, 1996, 1999). When the new HOP is used for this task, stress caused by handling and by novelty, which possibly may affect their performance, is

45 prevented. Here, we report the results of a pilot experiment in which this panel was used in an ID task protocol.

2 Method

2.1 Subjects Four adult male Wistar rats (Harlan, Italy; mean weight, 429 g) were kept in an air- 3 conditioned room (temperature, 21°±1°C; relative humidity, 60%±10%), on a 12:12-h reversed light:dark cycle (lights on at 8:00 p.m.). Prior to the experiments, the animals were housed in pairs in Macrolon III cages. From the start of the protocol, animals were singly housed, with the new apparatus being present in the cage (see below). Water was available ad lib, whereas food (Altromin-R, A. Rieper S.p.A., Vandoies, Italy) was available ad lib until the start of the protocol. The rats had previous experience in impulsivity tasks in a classical Skinner box setting, two months prior to the present pilot study.

2.2 Apparatus The operant-testing apparatus, consisting of four prototype computer-controlled panels (HOP; PRS Italia, Rome; see Figure 1), one for each of the subjects, was placed in a Macrolon III cage with sawdust bedding. The panel contained two nose-poking holes, hole lights, a chamber light, a feeder device, a food magazine where pellets (F0021-J Dustless Precision Pellet 45 mg; Bio-Serv, Frenchtown, NJ) were delivered, and a magazine light. The panel was connected through an interface to a PC, where a specific software (Sca020; PRS Italia, Rome) controlled and recorded all events. Nose poking in one of the two holes of the panel resulted in the delivery of five pellets (large reward, LL), whereas nose poking in the other hole resulted in the delivery of one pellet (small reward, SS). After nose poking and before food delivery, the hole light was turned on for 1 sec. Following food delivery, the magazine light was turned on for 90 sec, during which time nose poking was recorded but was without scheduled consequences (time-out, TO). The magazine light was then turned off, the chamber light was turned on, and the system was ready for the next trial. The following variables were recorded automatically: adequate hole poke visits (nose poking after a TO, resulting in the reinforcer delivery) and inadequate hole poke visits (nose poking during the length of the delay or during a TO interval, which were recorded but were without any consequences). For adequate nose poking, the dependent variable was the hole preference (calculated as the percentage of visits at the LL hole over total LL 1 SS adequate visits). For inadequate nose poking, the dependent variables were (1) the raw frequency of inadequate visits and (2) the proportion of inadequate to adequate visits. Data were averaged across the two daily sessions and, thus, will be presented as a function of the imposed delay level.

46 Home cage testing of delay-discounting in rats

Figure 1 Computer-controlled home cage operant panel (PRS Italia, Rome, Italy) (left), placed in Macrolon III cage (right).

2.3 Protocol of the ID Task for Testing Impulsivity Two days were required (see Figure 2) for pretraining to the experimental protocol. On Day 1, the animals were placed in the cages containing the panel, which occupied one fourth of the total living area. The adaptation period started with 24 h of access to ad lib regular food pellets (Altromin-R) placed in the food rack of the cage top, and, at the same time, rats could obtain Bio-Serv pellets from the panel by nose poking into the holes. On Day 2, the standard (Altromin-R) food was removed for 24 h, while the animals still could obtain precision (Bio-Serv) pellets from the panel. Then, 12 h of food deprivation followed, in order to increase the rats’ motivation to work for food delivery, and the animals entered the training phase of the protocol. During the subsequent training and testing phases, the animals had access only to precision (Bio-Serv) pellets during the 1-h sessions by operating the panel and had access to a limited amount of standard (Altromin-R) food after each session. Two daily sessions were run, from 9:00 to 10:00 a.m. and 6:00 to 7:00 p.m. (for arguments, see Strubbe and Woods, 2004). After each session, the total intake of precision pellets was calculated per individual, and additional standard food was given to meet the animals’ daily nutritional needs (details available upon request). The end of each session was indicated by switching off all the panel lights, plus the provision of the additional standard pellets in the food rack of the cage top. Training lasted four sessions (2 days), until all the subjects reached a significant preference for the large reward. During the testing phase (10 sessions, 5 days), a signaled delay was added to the 1-sec interval, normally scheduled between nose poking and large-reward delivery. The hole light was kept on during the entire length of this delay. The small-reward delivery was unchanged. Hence, the animals had a choice between an LL and an SS reward. The delay length was fixed for the two daily sessions and was changed over days: 0 sec on the last training day, followed by delays of 15, 45, 75, 105, and 150 sec on 5 subsequent days. The entire schedule for this protocol lasted 9 consecutive days in total. In summary, the present ID task was run in tight agreement with previous experiments at our lab (Adriani et al. 2003a, 2003b; Adriani and Laviola, 2006;

47 Figure 2 Protocol for home cage testing of impulsivity: pre-training (2 days) and first training day. The training 3 lasted just 2 days, whereas the testing phase lasted 5 days. The whole schedule lasted a total of 9 days.

Adriani et al. 2004), with a few minimal changes: first, the TO was elevated to 90 sec; second, the system was active daily during two sessions of 1 h each, rather than during 25-min sessions once daily. These changes allowed an overall reduction in the total duration of the schedule (i.e., only 9 days) and were indeed possible due to the fact that the panel was provided inside the home cage of the subject, rather than in dedicated operant chambers. Of course, this is not the only way an ID task protocol could be run, since many two-choice delayed reinforcement tasks exist. First of all, delay could progress within a session, and the same progression could be repeated across sessions (e.g., Evenden and Ryan, 1996, 1999). Also, the delay levels could be presented in a randomized, rather than ascending, order or could be titrated until the individual subject expressed indifference in his or her choice (i.e., 50% SS, 50% LL; see, e.g., Reynolds 2006). However, we adopted a gradual increase of delay interval over sessions, since this was done in our previous studies, the results of which were to be compared with the present results.

3 Results

Following four training sessions, all the rats showed a significant preference for the large over the small reward (average choice of 94.4% 6 5.3% for the large reward). Interestingly, instead of showing an exclusive preference (i.e., 100% for LL), they occasionally chose the alternative SS possibility. This finding, which replicated those of previous experiments in our lab (Adriani et al. 2003a, 2003b; Adriani and Laviola, 2006; Adriani et al. 2004), indicates that animals express a constant and active patrolling over the two alternative possibilities, as they adaptively probe whether there is any change over time in the outcome associated with nose poking at the nonpreferred hole. When delays were gradually increased over days (Figure 3), rats showed a shift toward SS choices at the longest delays (cf. Evenden and Ryan, 1996, 1999). A flatter or steeper shift toward SS choice is a classical index of reduced or increased impulsivity, respectively (Adriani and Laviola, 2006; Evenden and Ryan, 1996, 1999). Indeed, subjects may be individually characterized by delayavoidant traits or, vice versa, show an ability to cope with the delay constraint (Evenden, 1999). Accordingly, in all mammals, the most impatient among subjects are termed impulsive, in that they display intolerance to the forced waiting for a delayed reward. Thus, average group profiles are not entirely informative (Adriani et al. 2003a; Adriani et al. 2004), and

48 Home cage testing of delay-discounting in rats

precious information may emerge when individual profiles are considered. Indeed, each individual rat showed its own steepness profile (Figures 3 and 4), with Rat 1 showing nearly no shift and Rat 2 being the most impulsive one. Because nose poking in either hole during the course of the delay interval had no scheduled consequences, it was termed an inadequate response (Sagvolden, 2000; Sagvolden and Sergeant, 1998). This measure provides an index of the (in) ability to inhibit an unnecessary response. Data for delay = 0 reveal that inadequate responding was 67.6% of total nose poking, as if a couple of ineffective nose pokes were expressed per TO. More precisely, if we admit that ineffective nose pokes were released in bouts of two, then rats emitted a mean of 20.2 pairs of ineffective nose pokes per session, spaced an average of 106.2 sec apart. Such a long interresponse interval provides a precious internal control for rats’ understanding of TOs as windows of unavailability for their operant device. As the length of the delay was increased, inadequate nose pokes at the LL hole peaked and then decreased, whereas nose pokes at the SS hole progressively increased (Figure 5A). Such a finding suggests that, during the length of the delay interval, when they had to wait for the large reward to be delivered, the rats were (ineffectively) demanding more and more the smaller, immediate reward. However, the rate of ineffective nose pokes during delay intervals did not rise as delays were increased: If we compare delay = 15 sec with delay = 150 sec, when the waiting-constraint of delay was tenfold, it is clear that the SS ineffective nose poking was just doubled, and the total SS + LL inadequate visits were even slightly reduced. Interestingly, the percentage of inadequate nose pokes out of the total nose pokes remained fairly stable for each hole (around an overall mean of 75.3%; see Figure 5B). Since the number of effective choices for SS was increasing also, as Figures 3 and 4 indicate, both effective and ineffective SS nose poking apparently rose at a similar pace. As a consequence, the rats showed no specific increase in the rate of inadequate nose poking at the SS hole during the whole testing period, thus indicating no sign of motor impulsivity that could well have been released by this task.

Figure 3 Choice behavior by rats (n = 4) tested Figure 4 Choice behavior by four individual rats with the intolerance-to-delay protocol, shown tested with the intolerance-to-delay protocol, shown during daily sessions in the home cage situation. during daily sessions in the home cage situation. Data Data represent the mean (±SEM) choice per day represent the choice per day (as a percentage) for the (as a percentage) for the larger reward, delivered larger reward, delivered after a delay. after a delay.

49 3

Figure 5 Inadequate responding at both the small–soon (SS) and large–late (LL) holes (i.e., nose poking during the length of the delay interval being without any consequence) shown by rats (n = 4) tested in the intolerance-to-delay protocol. (A) Mean (±SEM) absolute quantity of inadequate nose pokes per session. (B) Mean (±SEM) percentages of inadequate nose pokes out of the total number of nose pokes.

4 Discussion

The aim of the present pilot was to test a new prototype of computer-controlled HOPs, currently designed to be used for testing impulsivity of rats (and mice) in their home cages. This pilot experiment showed that, in principle, it is possible to measure impulsive choice behavior in rats within a home cage setting. As was expected, when delays increased over days, rats showed a shift toward preference for SS at the longest delays. Similar results have been found in previous studies (cf. Adriani et al. 2003a, 2003b; Adriani and Laviola, 2006). To be able to compare the present results with those in previous studies, it is necessary to recalculate the outcomes with respect to TO duration. Classical definitions of impulsivity predict that a given factor (in the present study, delay) will discount the perceived value of the large reward, whereas the attractiveness of the small reward is unchanged. In the case of a very strong discounting process, which is warranted by the negative emotional impact of that factor (i.e., by the delay-generated aversion), animals will eventually express a preference for the SS reward. Such an impulsive choice is suboptimal and antieconomical, by definition, in that it leads to a lesser amount of food being gained in the long term (Evenden, 1999; Ho et al. 1999; Monterosso and Ainslie, 1999). Indeed, if preference shift from LL toward SS is convenient, it will be supported by “rational” criteria of payoff maximization. Thus, only decisions that are taken against economic convenience can be considered as an unbiased and reliable impulsivity index in the field of decision making (Adriani and Laviola, 2006, 2009). It may be useful to introduce the so-called indifferent point, at which either choice is mathematically identical in terms of total foraging, since it helps to clarify the economical aspects of the task. By definition, the long-term outcome is indifferent when rats can choose to receive either five SS deliveries of one pellet each ora single LL reward of five pellets during comparable intervals of time. As outlined by Adriani and Laviola (2006), the crucial unit to account for temporal features within the ID task is represented by the mean intertrial interval (mITI), composed of the TO

50 Home cage testing of delay-discounting in rats

interval (scheduled after food delivery) plus the spontaneous waiting by the animal (expressed after the end of TO and before the next nose poking). After each TO of 15 sec, animals performed the next nose poking after 10 sec, on average (Adriani and Laviola, 2006); thus, a fundamental descriptor of temporal features within the task, which could also be labeled crucial time unit, is represented by the task-specific parameter of mITI = 25 sec. Animals will, in fact, experience the same overall payoff either if they choose five consecutive SSs, during a time window lasting a total of five mITIs (5 times 25 sec = 125 sec) or if they choose LL just once and afford a sole delay, whose length is four times the mITI (i.e., a 100-sec delay) plus a further 25 sec (i.e., its own TO followed by spontaneous waiting). In the previous study, where the ID task was run inside the classical Skinner box setting, the TO interval was set at 15 sec, the mITI was 25 sec, and thus the indifferent point was encountered at a delay of 100 sec. In the present work, we set a TO of 90 sec, so that mITI was 100 sec, and the indifferent point was encountered, on average, at a delay of 400 sec. The reason for a shorter TO in previous studies resided mainly in shorter sessions, which are inherent to classical Skinner box testing. Indeed, when experiments with tens of subjects are run over several days, the experimenter needs to test several subjects per day in the same box. Thus, sessions usually range between 20 and 30 min, and the TO ranges between 15 and 25 sec. This allows animals to be presented with around 60 opportunities to make SS/LL selections. Animals living permanently within the reach of the operant panels will be offered the same quantity of selection opportunities while longer sessions are run. This can easily be done by using longer TO intervals, which obviously implies a longer mITI. Hence, the reinforcing rate inherent to the task was slower in the present home cage setting than it is in classical conditions, and this perhaps explains why longer delays were apparently tolerated by the rats. As a matter of fact, the present setting provided the rats with more time to elaborate their strategy, which is not the case with classical sessions. This is another advantage of home cage testing. In brief, for the present pilot work, receiving a single LL reward of five pellets after a delay of 400 sec would actually have been indifferent with respect to receiving five SS deliveries of one pellet each, with four mITIs averaging 100 sec between them. Note that, in the previous study, receiving a single LL reward of five pellets after a delay of 100 sec was indifferent with respect to receiving five deliveries of one pellet each, with four mITIs averaging 25 sec between them. This notion helps us to compare the present results with those in the previous study (Adriani and Laviola, 2006), where we proposed that the impact of a given delay could be expressed as a (sub)multiple of the mITI. By referring to delay-equivalent odds against discounting, using the formula odds = delay/mITI, the fact that mITI was fourfold in the home cage versus the classical setting implies that the present delays of the ID task should be divided by four. As such, the subjective indifference between LL and SS and the development of an SS preference, respectively, were clearly observed in the present pilot study at delays of 105 and 150 sec (odds = 1.05 and 1.50), which would therefore correspond to 26.25 sec (= 105/4) and 37.50 sec (= 150/4) of the previous study (Adriani and Laviola, 2006). In that study, a shift in preference was found between delays of 30 and 45 sec (equivalent odds = 1.2–1.8). Thus, a reference to the delay-equivalent odds values renders delay lengths fairly comparable across

51 studies, even though a further validation should be carried out. Currently, there is rising interest in comparing directly the results obtained from home cage testing versus the conventional Skinner box setting. Furthermore, the two kinds of setting will not necessarily lead to identical results: as underlined by a very recent report (Caprioli et al. 2009), comparing the performance of animals in their home cage versus a different testing apparatus may open avenues for very innovative findings. Apart from the rewarded nose pokes, unrewarded nose poking (i.e., during the length of the delay interval, when it had no scheduled consequences; Morgan, 3 1972) was recorded. This inadequate responding has been proposed as an index of inability to inhibit an unnecessary response—a measure of restlessness, in other words (Sagvolden, 2000; Sagvolden and Sergeant, 1998). The present results confirm a progressive elevation in absolute quantities (i.e., raw frequency of nose pokes) for SS inadequate responding (Adriani et al. 2003b). These data should be looked at carefully, since they might suggest a rise of restlessness, generated by the progressive increase of delay intervals. However, when we consider the relative proportion of inadequate nose pokes out of the total number of nose pokes, this parameter apparently shows a rather stable profile over the entire testing period, or even a slight nonsignificant decrease as delays got longer (Figure 5B). Thus, we conclude that, by exploiting the signaling cues provided by the panel, the rats were able to distinguish between specific portions of time—that is, when the panel was effectively available, as opposed to the delay-long windows of ineffectiveness. Indeed, if their response was random or aspecific and/or motor restlessness was generated by the waiting constraint, an increasing proportion of inadequate visits would emerge along the whole session. The present data, actually supporting a constant or even decreasing proportion of inadequate visits, indicate that rats refrained somewhat from nose poking after having expressed their choice. Furthermore, since we propose that no specific increase in restlessness was generated by the progression of delay intervals, the corollary is that cognitive, rather than motoric, impulsivity was assessed in the present study. The shift to SS was not a side effect of enhanced and aspecific SS nose poking during panel ineffectiveness. To study restlessness in more detail, other protocols should be implemented. For instance, in studies with spontaneously hyperactive and impulsive rats (Johansen et al. 2005), ineffective responding was punished somewhat, in that it led to resetting of delay length. In this kind of protocol, the real duration of delay can be dependent on rats’ own behavior. It is hence difficult to determine the effect of the delay interval alone (Johansen et al. 2005), since longer delays consist mainly of effective responses followed by short interresponse times. It is beyond the limits and scope of this article to discuss this issue in more detail. In our hands, the present ID task allows the possibility of extracting data on rewarded versus unrewarded nose pokes, which can respectively be used to distinguish betweencognitive and motor impulsivity (Adriani et al. 2003a, 2003b; Evenden 1999).

4.1 Advantages and Disadvantages of Home Cage Testing As compared with the classical Skinner box setting, a couple of advantages of home cage testing can be distinguished. First of all, two factors of the conventional protocol are clearly diminished—that is, the stress of handling and the novelty experience caused by transport to the test apparatus for several days. However, it is as yet

52 Home cage testing of delay-discounting in rats

unclear to what extent this effect can be traced back in the results. Second, prolonged training sessions are allowed in the home cage, thus shortening considerably the duration of the training period—that is, 4 days in the present protocol versus 1 week in the standard protocol (Adriani et al. 2003a, 2003b; Adriani and Laviola, 2006). A short training period is especially desirable for testing developing rodents, since developmental stages may last only 1 or 2 weeks for rats and mice (e.g., see Adriani et al. 2003a; Adriani and Laviola, 2003). As for the testing period, in the present pilot study, the same delay was tested twice a day, since it was unclear whether early and late sessions during the dark hours would impose a bias due to circadian rhythm fluctuations. Further studies could address more specifically and might lead to a shortened schedule as well. Another important advantage of the use of the HOP, as compared with other home cage systems, is its simplicity in use and relative low cost. HOPs can be placed in each standard Macrolon III cage, allowing one to house animals in common animal rooms, without any special requirements except for cables to an interface that then connects the panels to the controlling computer. Difficulties and disadvantages of the HOP can be found as well. In the present protocol, rats must be housed individually throughout the entire protocol duration. Stress through social isolation influences the reward system (Hall et al. 1997; Van den Berg et al. 1999) and decision making as such, as well. Impulsivity tasks are particularly interesting during adolescence, but social deprivation may produce changes in reward sensitivity (Van den Berg et al. 1999), as well as psychotic symptoms (Leussis and Andersen, 2008), particularly during this ontogenetic period. Solutions for this might be found in (partially) social housing, combined with technologies such as transponder-operated access to the operant panel(s) and/or communal cages. However, these adaptations to the model would involve extra animal training and might compromise the simplicity and easy use of the present system. The HOPs are placed in standard Macrolon III cages inside a standard animal room. Unlike the classical Skinner boxes, which are surrounded by soundproof boxes, no visual, audio, or olfactory barriers are used in the present system. It is as yet unclear to what extent individuals can influence each other’s choice behavior—for example, by making food calls.

4.2 Conclusion The present pilot study showed the possibility of testing impulsivity in the home cage, using an operant panel. In theory, other protocols can be run with this panel as well. For example, we recently proposed that a yoked probabilistic-discount (PD) protocol may exist for each ID task, as validated using the classical Skinner box setting (Adriani and Laviola, 2006). Thus, beside the present home cage ID task, a corresponding PD task has been run for investigating gambling behavior. Adult male rats were housed, trained, and tested as described in the present pilot study. However, during the training phase, large reward delivery was randomly released or omitted, by setting a percentage of probability (P = releases/demands * 100). The level of P was fixed for daily sessions and decreased progressively every other day, thus implementing a large-reward rarefaction (Adriani and Laviola, 2006). When the P level was decreased, some rats reduced their LL preference only slightly, and some other rats developed a clear preference for SS. According to Madden, Ewan, and

53 Lagorio (2007), indeed, there is a clear relationship between delay and probability discounting, in that the same individual aversion to delayed rewards comes along with greater value being attributed to unpredictable than to predictable rewards. Similarly, impulsive and/or gambling behavior is more likely to result from a reduced self-control ability, rather than from simplistic value-discounting functions (Monterosso and Ainslie, 1999). Current research is focusing on the further validation of both ID and PD task home cage protocols, as well as on finding solutions for the above-mentioned limitations. 3 For instance, it could be informative to determine impulsivity curves generated when the home cage ID task is run under several possible combinations of TO duration and session length. We assume, indeed, that the same individual would show more delay (in)tolerance if facing long (or short) sessions, whereas the role of TO has yet to be explored. Rats’ spontaneous waiting after a TO and before their next response (response time, RT) could be differentially affected if they are faced with ashort versus long TO after food delivery. If, indeed, rats track and do adapt to atask- specific reinforcing rate (Gallistel and Gibbon, 2000; Podlesnik and Shahan, 2008), their mITI average value should be critically paced by both the imposed TO duration and adaptive development of an individual RT value. Therefore, it would be valuable to further refine the present home cage ID task, as well as to validate other types of ID task protocols.

Author Note Research was performed along the lines of the “ADHD-sythe” young investigator project (to W.A.): “under-40 call” by the Italian Ministry of Health (assigned), and “ERC-StG” by EU-FP7-Ideas (pending). It was also supported by the bilateral Italy–U.S. Program on Rare Diseases (7NR1) and by the ERARE-EuroRETT Network (ERAR/6), EU (to G.L.). S.K. was a recipient of an Erasmus fellowship, within the Lifelong Learning Programme (to L.S.), supporting her stay at the Istituto Superiore di Sanità. The authors thank Pietro Serenellini (PRS Italia) for technical design and support. Correspondence concerning this article should be addressed to W. Adriani, Behavioral Neuroscience Section, Department of Cell Biology and Neurosciences, Istituto Superiore di Sanità, Viale Regina Elena 299, Rome 1-00161, Italy (e-mail: [email protected]).

Note—This article is based on a presentation made at Measuring Behavior 2008.

54 Home cage testing of delay-discounting in rats

55 4

56 CHAPTER 4

COMPROMISED DECISION-MAKING AND INCREASED GAMBLING PRONENESS FOLLOWING DIETARY SEROTONIN DEPLETION IN RATS

S. Koot a,b,c,* a Section of Behavioural Neuroscience, F. Zoratto a,* Department of Cell Biology and T. Cassano d Neurosciences, Istituto Superiore R. Colangeli e di Sanità, Roma, Italy G. Laviola a b Division of Behavioural Neuroscience, R. van den Bos b,c Department of Animals in Science and W. Adriani a Society, Faculty of Veterinary Medicine, Utrecht University, Utrecht, The Netherlands c Rudolf Magnus Institute of Neuroscience, UMCU, Utrecht, The Netherlands d Department of Biomedical Sciences, Medical School, University of Foggia, Foggia, Italy e Department of Physiology & Pharmacology “V. Erspamer”, Sapienza University, Rome, Italy

* Equally contributed to this work.

Neuropharmacology (2012) 62: 1640-1650.

57 Abstract

Psycho-genetic studies have revealed a role for the brain serotonin system in gambling proneness and poor decision- making. We assessed whether manipulation of brain serotonin levels in rats affected performance in operant-based tasks for decision-making and gambling proneness. Male Wistar rats were exposed to an L-tryptophan (TRP) deficient diet (0.0 g/kg; T- group) or to a control, L-tryptophan containing diet (2.8 g/kg; T+ group). The same rats were tested for decision- 4 making performance in the rodent Iowa Gambling Task (rIGT) using home-cage operant panels, and subsequently for gambling proneness in a Probabilistic Delivery Task (rPDT) using classic Skinnerboxes. At sacrifice, monoamines and metabolites were evaluated with HPLC analysis, confirming a drastically reduced serotonin synthesis, as well as altered dopamine turnover in the prefrontal cortex of T- rats. As expected, control rats (T+) progressively chose the option with the best long-term payoff in the rIGT, and also shifted from “Large & Luck-Linked” (LLL) to “Small & Sure” (SS) reinforcers in the rPDT. In contrast, depleted animals (T-) exhibited a weaker improvement of performance in the rIGT and maintained a sub-optimal attraction for LLL reinforcer in the rPDT. Comparing individual performances in both tests, we found a significant correlation between the two tasks in control (T+) but not in depleted (T-) rats. The present study revealed that (1) brain 5-HT depletion leads to poor decision- making and to gambling proneness; (2) the relationship between these two traits, shown in the control group, was disrupted in 5-HT depleted rats. The data are discussed in terms of changes within forebrain loops involved in cognitive and motivational/affective processes.

58 Compromised decision-making and increased gambling proneness following dietary serotonin depletion in rats

1 Introduction

The rapid worldwide growth of legalised gambling opportunities has raised concerns over the impact of gambling and its consequences on public health (Carragher and McWilliams, 2011; Shaffer and Korn, 2002). Epidemiological data suggest that 27.1% of adult people gambled more than 100 times in their lifetime, whilst a 10.1% gambled more than 1000 times (Kessler et al. 2008). Although gambling may remain a recreational activity for some people, it may become an overt problem for others. Such problematic gambling behaviour may be maladaptive or pathological, and disrupt personal, family, professional or vocational pursuits (DSM-IV, A.P.A., 2000; Potenza, 2001). Problematic gambling behaviour is also associated with poor decision-making performance, as measured for instance by the Iowa Gambling Task (IGT; Brand et al. 2005; Cavedini et al. 2002; Goudriaan et al. 2005). The IGT measures decision-making processes by simulating real-life decisions involving reward, punishment, and uncertainty of outcomes (Bechara et al. 1994, 1999). In this task, poor decision-making performance is associated with a choice for long- term disadvantageous options. Here, we focus on the relationship between gambling proneness and (poor) decision-making, as measured by two rodent operant tasks exploiting reward uncertainty. In general, the output of decision-making processes (i.e. which action is taken in the end), as well as the gambling temptations (caused by a lack of self-control abilities over impulsive attraction for “binging”) are determined by an interaction between two different forebrain loops: a limbic (affective/motivational) loop, encompassing the prefrontal cortex (PFC) in its orbital sub-region (i.e. orbito-frontal cortex, OFC), the amygdala and ventral striatum, versus a cognitive (executive/motor) loop, encompassing the dorso-lateral prefrontal cortex (dlPFC) and dorsal striatum (Bechara 2005; Doya 2008; McClure et al. 2004; Tanaka et al. 2004, 2007; Canese et al. 2011). These two loops exert different levels of control over decision-making behaviour. While the limbic loop is involved in immediate responding to (potential) rewards, losses or threats (i.e. impulsive behaviour) as well as in emotional control, the cognitive loop is more involved in long-term or future perspectives, i.e. in cognitive control (Bechara 2005; Doya 2008; McClure et al. 2004; Tanaka et al. 2004, 2007). Among others, serotonin (5-HT) plays a role in top-down control of behaviour, whereby these prefronto-cortical areas subserve inhibition of impulsive behavioural responses, through mediating the interplay between the limbic and cognitive loops (see reviews: Cools et al. 2011; Homberg 2012; Rogers, 2011). For instance, low levels of 5-HT have been associated with poor decision-making and/or poor impulse control (Baldwin and Rudge, 1995; Daw et al. 2002; Doya, 2008; Homberg et al. 2008; Lucki, 1998; Owens and Nemeroff, 1994; Soubrié, 1986; Tanaka et al. 2007; Winstanley et al. 2004b). Next to noradrenergic and dopaminergic dysfunction, serotonergic dysfunction has been reported as a key biological factor contributing to the pathophysiology of gambling proneness, which is characterized by a loss of impulse control (Ibanez et al. 2003; Pallanti et al. 2006, 2010). For instance, hypoactivity of the brain 5-HT system (Moreno et al. 2004) and low cerebrospinal- fluid levels of both L-tryptophan (TRP) and 5-HT have been found in pathological gamblers (Nordin and Sjodin, 2006). Furthermore, gambling proneness as well as

59 poor decision-making in the IGT have been associated with the short (s/s) allele of the 5-HT transporter promoter length polymorphism (5-HTTLPR; da Rocha et al. 2008; He et al. 2010; Homberg et al. 2008; Ibanez et al. 2003; Must et al. 2007; Stoltenberg and Vandever, 2010; van den Bos et al. 2009b). Given the considerations above, we experimentally manipulated the 5-HT brain availability, and investigated the consequences on decision-making and gambling proneness in rats. Several methods exist to deplete central 5-HT function, such as 5-HT agonist and antagonist drugs (e.g. 8-OH-DPAT and WAY 100635; Mobini et al. 2000b), lesions of the ascending serotonergic projection induced by intra-raphe injections of the selective neurotoxin 5,7-dihydroxytryptamine (5,7-DHT; Fletcher et al. 2001) and 4 systemic administration of the 5-HT synthesis inhibitor parachlorophenyl-alanine (PCPA; Dringenberg et al. 1995; Fletcher et al. 2001). Another way of depleting central 5-HT is nutritional manipulation of TRP. Since brain 5-HT synthesis depends on the availability of its precursor, dietary TRP depletion is considered an effective method to substantially reduce plasma and cerebral TRP levels and consequently to reduce brain 5-HT synthesis (Biggio et al. 1974; Vergnes and Kempf, 1981). As acute L-tryptophan depletion (ATD) only leads to moderate, transient depletion of TRP levels in adult rats (Brown et al. 1998; Lieben et al. 2004), we applied longterm 5-HT depletion (Lee et al.1999; Tanke et al. 2008; Uchida et al. 2005; Vergnes and Kempf,1981) using a TRP-free diet, allowing us to investigate how hypo-activity of the 5-HT system affects decision-making and gambling proneness in rats. Decision-making performance was assessed using a modified, automated version of a validated rodent version of the Iowa Gambling Task (rIGT, de Visser et al. 2011a; Homberg et al. 2008; van den Bos et al. 2006a): we developed an operant-based task using a modified home-cage operant panel (Koot et al. 2009, 2010). In this task, rats have to choose between a long-term advantageous option, containing a high probability of a small reward (two sugar pellets) and a low probability of punishments (two quinine pellets), versus a long-term disadvantageous option, containing a low probability of a large reward (four sugar pellets) and a high probability of punishments (four quinine pellets). After exploring the options, control rats normally develop a preference for the long-term advantageous one. Poor decision-making performance is thus characterized by a lack of developing this preference (de Visser et al. 2011a; Homberg et al. 2008; Koot et al. 2010; van den Bos et al. 2006a). To assess gambling proneness in rats, we used the rodent Probabilistic Delivery Task (rPDT), recently developed from temporal-discounting operant tasks (Adriani and Laviola, 2006). In this task, rats learn to prefer a large (six pellets) over a small (two pellets) reward. Subsequently, the probability of occurrence of the large reward decreases progressively to very low levels (i.e. a risky choice). Control rats normally change their preference towards the certain (“Small & Sure”, SS) reward, which is a “safer” option beyond the indifferent point (i.e. when the risky choice becomes mathematically disadvantageous in terms of total foraging; Adriani et al. 2012). As such, gambling proneness in rats is associated with a maintained preference for the highly uncertain (“Large & Luck-Linked”, LLL) reward, which becomes a sub-optimal option far beyond the indifferent point.

60 Compromised decision-making and increased gambling proneness following dietary serotonin depletion in rats

Depleted and control rats were tested in the rIGT and subsequently in the rPDT, allowing us to investigate (1) whether 5-HT depletion indeed disrupts rIGT performance and/or rPDT performance, and (2) whether poor decision-making in the rIGT is associated with gambling proneness in the rPDT, using correlational analysis.

2 Material and methods

All experimental procedures were approved by Institutional Animal Survey Board on behalf of the Italian Ministry of Health (licence to GL), and by the Animal Ethics Committee of Utrecht University. Procedures were in close agreement with the European Communities Council Directive (86/609/EEC) as well as with Italian and Dutch laws. All efforts were made to minimize animal suffering, to reduce the number of animals used, and to utilise alternatives to in vivo techniques, if available.

2.1 Subjects Twelve male adult Wistar rats (Charles River, Italy) were kept at the Istituto Superiore di Sanità (ISS, Rome, Italy) in an air-conditioned room (temperature 21 ± 1 °C) on a 12-h reversed light-dark cycle (lights off at 7:00 AM), where they were housed in pairs inside Makrolon III cages with sawdust bedding. Another twelve male adult Wistar rats (Harlan, The Netherlands) were kept in similar conditions at Utrecht University (UU, The Netherlands). The animals housed at UU were specifically intended to serve as an additional control group, in order to confirm the robustness of our novel operant version of rIGT. This is an adapted version of the validated maze-based protocol (de Visser et al. 2011a; Homberg et al. 2008; van den Bos et al. 2006a). Water was available ad libitum, whereas food (Rome: Altromin-R, A. Rieper S.p.A., Vandoies, Italy; Utrecht: 801730 CRM (E) Expanded, Special Diets Services, Witham, Essex, England) was available ad libitum unless stated otherwise. After four weeks of habituation to the housing conditions and handling by the experimenters, rats were randomly assigned to one of two experimental groups: one group (n = 6 at ISS) received a TRP-free diet (T-), while the other group (n = 6 at ISS and n = 12 at UU) received a control diet (T+). The TRP-free diet (DP/1069 mod., A. Rieper S.p.A., Vandoies, Italy) had a standard nutritional value, but with the complete lack of TRP. The control groups (at ISS and UU) were fed a similar diet, containing a standard amount of TRP (2.8 g/kg diet). Rats were tested in an adjusted operant version of the rodent Iowa Gambling Task (rIGT) and subsequently in the rodent Probabilistic Delivery Task for gambling proneness (rPDT), followed by forebrain sample collection at sacrifice (see Figure 1 for the entire experimental design). The rIGT test was presented first, involving a mild level of food restriction (95%), while the rPDT was presented afterwards, as a stronger food restriction was needed (85-90%). This order of testing was chosen since animals should proceed from less to more invasive behavioural tests, especially during dietary-manipulation periods (Zhang et al. 2006). The present experiment exploited a nutritional-deprivation approach plus a food- restriction schedule, which were however devoid of overtly adverse consequences. Constant but informal observations of rats (in both the rIGT and rPDT apparatuses

61 Figure 1 Experimental layout.

as well as when they returned to home-cages, until sacrifice) revealed no gross alterations in indexes of general welfare, nor emergence of stereotyped patterns of 4 behaviour. Several experiments have indeed assessed parameters of behaviour in mice and rats consuming a TRP-free diet for one month or more (Vergnes and Kempf, 1981; Uchida et al. 2005; Zhang et al. 2006).

2.2 The rodent Iowa Gambling Task (rIGT) in the home-cage 2.2.1 Apparatus The operant-testing apparatus, consisting of a prototype computer-controlled panel (2f-HOP; PRS Italia, Rome, Italy), was placed in a Makrolon IV cage with sawdust bedding, in which rats were housed individually. The current 2f-HOP is an adjusted version of the apparatus which we used in an earlier study (Koot et al. 2009). The panel contained two nose-poking holes, hole lights, a chamber light, two feeder devices, a food magazine where sugar pellets (F0042) and quinine pellets (F06498, quinine 4.44 g/kg diet; 45 mg Dustless Precision Pellet; Bio-Serv, Frenchtown, NJ, USA) were delivered, a little trapdoor to remove uneaten pellets, and a magazine light. The panel was connected through an interface to a PC, where specific software (Ska020 3.2.3; PRS Italia, Rome, Italy) controlled and recorded all events. Four subjects were tested at the same time. Nose-poking in the holes of the panel resulted in the delivery of sugar or quinine pellets (see below for ratio and amount). After nose-poking and before food delivery, the hole light was turned on for 1 s. Following food delivery, the magazine light was turned on for 15 s (timeout, TO), during which nose-poking was recorded but was without any scheduled consequence (i.e. inadequate nose-pokes). The trapdoor was opened 2 s before the end of the TO, to remove uneaten pellets from the food magazine. Then, the magazine light was turned off, the chamber light was turned on, and the system was ready for the next trial.

2.2.2 Protocol rIGT One week before the start of the training/testing protocol, rats were handled for 2 min daily, their body weight was taken, and they were familiarised with the sugar pellets (two pellets per animal per day) and T- or T+ diet respectively (4 g per animal per day) in the home-cage. Five days before the start of the training/testing protocol the standard food was removed and animals received ad libitum the T- or T+ diet. On the morning of the first day of training, rats were placed individually in the testing home-cages, with water available ad libitum but without food, where they were left undisturbed for an hour before the first session started. Two sessions were run per day, which took place around 9:00 AM and 5:00 PM respectively (for arguments, see Koot et al. 2009). Before each session, rats received 0.5 g of T- or T+

62 Compromised decision-making and increased gambling proneness following dietary serotonin depletion in rats diet respectively, and after each session they received the rest of their diet needed to maintain them on 95-98% of ad libitum body weight (determined before the first magazine training session). Daily after the AM-session, animals’ body weight was taken. Rats were food-restricted from the first day of training throughout the entire rIGT protocol in order to increase their motivation to work for food delivery. The training phase consisted of three 15 min-sessions of magazine training and one session of alternation training (i.e. two days in total). During the magazine training two sugar pellets were delivered automatically in the magazine with an interval of 60 s, i.e. a total of 15 trials. The trapdoor was opened to remove the uneaten pellets after an interval which was fixed for each session and progressively decreased (time to eat, TTE, 20-15-10 s for each magazine training session, respectively). Pilot experiments showed that these sessions were sufficient for rats to reliably learn to consume pellets within 10 s. During the magazine training, nose-poking holes were permanently covered by Plexiglas plates. During the entire experiment, the food magazine was covered by an aluminium plate which was only removed during sessions, to protect the underlying mechanics of the trapdoor against sawdust entering the food magazine. Subsequently, rats were trained to nose-poke in both holes, to prevent the development of a bias for one hole (alternation training, 10 trials per hole). This was done by training rats to emit alternating responses: just one hole was active while the other was inactive, during alternate trials. Nose-poking in the active hole led to the delivery of the reinforcer, whilst nose-poking in the inactive hole was without any consequence. One session was sufficient for rats to learn to nose-poke equally in both holes. After the training phase, the test phase started. The rIGT in the home-cage was based on the rIGT performed in a maze as described previously (de Visser et al. 2011a; Homberg et al. 2008; van den Bos et al. 2006a), modified to be adapted in the home- cage operant panels. Rats received two sessions per day, 40 trials per session for a total of 240 trials; thus this phase lasted 3 days. Each session started with switching on the chamber light accompanied by the free delivery of two sugar pellets. Nose- poking in either hole led to the delivery of reward and/or punishment, followed by the TO (15 s, including a TTE of 13 s before the uneaten pellets were removed). Rewards were represented by sugar pellets, punishments were represented by quinine pellets that were unpalatable but not uneatable. The (in the long run) disadvantageous hole presented occasionally large rewards (four sugar pellets, probability 30%) among series of quinine pellets (four, probability 70%), i.e. 12 sugar pellets per 10 trials. The (in the long run) advantageous hole presented regularly small rewards (two sugar pellets, probability 80%) among quinine pellets (two, probability 20%), i.e. 16 sugar pellets per 10 trials. The entire training and testing protocol lasted five days. After the last session, rats were housed again in their former pairs, and had ad libitum access to their diets (T- and T+ respectively) for five days until the rPDT protocol started.

2.3 The rodent Probabilistic Delivery Task (rPDT) 2.3.1 Apparatus Computer-controlled operant chambers (Coulbourn Instruments, Allentown, PA, USA) were placed in an experimental room, adjacent to the animal room. The

63 operant chambers were provided with two nose-poking holes, two chamber lights, two feeder devices, two food magazines where control (F06555, T+; TRP 2.8 g/kg diet) and depleted pellets (F06554, T-; TRP 0.0 g/kg diet; Dustless Precision Pellet 45 mg, Bio-Serv, Frenchtown, NJ, USA) were delivered, each with a magazine light, and five infrared photobeams on the bottom of the chamber (spaced apart 3 cm) to measure locomotor activity. Nose-poking in either hole was detected by a photocell. A computer, with custom-made software, controlled and recorded all events.

2.3.2 Protocol rPDT Five days after the last rIGT session, the rPDT protocol started. Daily sessions lasted 4 25 min and were run between 11:00 AM and 3:00 PM, for which animals were transported to the experimental room and placed in the operant chambers. After each session, animals were returned to their home-cage, where they were given an appropriate amount of their diet (4.5 g each) previously titrated in order to complete their daily caloric intake. During daily sessions, rats were able to eat (on average) 9 g of precision pellets, i.e. about two thirds of their daily needs to maintain their body weight on 85-90% of their ad libitum body weight (determined before the first rPDT training session). Animals were food restricted in order to increase their motivation to work for food delivery. After the final rPDT session, rats had again ad libitum access to their diets (T- and T+) until sacrifice (on average 16 ± 8 days after the last rPDT session). During the training phase (3 days) nose-poking in one of the two holes resulted in the delivery of two pellets, whereas nose-poking in the other hole resulted in the delivery of six pellets. After nose-poking and before food delivery, the chamber light above the nose-poked hole was turned on for 4 s. Following food delivery, the magazine light was turned on for 15 s (timeout, TO), during which nose-poking was recorded but was without scheduled consequences (i.e. inadequate nose-pokes). The magazine light was then turned off, and the system was ready for the next trial. The total number of completed trials and the inter-trial interval were not fixed, since rats were free to express nose-poking for food at their own, individually-variable rate during the 25 min session. During the testing phase (10 days) a probabilistic dimension was associated with the delivery of the large reward. The chamber lights were switched on after nose-poking following the usual schedule. However, sometimes the delivery of a large reward was omitted, while the magazine light was still switched on for 15 s, according to a given level of probability (p = percentage of actual food delivery over total demands). The small reward delivery was unchanged. Hence, animals had a choice between a “Large & Luck-Linked” (LLL) or a “Small & Sure” (SS) reward. The probability level was kept fixed for each daily session, and was decreased progressively over days (from p = 99% to 80%, 66%, 50%, 33%, 25%, 20%, 17%, 14%, 11% and finally p = 9%). One session was run for each p-level, which was changed daily. The indifferent point (at which either choice was mathematically identical in terms of total foraging) was p = 33%. We initially imposed a range of p-values before the indifferent point (99%, 80%, 66%, 50%) when LLL was always the optimal choice. Rats were then tested far beyond the indifferent point (25%, 20%, 17%, 14%, 11%, 9%) when LLL became asub- optimal option.

64 Compromised decision-making and increased gambling proneness following dietary serotonin depletion in rats

2.4 Monoamines and their metabolites: HPLC determination Rats were decapitated, their brains removed and rapidly dissected on ice to obtain the PFC, striatum, and hippocampus for HPLC analysis. We did not subdivide these areas into subnuclei, and thus the brain areas were collected as a whole. All samples were immediately flash frozen on dry ice, then stored at -80 °C until further processing. Brain samples of six T- and T+ rats were then analysed by determination of monoamines and their metabolites, according to previously published protocols (Cassano et al. 2009). In particular, each brain region was weighed and ultrasonicated in percloric acid (PCA) 0.1 M. Samples were centrifuged for 20 min at 15,000 g (4 °C), then the supernatant was used for monoamine and monoamine metabolite assays. The endogenous levels of 5-HT and 5-HT metabolite (5-hydroxyindolacetic acid, 5-HIAA), of dopamine (DA) and final DA metabolite (homovanillic acid, HVA), of noradrenaline (NA) and NA metabolite (4-hydroxy-3-methoxyphenyl-glycol, MOPEG) were assayed by microbore HPLC using a SphereClone 150-mm x 2-mm column (3- mm packing). The detection was accomplished with a Unijet cell (BAS) with a 6-mm- diameter glassy carbon electrode at þ650 mV versus an Ag/AgCl reference electrode, connected to an electrochemical amperometric detector (INTRO, Antec Leyden, The Netherlands). For each analysis, a set of standards containing various concentrations of each compound (monoamines and their metabolites) was prepared in the acid solution, and calibration curves were calculated by a linear regression. The retention times of calibration standards were used to identify peaks, and areas under each peak were used to quantify monoamine levels. Results were normalised to the weight of wet tissue.

2.5 Data analysis Significance level was set at p ≤ 0.05, ns = not significant; all statistics are two-tailed. Statistical analyses were performed using Statview II (Abacus Concepts, CA, USA), STATA Release 8.0 (Stata Corporation, College Station, TX, USA) and SPSS v16.0 for Windows (IBM corporation, Armonk, NY, USA). Data are expressed as mean ± SEM, unless otherwise indicated. Multiple comparisons were performed with the Tukey HSD Test.

2.5.1 Home-cage rIGT experiment Exclusion criteria were (1) rats that did not reliably eat the rewards, (2) rats that did not reliably nose-poke for pellet delivery, and (3) rats that consistently ate quinine pellets. The following variables were recorded automatically: adequate nose-poking (nose-pokes after a TO, resulting in the delivery of the pellets), inadequate nose- poking (nose-pokes during a TO interval, which were recorded but were without any consequences), and time needed to complete the session. As measure of decision-making performance, the dependent variable was the choice preference (calculated as the percentage of choices at the [in the long run] disadvantageous hole for each block of 40 trials). As measures of (motor) impulsivity (Sagvolden, 2000; Sagvolden and Sergeant, 1998), the dependent variables were the average frequency of inadequate nose-pokes per trial, calculated for each block of 40 trials, as well as the time needed to complete a trial. For all parameters, repeated measure ANOVAs were performed. Within-subject factor was trial block; the between-subjects factors were location (when comparing UU and ISS controls)

65 or treatment (when comparing T+ and T- groups on the whole experiment). As the two diets had differential effects on rats’ body weight, to account for a possible bias due to gross differences, body weight changes were included as a covariate in the ANOVAs. When these covariates yielded an effect on the statistical analysis, a Pearson’s correlational analysis was done to further reveal the biasing effect.

2.5.2 Standard-Skinnerbox rPDT experiment The inclusion criterion was defined as a preference for LLL of more than 60% at p = 66%, and was applied independently from rIGT exclusion criteria. As measure of gambling proneness, the dependent variable was the choice preference (%) for the 4 large-unlikely reward, i.e. the percentage of LLL over total choices expressed. As a measure of (motor) impulsivity (Sagvolden, 2000; Sagvolden and Sergeant, 1998), the dependent variable was the average frequency of inadequate nose-pokes per trial, calculated for each session, as well as the time needed to complete a trial. Locomotor activity inside operant chambers (measured as photobeam interruptions) was also measured to reveal general changes in motility. For all parameters, repeated measure ANOVAs were performed: day (i.e. session) was the within-subject factor; treatment was the between-subjects factor. As the two diets had differential effects on the rats’ body weight, to account for a possible bias due to gross differences, body weight changes were included as a covariate in the ANOVAs. When these covariates yielded an effect on the statistical analysis, a Pearson’s correlational analysis was done to further reveal the biasing effect.

2.5.3 The relationship between rIGT and rPDT The relationship between rIGT and rPDT was studied using correlational analysis. The last three sessions of both tasks were analysed, since they were characterized by similar conditions (stable performance in both experiments; probability values in the rIGT in the range of those in the rPDT). Moreover, these data are good indicators of decision-making performance (de Visser et al. 2011a) and gambling proneness (Adriani et al. 2009, 2010), respectively. Specifically, the individual values for choice preference (%) in the two tasks were correlated using Pearson linear correlation. Four Pearson’s R values were obtained by comparing the last three experimental sessions from rPDT (taken individually or as mean) with the mean value of the second half (last three experimental sessions) from rIGT (see Table 3). A further approach was undertaken for the purpose of graphical representation (see Figure 4). Each of the last three experimental sessions of the rPDT was combined with all the three rIGT sessions. We only showed those pairings of sessions that maximized the number of significant R values, avoiding day repetitions. This was done within each treatment separately, to avoid a bias introduced by the treatment itself.

2.5.4 Monoamines, monoamine metabolites and their ratios HPLC data were analysed by a set of two-tailed unpaired Student’s t-tests, to evaluate differences between experimental groups. The threshold for statistical significance was set at p ≤ 0.05.

66 Compromised decision-making and increased gambling proneness following dietary serotonin depletion in rats

3 Results

3.1 rIGT The control curves on decision-making performance were obtained from two independent locations: the labs and animal facilities at ISS and UU. One T+ subject at ISS and four T+ subjects at UU were excluded from further rIGT testing as they did not reliably nose-poke for pellet delivery. To assess the robustness of the rIGT protocol and whether the data of the control rats of ISS and UU could be pooled, these two locations were compared (see Table 1). Rats shifted their choice behaviour (across trial blocks) towards the long-term advantageous option, in an indistinguishable way (trial block x location: F(5,55) = 0.568, ns; location: F(1,11) = 0.259, ns). Therefore, results indicate that the rIGT protocol is robust and fully replicable across laboratories, allowing us to collapse these two locations into a single control group. Hence, data of six T- and thirteen T+ rats (n = 5 at ISS; n = 8 at UU) were analysed. The day before the start of the special diets, no difference was found in body weight between the groups (T+ rats, n = 13: 377.6 ± 7.5 g versus T- rats, n = 6: 367.9 ± 8.4 g; Student’s t= -0.776, df = 17, ns). After five days of ad libitum special diet, T+ rats (390.6 ± 6.8 g; 103.5 ± 0.6% relative to the start of diets) were heavier than T- rats (341.2 ± 8.4 g; 92.8 ± 1.5% relative to the start of diets; Student’s t= -4.295 (absolute values) and -7.809 (relative values), df = 17, p < 0.001). However, when the experiment-induced decrease was calculated on the last day of the rIGT (relative to the beginning of rIGT), no difference was found between the groups: the food restriction-induced body weight decrease (i.e. restriction level) was similar in T+ and T- rats across the five-day rIGT (T+ rats: 95.1 ± 0.3% versus T- rats: 96.0 ± 0.5% at the end of rIGT relative to the beginning; Student’s t = 1.474, df = 17, ns). Hence, the absolute and relative weights of rats differed at the end of the rIGT: T+ rats (371.5 ± 6.1 g, 98.5 ± 0.6% relative to the start of diets) were heavier than T- rats (327.6 ± 7.9 g; 89.1 ±1.4% relative to the start of diets; Student’s t = -4.169 (absolute values) and -7.051 (relative values), df = 17, p < 0.001).

Figure 2 Performance of depleted (T-) and control (T+) rats, as tested at ISS (Rome, Italy) and at UU (Utrecht, NL), in the home-cage rIGT. Shown are the mean (±SEM) proportions of “bad” choices (for the disadvantageous hole) per block of 40 trials. Rats could choose between two food pellets (the advantageous option: quinine at low probability, sugar at high probability) or four food pellets (the disadvantageous option: sugar at low probability, quinine at high probability). The test was conducted in the rats’ home-cages, with operant panels provided inside. *p < 0.05 Student t-test; depleted versus control rats.

67 Table 1 Performance of control rats at ISS (n = 5) and UU (n = 8) as tested in the home-cage rIGT. Shown are the mean (±SEM) proportions (%) of “bad” choices (for the disadvantageous hole) per block of 40 trials.

Trial blocks 1-40 41-80 81-120 121-160 161-200 201-240

T+ ISS (control) 47.00 ± 7.64 29.50 ± 5.39 32.00 ±12.58 17.50 ± 5.18 15.00 ± 5.48 13.13 ± 2.89

T+ UU (control) 35.94 ± 6.70 28.13 ± 3.77 27.19 ± 4.05 20.31 ± 5.38 11.25 ± 3.63 14.69 ± 3.55

Independent of treatment, all subjects improved their performance over trials 4 (see Figure 2; trial block: F(5,85) = 9.344, p < 0.001; trial block x treatment: F(5,85) = 0.919, ns). However, visual inspection of data strongly suggests that T- rats showed an overall weaker improvement across trials than T+ rats. Indeed, a near significant (treatment: F(1,17) = 3.828, p = 0.067) difference emerged. When considering the two halves of the rIGT (“exploration versus exploitation”; de Visser et al. 2011a) separately, clearly the difference between the depleted and the control groups was significant in the second half of the test, i.e. the last three sessions of the rIGT (treatment: F(1,17) = 5.143, p < 0.05). Accordingly, 5-HT depletion leads to a poor performance in the rIGT, specifically during the second half of the test (“exploitation” phase). To account for body weight differences, we included the rIGT-beginning and rIGT- end body weights (%, relative to ad libitum weight at start of the special diets) as covariates in the repeated measure ANOVA. It turned out that the treatment effect (without covariates: p = 0.067, see above) became significant, F(1,15) = 4.862, p < 0.05, indicating that relative body weight differences masked, but only slightly so, the observed test effect (main effect of rIGT-beginning relative weight covariate: F(1,15) = 1.407, ns; main effect of rIGT-end relative weight covariate: F(1,15) = 0.351, ns). No significant correlations were observed between the choice scores at each trial block and the relative body weight at the start or the end of the rIGT (R values between +0.112 and -0.383, n = 19, ns). Therefore, covariate and correlation approaches indicate that relative body weight differences ,if anything, slightly masked behavioural differences. Neither group showed a change across trials (trial block: F(5,9) = 0.581, ns; trial block x treatment: F(5,45) = 1.281, ns): T- rats showed more inadequate nose-pokes per trial (frequency: 3.18 ± 0.26) than T+ rats (1.80 ± 0.19; treatment: F(1,9) = 7.071, p < 0.05) throughout the experiment. While T+ rats decreased time needed to complete their trials across the sessions, T- rats showed the same score across trials, as per a floor effect (trial block x treatment: F(5,45) = 4.244, p < 0.01; data not shown). In fact, T- rats consistently needed less time to complete their trials (minutes needed per trial: 0.34 x 0.01) than T+ rats (0.55 ± 0.04; treatment: F(1,9) = 49.56, p < 0.001).

3.2 rPDT Before the rPDT started, all animals were weighed. On this baseline day, T- rats showed a markedly lower body weight compared to T+ controls (T+: 381.3 ± 9.5 g, 104.3 ± 1.9% relative to start of diets; T-: 326.8 ± 11.5 g, 89.3 ± 1.2% relative to start of diets; treatment: F(1,18) = 13.61, p < 0.01). However, when the task-

68 Compromised decision-making and increased gambling proneness following dietary serotonin depletion in rats induced decrease was calculated as percentage of the ad libitum baseline, taken just before the start of the rPDT, food restriction had a similar impact on body weight in either group (treatment: F(1,18) = 0.02, ns; day x treatment: F(6,108) = 1.39, ns). On average, the restriction level maintained during the rPDT was similar in T+ and T- rats (T+: 90.8 ± 1.0%; T-: 91.2 ± 1.0%; pool: 91.0 ± 0.7% of the ad libitum body weight, taken just before the start of rPDT). Hence, the weight of rats still differed at the end of the rPDT, with T+ rats (325.9 ± 13.4 g, 88.6 ± 2.4% relative to the start of diets) being heavier than T- rats (280.4 ± 5.3 g; 76.2 ±1.5% relative to the start of diets; treatment: F(1,18) = 48.19, p < 0.001). In contrast to T+ rats, T- rats showed only a slight reduction in their LLL preference, when the large reward decreased in probability. Specifically, a clear preference for SS was not developed by the T- group, even beyond the indifferent point (see Figure 3; treatment: F(1,18) = 0.39, ns; day x treatment: F(10,180) = 5.72, p < 0.001). At the lowest probability values, LLL choices were significantly higher in T- rats compared to T+ animals. Tukey posthoc showed a significant difference between groups on p = 80%, 14%, 11% and 9%, i.e. specifically during the final “gambling” part of the test. A separate ANOVA performed on the last three task sessions yielded a nearly significant treatment effect (F(1,18) = 4.25, p = 0.054). To account for body weight differences, we included as covariates the rPDT-mean weight (for the main effect of treatment) and the weights of each rPDT session (for the repeated measures factor). This ANCOVA revealed a completely similar profile (treatment: F(1,17) = 0.88, ns; day x treatment: F(10,179) = 5.97, p < 0.001; main effect covariate: F(1,17) = 0.99, ns; repeated measures covariate: F(1,179) = 2.22, ns), indicating that body weight differences between T+ and T- rats did not contribute to the observed effect. No significant correlations were observed between subject’s choice scores at each session and the corresponding body weight at each session (R values between -0.400 and +0.347, n = 20, ns), with the only exception of the second testing session (p = 80%). For this session, in which the decrease of probability was first introduced, a positive correlation was found (R-value = +0.528, p < 0.05). This significant correlation is in agreement with the significant group difference, revealed indeed at this session through Tukey post-hoc (see above). Therefore, covariate and correlational approaches confirm that body weight differences did not contribute at all to the behavioural profiles observed.

Figure 3 Performance of control and depleted rats as tested in the rPDT. Shown are the mean (±SEM) proportions of “risky” choices (for the LLL hole) per daily session. Rats could choose between two food pellets (small and sure, SS) or six uncertain food pellets (large and luck-linked, LLL). The test was conducted in classical operant (i.e. standard Skinnerbox) cages, with subjects daily transported from the housing room. Tukey HSD post-hocs: *p < 0.05; depleted versus control rats.

69 Table 2 Locomotor activity inside the operant chambers during the last three sessions of rPDT. Shown are the mean ± SEM amount of photobeam interruptions per session. Rats are the same as in Figure 3.

Probability level p = 14% p = 11% p = 9%

T+ 50.82 ± 5.19 58.54 ± 5.63 64.45 ± 6.89

T- 65.11 ± 4.68 53.00 ± 4.52 61.44 ± 6.82

4 While T+ rats showed, with decreasing levels of probability, a progressive increase in the total number of inadequate nose-pokes performed per trial, T- rats did not (data not shown). This was reflected by a near significant interaction term (day x treatment: F(10,180) = 1.87, p = 0.052). Overall, however, T- rats (frequency: 1.12 ± 0.04) did not differ from T+ rats (1.10 ± 0.03; treatment: F(1,18) = 0.05, ns) in the total number of inadequate nose-pokes performed per trial. T+ rats exhibited progressive reductions in time needed to complete their trials across daily sessions, whereas T- rats barely did so because of a floor effect (day x treatment: F(10,180) = 5.29, p < 0.001; data not shown). Moreover, T- rats consistently needed (on average) less time to complete their trials (seconds needed per trial: 30.43 ± 0.49) than T+ rats (34.76 ± 0.90; treatment: F(1,18) = 10.49, p < 0.01), especially at the beginning of the task. Indeed, post-hoc comparisons evidenced a significant difference between T+ and T- on p = 99%, 80%, 66% and 33%. In contrast, no significant difference was found between animal groups when considering the post-hoc comparisons performed on the last three task’s sessions (data not shown). No difference was observed between T- and T+ rats throughout the entire experiment for locomotor activity inside the operant chamber (treatment: F(1,18) = 0.99, ns; day x treatment: F(10,180) = 1.38, ns). Interestingly, a gradual increase of locomotor activity was observed over daily sessions, but similarly for both groups (day: F(10,180) = 11.27, p < 0.001). Noteworthy, the 5-HT depletion did not affect locomotion during the last three task sessions (see Table 2).

3.3 Relationship between rIGT and rPDT performance In the rIGT, the probability level was kept fixed throughout the experiment, while it was decreased progressively over daily sessions in the rPDT. Therefore, performance during the last three sessions of the rIGT was represented by the mean value, while the use of individual values from each session was more appropriate in the case of rPDT. A clear difference emerged between T+ and T- rats. A strong tendency for positive correlations emerged in T+ rats (see Table 3): slightly poorer decision-makers were likely to be more prone to gambling-like behaviour. In contrast, no correlations were found within the T- group. This result is also supported by a scatter plot of the last three task sessions, showing the correlations of individual performance parameters (within each treatment): the choice preference (%) for the disadvantageous hole in rIGT and for the LLL hole in rPDT (see Figure 4). For T+ rats, the relative choice for the disadvantageous hole in the rIGT was directly proportional to the relative choice for the LLL hole in the rPDT, that is, subjects behaving as good decision-makers in the rIGT were scoring

70 Compromised decision-making and increased gambling proneness following dietary serotonin depletion in rats lower for gambling proneness in the rPDT. In contrast, no correlations were found in the T- group. These data are suggesting a dissociation between decision-making and gambling proneness, at least when these emerge prominently as a consequence of 5-HT depletion in T- individuals.

Figure 4 In the cloud configuration, each individual rat of the depleted (T-, white symbols) and the control (T+, black symbols) group is represented by three symbols at the following coordinates: the parameter from the rIGT on X-axis and the parameter from the rPDT on Y-axis. The different shapes of the symbols (rhomb, triangle, square) identify the three selected comparisons, between pairs of sessions, ordered based on decreasing R values. The square refers to the third last (p = 14%), the triangle to the second last (p = 11%) and the rhomb to the last session (p = 9%) of the rPDT, for which the specific level of reward uncertainty is of particular relevance. The dotted lines, delineate significant correlations (within black triangles and within black rhombs) in the linear regression between rIGT and rPDT.

Table 3 R values of correlations between performance within each treatment group: for the rPDT, choice preference (%) for the LLL hole in each of the last three daily sessions, and their mean; for the rIGT, mean choice preference (%) for the disadvantageous hole calculated on the task’s second half (“exploitation”, last three experimental sessions). Threshold R value = 0.878 (n = 5).

Choice (%) for LLL hole in rPDT

p = 14% p = 11% p = 9% mean

Choice for disadvantageous hole Mean of the second half T+ 0.397 0.742 0.755 0.702 in rIGT

T- -0.174 -0.237 -0.311 -0.238

71 Table 4 Levels of monoamines and their metabolites as detected ex-vivo in the PFC (mean ± SEM; pg/mg of wet tissue). *p < 0.05, ***p < 0.001 between T- and T+ rats (n = 5-6).

5-HT 5-HIAA 5HIAA/ DA DOPAC HVA HVA/DA NA MOPEG MOPEG/ 5-HT NA T+ 23.3±1.4 271.9±16 11.6±0.6 16.9±2.6 1.8±0.6 31.2±2.8 2.6±1.0 172.8±3.9 47.8±1.8 0.28±0.01 T− 9.0±1.5 52.9±9.1 5.9±0.5 19.3±1.1 0.8±0.3 1.2±0.4 0.06±0.02 153.1±7.6 44.4±1.9 0.30±0.03 *** *** *** ns ns *** * * ns ns

4 3.4 Monoamine concentrations in forebrain areas HPLC data (see Table 4) demonstrated that dietary TRP depletion drastically reduced brain 5-HT synthesis, having marked consequences on levels of 5-HT and 5-HIAA in all brain areas analysed. Indeed, decreased 5-HT levels were observed in the PFC (t = 6.894, df = 9, p < 0.001, n = 5-6), striatum (t = 3.957, df = 9, p < 0.01, n = 5-6) and hippocampus (t = 3.000, df = 10, p < 0.05, n = 6) of T- rats compared to T+ animals. A comparable effect in the same direction was found on levels of the 5-HT metabolite 5-HIAA in the PFC (t = 11.90, df = 10, p < 0.001, n = 6), striatum (t = 3.799, df = 9, p < 0.01, n = 5-6), and hippocampus (t = 6.981, df = 9, p < 0.001, n = 5-6). The 5-HT turnover (calculated as the 5-HIAA/5-HT ratio) was significantly different only in the PFC (t = 7.022, df = 9, p < 0.001, n = 5-6), whereas no significant changes were observed in the striatum (t = 1.409, df = 8, ns, n = 4-6) and hippocampus (t = 0.6933, df = 9, ns, n = 5-6). No significant effect on dopaminergic and noradrenergic parameters was present in the striatum or hippocampus (data not shown). Moreover, no significant difference was observed in DOPAC concentrations detected in the PFC (t = 1.505, df = 10, ns, n = 6) (Table 4), the striatum (t = 1.710, df = 10, ns, n = 6), or the hippocampus (t = 0.4423, df = 10, ns, n = 6) (data not shown). However, in the PFC, a trend towards a decrease of DOPAC was observed (-53% in T- compared to T+ group). In the PFC of T- rats (see Table 4), both the levels of HVA and the DA turnover (calculated as the HVA/DA ratio) were significantly decreased (t = 11.93, df = 9, p < 0.001, n = 5-6; t = 2.684, df = 9, p < 0.05, n = 5-6, respectively), in the absence of significant alterations for levels of DA (t = 0.8576, df = 10, ns, n = 6). Furthermore, in the PFC of T- rats, the level of NA was significantly decreased (t = 2.309, df = 10, p < 0.05, n = 6), in the absence of significant alterations for levels of MOPEG and NA turnover (t = 1.315, df = 10, ns, n = 6; t = 0.5875, df = 10, ns, n = 6, respectively).

4 Discussion

The present study confirmed that serotonergic transmission was changed in PFC, hippocampus and striatum of TRP depleted subjects (Lee et al. 1999; Tanke et al. 2008; Uchida et al. 2005; Vergnes and Kempf, 1981), and revealed (1) that dietary 5-HT depletion in rats leads to both poor decision-making and gambling proneness; (2) that poor decision-making is related to gambling proneness in control but not in 5-HT depleted rats.

72 Compromised decision-making and increased gambling proneness following dietary serotonin depletion in rats

4.1 rIGT Reduced brain 5-HT synthesis, induced by dietary TRP depletion, disrupted rIGT performance in the home-cage setting, in that it led to a weaker improvement of decision-making performance over subsequent trials. Furthermore, it led to higher levels of motor impulsivity, as measured by inadequate nose-pokes and time needed per trial. The comparison of control groups run in Rome (ISS) and in Utrecht (UU) confirmed the robustness and replicability of the rIGT. These data also indicate that the home-cage operant task, which has been developed on the basis of a validated rIGT protocol (de Visser et al. 2011a; Homberg et al. 2008; van den Bos et al. 2006a), is well suited to study the decision-making performance in rats (Koot et al. 2010). Indeed, such home-cage panels are in general well-suited tools to measure operant behaviour-based performance (see also Koot et al. 2009). In the first half of the IGT, subjects explore the available options, while inthe second half of the task they switch to the option(s) which are advantageous in the long run. It has been hypothesized that the cognitive system, in which 5-HT plays a major role for the control of behaviour, is mainly activated during the second half of the task (de Visser et al. 2011a, 2011c; Homberg et al. 2008; van den Bos et al. 2006b, 2007). Recently, it was shown that the reduced rIGT performance of female rats, compared to male rats (van den Bos et al. 2006a, 2007), could be enhanced by increasing 5-HT levels through genetic engineering, i.e. in serotonin transporter knock-out rats (SERT KO; Homberg et al. 2008). Although data in the SERT KO rats may also be explained by 5-HT induced neuro-developmental changes (Homberg et al. 2008), the fact that enhanced performance was particularly expressed in the second half of the task may suggest a specific contribution of increased 5-HT levels. In line with the idea that 5-HT levels per se do matter, we show here that decreasing 5-HT levels in male rats led to exactly the opposite: a poorer performance in the second half of the task. Furthermore, after administering 8-OH-DPAT (a mixed 5-HT1A/7 receptor agonist), hence decreasing 5-HT release and acutely replicating a low 5-HT function, the subjects’ performance was impaired in an another IGT-like rat gambling task (where the duration of time-out periods was used as punishment, Zeeb et al. 2009). These data in rats collectively indicate that 5-HT levels do modulate decision- making performance as measured in IGT-like tasks (see de Visser et al. 2011c, for review). Data from human studies are in agreement with the present preclinical findings, in that differences in 5-HT transmission affect IGT performance. Healthy subjects being homozygous for the 5-HT transporter short allele (s/s) performed worse on the IGT than l/l carriers (He et al. 2010; Homberg et al. 2008; Stoltenberg and Vandever, 2010). The same was found in clinical patients with depression (Must et al. 2007), obsessive-compulsive disorder (da Rocha et al. 2008), borderline personality disorder (Maurex et al. 2009), especially in case of a TPH-1 (tryptophan hydroxylase-1, the rate-limiting enzyme in 5-HT bio-synthesis) haplotype gene (Maurex et al. 2009). Our data on motor impulsivity are in line with numerous studies that support the idea of an inverse relationship between 5-HT function and impulsivity, as was shown not only through a variety of human disorders, but also in a series of animal studies (see Bizot et al. 1999; Clarke et al. 2004; Evenden, 1999; Leyton et al. 2001; Rogers et al. 1999a; Soubrié, 1986). Cools et al. (2011) have pointed out that

73 manipulating 5-HT levels seems to affect performance only on impulsivity tasks that have a clear affective (reward versus punishment) component (e.g. reversal learning, conditioned suppression, premature responding and intertemporal choice), while the performance on tasks without a clear affective component (e.g. the stop-signal reaction-time task and the go-nogo task) was not affected by 5-HT manipulation. The rIGT clearly belongs to the first category.

4.2 rPDT Our results demonstrate that a reduced brain serotonergic function strongly elicits a trait of gambling proneness, as evidenced in the rPDT. At the lowest probability 4 values, far beyond the indifferent point, depleted rats showed only a slight reduction in their LLL preference: these rats were not only tolerant to reward uncertainty, but also sub-optimally attracted by its random-and-binge delivery. Although brain 5-HT depletion is known to affect processing of delayed reinforcement (temporal-discounting), so far it does not appear to consistently influence choices involving uncertain reinforcement (probabilistic discounting; Cardinal, 2006; Mendelsohn et al. 2009). Indeed, experimental evidence demonstrating a link between 5-HT and risky decision-making is rather contradictive, in part due to the lack of a good animal model (Long et al. 2009). Similarly, a study performed in rats did not seem to support a role for central 5-HT in sensitivity to probabilistic reinforcement (Mobini et al. 2000b). However, other experimental studies in both humans, monkeys and rats have demonstrated the opposite (Long et al. 2009; Murphy et al. 2009; Rogers et al. 1999a; Zeeb et al. 2009). Recently, Mendelsohn et al. (2009) have reviewed the existing literature on the effects of acute L-tryptophan depletion (ATD) on memory, attention and executive functions in humans. Distinctively, decreasing central 5-HT by means of ATD can both improve (Rogers et al. 2003; Talbot et al. 2006) and decrease (Rogers et al. 1999a), or even has no effect on (Anderson et al. 2003) probabilistic discounting. ATD may subtly alter the ability to discriminate differences in the magnitude of rewards, but not of losses (Rogers et al. 2003). More recently, other authors concluded that 5-HT activity plays a significant role in irrational, risky decision-making under conditions of uncertainty. Indeed, it has been found that the subjective value of the risky option increased (or the subjective value of the safe option decreased) following serotonin depletion (Long et al. 2009). The role of 5-HT in decision-making (under conditions of reward uncertainty) can also be investigated by means of dietary TRP supplementation instead of depletion. Such supplements were associated with alterations in the impact of gains and losses and with a marked unbalance between risk-averse and risk-seeking choices (Murphy et al. 2009). In line with findings in the rIGT, 5-HT depleted rats needed less time per trial than control rats at the beginning of the rPDT, i.e. before the indifferent point. They also tended to show slightly more inadequate nose-pokes. Apparently, 5-HT depletion leads to motor impulsivity and quick choice selection in both tasks. However, in the rPDT beyond the indifferent point (i.e. in the final “gambling” part), 5-HT depleted and control rats did not differ in time needed to complete a trial. Furthermore, when considering only the SS hole (data not shown), the number of inadequate nose-pokes increased more in controls than in the 5-HT depleted rats with progressive reward rarefaction, suggesting that the former may have been disturbed by punishment

74 Compromised decision-making and increased gambling proneness following dietary serotonin depletion in rats

(consisting in reward-delivery omission) while the latter did not. Finally, locomotor activity increased over sessions, which could be interpreted as a reaction tothe progressive rarefaction of rewards in both groups.

4.3 Roles of serotonin and PFC in decision-making/gambling proneness The notion that 5-HT depletion modulates both decision-making and gambling proneness may be related to reward sensitivity (Hayes and Greenshaw, 2011), punishment sensitivity (Cools et al. 2008, 2011) or behavioural flexibility, and points to the interplay among different cortico-basal ganglia loops (Bechara, 2005; Doya, 2008; Homberg, 2012; Tanaka et al. 2004, 2007, 2009). Serotonin plays an important role in the regulation of reward-related processing, as it is involved in natural reward-related physiology and behaviour, from feeding and sexual activity to emotional regulation. Altered reward processing has been proposed in many psychiatric disorders (see for review Hayes and Greenshaw, 2011). In the present study, data obtained in the rIGT suggest that the reward-system might have been affected by TRP depletion. However, this does not appear the case for the performance in the rPDT, as no significant difference between T+ and T- rats was observed in learning curves during the training phase and the initial testing phase when uncertainty level was low (see Figure 3, sessions until p = 33%). That is, both control and depleted rats were perfectly able to differentiate large from small rewards. Accordingly, we conclude that differences in reward sensitivity per se do not seem to account for the differences between 5-HT depleted and control rats in rIGT and rPDT performance. Serotonin plays also a role in the perception of punishment and in negative reinforcement, as depletion of 5-HT has been suggested to enhance (see Cools et al. 2008, 2011) and/or to decrease (Graeff, 2002; Soubrié, 1986; see also Crockett et al. 2009) the behavioural and brain responsiveness to punishment. According to the first suggestion, a reduction of 5-HT levels could enhance aversive processing (Cools et al. 2008, 2011). However, in both rIGT and rPDT the 5-HT depleted rats choose for the option with the highest punishments, i.e. a high frequency of quinine pellets’ presentation and of reward omission, respectively. Apparently, 5-HT depletion leads to a lower sensitivity to punishments, as indicated above. Alternatively, or in addition to such punishment insensitivity, it is well known that prefrontal 5-HT mediates attention to environmental stimuli arriving as outcome of actions (i.e. reward, but also punishments, Branchi, 2011; Homberg, 2012). It may be proposed that 5-HT depleted rats have a lower capacity to notice the outcome of their actions in both tasks, and therefore do not adapt their behaviour. Finally, another way to explain the observed differences might be that depleted rats are inflexible or rigid in their responding. That is, while the behaviour of control rats shifts under the guide of a cognitive/motor loop, which allows them to develop and maintain a long-term strategy (Tanaka et al. 2004, 2007, 2009; see next section), depleted rats remain driven by the limbic loop, which is actually rendered less sensitive to changes in reward punishment contingencies. Chronic dietary TRP depletion led to extremely reduced levels of HVA, and impaired DA turnover in the PFC. An effect on DA and/or its metabolites after 5-HT manipulation was expected, because of the well known influence exerted by 5-HT upon central

75 dopaminergic systems (see for review Di Giovanni et al. 2010). Indeed, it has been suggested that changes in the relative levels of these two neurotransmitters could be one of the crucial factors in ADHD (Oades, 2002). The present strong reduction of DA turnover may suggest a compromised COMT-mediated catabolism, which is the major metabolic fate of prefrontal DA (Morón et al. 2002). Adequate decision- making requires the integrity of the anterior cingulate cortex (ACC) and the OFC (Cohen et al. 2005), as well as associated subcortical circuitry including the basal ganglia (Krawczyk, 2002) and amygdala (Bechara et al. 2003). A deficit in the OFC may be particularly critical in decisions involving reward and punishment, including gambling (Rogers et al. 1999a). Patients with OFC lesions, but not patients with other 4 prefrontal lesions, showed impaired decision-making in a decision-gamble task. Therefore, altered prefrontal 5-HT together with compromised DA transmission may have contributed to the behavioural performance observed for depleted rats (for a review, see Rogers, 2011). Chronic dietary TRP depletion also reduced the levels of NA in the PFC. Relatively little is known, so far, about the role of NA in probabilistic reinforcement (Cardinal, 2006). It has been suggested that NA neurons encode some aspects of uncertainty in the general process of making predictions in a given context (Yu and Dayan, 2005). Systemic NA blockade has been shown to affect decision-making under uncertainty in humans, by reducing the discrimination between magnitudes of different losses (Rogers et al. 2004). Conversely, NA re-uptake inhibition had no effect in the human IGT (O’Carroll and Papps, 2003). In our hands, reduced NA in the PFC of T- rats may hence favour insensitivity to reward omission.

4.4 Relationship between rIGT and rPDT performance Significant correlations between the performance parameters of the rIGT and rPDT were found within controls but not within the 5-HT depleted rats. The fact that we did not observe a correlation between these two traits in depleted rats may suggest that poor decision-making does not necessarily predict gambling proneness. That is, in control rats, lower choice values in the rIGT, identifying good decision-makers, were associated with lower choice values in the rPDT, identifying risk-averse subjects. When 5-HT systems were intact, these two traits did correlate, while this link disappeared when rats were depleted of 5-HT. This may indicate that a dissociation between decision-making and risk-proneness can only be unmasked by altered function of the 5-HT system. As such, poor decision-making emerges in some individuals whereas a risk-prone trait emerges in other individuals, only when experimental subjects suffer from 5-HT depletion. There are three types of choosing under uncertainty, distinguished by the degree of awareness about probabilistic rules and choice outcome. Decisions “under ambiguity” are those with unknown rules, corresponding to the early (training) part of both the rIGT and rPDT (Stoltenberg and Vandever, 2010). When the contingencies have been learnt, decisions under risk imply a possible detrimental impact on outcome, corresponding to the last sessions of both tasks. We underline that actual details of task options can be designed to produce low versus high levels of risk, as we did presently by using rIGT and rPDT, respectively. Decision-making under low versus high risk may involve dissociable brain 5-HT sub-circuits: choosing under low risk may involve a cognitive loop (Brand et al. 2006), whilst we proposed that high-

76 Compromised decision-making and increased gambling proneness following dietary serotonin depletion in rats risk options may trigger a “temptation” (Adriani and Laviola, 2006) and therefore involve a limbic loop of motivational/affective processing. Tanaka et al. (2004, 2007) suggested a functional shift in corticobasal ganglia loops, from predicting immediate outcomes towards predicting long-term or future outcomes, in subjects performing a decision-making task: activity of the lateral OFC - ventral striatum (“limbic”) loop was correlated with reward prediction in shorter time scales, mainly at low 5-HT levels, while activity of the dorsolateral prefrontal cortex (dlPFC) e dorsal striatum (“cognitive/motor”) loop correlated with longer time scale predictions, being stronger at high 5-HT levels (Tanaka et al. 2007). This was recently confirmed in a study using immediate and delayed punishments after TRP depletion (Tanaka et al. 2009). Low levels of 5-HT would be associated with loss of prefrontal top-down control from OFC (and dlPFC) over ventral (and dorsal) striata (Homberg, 2012). These two independent sub-sets (i.e. “limbic” versus “cognitive” loop) have recently been imaged (Canese et al. 2011), by the use of serotonergic compounds. Accordingly, in the rIGT, 5-HT depleted animals would remain focused on immediate wins and losses; they would not develop a choice for the long-term advantageous option since this would require both: a) integrating the wins and the losses into combinations differing in values (as normally done by the OFC), and b) shifting towards the above-mentioned control by the cognitive loop (de Visser et al. 2010, 2011a, 2011c; Homberg et al. 2008; van den Bos et al. 2006b, 2007). In the rPDT, however, a lack of affective feedback within the limbic loop would lead, in the 5-HT depleted animals, to a lower ability to detect changes in contingencies: these rats would thus continuously choose for the high-magnitude reward, although coming only rarely. These cognitive versus limbic loops would be both affected, though independently, and would impede a shift of behaviour towards a “safer” option, represented in both cases by the small reward. In other words, the5-HT depleted rats were stuck on a short-term focus and unable to recruit cognitive self- control (over the limbic system) effectively; the latter conversely allowed control subjects to change their strategy towards a long-term advantageous behaviour. As far as present data are concerned, we may hypothesize that, under normal conditions, there is a crosstalk and reciprocal balance between these two looping systems, mediated by 5-HT. Thus, good decision-making and risk aversion are both associated with a correct function of the limbic and cognitive loops. Conversely, when 5-HT is depleted, both sub-sets of 5-HT system (for images, see Canese et al. 2011) become actually impaired and possibly uncoupled, so that the observed phenotype may depend on which system loop is affected most. Given the relatively low number of individual animals in the present correlation analysis, further research would be necessary to substantiate these conclusions.

4.5 Potential experimental caveats The final remark about our present TRP depletion approach is to rule out a potential “bias” of the study, due to the fact that the TRP-free diet (in combination with the food-restriction schedules) not only decreased 5-HT synthesis in the brain (thus affecting cognition and risk aversion), but also led to whole-body metabolic adaptation processes, leading to weight loss. A diminished food intake, and consequently a weight loss, is observed in animals when fed with a diet lacking only

77 one amino acid (Leung and Rogers, 1973), as in the case of tryptophan (Walters et al. 1979). It is however unlikely that reduced body weight alone may account for the different choice patterns shown presently. In our hands, covariate and correlation approaches confirm this notion. In agreement with our result, authors who also assessed the overall consequences of a TRP-free diet, concluded that decreased food intake, and consequently weight loss in T- animals, could not account for their observed behavioural profiles (Tanke et al. 2008). Such a conclusion is, at least for the rIGT, in line with recent data (Rivalan et al. 2009) showing that rats’ performance in a closely-related task was independent of deprivation levels (0e20% loss of body weight). 4 Also, we assume that the metabolic impact of the TRP-free diet was similar throughout the entire one-month deprivation period, and likely did not differ between the first and second operant tests. It is well known that the 5-HT system tends to reach very quickly, and to maintain thereafter, a “stable minimum” until sacrifice. Indeed, Fadda et al. (2000) reported that, already after four days, a TRP- free diet induced an almost total disappearance of 5-HT extracellular levels in the frontal cortex. Similarly, Tanke et al. (2008) observed a marked decrease in blood 5-HT levels and neuronal 5-HT content on day 6 already. Accordingly, in our present experiment, we decided to wait for five days (after providing diets), thus starting operant protocols on the sixth day.

4.6 Conclusion The present data show that L-tryptophan depletion leads to changes in behavioural responses within a decision-making as well as a gambling-proneness task. Future studies should investigate in more detail the 5-HT mediated changes, occurring within forebrain loops involved in cognitive control and motivation/affective processes.

78 Compromised decision-making and increased gambling proneness following dietary serotonin depletion in rats

Acknowledgements

Funding source: Italian Ministry of Health, with “under 40” Young-Investigator Project “ADHD-sythe” and EU-FP7 “Prio-Med-Child” ERAnet Project “NeuroGenMRI” (both coordinated as PI by WA); Utrecht University, PhD support (to SK). We wish to thank Pietro and Alessio SERENELLINI (PRS Italia, Rome, Italy) for manufacturing and technical support with the Home-Cage Operant Panels; Dr Flavia CHIAROTTI and Dr Hein VAN LITH for their expert statistical advice; Luigia CANCEMI for valuable assistance with animal care; Nadia FRANCIA for precious technical support. Finally, we really wish to thank the two anonymous referees for constructive comments, which helped to improve the quality of the manuscript.

Note: Part of these data have been published as an abstract in: Proceedings of Measuring Behaviour (Eindhoven, The Netherlands, August 24-27), 2010.

79 5

80 CHAPTER 5

TIME-DEPENDENT EFFECTS OF CORTICOSTERONE ON REWARD-BASED DECISION-MAKING IN A RODENT MODEL OF THE IOWA GAMBLING TASK

S. Koot a,b a Department of Neuroscience & A. Baars b Pharmacology, Rudolf Magnus Institute, P. Hesseling b University Medical Centre Utrecht, R. van den Bos a,c The Netherlands M. Joëls a b Division Behavioural Neuroscience, Department of Animals in Science and Society, Faculty of Veterinary Medicine, Rudolf Magnus Institute, Utrecht University, The Netherlands c Department of Organismal Animal Physiology, Radboud University Nijmegen, The Netherlands

Neuropharmacology (2013) 70: 306-315.

81 Abstract

Corticosteroid hormones, released after stress, are known to change neuronal activity in two time-domains: within minutes via non-genomic pathways and with a delay of >1 h through pathways involving transcriptional regulation. Recent evidence in rodents and humans indicates that these two modes of corticosteroid action differently affect cognitive tasks. Here, we investigated whether reward-based decision- making, in a rat model of the Iowa Gambling Task (rIGT), is also differently altered by rapid versus delayed actions 5 of corticosterone. We targeted the rapid and delayed time domain by injecting corticosterone (CORT, 1 mg/kg, s.c.) at 30 min (rapid) or 180 min (delayed) respectively prior to behavioural testing, during the final 3 days of the behavioural paradigm. In saline treated rats, the number of visits to the disadvantageous arm decreased over trial blocks, whilst this was attenuated when CORT was administered 30 min before testing. This attenuation was associated with a significantly increased c-Fos expression in the lateral orbitofrontal cortex and insular cortex, and a trend for an increase in the infralimbic cortex. The rapid corticosteroid effect contrasted with treatment 180 min before testing, where the number of visits to the disadvantageous arm as well as c-Fos labelling was not affected. These findings indicate that rapid corticosteroid actions impair reward-based decision-making.

82 TIME-DEPENDENT EFFECTS OF CORTICOSTERONE ON REWARD-BASED DECISION-MAKING IN A RODENT MODEL OF THE IOWA GAMBLING TASK

1 Introduction

Upon stress, corticosteroid hormones are released in high amounts. Shortly after stress onset, corticosteroids change neuronal cell function through non-genomic pathways (Joëls et al. 2012). Simultaneously a gene-mediated cascade is started which eventually will normalise increased neuronal activity. Several studies over the past years have shown that rapid (non-genomic) versus delayed (genomic) corticosteroid actions differentially affect cognitive function. For instance, through slow genomic actions stress improves spatial, declarative and working memory formation (Cornelisse et al. 2011; Henckens et al. 2011; Oitzl et al. 2001; Sandi and Pinelo-Nava, 2007; Smeets et al. 2009; Yuen et al. 2009). In the rapid time-domain, stress causes a shift from spatial and instrumental towards habitual learning strategies, in rodents as well as humans (Schwabe and Wolf, 2011). When targeting these two time-domains specifically by administering cortisol in human subjects either shortly or several hours before behavioural testing, working memory was found to be improved by slow compared to rapid corticosteroid actions, and this improved performance was linked to enhanced activity in the dorsolateral prefrontal cortex (Henckens et al. 2011). These and other studies have led to the hypothesis that prefrontal cortical functioning is impaired by corticosteroids acting via rapid non-genomic pathways, but enhanced by slow corticosteroid actions (Joëls et al. 2012). One of the most important cognitive functions in daily life is decision-making. It comprises a complex assessment of short-term and long-term costs and benefits of competing actions. Recently, men were found to become more risk-taking under acutely stressful conditions in formal tests of risk-based decision-making, using e.g. the IowaGambling Task (IGT; Preston et al. 2007; van den Bos et al. 2009), the Balloon Analogue Risk Task (Lighthall et al. 2009) or the Game of Dice Task (Starcke et al. 2008; Starcke and Brand, 2012). In none of these studies, though, the two time-domains of corticosteroid actions were considered. We hypothesized that decision-making is impaired by rapid actions of corticosteroids but improved by delayed effects. In the current study we tested this hypothesis in male Wistar rats, using a well validated rodent model of the IGT (rIGT; Homberg et al. 2008; de Visser et al. 2011a, 2011b; van den Bos et al. 2006a, 2012b; van Hasselt et al. 2012; review: de Visser et al. 2011c), which assesses risk-based decision-making. In general, the output of decision-making processes is determined by an interaction between two different forebrain loops: i) a limbic (affective/motivational) loop, encompassing the orbitofrontal cortex, the amygdala and ventral striatum/nucleus accumbens, and ii) a cognitive (executive/motor) loop, encompassing the dorsolateral prefrontal cortex, anterior cingulate cortex and dorsal striatum (Bechara, 2005; de Visser et al. 2011a, 2011b, 2011c; Doya, 2008; McClure et al. 2004; Rivalan et al. 2011; Tanaka et al. 2004, 2007; Yin and Knowlton, 2006; Yin et al. 2008; Zeeb and Winstanley, 2011). While the limbic loop is involved in the early stages of the IGT, adjusting behaviour to changing contingencies, the cognitive loop is more involved in later stages, i.e. in future perspectives and cognitive control. The medial prefrontal cortex (mPFC) in rats has been suggested to share an anatomical and functional homology to the anterior cingulate cortex and dorsolateral prefrontal cortex in humans (Brown and Bowman, 2002; Uylings and van Eden, 1990; Uylings et al. 2003). We have previously shown in male Wistar rats, using transient inactivation of the mPFC with the GABA-

83 agonists muscimol and baclofen in the last three daily sessions of the rIGT, that the mPFC becomes indeed more strongly involved as rats (express to) have learned task- contingencies, i.e. choose for the best long-term option (de Visser et al. 2011b). Therefore, in view of our hypothesis, CORT (1 mg/kg s.c.) was applied during the later stage of the task (final three daily sessions) either 30 or 180 min prior to behavioural testing, to probe the rapid and delayed effects on task progression respectively. As corticosteroids are known to affect working memory and consolidation (Cazakoff et al. 2010; Roozendaal and McGaugh, 2011), we included parameters indicative hereof in our task. After the final session, c-Fos expression was determined, to examine the involvement of specific brain regions in the behavioural effects (de Visser et al. 2011a; van Hasselt et al. 2012). 5 2 Materials and methods

2.1 Animals Male Wistar rats (Harlan, the Netherlands; Charles River, Sulzfeld, Germany; 9-11 weeks of age) were kept under controlled conditions (temperature 21 ± 2 °C, relative humidity 60 ± 15%) and a reversed 12 h light/dark cycle (lights off at 7:00 h or 8:00 h; depending on the interval between testing and corticosterone injections, see below). Rats were housed in pairs in Macrolon type-IV cages with a shelter and paper tissues as cage enrichments. Food and water were available ad libitum except during testing (see below). A radio provided background noise. All experiments were approved by the Animal Ethics Committee of Utrecht University and conducted in agreement with Dutch laws (Wet op de Dierproeven, 1996) and European regulations (Directive 86/609/EEC).

2.2 Behavioural procedures Experiments were run as described previously (de Visser et al. 2011a, 2011b) in order to keep the procedure similar to earlier used protocols. Therefore, after a habituation period of 2.5 weeks, all rats were first tested on the elevated plus maze (EPM) and subsequently in the rodent version of the Iowa Gambling Task (rIGT). Because anxiety, as measured by EPM performance, was earlier found to correlate with rIGT behaviour (Rivalan et al. 2009; de Visser et al. 2011a), we compared anxiety levels between batches of rats to promote that they were comparable prior to saline or corticosteroid treatment (see 2.5 Statistical Analyses). We refer to de Visser et al. (2011a) for detailed descriptions of the EPM protocol and analyses. We analysed percent time spent on the open arm as a measure of anxiety (de Visser et al. 2011a). Rats were tested in the rIGT 1.5 week after EPM exposure (starting on Mondays; see de Visser et al. 2011a). The rIGT apparatus, made of grey PVC, consisted of a start box, choice area and four arms. Before testing, rats were habituated to the apparatus in a 10 min free exploration trial. Two days later they were mildly food deprived by providing for one day 50% of ad libitum food intake (on Sunday), followed by 30% of ad libitum food intake (for at least two days), which was subsequently individually adjusted to maintain rats at 90-95% of free feeding body weight during testing. Testing took place for 9 days, between either 9:00-12:00 h (lights off at 7:00 h) or 11:00-14:00 h (lights off at 8:00 h) for the rapid CORT/saline (30 min) and delayed

84 TIME-DEPENDENT EFFECTS OF CORTICOSTERONE ON REWARD-BASED DECISION-MAKING IN A RODENT MODEL OF THE IOWA GAMBLING TASK

CORT/saline (180 min) groups, respectively. Testing did not occur during weekend days. Food was freely available on these days, but rats were returned to their restricted diet the day before testing continued. A trial started by lifting the slide door of the start box. Rats could freely enter the choice area of the apparatus and choose one of the four arms. To help rats differentiating arms, visual cues (10 x 10 cm; cross or circle in black or white) were placed to the side of the wall at the entrance of arms. When rats had entered a choice arm with their full body, including their tail, the arm was closed. At the end of the arm, rats could obtain pellets (baited arms) or nothing at all (empty arms). Each trial lasted max 6 min (inter-trial interval: 30 s). Rats received a total of 120 trials: 10 trials on days 1-6 and 20 trials on days 7-9. By the time rats reached the second half of the task, i.e. days 7-9, a session lasted 12 min at most. Rewards were 45 mg sugar pellets (F0042, Bio-serv Inc, Frenchtown, NJ, USA); punishments were quinine-treated sugar pellets that were unpalatable but not uneatable. Rats were habituated to the sugar pellets in the week before the first rIGT session in their home cage daily, followed by a single session of providing two sugar pellets in a novel empty Macrolon type-III cage. Most rats consumed the quininetreated sugar pellets once, but left them uneaten after briefly tasting them. Rats consistently eating quinine-treated sugar pellets were excluded from analysis. No animals were excluded because of this in the present experiments. Of the four arms in the maze, two arms were baited and two arms were empty. The two empty arms were included to measure non-reward related exploration (de Visser et al. 2011a, 2011b, 2011c). The two baited arms consisted of a long-term disadvantageous arm and a long-term advantageous arm. In the disadvantageous arm, rats received occasional big rewards (three sugar pellets in 1 out of 10 trials) among frequent punishments (three quinine-treated sugar pellets in 9 out of 10 trials). In the advantageous arm, rats received frequent small rewards (one sugar pellet in 8 out of 10 trials) and infrequent punishments (one quinine-treated sugar pellet in 2 out of 10 trials). The positions of the baited and empty arms, as well as the advantageous and disadvantageous arm were counterbalanced across rats. Performance parameters were: (1) the number of visits to the empty arms as a fraction of the total number of trials per block of 20 trials and (2) the number of visits to the disadvantageous arm as a fraction of the total visits to the baited arms per block of 20 trials. To unravel the potential mechanisms underlying behavioural effects of CORT injections, we measured several parameters in addition. First, for the disadvantageous and empty arms the average scores over the first and final 5 trials per block of 20 trials were compared as a measure of working memory, whilst the first 5 trials of a block of 20 trials were compared with the final 5 trials of the previous block of 20 trials as a measure of consolidation. Although these measures have not been validated in this particular task, they are comparable to approaches used in e.g. a radial maze (Butts et al. 2011; Conrad, 2010; Janitzky et al. 2011). Second, the total number of switches between different arms per block of 20 trials was calculated as measure of exploratory behaviour (de Visser et al. 2011a). Third, responses to encounters with sugar pellets or quinine-treated sugar pellets in the advantageous and disadvantageous arm were measured as win-stay/lose-shift behaviour per block of 20 trials (de Visser et al. 2011a). Blocks of 20 trials were chosen to obtain a sufficient number of encounters with sugar and quinine-treated sugar pellets. Thus,

85 when rats encountered a sugar reward, their subsequent choice was scored as a win-stay when they revisited the same advantageous or disadvantageous arm. When rats encountered a quinine punishment, their subsequent choice was scored as a lose-shift when they switched to another arm. Win-stay and lose-shift behaviour were calculated as a fraction of the number of encounters with either sugar (win) or quinine (loss). Thus, values of 1 indicate win-stay and lose-shift tendencies only, while values of 0 indicate win-shift and lose-stay tendencies only.

2.3 Corticosterone injections Saline or corticosterone (1 mg/kg CORT, stored at -20 °C until use; C174 Corticosterone HBC-complex, SigmaeAldrich Inc.) injections were given subcutaneously on the last 5 3 days of the rIGT, either 30 or 180 min before testing. After injection rats were returned to their home cage, before they were tested in the rIGT. We chose to administer CORT s.c. in the second half of the task, i.e. the final three sessions of 20 trials each, because the mPFC becomes more strongly involved as rats have learned task-contingencies within the rIGT (de Visser et al. 2011b). The prefrontal cortex is particularly sensitive to the effects of stress exposure (Arnsten, 2009). The dose of CORT was chosen to induce plasma CORT levels resembling those achieved after stress (de Quervain et al. 1998; Shafiei et al. 2012). To confirm this, the levels of plasma CORT were determined 30 min after CORT or saline injection in an additional group of rats that was exposed to the same experimental procedure (rIGT testing days 1-5), but from which blood samples were drawn through tail vein incision on days 6-9 instead of testing them in the rIGT. Levels of plasma CORT were not determined after 180 min as levels have been shown to peak after 30 min but return to nearly baseline within 90 min after 1 mg/kg (see e.g. Shafiei et al. 2012). Blood was collected in heparinised (500 IU ml-1) tubes and stored on ice. After centrifugation at 5000 r.p.m. for 10 min, the supernatant was stored at-20°C until assay. Corticosterone plasma concentrations were determined witha radioimmunoassay (07-120102, ImmuChem Double Antibody Corticosterone RIA Kit, MP Biomedicals, New York, USA). Coefficients of variation within and between assays were less than 10%.

2.4 c-Fos immuno-histochemistry Two hours after the final rIGT session, rats were decapitated, blood was collected and corticosterone plasma concentrations were determined as described above. Brains were quickly frozen in liquid (-80 °C) 2-methylbutane and stored at -80 °C. To distinguish between the effect of treatment per se (CORT or saline) and treatment in relation to rIGT testing (30 min before testing) on c-Fos labelling, an experimental group (n = 14 and 14 for CORTand saline, respectively) receiving the same treatment but not exposed to the rIGT was added. Coronal sections (20 mm) were cut on a cryostat (Leica CM3050S) and mounted on Starfrost adhesive slides (Knittel Glaser,Waldemar Knittel, Germany) and stored at -20 °C. Sections were analysed by immuno-histochemical detection of c-Fos according to previously published protocols (de Visser et al. 2011a; van Hasselt et al. 2012). For the immuno-histochemical detection of c-Fos, rabbit anti-c-Fos (Calbiochem, Darmstadt, Germany) was used. The anatomical localization of c-Fos was aided by use of adjacent Nissl stained sections and illustrations in a stereotaxic atlas (Paxinos and

86 TIME-DEPENDENT EFFECTS OF CORTICOSTERONE ON REWARD-BASED DECISION-MAKING IN A RODENT MODEL OF THE IOWA GAMBLING TASK

Watson, 2005). For each region analysed (see Results) at least two overt landmarks were used. For quantitative analysis of c-Fos positive cells, the program Leica QWIN (image processing and analysis software, Cambridge, UK) was used. Left and right hemispheres were separately analysed in one section. The number of positive cells was then averaged for each animal and expressed per mm2.

2.5 Statistical analyses Statistical analyses were carried out using SPSS 16.0 for Windows. We noted that in one batch of rats in the rapid CORT experiment the CORT effect was clearly different from the other three batches, which was confirmed by statistical analysis (three- way repeated measures ANOVA: treatment, batch; repeated: trial block (last three sessions): trial block*treatment*batch: F(6,56) = 2.488, p < 0.05; trial block: F(2,56) = 10.176, p < 0.001; treatment: F(1,28) = 4.778, p < 0.05). As this batch (also) consisted of animals with very low levels of anxiety as measured on the EPM compared to the other batches (per cent open arm time >40%) we excluded this batch (n = 9 rats: CORT n = 5, saline n = 4) from further analysis, leaving 27 rats (CORT n = 15, saline n = 12) for final behavioural analysis in this experiment. Specific tests are indicated in the Results section. Whenever sphericity was violated in an Analysis of Variance (ANOVA), degrees of freedom were adjusted. Statistical significance was set at p ≤ 0.05 (two-tailed).

3 Results

3.1 Corticosterone injections The sample on day 6 served as a baseline andwas compared with levels on days 7-9 (Figure1). No differences existed in baseline plasma CORT levels for the two injection groups; values were at a level that can be expected at the start of the active phase. Plasma CORT levels were clearly higher after CORT injections than after saline injections across the three day period, with a slight decrease in CORT plasma levels on day 3 (repeated measures ANOVA: day*treatment: F(2,34) = 4.643, p < 0.05; treatment: F(1,17) = 119.957, p < 0.001). Plasma CORT values were clearly higher than baseline values 30 min after CORT injections (paired t-test: 1st: t=12.525, df=8, p<0.001; 2nd: t = 14.431, df = 8, p < 0.001; 3rd: t = 3.329, df = 8, p < 0.01). Apart from a trend for a decrease on the first day (paired t-test, t = 2.217, df = 9, p = 0.054) no differences were found on the other two days following saline injections. When rats were decapitated on the final day of the rIGT paradigm, rats treated with CORT 30 min before rIGT testing did not show elevated plasma corticosterone levels compared to controls when examined 120 min later, i.e. in total 150 min after injection; rather their values appeared to be lower: CORT treated rats (mean ± SEM): 56.01 ± 10.01 ng/ml, saline treated rats: 271.04 ± 34.09 ng/ml (t(25) = -5.857, p < 0.0001).

87 5

Figure 1 Mean (±SEM) plasma CORT values during baseline and following three injections of CORT or saline (determined at 30 min after injection). **: p < 0.01; ***:p < 0.001.

Table 1 Baseline rIGT performance prior to treatment. Shown are means (±SEM) of fractions of visits to (a) empty and (b) disadvantageous arm of rats of the 30 and 180 group before treatment started, i.e. trial blocks 1-60. No significant differences between the intended treatment groups were found on the last session before the injections started for the empty arms (two-way ANOVA: timing, treatment) and for the disadvantageous arm (two-way ANOVA: timing, treatment). This was further confirmed by separate Student t-tests (treatment) for each timing condition.

a. Fraction of visits to empty arms

30 group 180 group

trial block CORT saline CORT saline

1 – 20 0.38 ± 0.02 0.43 ± 0.04 0.43 ± 0.03 0.48 ± 0.02

21 – 40 0.38 ± 0.04 0.40 ± 0.03 0.44 ± 0.02 0.41 ± 0.03

41 – 60 0.35 ± 0.04 0.29 ± 0.04 0.35 ± 0.04 0.44 ± 0.04

b. Fraction of visits to disadvantageous arms

30 group 180 group

trial block CORT saline CORT saline

1 – 20 0.51 ± 0.05 0.51 ± 0.05 0.49 ± 0.05 0.40 ± 0.05

21 – 40 0.48 ± 0.06 0.49 ± 0.07 0.49 ± 0.06 0.39 ± 0.05

41 – 60 0.43 ± 0.05 0.44 ± 0.05 0.55 ± 0.04 0.42 ± 0.07

88 TIME-DEPENDENT EFFECTS OF CORTICOSTERONE ON REWARD-BASED DECISION-MAKING IN A RODENT MODEL OF THE IOWA GAMBLING TASK

3.2 Rodent Iowa Gambling Task (rIGT) 3.2.1 Baseline Table 1 shows the performance of the rats in the first three trial blocks before injections were given. In the final block prior to treatment (trial block 41-60) neither for the empty arms nor for the baited arms significant performance differences existed between (intended) CORT or saline groups across timing conditions (see Table 1).

3.2.2 Empty arms Regardless of treatment, all rats visited the empty arms less often across trial blocks; rats in the 180 min groups had higher values than rats in the 30 min groups (Figure 2, panels A and B; three-way repeated measures ANOVA (timing, treatment; repeated: trial block; Greenhouse-Geisser adjusted): trial block: F(1.54, 75.23) = 5.270, p < 0.05; timing: F(1,49) = 4.620, p < 0.05). When the 30 min and 180 min groups were analysed separately with two-way ANOVAs, no differences between CORT and saline treatment appeared in either timing condition; only in both cases a trend in performance improvement across trial blocks was found (30 min group: trial block, F(2,50) = 3.066, p = 0.055; 180 min group (Greenhouse-Geiser adjusted): trial block, F(1.45, 34.73) = 2.819, p = 0.088). Thus, corticosterone had no effect on empty arm choices either injected 30 min or 180 min before testing.

3.2.3 Disadvantageous arm Figure. 3, panel A and B, show that following injections, saline rats and rats treated with CORT 180 min before the task decreased their fraction of visits to the disadvantageous arm, while rats treated with CORT 30 min before the task did not. This was confirmed by a three-way interaction term(three-way repeatedmeasuresANOVA (timing, treatment; repeated: trial block): trial block*treatment*timing: F(2,98) = 4.728, p < 0.05; trial block: F(2,98) = 15.994, p < 0.001). Moreover, this was confirmed when the 30 min and 180 min groups were analysed separatelywith two-way repeated measures ANOVAs (treatment; repeated: trial block): within the 30 min group saline treated rats decreased their fraction of visits to the disadvantageous arm across trial blocks, while the 30 min CORT group did not (30 min group: trial block: F(2,50) = 5.108, p < 0.01; trial block*treatment: F(2,50) = 4.247, p < 0.05; treatment: F(1,25) = 2.955, p = 0.098). Within the 180 min group there was no significant difference between treatments; all rats decreased their fraction of visits to the disadvantageous armacross trial blocks (trial block: F(2,48)=12.143, p < 0.001). We attempted to understand which behavioural mechanism underlies the significant difference between CORT (30 min) and saline treated rats in the baited arms. Therefore, we analysed working memory, consolidation, switches as well as win-stay and lose-shift behaviour in trial block 101-120 for the 30 min group, the only trial block revealing post hoc a significant difference between the two treatment groups. CORT treated rats did not differ from saline treated controls on working memory, i.e. the fraction of choices for the disadvantageous arm in the last 5 trials minus fraction of choices for the disadvantageous arm in the first 5 trials in trial block 101-120 (CORT 0.05 ± 0.11 versus saline -0.01 ± 0.08). Also, CORT treated rats did not differ from saline treated controls on consolidation, here defined as the fraction of choices for the disadvantageous arm in the first 5 trials of trial block 101-120 minus

89 5

Figure 2 Visits to empty arms in the rodent Iowa Gambling Task (rIGT) are not affected by corticosterone. Shown is the performance during the second half of the rIGT, in which rats received CORT (1 mg/kg s.c.) or saline prior to each trial block. (A) Mean (±SEM) fraction of visits to empty arms 30 min after injection. (B) Mean (±SEM) fraction of visits to empty arms 180 min after injection.

Figure 3 Visits to the disadvantageous arm in the rodent Iowa Gambling Task (rIGT) are altered by corticosterone treatment. Shown is the performance during the second half of the rIGT, in which rats received CORT (1 mg/kg s.c.) or saline prior to each trial block. (A) Mean (±SEM) fraction of visits to disadvantageous arm 30 min after injection. (B) Mean (±SEM) fraction of visits to disadvantageous arm 180 min after injection.

90 TIME-DEPENDENT EFFECTS OF CORTICOSTERONE ON REWARD-BASED DECISION-MAKING IN A RODENT MODEL OF THE IOWA GAMBLING TASK fraction of choices for the disadvantageous arm in the last 5 trials of trial block 81- 100 (CORT: 0.06 ± 0.11 versus saline -0.12 ± 0.06). CORT treated rats showed a trend towards more switches in trial block 101-120 than saline treated rats (CORT 11.13 ± 1.095 versus saline 8.25 ± 1.045; Student t-test: t = 1.871, df = 25, p = 0.073). No differences were present between the CORT treated rats and saline treated rats with respect to lose-shift behaviour in trial block 101-120 on the advantageous (CORT 0.74 ± 0.10 versus saline 0.64 ± 0.18) or disadvantageous arm (CORT 0.81 ± 0.06 versus saline 0.71 ± 0.13). However, on the advantageous arm CORT treated rats displayed a trend towards less win-stay behaviour compared to the saline treated controls in trial block 101-120 (CORT 0.49 ± 0.07 versus saline 0.65 ± 0.06; Student t-test: t = -1.795, df = 23, p = 0.086). We could not measure win-stay behaviour reliably in the disadvantageous arm, as the number of encounters with rewards was too low to obtain reliable scores. Thus, corticosterone affected reward-based decision-making when injected 30 min but not when injected 180 min before testing.

3.3 c-Fos immunohistochemistry Figure 4, panel A, depicts the c-Fos analysis of the different brain areas in the rats treated 30 min prior to testing. In two structures significant differences were observed in c-Fos expression following CORT versus saline injections in combination with rIGT exposure (Figure 4, panel B): the lateral orbitofrontal cortex (Student t-test: t = 2.105, df = 25, p < 0.05) and the insular cortex (Student t-test: t = 2.193, df = 25, p < 0.05). In the infralimbic area we observed a trend (Student t-test: t = 1.920, df = 24, p = 0.067). Interestingly, when rats were not exposed to the rIGT, c-Fos expression levels in these and other areas were comparable for CORT treated rats and saline treated rats (Figure 4, panel C). For the c-Fos analysis of the brain areas of the rats treated 180 min prior to testing, we concentrated on those areas that turned out to be sensitive to CORT treatment 30 min before behavioural testing, i.e. a number of subareas in the PFC. Figure 5 shows that no effects were observed between CORT treated and saline treated rats for any of the tested prefrontal areas, including the lateral orbitofrontal cortex, the insular cortex and the infralimbic cortex. Thus, changes in reward-based decision-making when corticosterone was injected 30 min before testing were accompanied by significant changes in the lateral orbitofrontal cortex and insular cortex and a trend in the infralimbic cortex.

91 5

Figure 4 (A) Anatomical localization of brain regions used for analysis of c-Fos expression. Anterior-posterior coordinates relative to Bregma are stated below each coronal section. Abbreviations: mOFC = medial orbital frontal cortex (OFC), vOFC = ventral OFC, lOFC = lateral OFC, Cg1 = cingulate cortex, PrL = prelimbic cortex, IL = infralimbic cortex, INS = insular cortex, DLS = dorsolateral striatum, DMS = dorsomedial striatum, NaC = nucleus accumbens core, NaS = nucleus accumbens shell, BLA = basolateral amygdala, CeN = central nucleus of the amygdala, DG = dentate gyrus and CA1 = CA1 region of the hippocampus. Based on Paxinos and Watson (2005). (B) Mean (±SEM) c-Fos expression levels (# c-Fos positive cells/mm2) after CORT and saline injections and rIGT performance (30 min after injection). *: p < 0.05, t: p < 0.10. (C) Mean (±SEM) c-Fos expression levels (# c-Fos positive cells/mm2) after CORT and saline injections, but without rIGT testing.

92 TIME-DEPENDENT EFFECTS OF CORTICOSTERONE ON REWARD-BASED DECISION-MAKING IN A RODENT MODEL OF THE IOWA GAMBLING TASK

Figure 5 Mean (±SEM) c-Fos expression levels (# c-Fos positive cells/mm2) after CORT and saline injections and rIGT performance (180 min after injection).

4 Discussion

In this study we tested the hypothesis that corticosterone (CORT) via a rapid pathway impairs reward-based decision making, while improving this process when acting via a delayed pathway. The main findings are: (1) in saline treated rats the number of visits to the disadvantageous arm decreased over time, while this improvement in performance was not seen in rats treated with CORT 30 min before testing. This difference in rIGT performance was accompanied by a significant increase in c-Fos expression in the lateral orbitofrontal cortex and insular cortex and a trend for an increase in the infralimbic cortex. (2) When CORTwas administered 180 min before testing, the number of visits to the disadvantageous arm did not differ from that of saline treated rats. This lack of a behavioural effect was accompanied by the absence of an effect on c-Fos expression in the prefrontal and subcortical areas that we analysed. The present findings therefore confirm that rapid corticosteroid actions impair reward-based decision-making, but they do not support that decision-making is improved when animals are tested several hours after the peak of corticosterone.

4.1 Corticosterone: dose and timing The dose of CORT that we used, i.e. 1 mg/kg s.c., was previously shown to induce plasma levels resembling those seen after stress (de Quervain et al. 1998; Shafiei et al. 2012). The high CORT levels were confirmed here in a separate group of rats, which, like rats used for behavioural testing, were exposed to the rIGT during days 1-5 but from which blood was sampled via the tail vein on day 6 (baseline) and 30 min after receiving CORT on day 7, 8 and 9. The plasma CORT levels that we observed 30 min after injection of 1 mg/kg CORT were comparable to those published by others (Shafiei et al. 2012). Our baseline levels were already quite high, most likely due to the fact that rats were tested during the active phase of the circadian cycle and were mildly food-deprived (Dallman et al. 1999; Djordjević et al. 2008; Mantella et al. 2005; Sarabdjitsingh et al. 2012). The response was still high but slightly attenuated after the third injection. This may indicate that metabolic conversion and/or clearance alters over the course of days (Sarabdjitsingh et al. 2012). The 30 min delay after peripheral CORT injection is probably too short to induce genomic actions (de Kloet et al. 2008; Joëls et al. 2012). This contrasts with the 180 min interval, which gives ample time to allow gene-mediated actions to develop (e.g. Joëls et al. 2012). We cannot exclude that corticosterone levels in this group of rats were

93 still elevated at the time of behavioural testing, so that non-genomic actions may have existed in addition to the delayed, presumably genomic, effects. However, it has been shown that plasma CORT levels return to baseline after 90 min following injections of 1 mg/kg CORT (Shafiei et al. 2012). Furthermore, indirect evidence from our experiments also argues against this: rats treated with CORT 30 min before rIGT testing did not show elevated plasma corticosterone levels compared to controls when examined 120 min later, i.e. in total 150 min after injection; rather their values appeared to be lower.

4.2 Behavioural effects of corticosterone Thus far, experiments reporting on this rIGT have shown that male rats start visiting the long-term advantageous arm more often from trial 61 onwards (de Visser et al. 2011a, 5 2011b; van den Bos et al. 2006a, 2012b; van Hasselt et al. 2012). Here, we observed that from trial 61 onwards saline treated rats decreased their number of choices for the empty arms and increased their choices for the long-term advantageous arm. This suggests that the injection procedure per se had no effect on rIGT performance. The shift from exploration to the expression that task contingencies have been learned may be associated with a shift from predominant involvement of the limbic (affective/ motivational) loop, encompassing the orbitofrontal cortex, the amygdala and nucleus accumbens/ventral striatum, to a more dominant activity of a cognitive control loop, encompassing the dorsolateral prefrontal cortex, anterior cingulate cortex (medial prefrontal cortex in rats; see below) and dorsal striatum (Bechara, 2005; de Visser et al. 2011a, 2011b, 2011c; Doya, 2008; McClure et al. 2004; Rivalan et al. 2011; Tanaka et al. 2004, 2007; Yin and Knowlton, 2006; Yin et al. 2008; Zeeb and Winstanley, 2011). Previously we have shown that successful task progression in male rats requires activation of the medial prefrontal cortex (mPFC; prelimbic and infralimbic region; de Visser et al. 2011a, 2011b). The mPFC in rats has been suggested to share an anatomical and functional homology to the anterior cingulate cortex and dorso-lateral prefrontal cortex in humans (Brown and Bowman, 2002; Uylings and van Eden, 1990; Uylings et al. 2003). Importantly, the day of the first CORT injection was chosen to target the moment that animals may start to switch from the limbic to the cognitive loop, with an involvement of prefrontal areas that are expected to be affected by corticosterone (reviewed in Joëls et al. 2012). Administering CORT in the second phase of the task appeared to be sufficient to demonstrate the rapid effects of corticosterone. The observation that task progression was hampered by CORT administration 30 min prior to rIGT testing suggests an effect on the switch between the limbic and cognitive loop or the activity of the latter. The fact that we observed corticosterone related changes dependent on rIGT exposure in the lateral orbitofrontal cortex (lOFC) and a trend in the infralimbic (IL) cortex supports this notion (see Section 4.4). Following a strong interaction term only on the final trial block (101-120) a significant difference was revealed between CORT and saline treated rats. A lack of significant effects on trial block 61-80 and 81-100 can be explained by the fact that saline treated rats were still learning to avoid the disadvantageous arm. This would be in line with earlier findings in (control) male rats (de Visser et al. 2011a, 2011b; van den Bos et al. 2006a, 2012b). Thus, CORT injections hampered rats to improve their performance, such that the difference between CORT and saline treated rats became only apparent once saline treated rats reached a sufficiently low fraction of visits to the disadvantageous arm (trial block 101-120).

94 TIME-DEPENDENT EFFECTS OF CORTICOSTERONE ON REWARD-BASED DECISION-MAKING IN A RODENT MODEL OF THE IOWA GAMBLING TASK

Given that saline treated rats (180 min before rIGT testing) showed a steady improvement in the second phase of the task, the administration schedule was perhaps less optimal to demonstrate a further improvement in rIGT performance as might be expected for the slow effects of corticosterone (Joëls et al. 2012). The effect of corticosterone was selective for the baited arms, as no effects were observed for the empty arms. This resembles earlier reported effects in a study (de Visser et al. 2011b) where the mPFC was inactivated in the second half of the task (see further below). We only observed that rats in the 180 min groups had overall a slightly higher preference for the empty arms than rats in the 30 min groups. However, in both timing conditions corticosterone had no significant effect on choosing emptyarms compared to saline. It should be noted that two studies thus far have failed to find any behavioural effects of CORT injections (1 and 3 mg/kg) on reward-based decision-making tasks in rats (Graham et al. 2010; Shafiei et al. 2012). Several factors may account for the different outcome. First, the design in these studies was not optimal for detecting rapid corticosteroid actions, the only condition where we observed effects of CORT. Thus, while we tested our animals 30 min after injections (peak CORT level), Shafiei et al. (2012) did so after 90 min (when CORT levels had already returned to baseline). Graham et al. (2010) suspended CORT in sesame oil and injected this 30 min s.c. before testing, while we used a water soluble form of CORT. A 30 min interval may have been too short to observe strong behavioural effects in their experiment, due to slow diffusion of the compound. Unfortunately, they did not determine plasma CORT levels after 30 min. Second, differences in the kind of decision-making tasks may play a role. Thus, the data of for instance Shafiei et al. (2012) suggest that delay-discounting and effort-based decision-making tasks are differently sensitive to stress. Whatever the reasons for the differences between the present study and these two studies (Graham et al. 2010; Shafiei et al. 2012) both our behavioural and neural data (discussed below) indicate a highly specific effect of CORT administered 30 min before rIGT testing.

4.3 c-Fos analysis The brain areas included in our analysis of c-Fos labelling, an often-used global assay for neural activity in the brain, were based on our earlier studies on rIGT performance (de Visser et al. 2011a; van Hasselt et al. 2012). Correlational analysis of rIGT performance in the last session with c-Fos activity in saline treated rats (30 min prior to testing; data not shown) supported our earlier observations on the involvement of e.g. the mPFC and striatal areas in rIGT performance. Administration of CORT at 30 min, but not 180 min, before rIGT testing showed a significantly enhanced c-Fos expression in the lOFC and insular cortex and a trend in the infralimbic part of the mPFC compared to the saline treatment. Importantly, CORT treatment without rIGT exposure did not lead to any significant differences. The observed effects were thus specific for the injection of CORT combined with exposure to the rIGT, i.e. corticosterone is affecting structures which are active during task performance of the rats. Although the observed increase in c-Fos activity does not specify whether the treatment caused an increase in inhibition or excitation, an inhibition is most likely, as inactivation of the mPFC with GABA-receptor agonists resulted in a similar behavioural profile in the rIGT (de Visser et al. 2011b).

95 4.4 Underlying mechanism Administering CORT on 3 consecutive days 30 min before behavioural testing could have affected working memory (within sessions) and/or consolidation of information (Cazakoff et al. 2010; Roozendaal and McGaugh, 2011). However, we did not find any evidence for this. First, we did not observe a difference between CORT treated rats and saline treated rats when we compared the first and last 5 trials of the last trial block. This suggests no effect on working memory. Second, corticosterone effects were specific for the baited arms (disadvantageous and advantageous arm); no effects were observed for learning to differentiate the baited from the non-baited (empty) arms. This also suggests no effect on working memory or consolidation. Third, CORT compared to saline treated rats did not differ in their performance after the first injection (trials 5 61-65 compared to trials 56-60; data not shown) or when performance in the first 5 trials on the final testing day was compared with performance in the final 5 trials on the preceding day. This suggests no effect on consolidation. Fourth, no effect was observed of CORT injections on c-Fos expression in two hippocampal areas (CA1, DG) compared to saline injections; areas involved in consolidation (Frankland and Bontempi, 2005; Kelly et al. 2003). The observation that only within the baited arms an effect of CORT (injected 30 min prior to rIGT testing) occurred suggests that this could be related to effects on e.g. reward sensitivity, punishment sensitivity, the integration of trial-by-trial information into a long-term advantageous versus long-term disadvantageous option, in an inability to suppress responding to punishments in the context of series of wins in the advantageous arm, or – as mentioned above – in the switch between the limbic and cognitive loop. Indeed, within the limbic (subcortical) system, high levels of cortisol are thought to shift the balance from the activity of the ventral striatum (reward-related behaviour) and amygdala (punishment-related behaviour) towards the ventral striatum (Dellu et al. 1996; Mather et al. 2010; Piazza et al. 1993; Piazza and Le Moal, 1997; Wager et al. 2008). Putman et al. (2010) have shown that this may lead to an increase in reward-related and decrease in punishment-related behaviour in tasks with a large contrast between rewards and punishments. The IGT might be such a task. Furthermore, it has been shown that cortisol-induced increase in risk-taking behaviour in men may be associated with a loss of top-down control of prefrontal over subcortical areas, such as mediated by the lOFC and dorsolateral prefrontal cortex (Arnsten, 2009; Dias-Ferreira et al. 2009; Erickson et al. 2003; Goldstein et al. 2010; Kern et al. 2008; Piazza and Le Moal, 1997; Stark et al. 2006;Wang et al. 2007). While we did find evidence for changes in prefrontal activity we did not find evidence for changes in limbic, subcortical, activity. CORT administered 30 min before rIGT testing did not lead to more risk-taking behaviour as may have been expected when it would have increased reward-sensitivity and decreased punishment sensitivity (Mather and Lighthall, 2012; Putman et al. 2010). Thus, we did not observe that CORT treated rats increased their choices for the disadvantageous option with an occasional high reward (three sugar pellets) and high frequency of punishments (9/10). Furthermore, corticosterone did not seem to affect punishment sensitivity per se as measured by a lack of effect on lose-shift behaviour and as indicated by the lack of effect on amygdala c-Fos activity. Corticosterone tended to decrease win-stay behavior in the advantageous arm. We could not measure win- stay behavior reliably in the disadvantageous arm as the number of encounters with rewards was too low to obtain reliable scores. Still, changes in reward-related behaviour

96 TIME-DEPENDENT EFFECTS OF CORTICOSTERONE ON REWARD-BASED DECISION-MAKING IN A RODENT MODEL OF THE IOWA GAMBLING TASK were not accompanied by changes in accumbens or ventral striatum c-Fos expression. Collectively, these findings suggest that sensitivity to rewards or punishments per se may not have been changed by CORT. The differences in the profile of task progression after saline and CORT administered 30 min before rIGT testing suggest that CORT treated rats did not improve their (existing) performance within the baited arms while saline rats did. This is in line with the differences in c-Fos expression in the lOFC and IL region of the mPFC. The lOFC has been implicated in emotional control, i.e. adjusting behaviour as contingencies change, with a focus on distant or longterm rewards (Elliott et al. 2000b; Frank and Claus, 2006; Kable and Glimcher, 2009; Kringelbach, 2005; Mar et al. 2011; McClure et al. 2004; Noonan et al. 2010; O’Doherty et al. 2001; Rushworth et al. 2011; Sescousse et al. 2010; Sul et al. 2010; Windmann et al. 2006). Also the mPFC, including the IL, is involved in the maintenance of a long-term perspective by withholding to respond to immediate rewards or losses (McClure et al. 2004; Sierra-Mercado et al. 2011; Tanaka et al. 2004), and through its role in strategy shifting, behavioural flexibility and goal- directed learning behavior (Balleine and O’Doherty, 2010; Birrell and Brown, 2000; Dias et al. 1996; Floresco et al. 2008; Ragozzino et al. 1999; Sul et al. 2010; Tran-Tu-Yen et al. 2009; Young and Shapiro, 2009). Furthermore, the insular cortex has an important role in this process, as it signals incentive or reward value (Cardinal et al. 2003; Kesner and Gilbert, 2007; Weller et al. 2009), risky aversive stimuli (Clark et al. 2008; Preuschoff et al. 2008), and integrates representations of the bodily state (Contreras et al. 2007; Naqvi and Bechara, 2009; Xue et al. 2010). Thus, corticosterone induced changes in lOFC and insular cortex c-Fos activity and trend towards change in the mPFC may lead to an inability to integrate new information into a long-term perspective, i.e. labelling one option as more profitable in the long run than another. If so, this would be in line with the hypothesis that shortly after stress corticosterone promotes a shift to more habitual rather than goal-directed behaviour (Schwabe and Wolf, 2011). This may lead to an inability to integrate punishments and rewards (Ossewaarde et al. 2011). If rats fail to distinguish the two baited arms from each other, they are more likely to keep switching between these arms, for which we obtained suggestive evidence.

4.5 Concluding remarks In humans, stress increases risk-taking in the IGT and this depends on cortisol: subjects responding with high cortisol levels to stressful conditions perform poorly, while low- cortisol responders do not (van den Bos et al. 2009). Our study in a rodent IGT model (rIGT) indicates that stress-induced interference with adequate decision-making may depend on rapid actions of corticosteroid hormones, targeting the lateral OFC as well as the insular cortex and possibly the infralimbic cortex. Interestingly, both the lateral OFC and insular cortex were also implicated in the effect of maternal care on rIGT performance later in life (van Hasselt et al. 2012).

Acknowledgements

We thank Dr. Louk Vanderschuren for fruitful discussions, and Marla Lavrijsen as well as Susanne Kirchhoff for technical assistance.

97 6

98 CHAPTER 6

CORTICOSTERONE AND DECISION-MAKING IN MALE WISTAR RATS: THE EFFECT OF CORTICOSTERONE APPLICATION IN THE INFRALIMBIC AND ORBITOFRONTAL CORTEX

S. Koot a,b a Rudolf Magnus Institute of Neuroscience, M. Koukou a,b Department of Neuroscience & A. Baars b Pharmacology, University Medical Centre P. Hesseling b Utrecht, the Netherlands J. van ’t Klooster b b Division Behavioural Neuroscience, R. van den Bos a,c* Department of Animals in Science and M. Joëls a* Society, Faculty of Veterinary Medicine, Utrecht University, the Netherlands c Department of Organismal Animal Physiology, Radboud University Nijmegen, the Netherlands

* Both authors contributed equally

Submitted

99 Abstract

Corticosteroid hormones, released after stress, are known to influence neuronal activity and produce a wide range of effects upon the brain, like affecting cognitive tasks as decision- making. Recently it was shown that systemic injections of corticosterone disrupt reward-based decision-making in rats when tested in a rat model of the Iowa Gambling Task (rIGT), i.e. rats do not learn over time to avoid the long-term disadvantageous option. This effect was associated with a change in neuronal activity in prefrontal brain areas, i.e. the infralimbic (IL), lateral orbitofrontal (lOFC) and insular cortex, as assessed by changes in c-Fos expression. Here, we 6 studied whether injections of corticosterone in the IL and lOFC lead to similar changes in decision-making. As in our earlier study corticosterone was injected during the final 3 days of the behavioural paradigm prior to behavioural testing (25 min). Infusions of vehicle into the IL led to a decreased number of visits to the disadvantageous arm over trial blocks, while infusion with corticosterone did not. Infusions into the lOFC did not lead to differences in the fraction of visits to the disadvantageous arm between vehicle treated and corticosterone treated rats. However, compared to vehicle treated rats of the IL group, performance of vehicle treated rats of the lOFC group was impaired, suggesting cannulation/ infusion-related damage to the lOFC affecting decision- making. These results show that infusions with corticosterone into the IL result in disrupted decision-making performance, but they do not directly support that the same holds true for infusions into the lOFC. These findings, together with the earlier findings of systemic corticosterone administration, provide direct evidence that corticosterone disrupts decision- making, at least through changing IL activity.

100 Corticosterone and decision-making in male Wistar rats: the effect of corticosterone application in the infralimbic and orbitofrontal cortex

1 Introduction

Stress is the subjective experience of actual or potential, physical or psychological threats. Upon stress, a physiological response is triggered which prepares the organism to deal with the changing environment (Joëls and Baram, 2009; Joëls et al. 2011). As part of the stress response, the adrenal cortex secretes glucocorticosteroid hormones in response to ACTH from the pituitary gland. Neurons receive high levels of corticosteroids (cortisol in humans, corticosterone in most rodents), which influence neuronal activity and produce a wide range of effects upon the brain (McEwen, 2006; Groeneweg et al. 2011). Altered gene transcription mediates many of their actions, but a range of effects on behaviour and endocrine output occurs within minutes (Groeneweg et al. 2011). Ultimately, glucocorticosteroids restore homeostasis in the aftermath of stress by diverting energy supply to challenged tissues and control the excitability of neuronal networks (de Kloet et al. 1999). Cognitive processes, like memory (Joëls et al. 2006; Schwabe et al. 2012a) and decision-making (Starcke and Brand, 2012), have been shown to be affected by stress. Effective decision-making requires the cooperation of several cognitive skills for planning, initiating and monitoring goal-directed behaviour (Rivalan et al. 2011). One way to study decision-making performance in humans is through the Iowa Gambling Task (IGT; Bechara et al. 1994). Due to combining the uncertainty of outcome with a conflict between short-term and long-term benefit, this task is widely used as a simulation of real-life decision-making (Brand et al. 2007). Stress has been repeatedly linked to suboptimal decision-making. Men became more risk- taking under acutely stressful conditions in both the IGT (Preston et al. 2007; Van den Bos et al. 2009) and other decision-making tasks (Balloon Analogue Risk Task, Lighthall et al. 2009; Game of Dice Task, Starcke et al. 2008; Starcke and Brand, 2012). Specifically, cortisol reactivity was associated with decision-making performance, i.e. the higher the salivary cortisol levels, the poorer the performance in the IGT (Van den Bos et al. 2009). In order to delineate the underlying mechanism of the effect of acute stress on decision-making we tested the effect of corticosterone treatment in a rodent version of the IGT (rIGT; Van den Bos et al. 2006a) in a previous study (Koot et al. 2013 (chapter 5)). In this study we found impaired decision-making of male rats tested 30 minutes after systemic injections of corticosterone (CORT; 1 mg/kg subcutaneously): unlike control rats, CORT treated rats did not decrease their visits to disadvantageous arms over trial blocks. CORT administration did not affect choices for empty arms, which indicates that only reward-related decision-making was affected by CORT. No effects were found when CORT was administered 180 minutes prior to testing, so only rapid corticosteroid actions impaired decision-making. The effect of CORT injections was accompanied by a significant increase inc-Fos expression in the lateral orbitofrontal cortex (lOFC) and insular cortex and a trend for an increase in the infralimbic cortex (IL); CORT treatment in itself did not change neuronal activity. These areas have previously been linked to decision-making performance (Pais-Vieira et al. 2007; Rivalan et al. 2011; Zeeb and Winstanley, 2011; de Visser et al. 2011b). Furthermore, the medial prefrontal cortex (mPFC; prelimbic and infralimbic region) was shown to be involved in glucocorticoid-mediated memory consolidation and working memory processes (Roozendaal et al. 2004, 2009; Barsegyan et al. 2010).

101 Although the IL, lOFC and insular cortex seem to be involved in the effect of systemic CORT administration on decision-making (Koot et al. 2013 (chapter 5)), it is not clear whether the altered activity in these areas was due to direct or indirect effects of CORT. Therefore, here we applied CORT locally in the IL and lOFC. We focused on these two areas, as pilot experiments showed that it was currently not possible for us to target the insular cortex in the region which was found to be associated with disrupted decision-making after CORT treatment (Koot et al. 2013 (chapter 5)). Similar to the design of our previous study (Koot et al. 2013 (chapter 5)), we infused CORT in the second half of the rIGT (cf. de Visser et al. 2011b) bilaterally in either the IL or the lOFC. When the effects of systemic CORT injections on decision-making would result from direct effects of CORT in these areas, CORT infusions would result in disrupted reward-related decision-making performance, i.e. a failure to decrease of choices for the disadvantageous arm and no effects on empty arm choices. 6 2 Materials and methods

2.1 Animals Male Wistar WU rats (n = 96; Charles River, Sulzfeld, Germany; 9 weeks of age) were housed individually in Macrolon type IV cages with sawdust bedding under a reversed 12h light/dark cycle (lights off at 07:00h) under controlled conditions (temperature 21±2°C, relative humidity 60±15%). Paper tissues and a shelter were provided as cage enrichment. Food and water were available ad libitum, except during testing (see below). A radio provided background noise. Behavioural testing started at least 1 hour after dark onset. All experiments were approved by the Animal Ethics Committee of Utrecht University and were conducted in agreement with Dutch laws (Wet op de Dierproeven, 1996) and European regulations (Guideline 86/609/EEC).

2.2 Surgery After 2.5 weeks of habituation to the facility and human handling, surgeries took place. Rats were anaesthetised under red-light conditions using a mixture of ketamine (60 mg/kg, i.p., Narketan, Vétoquinol S.A., France, 100 mg/ml ketamine) and dexmedetomidine (0.15 mg/kg, i.p., Dexdomitor, Pfizer Animal Health BV, Capelle a/d IJssel, The Netherlands, 0.5 mg/ml dexmedetomidine hydrochloride), including buprenorphine (0.05 mg/kg, i.p., Temgesic, RB Pharmaceuticals Limited, 0.3 mg/ ml buprenorphine) as additional analgesia. As soon as the pedal reflex was absent, the rat was transported to the surgery room and, after intubation, anaesthesia was maintained with isoflurane in 100% O2. The animals received 8 ml of saline (s.c.) to support normal fluid balance, and eye ointment (Ophtosan Oogzalf, Produlab Pharma Raamsdonkveer, ASTfarma BV, Oudewater, The Netherlands; 10,000 IE of vitamin A palmitate per gram). For the surgery, the animal was positioned in the stereotactic apparatus (model 963, Ultra Precise Small Animal Stereotaxic, David Kopf Instruments, Tujunga, CA, USA). Body temperature was monitored using a rectal probe thermometer and maintained at 37–38°C with an adjustable electrically heated mattress. Respiratory rate, and inspired and expired CO2 were monitored continuously and anaesthetic administration was adjusted appropriately. Rats were implanted bilaterally with stainless steel guide cannulas (length: 5 mm; 22 ga; Plastics

102 Corticosterone and decision-making in male Wistar rats: the effect of corticosterone application in the infralimbic and orbitofrontal cortex

One type C313GRL, Plastics One Inc., Roanoke, VA, USA) aimed at the IL or lOFC. The IL was targeted under a lateral angle of 30° using the following coordinates: anteroposterior (AP): +3.2 mm relative to bregma; mediolateral (ML): ±2.8 mm; dorsoventral (DV): 4.4 mm below skull surface. The coordinates used for the lOFC were: AP: +4.3 mm; ML ±2.7 mm; DV: -3.6 mm; no angle. In both cases, the nose was set at -3.3 mm. All coordinates were adapted from the atlas of Paxinos and Watson (2005) to our rats. After surgery, anaesthesia was antagonized with atipamezole (0.6 mg/kg, i.p., Antisedan, Pfizer Animal Health BV, Capelle a/d IJssel, The Netherlands). Rats were returned to their home cages once they regained locomotion. Postoperative analgesia consisted of meloxicam (0.2 mg/kg, s.c., Metacam, Boehringer Ingelheim, Alkmaar, The Netherlands, 5 mg/mL) at 24h intervals for two days after surgery, plus buprenorphine (0.05 mg/kg, s.c.) at 12h intervals for three days after surgery. All rats were allowed to recover for two weeks before behavioural testing started.

2.3 Behavioural procedures Experiments were run as described previously (De Visser et al. 2011a; 2011b; Koot et al. 2013 (chapter 5)) in order to keep the procedure similar to earlier used protocols. Therefore, after a habituation period of 2.5 weeks, all rats were first tested on the elevated plus maze (EPM) and subsequently in the rodent version of the Iowa Gambling Task (rIGT). Because anxiety, as measured by EPM performance, was earlier found to correlate with rIGT behaviour (Rivalan et al. 2009; De Visser et al. 2011a), we compared anxiety levels between batches of rats to promote that they were comparable prior to saline or corticosteroid treatment. We refer to De Visser et al. (2011a) for detailed descriptions of the EPM protocol and analyses. We analysed per cent time spent on the open arm as a measure of anxiety (De Visser et al. 2011a). No significant differences were found between batches (data not shown), thus all batches were included for further analysis. Rats were tested in the rIGT 1.5 week after EPM exposure (starting on Mondays; see De Visser et al. 2011a). The rIGT apparatus, made of grey PVC, consisted of a start box, choice area and four arms. Before testing, rats were habituated to the apparatus in a 10 min free exploration trial. Two days later they were mildly food deprived (90- 95% of free feeding body weight) and tested for 9 days. Testing took place between 9:00-15:00h (lights off at 7:00h). Testing did not occur during weekend days. Food was freely available on these days, but rats were returned to their restricted diet the day before testing continued. A trial started by lifting the slide door of the start box. Rats could freely enter the choice area of the apparatus and choose one of the four arms. To help rats differentiating arms, visual cues (10x10cm; cross or circle in black or white) were placed to the side of the wall at the entrance of arms. When rats had entered a choice arm with their full body, including their tail, the arm was closed. At the end of the arm, rats could obtain pellets (baited arms) or nothing at all (empty arms). Each trial lasted max 6 min (inter-trial interval: 30 sec). Rats received a total of 120 trials: 10 trials on days 1-6 and 20 trials on days 7-9. By the time rats reached the second half of the task, i.e. days 7-9, a session lasted 12 min at most. Rewards were 45mg sugar pellets (F0042, Bio-serv Inc, Frenchtown, NJ, USA); punishments were quinine-treated sugar pellets that were unpalatable but not

103 uneatable. Rats were habituated to the sugar pellets in the week prior to the first rIGT session in their home cage daily, followed by a single session of providing two sugar pellets in a novel empty Macrolon type-III cage, which all rats did eat. During rIGT testing most rats consumed the quinine-treated sugar pellets once, butleft them uneaten after tasting them briefly. Rats consistently eating quinine-treated sugar pellets were excluded from analysis. No animals were excluded because of this in the present experiment. Of the four arms in the maze, two arms were baited and two arms were empty. The two empty arms were included to measure non-reward related exploration (De Visser et al. 2011a, 2011b, 2011c). The two baited arms consisted of a long-term disadvantageous arm and a long-term advantageous arm. In the disadvantageous arm, rats received occasional big rewards (three sugar pellets in 1 out of 10 trials) among frequent punishments (three quinine-treated sugar pellets in 9 out of 10 6 trials). In the advantageous arm, rats received frequent small rewards (one sugar pellet in 8 out of 10 trials) and infrequent punishments (one quinine-treated sugar pellet in 2 out of 10 trials). The positions of the baited and empty arms, as well as the advantageous and disadvantageous arm were counterbalanced across rats. Performance parameters were: (1) the number of visits to the empty arms as a fraction of the total number of trials per block of 20 trials and (2) the number of visits to the disadvantageous arm as a fraction of the total visits to the baited arms per block of 20 trials. To unravel the potential mechanisms underlying behavioural effects of CORT injections, we measured some additional parameters. First, the total number of switches between different arms per block of 20 trials was calculated as measure of exploratory behaviour (De Visser et al. 2011a). Second, responses to encounters with sugar pellets or quinine-treated sugar pellets in the advantageous and disadvantageous arm were measured as win-stay/lose-shift behaviour per block of 20 trials (De Visser et al. 2011a). Blocks of 20 trials were chosen to obtain a sufficient number of encounters with sugar and quinine-treated sugar pellets. Thus, when rats encountered a sugar reward, their subsequent choice was scored as a win- stay when they revisited the advantageous arm. When rats encountered a quinine punishment, their subsequent choice was scored as a lose-shift when they switched to another arm. Win-stay and lose-shift behaviour was calculated as a fraction of the number of encounters with either sugar (win) or quinine (loss). Thus, values of 1 indicate win-stay and lose-shift tendencies only, while values of 0 indicate win-shift and lose-stay tendencies only.

2.4 Intracerebral infusions On the last three days of the rIGT, i.e. the second half of the rIGT, rats were infused bilaterally with a solution of CORT (5 ng/μL; Sigma-Aldrich) or vehicle solution in a volume of 0.5 μL per hemisphere, approximately 25 min before testing. The dose of 5 ng/μL CORT was chosen as in previous infusion studies this dose resulted in significant effects compared to other doses, i.e. 5ng/μL as opposed to 2 or 10ng/μL resulted in enhanced memory when infused in the insular cortex, basolateral amygdala or dorsal striatum (Miranda et al. 2008; Quirarte et al. 2009). CORT was first dissolved in ethanol and then diluted in saline to reach the intended concentration. The final concentration of ethanol was 2%. The vehicle solution consisted of 2% ethanol in saline. The infusions targeted either the IL or the lOFC. Hamilton 10-μL syringes were

104 Corticosterone and decision-making in male Wistar rats: the effect of corticosterone application in the infralimbic and orbitofrontal cortex used, attached to tubing and an injection needle of 6mm long, thus protruding 1mm beyond the tip of the guide cannulas. Before the infusions, longer injection needles (7mm) were inserted to the cannulas to prevent blockage. Then, a mechanical pump (Harvard Apparatus, serial number: B-17020) was used to perform the infusions. A volume of 0.5 μL per hemisphere was infused at a rate of 0.4 μL/min. The needles were left in place for 60 seconds in addition to allow for diffusion. During the whole procedure, the rats were held by the experimenter and slightly restrained to prevent abrupt movements. After infusion rats were returned to their home cage, before they were tested in the rIGT.

2.5 Histology After completion of the experimental protocol, rats were decapitated and brains were quickly frozen in liquid (-80˚C) 2-methylbutane and stored at -80˚C. Coronal sections (20 μm) were cut on a cryostat (Leica CM 3050S), mounted on Menzel SuperFrost Plus slides (MenzelGmbH&Co, Braunschweig, Germany) and stained with cresyl violet. Injection sites were verified with reference to the neuro-anatomical atlas of Paxinos and Watson (2005).

2.6 Statistical analyses IBM SPSS Statistics 20 for Windows was used for the data analysis. Specific tests are indicated in the Results section. Whenever sphericity was violated, degrees of freedom were corrected. Significance was set at p≤0.05 (two-tailed). Results are expressed as mean ± SEM.

3 Results

3.1 General Sixty rats were cannulated targeting the IL and thirty-six targeting the lOFC. Seven rats died during surgery or recovery (IL n=3, lOFC n=4). In the IL-infused group, twelve additional rats were excluded; five due to incorrect placement of the cannulas, one due to loss of one guide cannula, one because the target region of the cannulas was impossible to verify, two due to large infections in the brain, two due to not completing the rIGT and one due to health problems affecting the performance. This resulted in n=34 rats used for the analysis (CORT=21, vehicle=13). In the lOFC- infused group, nine additional rats were excluded; five due to incorrect placement of the cannulas, one due to difficulties verifying the location of the cannulas and three due to not completing the rIGT. This resulted in n=23 rats used for the analysis (CORT=11, vehicle=12). Figure 1 shows the location of injection needle tips in the IL and lOFC of rats tested in the rIGT and included in the analyses.

105 6 Figure 1 Representation of the location of the infusions into the (A) infralimbic cortex (CORT: black circles, n=21; vehicle: open squares, n=13), and (B) lateral orbitofrontal cortex groups (CORT: black circles, n=11; vehicle: open squares, n=12). Sections correspond to the atlas of Paxinos and Watson (2005).

Figure 2 Infralimbic cortex (IL) cannulated group. (A) Mean (±SEM) fraction of visits to empty arms. (B) Mean (±SEM) fraction of visits to disadvantageous arm. ** p=0.001.

Figure 3 Lateral orbitofrontal cortex (lOFC) cannulated group. (A) Mean (±SEM) fraction of visits to empty arms. (B) Mean (±SEM) fraction of visits to disadvantageous arm.

106 Corticosterone and decision-making in male Wistar rats: the effect of corticosterone application in the infralimbic and orbitofrontal cortex

3.2 Rat Iowa Gambling Task (rIGT) 3.2.1 Baseline (trials 1-60) Figure 2 and 3 show the performance of rats infused in the IL and lOFC, in the first three trial blocks (t1-60) and in the last three trial blocks (t61-120) when injections were given, both for visits to the empty and disadvantageous arms. To check whether any baseline differences between treatment groups prior to the infusions existed, a three-way repeated measures ANOVA was run for the fractions of visits to empty and disadvantageous arm for all animals on the first three trial blocks. Trial block was set as within-subjects factor, brain region (IL, lOFC) and treatment (CORT, vehicle) as between-subjects factors. No differences in performance existed between the IL and lOFC-cannulated groups or between the intended treatment groups before treatment started, and all rats visited the empty arms less often across trial blocks (empty arms: trial block p=0.001, all other p≥0.198; disadvantageous arm all p≥0.267). A two- way ANOVA on the last trial block before treatment (t41-60) with brain region and treatment as between-subjects factors confirmed this, as no significant differences between the above-mentioned groups were found (all main effects and interactions: p≥0.136).

3.2.2 Treatment (trials 61-120) In the second half of the task, regardless of treatment, all rats decreased their fraction of visits to the empty arms, with IL-cannulated rats tending to visit the empty arms less often than lOFC-cannulated rats (Figure 2A and 3A; three-way repeated measures ANOVA (brain region, treatment; repeated: trial block; Huynh-Feldt adjusted): trial block F(1.89, 100.18)=3.407, p=0.040; brain region F(1,53)=3.198, p=0.079; all other interactions and main effects: p≥0.509). When the IL and lOFC infused groups were analysed separately with two-way repeated measures ANOVAs, no differences between CORT and vehicle treatment appeared either (IL: all interactions and main effects p≥0.164; OFC: all interactions and main effects p≥0.189). Thus, CORT had no effect on empty arm choices either infused in the IL or lOFC. In contrast, figure 2B and 3B show that rats infused with vehicle in the IL visited the disadvantageous arm less often over trial blocks in contrast to the rats in the other groups. This was confirmed by a three-way interaction term (three-way repeated measures ANOVA (brain region, treatment; repeated: trial block): trial block * treatment * brain region: F(2,106)=5.836, p=0.004; trial block: F(2,106)=7.567, p=0.001; all other interactions and main effects p≥0.181). Moreover, this was confirmed when IL and lOFC infused groups were analysed separately withtwo- way ANOVAs (IL infused group: trial block: F(2,64)=4.461, p=0.015; trial block * treatment: F(2,64)=4.107, p=0.021; treatment: F(1,32)=2.930, p=0.097; lOFC infused group: trial block: F(2,42)=3.590, p=0.036; trial block * treatment: F(2,42)=2.198, p=0.124; treatment: F(1,21)=0.000, p=0.999). In an attempt to delineate the mechanisms underlying the significant difference between CORT and vehicle infused IL rats, we analysed switches and win-stay and lose-shift behaviour in trial block 101-120 in the IL infused group. No differences were present between CORT infused rats and vehicle infused rats with respect to switches (CORT 9.10±0.94 versus vehicle 8.77±1.19; one-way ANOVA, treatment F(1,33)=0.046, p=0.831). Similarly, no differences were found between CORT infused

107 Table 1 rIGT performance under treatment. Shown are means (± SEM) of fractions of visits to empty and disadvantageous arm of vehicle treated rats of the infralimbic cortex (IL) and lateral orbitofrontal cortex (lOFC) cannulated group after infusions, i.e. trial blocks 61-120.

rIGT performance (fraction of visits) empty arms disadvantageous arms trial block IL vehicle lOFC vehicle IL vehicle lOFC vehicle 61 – 80 0.33 ± 0.05 0.39 ± 0.05 0.40 ± 0.07 0.45 ± 0.07 81 – 100 0.29 ± 0.06 0.34 ± 0.06 0.29 ± 0.07 0.40 ± 0.07 101 – 120 0.29 ± 0.07 0.33 ± 0.07 0.14 ± 0.04 0.42 ± 0.04

6 rats and vehicle infused rats with respect to lose-shift (CORT 0.65±0.07 versus vehicle 0.81±0.12; one-way ANOVA, treatment F(1,25)=1.176, p=0.289) or win-stay behaviour (CORT 0.50±0.06 versus vehicle 0.65±0.07; one-way ANOVA, treatment F(1,26)=2.303, p=0.142). Visual inspection of the dataset suggested that vehicle treated rats within the lOFC infused group did not decrease their visits to the disadvantageous arm as could be expected based on data of control rats in the current (vehicle infused IL rats) and past experiments (De Visser et al. 2011a; Koot et al. 2013 (chapter 5)). Therefore, we compared the data of the vehicle infused IL rats and vehicle infused lOFC rats (Table 1). Vehicle treated rats of the IL infused group decreased their fraction of visits to the disadvantageous arm, while vehicle treated rats of the lOFC infused group did not, which was confirmed by a repeated measures ANOVA (trial block: F(2,46)=4.389, p=0.018; trial block * brain region: F(2,46)=3.280, p=0.047; brain region: F(1,23)=4.916, p=0.037). Remarkably, no differences were found for the empty arms (all p≥0.401).

4 Discussion

Using a rodent version of the IGT to assess decision-making, this study resulted in two main findings. First, targeting the infralimbic cortex (IL) vehicle infused rats decreased the number of visits to the disadvantageous arm across trial blocks, while corticosterone (CORT) infused rats did not. Infusions of CORT into the IL didnot affect choices for empty arms. These data indicate that only reward-related decision- making was affected by CORT infusions. Second, targeting the lateral orbitofrontal cortex (lOFC) no differences were observed between vehicle and CORT treated rats in the number of visits to the disadvantageous or empty arms. However, compared to vehicle treated rats of the IL group vehicle treated rats of the lOFC group showed no decrease in the number of visits to the disadvantageous arm. No differences were found for the empty arms, again suggesting that also here only reward-related decision-making was affected. The present results confirm that intracerebral infusions with CORT targeting the IL result in disrupted decision-making performance, but they do not directly support that the same holds true for infusions targeting the lOFC.

108 Corticosterone and decision-making in male Wistar rats: the effect of corticosterone application in the infralimbic and orbitofrontal cortex

4.1 Corticosterone dose and brain regions The dose of CORT used for the infusions (5 ng/μL) was previously shown to enhance the effect on taste aversion memory when infused in the insular cortex or basolateral amygdala (Miranda et al. 2008) or on memory consolidation when infused in the dorsal striatum (Quirarte et al. 2009). Although in these studies other brain areas were targeted, the dose of 5 ng/μL in the present study was sufficient to induce significant differences between IL infused groups. As both vehicle and CORT treated rats of the OFC targeted groups showed a less than optimal decision-making performance, no conclusions on the dose for this brain region can be drawn. The selection of brain areas for infusion was based on a previous study (Koot et al. 2013 (chapter 5)): disrupted decision-making performance, as identified by a decrease of visits to the disadvantageous arm 30 min after systemic CORT injections, was related to significantly increased levels of c-Fos expression in the lOFC and insular cortex and a trend in the IL. The current study aimed for elucidation of the effects of direct CORT administration into the IL and lOFC on decision-making in the rIGT. The insular cortex was left out from the study due to technical difficulties targeting the insular cortex at the specific site that was studied for c-Fos expression. Previous studies with this variant of the rIGT have shown that male rats shift from exploratory behaviour in the first half of the task to visiting the disadvantageous arms less often in the second half of the task (De Visser et al. 2011a, 2011b; Homberg et al. 2008; Van den Bos et al. 2006a, 2012b; Van Hasselt et al. 2012). In the present study, we found that from trial 61 onwards vehicle treated rats within the IL infused group decreased their number of visits to the long-term disadvantageous arm. This suggests that cannulation and/or infusion procedure in the IL per se had no effect on rIGT performance. In contrast, cannulation and/or infusion procedure in the lOFC appeared to have an effect on rIGT performance as indicated by the poor decision- making performance (see below).

4.2 Behavioural effects of CORT infusions targeting infralimbic cortex Previously successful task progression in male rats was shown to be associated with activation of the mPFC (prelimbic and infralimbic region; De Visser et al. 2011a, 2011b; Van Hasselt et al. 2012). Furthermore, De Visser et al. (2011b) showed that transient inactivation of the prelimbic region (PrL) of the medial prefrontal cortex (mPFC) prior to the last three sessions of the rIGT hampered task-progression within the baited arms, i.e. rats did not decease their number of visits to the disadvantageous arm, while no effect was found for the empty arms. The data from this study are in line with the current data as similar effects emerged at the moment vehicle treated IL rats started to improve upon their performance in the second half of the task. Although the study by De Visser et al. (2011b) aimed at targeting the PrL, the large injections volumes used (1 μL per hemisphere), could well have caused the compounds to affect the functioning of the IL. When considering the injection angle (30°) and volume (0.5 μL per hemisphere) used in the current study, it is unlikely that the present CORT injections reached the PrL. It has been shown that both the IL and PrL play a role in task performance as assessed by c-Fos expression (De Visser et al. 2011a; Van Hasselt et al. 2012). One reason for the differential effect of the PrL and IL for the effect of peripheral CORT (Koot et al. 2013 (chapter 5)) could be that the IL region is more sensitive to CORT than the PrL region.

109 Corticosteroid actions in the brain are mediated by mineralocorticoid (MR) and glucocorticoid (GR) receptors (de Kloet et al. 2005) and the mPFC contains both MR and GR (Chao et al. 1989; Herman et al. 2005; Patel et al. 2000). After becoming activated by stress (Cullinan et al. 1995; Figueiredo et al. 2003) these receptors contribute to the neural control of the endocrine response and behavioural adaptation to stress (Diorio et al. 1993; Herman et al. 2005; Sullivan and Gratton, 2002). Only a few studies (Barsegyan et al. 2010; Butts et al. 2011; Roozendaal et al. 2009) delineated the direct effect of CORT applied within the mPFC (both PrL and IL), and found that impairments on working memory and improvements in memory consolidation were GR mediated. Additional experiments are needed to unravel whether the observed rapid effects of CORT infusion in the IL on decision-making are MR or GR mediated. In an attempt to delineate the behavioural mechanisms underlying the significant 6 difference between CORT and vehicle infused IL rats, the number of switches, win- stay and lose-shift behaviour were examined, but no effects of CORT treatment were found on these parameters. This suggests that sensitivity to reward or punishment was not affected by the CORT infusions, as measured by a lack of effect on lose- shift or win-stay behaviour. This is in line with our previous study (Koot et al. 2013 (chapter 5)), in which also working memory and memory consolidation did not seem to be affected. Furthermore, the selective effect on the baited arms supports this notion. These data show that CORT induced disrupted decision-making is caused by at least direct effects on the IL. The mPFC, including the IL, is critically involved in strategy shifting, behavioural flexibility and goal-directed learning behaviour by encoding task-rules (Balleine and O’Doherty, 2010; Birrell and Brown, 2000; Dias et al. 1996; Floresco et al. 2008; Ragozzino et al. 1999; Sul et al. 2010; Tran-Tu-Yen et al. 2009; Young and Shapiro, 2009). Particularly, the IL promotes instrumental discrimination learning and suppresses impulsive responding (Chudasama and Robbins, 2003; Murphy et al. 2005, 2012). Inactivation of the IL leads to deficits in retrieval of emotional (extinction) memory (Sierra-Mercado et al. 2006). It has been suggested that the rodent IL constitutes a functional homologue of the human ventromedial PFC (Milad and Quirk, 2012). In line with this suggestion, patients with ventromedial PFC lesions also have difficulties in recalling the difference in emotional value between options prior to making the next choice within the IGT, hampering thereby task progression (Bechara et al. 1999). Thus, in the present study CORT infusion in the IL could have led to a disrupted IL functioning leading to a lack of inhibitory control (Quirk and Beer, 2006). CORT infused rats may therefore continue to explore the different options in the rIGT rather than integrating and retrieving earlier information into a behaviour necessary for successful completion of the rIGT.

4.3 Behavioural effects of CORT infusions targeting lateral orbitofrontal cortex Infusions of both vehicle and CORT into the lOFC resulted in a different behavioural pattern than what was seen in IL cannulated rats. In the lOFC infused group,no difference between vehicle and CORT treated rats in visits to the disadvantageous arm was found. Also no difference between vehicle and CORT treated rats in visits to

110 Corticosterone and decision-making in male Wistar rats: the effect of corticosterone application in the infralimbic and orbitofrontal cortex the empty arms was observed. Therefore, we cannot draw conclusions as yet on the effect of CORT infusions targeting the lOFC. When comparing the two vehicle groups (IL and lOFC), vehicle treated lOFC cannulated rats did not decrease their visits to the disadvantageous arm, while vehicle treated IL cannulated rats did, just as other control rats in our previous studies (De Visser et al. 2011a, Koot et al. 2013 (chapter 5)). No differences were found in visits to the empty arms. Based on this observation, it could be that the lOFC was damaged in all rats of this group, as a result of the region’s cannulation, the volume infused on three consecutive days, or the insertion of longer injection needles prior to infusion. It is difficult to define the effect of each of these procedures on the function of the lOFC, as to our knowledge no studies have been performed cannulating the lOFC for other purposes than inactivation or lesions of this area. Nonetheless, inactivation studies showed that the lOFC processes information about rewarding values (Kantak et al. 2009), and enables task switching by supporting initial inhibition of previously relevant choice patterns (Kim and Ragozzino, 2005; Ragozzino, 2007). The disruption of reward-based decision-making in all lOFC cannulated rats in the present study fits within the frame of current knowledge of the role of the OFC in reinforcement-guided decision-making (Schoenbaum and Roesch, 2005; Fellows, 2007; Murray and Izquierdo, 2007). The OFC is important in signalling the value of an expected outcome (Frank and Claus, 2006; Stalnaker et al. 2009), and damage to the OFC may prevent the adequate integration of information about the consequences of responding for a reward with the subjective value of that rewarding outcome (Ostlund and Balleine, 2007; Rivalan et al. 2011; Schoenbaum et al. 1999). Furthermore, the lOFC is important when a shift in choice pattern is demanded after a change in contingencies occurred and as such a suppression of previously rewarded behaviour is required (Bohn et al. 2003; Chudasama and Robbins, 2003; Elliott et al. 2000a; Fellows, 2007; McAlonan and Brown, 2003; Zald et al. 2012). Thus, our data suggest that the supposed lesions of the lOFC may have interfered with the ability to integrate knowledge of task contingencies into goal-directed behaviour.

4.4 Concluding remarks The present study showed the crucial involvement of IL in the performance of male rats in the rIGT, as parts of a network regulating decision-making. Similar to systemically injected CORT (Koot et al. 2013 (chapter 5)), locally applied CORT on the IL resulted in disrupted reward-based decision-making performance of male rats in the rIGT. These findings complement previous studies indicating the role of prefrontal areas in decision-making and build on the existing knowledge of how corticosteroids can affect cognitive function. Future studies should focus on whether CORT affects decision-making through mineralo- or glucocorticoid receptors. We were unable to delineate the effect of direct application of CORT on the lOFC in terms of behavioural outcome in the rIGT. Refined techniques, such as adjusted cannulation coordinates (less deep cannula, longer infusion needle) would be one option to circumvent possible lesions.

111 7

112 CHAPTER 7

TEMPORAL DOMAINS IN SOCIAL DECISION- MAKING FOLLOWING STRESS IN WOMEN

S. Koot a a Rudolf Magnus Institute of Neuroscience, C.H. Vinkers b,c Department of Neuroscience & Pharmacology, J.V. Zorn c University Medical Centre Utrecht, the B. Olivier c Netherlands M.P. Boks b b Department of Psychiatry, Rudolf Magnus R. van den Bos a,d Institute of Neuroscience, University Medical T. Kalenscher e Center Utrecht, Utrecht, The Netherlands M. Joëls a c Division of Pharmacology, Utrecht Institute for Pharmaceutical Sciences and Rudolf Magnus Institute of Neuroscience, Utrecht University, Utrecht, The Netherlands d Department of Organismal Animal Physiology, Radboud University Nijmegen, the Netherlands e Comparative Psychology, Institute of Experimental Psychology, Heinrich-Heine University Düsseldorf, Düsseldorf, Germany

In preparation

113 Abstract

In modern society, decisions are often made under stressful conditions. Under such circumstances, stress may result in suboptimal decision-making. Corticosteroid hormones, released after stress, influence brain function in atime- dependent manner, causing rapid (non-genomic) and slow (genomic) effects. We recently showed that the effects of stress on social decision-making are time-dependent in male individuals with a lower propensity for altruistic punishment 75 min after stress compared to directly after stress, i.e. behavioural response to unfairness changed depending on the time interval after stress. Since gender differences have been found in stress reactivity and decision-making, we here aimed to investigate how stress affects altruistic punishment 7 and trust in women. Healthy female participants (n=80) were exposed to either a validated grouped social stress test (Grouped Trier Social Stress Test, TSST-G) or a control condition. Subsequently, participants completed several social decision-making tasks either directly after stress or 75 min later. General altruism was assessed with a one-shot version of the Dictator Game in which an anonymous donation could be offered to a charitable organization. Altruistic punishment was assessed using the Ultimatum Game. Reciprocal exchange was assessed in the Trust Game. As expected, cortisol and alpha-amylase concentrations as well as subjective negative affective states increased significantly in the stress condition. However, exposure to psychosocial stress did not markedly affect general altruism, altruistic punishment, or trust in women, neither directly after stress nor 75 min later. Yet a trend was found for naturally cycling women being more generous than women using contraceptives. The absence of influence of stress on social decision-making in women, in contrast to men, may be explained by the use of hormonal contraceptives, which was associated with a more blunted stress-induced cortisol response, possibly reducing the effect on social decision-making. Overall, our results indicate that in women, in contrast to men, levels of altruistic punishment and trust remain relatively stable in the direct and late aftermath of stress.

114 Temporal domains in social decision-making following stress in women

1 Introduction

Decisions are often made under stressful conditions, such as in combat situations in the military, during financial business, and in health care settings. Decision-making is one of the most important cognitive functions in daily life. Stress was earlier shown to result in suboptimal decision-making in humans tested directly after being stressed (Lighthall et al. 2009; Preston et al. 2007; Starcke et al. 2008; Starcke and Brand, 2012; Van den Bos et al. 2009). For instance, in men stress-induced cortisol reactivity correlated negatively with decision-making performance, i.e. the higher the salivary cortisol levels, the poorer the performance in the Iowa Gambling Task (Van den Bos et al. 2009). Most studies have examined decision-making performance shortly after exposure to stress. However, recent evidence in rodents and humans suggests that stress- induced changes in behaviour may follow a temporal pattern (de Kloet et al. 2005). Shortly after stress, catecholamines and the fast (non-genomic) effects of corticosteroids are likely to be involved in the rapid adjustment of behaviour (Joëls and Baram, 2009), promoting instrumental and habitual behaviour (Schwabe et al. 2010). In contrast, several hours after stress exposure, the slower (genomic) cascade has fully developed, which (among others) contributes to the overall restoration of homeostasis and allows more flexible behaviour aimed at long-term goals (de Kloet et al. 2005; Diamond et al. 2007; Henckens et al. 2010, 2011; Williams and Gordon, 2007). These temporal patterns have recently also been observed with regard to decision-making processes. In male rats, reward-based decision-making was impaired shortly after corticosterone administration but not when tested 3 h later (Koot et al. 2013 (see chapter 5)). Moreover, in healthy male individuals, a lower propensity for altruistic punishment, i.e. the costly punishment of unfair behaviour, was found 75 min after stress compared to directly after stress (Vinkers et al. 2013). So far, the effects of stress on decision-making and particularly its temporal aspects have mainly been studied in male individuals. However, gender differences in decision-making performance (under both stressful and non-stressful circumstances) have been observed in both humans and animals (Bolla et al. 2004; Overman et al. 2004, 2006; Reavis and Overman, 2001; Van den Bos et al. 2006a, 2007, 2012b, 2013). In these studies, male individuals focused on the long-term pay-off, while female participants sampled both short- and long-term beneficial options, i.e. remained exploratory. These gender effects in decision-making may be the result of differences in risk-adjustment (adjusting betting behaviour according to the likelihood of winning; Deakin et al. 2004; Lee et al. 2009; Sapienza et al. 2009; Schubert et al. 1999; Van den Bos et al. 2012b), punishment and reward sensitivity (Eriksson and Simpson, 2010; Lee et al. 2009; Powell and Ansic, 1997), and cognitive control over affective processes in general (Mak et al. 2009). Moreover, the social environment may lead to gender differences in (risky) decision-making. Compared to female individuals, male subjects express increased risk-taking tendencies in the presence of bystanders and when shown pictures of attractive individuals of the opposite sex (d’Acremont and Van der Linden, 2006; Gardner and Steinberg, 2005; Wilson and Daly, 2004). The outcome of decision-making under stress, and gender-differences therein, may thus differ depending on the social context. To test the relevance of social context in a laboratory setting, several simple but sophisticated tasks have been

115 developed (Rilling and Sanfey, 2011): reciprocal exchange can be tested in the Trust Game (TG), fairness and inequity (altruistic punishment) with the Ultimatum Game (UG), and altruism with the Dictator Game (DG) (Rilling and Sanfey, 2011). Given that the temporal domains of stress on social decision-making in male individuals and the evidence that gender-differences may exist, we here investigated whether acute social stress exerted time-dependent effects on altruistic punishment and trust in female individuals. To this end, eighty healthy female participants were exposed to either a grouped stress test (Grouped Trier Social Stress Test, TSST-G) or a control condition (von Dawans et al. 2011). Social decision-making tasks (DG, UG, TG) were carried out either directly after stress or 75 min later, which is sufficiently long to allow the development of gene-mediated events. To assess the stress response and the possible relation of the autonomic nervous system and hypothalamic-pituitaryadrenal axis with the social decision-making tasks, salivary samples were repeatedly taken during the experiment and analysed on alpha-amylase and cortisol levels.

7 2 Methods

2.1 Participants Female adult healthy participants were recruited. The study was approved by the Utrecht Medical Center ethics committee and performed according to the ICH guidelines for Good Clinical Practice and the Declaration of Helsinki and its latest amendments. All participants gave their written informed consent prior to their inclusion in the study. Participants were not eligible to participate in case of current drug use, use of self-reported psychoactive medication, physical or mental illness, smoking, or not being fluent in Dutch. Current use of psychoactive substances (amphetamines, MDMA, barbiturates, cannabinoids, benzodiazepines, cocaine, and opiates) was determined with a urine multi-drug screening test (InstantView) and based on a positive score one participant was excluded. Thus, 79 individuals were included in the final analysis (Table 1). Several studies show that the increase in salivary cortisol levels in response to stressors is dampened in individuals taking hormonal contraceptives, and that salivary cortisol levels are higher in the luteal phase than in the follicular phase (Kirschbaum et al. 1999; Kudielka and Kirschbaum, 2005). Fifty-four women were using hormonal contraceptives during their participation; information on theuse of contraceptives was missing for three women. Of the twenty-two naturally cycling women, eleven were in the follicular phase of the menstrual cycle and seven in the luteal phase. Information on cycling day was missing for four natural cycling women. Participants were financially compensated (see below) and were instructed to refrain from eating, drinking and heavy exercise at least 2 h before the experimental session, as well as to refrain from caffeine use at least 4 h before the experimental session. All participants reported adherence to these instructions. All individuals had not previously been enrolled in stress-related research and were unfamiliar with each other.

116 Temporal domains in social decision-making following stress in women

Table 1 Baseline sociodemographic characteristics of all groups. Data are presented as mean (standard deviation). BMI: body-mass index. BIS/BAS: behavioral inhibition/avoidance system. STAI: Spielberger State and Trait Anxiety Inventory.

control early control late stress early stress late

n = 20 n = 20 n = 20 n = 19 F value p value

age 22.7 (2.3) 23.1 (2.4) 21.8 (2.0) 22.6 (2.1) 0.23 0.63

BMI 22.9 (2.7) 22.6 (3.0) 21.0 (2.3) 22.4 (4.1) 1.44 0.23

BIS 20.2 (2.6) 21.1 (3.4) 20.9 (3.1) 21.2 (2.5) 0.23 0.63

BAS Drive 10.1 (2.4) 11.1 (1.8) 11.0 (2.2) 10.8 (2.0) 1.55 0.22

BAS Reward Responsiveness 17.2 (2.1) 18.0 (1.1) 17.8 (1.8) 17.8 (2.1) 0.88 0.35

BAS Fun Seeking 10.9 (1.5) 11.0 (1.7) 10.7 (1.5) 11.2 (1.8) 0.24 0.62

STAI Trait 30.1 (5.7) 32.0 (6.7) 33.3 (5.6) 33.5 (8.3) 0.29 0.59

testosterone (pmol/l) 60.4 (33.4) 45.1 (22.5) 51.4 (22.0) 61.8 (38.9) 3.58 0.06

baseline cortisol (nmol/l) 8.2 (1.7) 7.8 (1.4) 8.4 (1.9) 7.5 (2.7) 0.24 0.63

baseline alpha-amylase 1.8 (0.9) 1.4 (1.1) 1.9 (1.5) 1.9 (1.6) 0.49 0.49 (100.000 U/ml)

2.2 General Procedure All procedures were carried out as previously described (Vinkers et al. 2013). All experiments occurred between 13:00 and 17:00 h to control for diurnal variations of cortisol secretion. Participants were instructed not to communicate with each other. After inclusion, participants were randomised to either the stress condition (group-wise Trier Social Stress Test; TSST-G; von Dawans et al. 2011) or a validated non-stressful control condition (Figure 1A). Five minutes before the stress or control intervention, all participants received written instructions for the appropriate test condition. In the stress condition, up to four participants sequentially delivered a public speech in a randomly assigned order (Vinkers et al. 2013; von Dawans et al. 2011) and performed mental arithmetic in front of an evaluative panel while being videotaped and recorded. The control condition consisted of a speech and arithmetic performance carried out simultaneously by all participants, ensuring a comparable cognitive load but without the social evaluative aspects. The DG, TG and UG were completed either directly after the stress/control condition (early groups) or 75 min later (late groups, 90 min after the onset of the TSST-G) (Figure 1A).

117 Figure 1 (A) Timeline of the experimental session. S = salivary sampling to determine cortisol and alpha-amylase levels; Q = general questionnaires; V = subjective stress assessment. Salivary cortisol (B, both conditions) (C, within stress condition only) and alpha-amylase (D) levels were measured throughout the experiment. Data points represent the mean values, error bars indicate SEM. **p<0.01; ***p<0.001

118 Temporal domains in social decision-making following stress in women

2.3 Behavioural tasks 2.3.1 General All behavioural tasks were programmed and presented using Presentation software (Neurobehavorial systems, Albany, CA, USA). After inclusion on the experimental day, participants received written instructions on the aim and methodology of the behavioural tasks. These instructions were accompanied by test exercises to ensure that all participants had understood the instructions. Also, participants were informed that financial compensation would be partially determined by randomly averaging several trials at the end of the procedure and paying out whatever choice the individual made in that trial. In addition to written instructions prior to the tasks, an instruction screen preceded the DG, TG and UG during the actual test, briefly explaining the specific task again. The UG was presented first, immediately followed by the DG and the TG.

2.3.1 Dictator Game To measure altruism, we chose a variant of the DG in which a participant could choose to anonymously donate an amount to a charitable organization. On a computer screen, participants were offered €10 with the possibility to donate any amount to Unicef and keep the remaining amount to themselves. Participants were asked to enter the amount which they would like to donate ranging from €0 to €10.

2.3.2 Ultimatum Game The principle of the UG is that two players must divide a sum of money, with one subject (the proposer) proposing the specific division and the other subject (the responder) then decides to accept or reject this offer. If the offer is accepted, the sum is split as proposed, but if it is rejected, neither player receives anything. As such, the UG was used to examine altruistic punishment behaviour. Here, each participant was offered a proposed division of a sum of money by different anonymous players via the computer. The participant then had the option of accepting or rejecting this offer. In order to make participants believe they were facing real opponents, instructions stated that participants were part of a larger study and that offers were actual offers by previous participants. To further increase credibility, participants first received €10, €15, €20 and €25 in a random order and were asked to share a certain amount with future participants (i.e. to act as proposer). Thereafter, participants acted as responder and received a total of 20 trials in random order during which they could either accept or reject a proposed division. Every proposal offered 10, 20, 30, 40 or 50% of €10, €15, €20 and €25 in a way that all percentages were proposed for each amount. In each trial, an offer was made by a different proposer whose putative identity was revealed by showing a first name accompanied by the first letter of their last name. All identities were hypothetical and equally balanced for gender. The design of each trial consisted of the presentation of the proposer’s name (2 s), the total offered stake (5 s) and the proposed division of the offer that they could accept or reject (5 s). The main outcome parameter was the relative fraction of rejected offers per offered percentage, or the absolute offers, respectively.

119 2.3.3 Trust Game In the TG, a player (the investor) must decide how much of an endowment to invest with a partner (the trustee). Once transferred, this money is multiplied by some factor and then the trustee has the opportunity to return money to the investor but does not need return anything. If the trustee honours trust and returns money, both players end up with a higher monetary pay-off than the original endowment. However, if the trustee abuses trust and keeps the entire amount, the investor takes a loss (Rilling and Sanfey, 2011). Thus, the TG was used to examine trust and predictions how trust was reciprocated. Here, participants acted only in the role of investor in the TG. They played against four different partners, and began each trial by being endowed with €10. On each trial, participants could decide to transfer any amount of this money, i.e. they could offer some amount between €0 (placing no trust) and €10 (placing full trust). Any amount of money larger than zero offered to the partner was multiplied by three and transferred to the partner. The participant was asked how much she expected to receive in return from the partner, who could then return 7 any amount between €0 and the full sum to the investor. If the partner returned money, both the investor and the partner ended up with a higher monetary payoff than the original €10, but if the partner kept the entire amount to him/herself, the participant received nothing in return. The participant was informed to be notified of their partner’s decision after the task and as such was not instantly informed about the total amount earned in the trial. On each trial, the participant was asked to make an offer to a different partner whose putative identity was revealed by showing a first name accompanied by the first letter of their last name. All identities were hypothetical and equally balanced for gender.

2.4 Questionnaires 2.4.1 Subjective mood At baseline (t = −40 min), during the stress/control condition (t = +8 min) and after the experimental condition (t = +35 min), perceived and subjective levels of stress, anxiety and insecurity, and feelings of warmth and sweating were assessed using visual analogue scales (VAS, 118mm scale) ranging from 0 (not at all) to 100 (maximum) (Figure 1A).

2.4.2 Reward- and punishment sensitivity Differences in punishment and reward sensitivity have been shown to influence decision-making in the UG and the DG (Scheres and Sanfey, 2006). Therefore, participants completed the BIS/BAS (behavioural inhibition/activation system) questionnaire at baseline (t = -40 min). This questionnaire contains a BIS scale that measures punishment sensitivity and three BAS scales measuring different elements of reward sensitivity (drive, reward responsiveness, and fun seeking) (Carver and White, 1994).

2.4.3 Trait anxiety To rule out a putative bias due to pre-existing differences in anxiety we assessed participants’ trait anxiety at baseline (t = -40 min) using the Spielberger State Trait Anxiety Inventory (STAI) (Spielberger, 1989).

120 Temporal domains in social decision-making following stress in women

2.5 Saliva sampling To examine hormonal levels, saliva samples were collected at various time points (Salivette, Sarstedt, Nümbrecht, Germany). In total, eight saliva samples were collected, i.e. at baseline (t = -10 min relative to the initiation of the stress/control condition), during the stress/control condition (t = +8 min), and at various time points afterwards (t = +16, +35, +50, +65, +90, and +125) (Figure 1A). Samples were stored at -20°C until analysis. Cortisol in saliva was measured without extraction using an in house competitive radio-immunoassay employing a polyclonal anticortisol- antibody (K7348). [1,2-3H(N)]-Hydrocortisone (PerkinElmer NET396250UC) was used as a tracer. The lower limit of detection was 1.0 nmol/l and inter-assay variation was <6% at 4-29 nmol/l (n=33). Intra-assay variation was <4% (n=10). Samples with levels >100 nmol/L were diluted 10x with buffer. Concentrations of salivary alpha- amylase (sAA) were determined as an index for adrenergic activity in response to stress (Nater and Rohleder, 2009). Amylase was measured on a Beckman-Coulter AU5811 chemistry analyzer (Beckman-Coulter Inc, Brea, CA). Saliva samples were diluted 1000x with 0.2% BSA in 0.01 M phosphate buffer pH 7.0. Inter-assay variation was 3.6% at 200.000 U/L (n=10). Additionally, one baseline saliva sample was used to determine testosterone levels because testosterone has been reported to influence decision making in the UG (Burnham, 2007). Testosterone in saliva was measured using an in house competitive radio-immunoassay employing a polyclonal anti- testosteron-antibody (Dr. Pratt AZG 3290). [1,2,6,7-3H]-Testosterone (NET370250UC, PerkinElmer) was used as a tracer following chromatographic verification of its purity. The lower limit of detection was 20 pmol/L. Inter-assay variation was 15.5-6.8% at 36-160 pmol/L respectively (n=20).

2.6 Data Analysis All statistics were carried out using SPSS version 20 (SPSS Inc., Chicago, IL, USA). One- way ANOVAs were used to check for differences in baseline parameters between groups. Changes in VAS scales (value during/after the experimental condition – baseline value) were analysed using a one-way ANOVA with condition (stress or control) as between-subject factor. Changes in salivary cortisol and alpha-amylase concentrations were analysed using a 8 (time: -10 vs +8 vs +16 vs +35 vs +50 vs +65 vs +90 vs +125) x 2 (condition: stress vs control) x 2 (contraceptives: naturally cycling vs women using contraceptives) general linear model (GLM) repeated measures ANOVA, where time was a repeated measure. Results were corrected by the Greenhouse- Geisser procedure where appropriate (indicated by an ε value). Also, the area under the curve increase (AUCi) of cortisol and alpha-amylase were calculated for each participant as previously described (Pruessner et al. 2003). Moreover, the percentage increase was calculated for cortisol [(4th sample − 1st sample ) / 1st sample * 100] and alpha-amylase [(2nd sample − 1st sample ) / 1st sample * 100] as a measure for the temporal changes in both parameters. The effect of stress on the percentage increase and the AUCi was analysed between groups using a one-way ANOVA. For the DG, the donated amount was analysed with a 2 (condition: stress vs control) x 2 (time: early vs late) x 2 (contraceptives: naturally cycling vs women using contraceptives) GLM ANOVA. For the UG, mean rejection rates were calculated for offers in the 10, 20, 30, 40, and 50% category, and were analysed using a 5 (percentage: 10% vs 20% vs 30%

121 vs 40% vs 50%) x 2 (condition) x 2 (time) x 2 (contraceptives) GLM ANOVA with percentage as a repeated measure. The average amount offered was analysed with a 2 (condition: stress vs control) x 2 (time: early vs late) x 2 (contraceptives: naturally cycling vs women using contraceptives) GLM ANOVA. For the TG, the offered amount and the amount predicted to receive in return were analysed using a 2 (condition) x 2 (time) x 2 (contraceptives) GLM ANOVA. For comparison of the average amount offered in the UG and TG, percentages from the amount available were calculated, i.e. in the UG €10, €15, €20 or €25, in the TG always €10, and analysed using a 2 (test: offer UG vs offer TG) x 2 (condition) x 2 (time) x 2 (contraceptives) GLM ANOVA with test as a repeated measure. Donated amount (DG), mean rejection rates (UG), overall accepted (UG), average amount offered (UG and TG) and amount predicted to receive in return (TG) were correlated with the AUCi and the percentual change in cortisol and alpha-amylase, as well as baseline testosterone, using Spearman’s correlation coefficients. The significance level was set at p ≤ 0.05. 7

3 Results

3.1 Baseline parameters The experimental groups did not differ in any of the measured baseline parameters regarding personal variables (age, body mass index), baseline hormonal values (cortisol, alpha-amylase, testosterone) and personality characteristics (BIS, BAS drive, BAS reward responsiveness, BAS fun seeking and STAI trait) (all p values ≥ 0.05, Table 1).

3.2 Subjective self-reported parameters As expected, compared to the control condition, stress exposure significantly increased levels of self-reported perceived levels of stress during the experimental condition compared to baseline stress levels (Table 2; p<0.001). This was also found for anxiety (VAS increase, p=0.004), insecurity (VAS increase, p<0.001), and sweatiness (p=0.001). With regard to the effects of time, the early groups reported increased levels of insecurity compared to late group (early group 22.6 (24.5) versus late group 12.5 (19.3) [mean (SD)]; time, F(1,78)=4.601, p=0.035) independent of stress. No other significant effects or interactions of timing (early and late groups) were present (all p values ≥ 0.069). A return to baseline for all self-reported parameters was ascertained by comparing the baseline value (t = -40 min) to the self-report after the experimental condition (t = +35 min), showing no significant differences in self-report levels of stress (p=0.62), anxiety (p=0.77), insecurity (p=0.52), warmth (p=0.35), and a trend for sweatiness (p=0.053).

3.3 Hormone levels in saliva Stress resulted in increased cortisol levels compared to the control condition (main effect of time, F(2.14, 164.99)=30.051, p<0.001; main effect of condition, F(1,77)=9.868, p=0.002; condition x time interaction, F(2.14, 164.99)=15.606, p<0.001; figure 1B). Further support that the stress procedure was effective stems from significant increases in the AUCi (F(7,78)=22.947, p<0.001) and the percentage

122 Temporal domains in social decision-making following stress in women

Table 2 Changes in visual analogue scale (% change during experimental condition compared to baseline, VAS) in parameters of subjective stress. Data are presented as mean (standard deviation).

control stress

n = 40 n = 39 F value p value

stress 6.4 (17.7) 30.3 (20.2) 31.43 <0.001

anxiety 0.8 (7.3) 7.7 (12.7) 8.75 0.004

insecurity 9.1 (18.8) 26.4 (22.9) 13.47 <0.001

warmth -1.5 (17.9) 7.1 (26.4) 2.87 0.094

sweatiness 1.6 (16.1) 13.9 (17.0) 10.99 0.001

cortisol increase (F(1,78)=25.113, p<0.001) in the stress group compared to the control condition. Additionally, within the stress group naturally cycling women displayed increased cortisol levels in response to stress compared to women taking contraceptives (main effect of time, F(1.65, 59.45)=28.919, p<0.001; main effect of contraceptives, F(1,36)=4.113, p=0.050; time x contraceptives interaction, F(1.65, 59.45)=4.827, p=0.016; figure 1C). No significant effect of menstrual phase (follicular compared to the luteal phase) on the stress-induced cortisol response was found in naturally cycling women (main effect of time, F(1.63, 16.32)=11.816, p<0.001; main effect of cycle, F(1,10)=0.359, p=0.562; time x cycle interaction, F(1.63, 16.32)=0.122, p=0.847). Stress significantly increased alpha-amylase levels (main effect of time, F(1.98, 150.13)=12.545, p<0.001; main effect of condition, F(1,76)=3.232, p=0.076; condition x time interaction, F(1.98, 150.13)=6.651, p=0.002; figure 1D). The AUCi of alpha- amylase did not significantly differ between stress and control condition (p=0.167), but the percentage increase in alpha-amylase due to the condition was larger in the stress group than in the controls (F(1,77)=20.442, p<0.001). These data indicate a rather short-lived increase in salivary alpha-amylase levels due to stress. Within the stress group no significant differences between naturally cycling women and women on contraceptives with respect to alpha-amylase levels were found (main effect of time, F(1.67, 58.52)=7.806, p=0.002; main effect of contraceptives, F(1,35)=0.271, p=0.606; time x contraceptives interaction, F(1.67, 58.52)=0.363, p=0.659).

3.4 Behavioural tasks 3.4.1 Dictator Game Data for one participant (DG, stress-late group) was not available due to technical problems. No significant differences in donated amount were found between conditions or time (figure 2; table 3). As women taking contraceptives showeda blunted cortisol response when exposed to stress compared to naturally cycling women, the effect contraceptive use was tested. This did not affect the donated amount (table 3).

123 Figure 2 In the Dictator Game, no differences were found in the donated monetary amount (€) to Unicef between conditions, time or use of contraceptives. Bars represent the mean values, error bars indicate SEM.

Figure 3 In the Ultimatum Game, lower offers were rejected and higher offers were accepted. We observed no differences in acceptance rates between conditions [(A) control; (B) stress], time, or use of contraceptives. Data points represent the mean values, error bars indicate SEM.

Figure 4 Results in the Trust Game. (A) The average amount offered to partner in € was not affected by condition, stress or the interaction. (B) A main effect of time was observed in the average amount predicted to receive in return from the partner in €. However, we observed no main effect of condition, nor an interaction between condition and time. Bars represent the mean values, error bars indicate SEM. *p<0.05

124 Temporal domains in social decision-making following stress in women

Table 3 Behavioural tasks: statistics of different parameters measured in the Dictator Game (DG; amount offered), Ultimatum Game (UG; rejection rate and amount offered) and Trust Game (TG; amount offered and amount predicted to receive in return).

DG offer UG rejection UG offer TG offer TG predict rate return F value 0.071 0.196 0.210 0.398 1.006 condition p value 0.791 0.659 0.648 0.530 0.319 F value 0.001 0.613 0.763 0.596 2.310 time p value 0.972 0.436 0.385 0.443 0.133 F value 0.025 0.300 3.523 3.671 3.779 contraceptives p value 0.875 0.586 0.065 0.060 0.056 F value 0.001 1.003 0.256 1.791 0.303 condition x time p value 0.980 0.320 0.614 0.185 0.584 F value 0.106 - 1.051 0.012 0.262 condition x contraceptives p value 0.746 - 0.309 0.912 0.610 F value 0.274 - 0.299 0.942 0.223 time x contraceptives p value 0.602 - 0.587 0.335 0.639 F value 0.042 - 2.800 0.548 0.181 condition x time x contraceptives p value 0.838 - 0.099 0.462 0.672 F value - 5.896 - - - percentage p value - 0.001 - - - F value - 0.530 - - - percentage x condition p value - 0.659 - - - F value - 0.772 - - - percentage x time p value - 0.509 - - - F value - 1.584 - - - percentage x contraceptives p value - 0.195 - - - F value - 0.635 - - - percentage x condition x time p value - 0.591 - - -

Table 4 Effect of the use of contraceptives on the average amount offered in the Ultimatum Game (UG) and Trust Game (TG); average amount predicted to receive in return in the TG; average of the percentages in the UG and TG. Shown are mean (SEM) amounts in € or percentages.

naturally cycling contraceptives n = 22 n = 54 F value p value DG offer € 5.4 (0.7) 5.3 (0.4) 0.025 0.875 UG offer € 8.4 (0.5) 7.4 (0.3) 3.523 0.065 TG offer € 6.3 (0.4) 5.3 (0.3) 3.671 0.060 TG predict return € 9.1 (0.9) 7.1 (0.5) 3.779 0.056 mean offer of UG and TG % 54.9 (2.5) 47.4 (1.6) 6.209 0.015

125 3.4.2 Ultimatum Game Participants were more likely to reject lower offers (e.g. 10-20% of the stake) and accept higher offers (e.g. 40-50% of the total stake) as indicated by a significant percentage effect (F(2.95, 221.31)=133.935, p<0.001; figure 3). Stress did not affect rejection rates in the UG, nor did time or their interaction (table 3). TheANOVA showed that the use of contraceptives did not affect rejection rates either (table 3). When being in the investor role, the experimental groups did not differ in the average amount offered over four trials depending on condition or time (table 3). However, when considering the use of contraceptives, naturally cycling women tended to offer more on average than women using contraceptives (table 4; main effect of contraceptives, F(1,75)=3.523, p=0.065; all others see table 3).

3.4.3 Trust Game Stress and timing did not significantly alter the amount offered (figure 4A;table 3). However, when considering the use of contraceptives, naturally cycling 7 women tended to offer more than women using contraceptives (main effect of contraceptives, F(1,75)=3.671, p=0.060; all others see table 3; table 4). Participants in the late group predicted a higher returned amount compared to participants in the early group irrespective whether participants were exposed to the stress or control condition (figure 4B; main effect of time, F(1,78)=4.023, p=0.048; all others see table 3). Naturally cycling women tended to predict higher amounts than women using contraceptives (table 4; main effect of contraceptives, F(1,75)=3.779, p=0.056; all others see table 3).

3.4.4 Amount offered in Ultimatum Game versus Trust Game When comparing the average percentage of the offer between the UG and the TG (calculated as the average percentage of the available amount , i.e. in the UG the offered percentage of €10, €15, €20 or €25, in the TG €10), women offered on average more in the TG (57.57 ± 2.41%) than in the UG (44.69 ± 1.52%; test F(1,68)=21.867, p<0.001). Furthermore, naturally cycling women offered a significantly increased amount compared to women using contraceptives when combining the UG and TG (main effect of contraceptives, F(1,74)=6.209, p=0.015; all other p≥0.168; table 4).

3.4.5 Correlations between behavioural tasks and hormones If activation of the autonomic nervous system and/or the hypothalamo-pituitaryadrenal axis plays an important role in social decision-making in women, one would expect to see significant correlations between the stress-induced hormonal response and the behavioural outcome after stress dependent on the time point. Therefore, we separately tested correlations in the stress early and stress late groups. This of course leaves the possibility open that behaviour was similarly affected at these two points in time after stress exposure. We tested for correlations between the different parameters of the behavioural tasks and AUCi for cortisol, AUCi for alpha-amylase, percentage increase in cortisol, and percentage increase in alpha-amylase. The parameters of the behavioural tasks were for the DG the donated amount, for the UG the average acceptance rate as well as the average amount offered, and for the TG the amount offered as well as the amount predicted to receive in return. None of these correlational analyses

126 Temporal domains in social decision-making following stress in women

between the behavioural parameters and the different measures for cortisol and alpha-amylase reached significance (all p>0.108)

4 Discussion

In the present study we examined the temporal patterns of stress on altruistic punishment and trust in female individuals by exposing them to a group-wise stress test or a control condition and examining social decision-making either directly after the experimental condition or 75 min later. Our results show that exposure to psychosocial stress does not markedly affect altruistic punishment or trust in women, neither directly after stress nor approximately one hour later. In support, the AUCi for cortisol or alpha-amylase did not correlate with stress-induced performance in the DG, UG or TG. Collectively, these data indicate that, contrary to recent findings in male individuals, altruistic punishment and trust in females is not significantly affected due to exposure to stress, nor in the direct or late aftermath. Behavioural studies in rodents and humans indicate that stress and / or cortisol exert time-dependent effects on cognitive performance. Shortly after corticosteroid hormone levels rise, instrumental and habitual behaviour is promoted (Schwabe et al. 2010). Several hours later, however, corticosteroids allow more flexible behaviour to meet long-term goals, which is thought to contribute to the restoration of homeostasis (de Kloet et al. 2005; Diamond et al. 2007; Williams and Gordon, 2007; Henckens et al. 2010, 2011). Time-dependent effects on decision-making after exogenous administration of corticosteroids were reported in rodents (Koot et al. 2013 (see chapter 5)). Moreover, healthy male human individuals showed a lower propensity for altruistic punishment 75 min after stress compared to directly after stress (Vinkers et al. 2013). Yet, the effects of stress on decision-making, as tested directly after stress, are known to be gender-dependent (Lighthall et al. 2009, 2012; Preston et al. 2007; Van den Bos et al. 2009; but see Mather et al. 2009). Therefore, the apparent absence of stress-induced effects on social decision-making in females was not entirely unexpected. Gender differences in decision-making are particularly pressing in tasks examining decision-making under ambiguity (i.e. no prior information indicating choice contingencies) and under conditions in which losses are inherent to awinning strategy (reviewed in Van den Bos et al. 2013). In these tasks, men tend to decide on global information, while women decide on detailed information (Van den Bos et al. 2013). Under stress, women appear to become more risk-averse and chose more advantageously in the long run in decision-making tasks as compared to controls or men (Lighthall et al. 2009, 2012; Preston et al. 2007; Van den Bos et al. 2009). In the present study, concerning a social setting involving trust and altruistic punishment, no such differences were observed in decision-making after stress or control situation. Several reasons may underlie our finding that no effect of stress on decision-making was found in female individuals. One factor could be related to the fact that the majority of our female individuals used contraceptives. In the individual version of the TSST (Cornelisse et al. 2011; Kirschbaum et al. 1999; Kudielka and Kirschbaum, 2005; but see Van Stegeren et al. 2008) cortisol responses to stress were more pronounced in men than in women, particularly in women using contraceptives. We

127 found comparable results in the TSST-G used here. Thus, in males the TSST-G was found to be highly effective in activating the sympatho-adrenomedullar system as well as the HPA-axis (Vinkers et al. 2013; von Dawans et al. 2011, 2012). In females, we observed small (though significant) cortisol responses, which were more pronounced in naturally cycling females than in women using contraceptives (Figure 1). Interestingly, the effects of stress on decision-making in female individuals were found either in natural cycling women only (Lighthall et al. 2009, 2012) or only in women with a high cortisol response (as defined by median split; Van den Bos et al. 2009). Since the majority of our participants used contraceptives, the lack of effect of stress on any of the behavioural tasks may be explained by the relatively low rises in cortisol after stress. In theory, compared to natural cycling participants, women using contraceptives could be less stressed in general in the present cross-sectional study. However, this is unlikely, since the finding that women using contraceptives show a blunted free cortisol response to psychosocial stress compared to men and natural cycling women (Kirschbaum et al. 1999), may be explained by increased 7 levels of corticosteroid-binding globulin (Wiegratz et al. 1995; Crook, 1997), while total plasma levels of cortisol did not differ between these groups . Hormonal contraceptives per se were also found to affect behavioural performance in the tasks, independent of stress exposure. Thus, in the DG naturally cycling women on average tended to offer more than women using contraceptives (Table 4). In agreement, when combining the offers in the setting of the TG and UG, naturally cycling women overall offered more than women using contraceptives (Table 4). Note however, that the TG and UG are economically not equivalent, and therefore one should be cautious when interpreting this result. Nonetheless, it suggests that altruism in general is negatively affected by contraceptive use; this is most likely caused by the gonadal hormone balance induced by contraceptives, although this was not specifically tested. In women using contraceptives, oestrogen excretion by the body is reduced and progesterone excretion ceases almost completely (Rivera et al. 1999). By contrast, in natural cycling women progesterone peaks during the luteal phase, and cycle effects have been found for altruism, inequity aversion and competitiveness (Ball et al. 2013; Buser, 2012a, 2012b; Wozniak et al. 2010). The gonadal hormone state of the participants in the current study may have added to the variance in outcome and hence reduced the power of the study. The fact that in previous studies the effects of stress on decision-making have been examined in an individual context, while in the present we focused on social decision-making in particular, may have contributed to the discrepancies between the present and earlier studies in females. Men and women might react differently to stress in a social context. Taylor et al. (2000) proposed a biological (evolutionary) theory which distinguishes between the male “fight-or-flight” or freeze response and the female “tend-and-befriend” reaction to stress. This theory is supported by indications that men use more individualistic and antisocial forms of coping after stress (Carver et al. 1989; Hobfoll et al. 1994), while women are more likely to seek and provide social support (Carver et al. 1989; Rosario et al. 1988; Stone and Neale, 1984) and join with others (Hobfoll et al. 1994). In a putatively chronic stress situation, men tended to be less generous than women in the TG and DG, yet women’s generosity per se was not affected by chronic stress (Lindova et al. 2010). Also, in the presence of a disgusting smell, a negative emotion involved in rejection

128 Temporal domains in social decision-making following stress in women

of unfair offers, men judged a monetary offer as less unfair than women as measured in the UG; by contrast, women’s judgement was not affected by the disgusting smell (Bonini et al. 2011). Therefore, the lack of stress effects in female individuals on social decision-making is not unprecedented. As the effects of stress on decision-making in women in previous studies were only found in naturally cycling women (Lighthall et al. 2009, 2012), future studies on social decision-making in female individuals should aim at including sufficient numbers of naturally cycling women. Furthermore, as cycle phase appeared to influence social decision-making (see above) the menstrual cycle should be assayed carefully, to prevent misestimating individuals’ cycle length (Small et al. 2007). In conclusion, we here report that in female individuals altruistic punishment and trust remain relatively stable in the direct and late aftermath of stress. This contrasts with a recent study in males, showing that the propensity for altruistic punishment is enhanced directly after stress, though not some time later (Vinkers et al. 2013). These discrepancies in men and women may critically depend on the experimental tasks and the fact that most females in our study used contraceptives. The data nevertheless underlines the importance of gender effects in social stress- induced effects on decision-making in a representative group of young Dutch males and females. Our data adds to a growing body of evidence that the effect of stress on decision-making depends on gender (see for reviews Starcke and Brand, 2012; Van den Bos et al. 2013). This is highly relevant given that stress is an inherent aspect of many decision-making processes in daily life.

Acknowledgements

The authors would like to thank Lotte Houtepen, Sandra Cornelisse, Remi Stevelink, Robin Buijs, Simeon van Tongeren, Chantal van de Ven, and Loes Mehlkopf for their help with the stress protocol, and Inge Maitimu for the analysis of the saliva samples.

129 8

130 CHAPTER 8

GENERAL DISCUSSION

1 Summary 132

2 Modelling decision-making: value of animal models? 133

3 Stress and decision-making 136

4 Gender and decision-making 139

5 Final remarks 141

131 1 Summary

Decision-making is related to assessing costs and benefits of competing actions with either a known outcome or an uncertain result. Decision-making is comprised of several abilities, such as flexibility in behaviour, inhibition of responses to immediate rewards in order to weigh long-term and short-term losses and benefits, and continuous evaluation of the risks and rewards of various options. Several factors can affect these abilities, leading to differences in the outcome of the decision-making process. In this thesis I have focused on two of these factors, i.e. stress and gender. The overall aim here was to improve our understanding of the relevance of stressful conditions and gender on decision-making, in rodents and in humans.

In chapter 2 one aspect of decision-making, i.e. impulsive choice, was examined by testing mice in the delay-discounting task. As only a few studies have been published on differences between males and females in this task, in this chapter both genders were tested. Male and female mice showed a partial overlap in their impulsive choices, but more “impatient” female mice (shifting from the large to the small 8 reward at short delays) seemed to choose more often the small immediate reward than the more “impatient” male mice. We discussed the results in the context of impulsivity, explorative behaviour and the effect of additional features possibly affecting the behaviour, such as the level of food deprivation to motivate animals to perform the task. Besides factors like gender and internal state, also the level of handling of the animals prior to testing may affect the task outcome. Therefore, home cage testing has become increasingly popular. To work on a less invasive or interfering method of testing rodent decision-making, in chapter 3 a pilot study on home cage testing of delay-discounting in rats was described. The main result of chapter 3 was that testing delay-discounting in the home cage results in behavioural patterns similar to those found in traditional Skinner box testing. This paved the way to continue exploring the possibilities of testing rodents in a home cage setting on more complex decision-making. This was done in chapter 4, where a new protocol to run the rodent Iowa Gambling Task (rIGT; Van den Bos et al. 2006a) in a home cage setting was presented. In this chapter we assessed whether manipulation of brain serotonin levels in rats affected their performance in operant-based tasks on both decision-making (rIGT) and gambling proneness (probabilistic-delivery task, PD). Lowering serotonin levels led to both poor decision-making and gambling proneness. Moreover, poor decision-making was related to gambling proneness in control rats, but not in rats with low serotonin levels. After exploring the possibilities to reduce the level of arousal due to testing itself, the focus was shifted towards the effect of deliberately induced stress on decision-making. In chapter 5 the time-dependent effects of the stress-hormone corticosterone (the rodent equivalent of cortisol) on decision-making were addressed by treating rats at two different time-points prior to being tested in the rIGT (Van den Bos et al. 2006a). Administration of corticosterone 30 min prior to testing disrupted reward-based decision-making. This was associated with increased c-Fos expression in a number of prefrontal areas, i.e. the lateral orbitofrontal cortex, infralimbic cortex and insular cortex. Administration of corticosterone 3 hours prior to testing did not disrupt reward-based decision-making. The importance

132 General Discussion

of these brain areas for reward-based decision-making under stressful conditions is emphasized in chapter 6. Here we showed that corticosterone applied directly to the infralimbic cortex disrupted reward-based decision-making, similarly to the findings presented in chapter 5. The data for the lateral orbitofrontal cortex were inconclusive, possibly due to (unintended) damage, preventing treatment differences to be observed. In chapter 7 the time-dependent effects of stress on social decision- making were addressed. Vinkers et al. (2013) showed that experimentally induced stress in men causes lowered generosity, and altered altruistic punishment in a time-dependent manner. More specifically, when tested 75 min after stress men had a lower propensity for altruistic punishment compared to men tested directly after stress. In a follow-up study we investigated whether women were affected by experimentally induced stress in a similar manner. It appeared that unlike the data in men stress did not affect altruistic punishment or general altruism in women. The results were discussed in the context of, among other things, the use of hormonal contraceptives, a condition that was associated with a more blunted stress-induced cortisol response.

2 Modelling decision-making: value of animal models?

Decision-making in humans and animals has been modelled in several ways (for reviews see e.g. Brand et al. 2006; De Visser et al. 2011c; Floresco et al. 2008; Weber and Johnson, 2009). As human decision-making is largely mirrored in animal choice behaviour (Van Wingerden and Kalenscher, 2012) animal models seem to be suitable for the investigation of the mechanisms underlying decision-making. One important reason to use animal models of decision-making behaviour is the wide range of techniques available in animal modelling, which makes it possible to resolve causal contributions of brain regions, tease apart subcomponents and to perform pharmacological manipulations which are technically or ethically impossible in human subjects (Kalenscher and Van Wingerden, 2011). Another reason for using animal models of decision-making is the study of the evolutionary roots of human decision-making, sharing neural substrates underlying choice behaviour with animals (Kalenscher and Van Wingerden, 2011). In this thesis animal models have been used mainly for the first reason. However, although the used techniques have shed light on the different mechanisms involved in decision-making, using animals to model human behaviour also has some drawbacks. The following aspects of animal models for decision-making should be taken into account when interpreting the results presented in this thesis. In the reward-based decision-making paradigms used in this thesis, i.e. delay- discounting task, probabilistic delivery task and the rIGT, food pellets were used as reward. In the (adult) human equivalents of these tasks, usually money is used a reward. This difference in rewards leads to the following translational issues to be discussed. First, food is a primary reward, immediately satisfying the needs of the hungry (motivated) animal, i.e. it directly contributes to the homeostatic state of the hungry subject, and elicits a strong hedonic response, but only until satiation. In contrast, money is an abstract secondary reward, only rewarding in association with a primary reinforcer, which does not satisfy the immediate needs of the individual

133 as subjects in general are not deprived of money. Additionally, food rewards are typically consumed immediately, while monetary rewards are accumulated before being paid out or spend. Furthermore, subjects will only receive a small amount of money at the end of the task, often not related to their actual task-performance. However, studies in humans showed that both food and monetary rewards result in the same behavioural patterns and activate similar neural networks in delay- discounting paradigms (Hayden et al. 2007; McClure et al. 2004, 2007), suggesting that money and food rewards are at least comparable in this respect when assessing decision-making processes. Yet, very recently (Sescousse et al. 2013) it was shown in humans that although both primary (food and erotic pictures) and secondary (money) rewards engage a common brain network – including the ventromedial prefrontal cortex, ventral striatum, amygdala, anterior insula and mediolateral thalamus – primary rewards are more strongly presented in the anterior insula and amygdala, while secondary rewards are represented in the anterior part of the orbitofrontal cortex, an evolutionarily more recent brain region. This suggests at least some reward-type dependency of the activation of subregions. Such a dissociation has also been found in studies using rodents: the primary rewarding aspects of cocaine 8 (cocaine itself) are neuroanatomically dissociated from the secondary rewarding aspects of cocaine (to cocaine conditioned stimuli), i.e. the basolateral amygdala was implicated in reinstatement of cocaine-conditioned reward, while the nucleus accumbens (core) played a role in primary cocaine reward (Grimm and See, 2000). Second, an aspect related to the previous, is the fact that an abstract monetary reward gives the possibility to “punish” the subject by losing money and thus reducing their momentary budget, while this is impossible with a food reward that already has been consumed and “added” to the energy budget. As such, other “punishments” have been developed for animals, e.g. delays, omissions of reward, unpalatable tastes, air-puffs and shocks, all resulting in a negative impact on the total amount of food pellets to be gained by the subject (e.g. De Visser et al. 2011c; Purgert et al. 2012; Voikar et al. 2010). Nonetheless, the punishments in rodent models are not entirely equivalent to a loss of money, as these punishments are not a real loss of rewards, hindering comparisons between animal and human models. Therefore, to satisfy these comparisons adjustments of human tasks to use food or liquids instead of monetary rewards could be further explored (e.g. Jimura et al. 2013; Valentin et al. 2007). The fact that animals only work for food when they are hungry, and thus have to be food deprived to increase their motivation to perform the task, leads to the following aspect of using animal models to be discussed, i.e. arousal. Food deprivation itself – especially when sustained – may lead to stress, dependent on the method and severity of the deprivation protocol. Particularly, clear effects involving stress systems are seen in anticipation of access to food (Belda et al. 2005; Díaz-Muñoz et al. 2000; Krieger, 1974; Poulin and Timofeeva, 2008). Still, it should be mentioned that also when testing human subjects some acute arousal may be present, not related to any deprivation protocols, but rather to the exciting and unpredictable/uncontrollable features of the test setting as such. Yet, this is easier to be controlled for by e.g. increased predictability through oral or written task descriptions and knowledge that a test situation is not similar to real life. Apart from arousal related to food deprivation, also other ways of enhanced

134 General Discussion

arousal can be inherent to the task design of an animal model. For instance, handling the animals prior to testing can influence their behaviour. A period of habituation to the experimenter to reduce arousal can therefore be considered, depending on whether or not one expects arousal to be a necessary element to see behavioural effects (review: Roozendaal et al. 2006). However, even a period of handling will not fully rule out the influence of handling by this person. To rule out the impact of experimenters, methods have therefore been developed to test animals in their own environment, i.e. with home-cage systems, without interference of the experimenter (e.g. De Visser et al. 2006; Knapska et al. 2006). In chapters 2 and 3 it was shown that reward-based decision-making tasks can be adapted to home cage testing situations. However, a drawback of these systems as yet is that animals have to be housed individually, leading to changes in reward sensitivity due to isolation stress (Van den Berg et al. 1999). Especially during juvenile and adolescent periods, social deprivation may greatly disrupt normal development (Leussis and Andersen 2008; Van den Berg et al. 1999) and impair cognitive control (Baarendse et al. 2013). To tackle this problem, Zoratto et al. (2013) developed a living-together/testing-apart system, only separating the animals during a short testing period daily, which is a promising step forward for future home-cage systems. Another methodological difference between the animal models used in this thesis and the original human based tasks, is that the duration of the tests in the animal models used here is much longer. In human tasks, testing lasts one single session and is preceded by a written or oral explanation of the assignment. In animal models instead, rats and mice may need to be trained beforehand in order to make them familiar with the different task features before actual testing commences (e.g. delay-discounting); these features include where to find a food reward, that lever-pressing/nose-poking leads to food access and that differences may exist in the number of food-pellets between right and left levers/nose-poking holes. In addition, even without a training phase tests may take several days to be completed (e.g. rIGT). It is unclear whether these differences in task duration (one session versus multiple sessions across several days) have a strong effect on the final results. However, at least in the IGT human subjects and rats show a similar profile across the trial-blocks of the task, which seems thus to be independent from task duration. Nevertheless, task duration might have a strong impact on pharmacological intervention paradigms (acute in humans versus multi-days intervention in rodents). However, when comparing the effect of a longer lasting state on decision-making performance, similar results in rats and humans can be obtained. For example, our preclinical findings in rats showing that lowered serotonin levels affect rIGT performance (Koot et al. 2012, chapter 4) are in agreement with data from human studies, indicating a role for serotonin in IGT performance (He et al. 2010; Homberg et al. 2008; Stoltenberg and Vandever, 2010). Apart from the methodological remarks on the use of animal models in this thesis, it should be noticed that animal models are not available for all human characteristics reflected in decision-making. For example, complex aspects of human decision-making, like the involvement of certain values and principles, pressure of modulation by complex social environments, and cross-cultural differences, are hard to capture in animal models. Another limitation of the studies presented here, is that all subjects – humans and rodents – have been tested during young adulthood. As such one cannot directly generalise the results to juvenile, adolescent or aging subjects.

135 The animal models that were used here, gave us insight into factors influencing decision-making, but certain aspects have to be kept in mind when using them. Apart from the fact that human choice behaviour is modelled, the behaviour of each species should be considered within its own context. The models used here seem to demand biologically relevant behaviour in both mice and rats regarding foraging behaviour. If we would wish to model social decision-making, differences in the social organisational structure in different species should be taken into account. In both rats and mice, female subjects are social, and require social housing, but the male subjects of these species differ: while male rats generally are social, male mice may show high levels of aggression towards male conspecifics (Van Loo et al. 2001). Thus, although human and non-human mammals used here share several characteristics, it should be kept in mind when comparing behavioural results across species that rats are not humans and that mice are not rats.

3 Stress and decision-making

8 In this thesis the effects of stress and corticosterone on reward-based and social decision-making were explored. Systemically applied corticosterone disrupted reward-based decision-making in rats measured (relatively) shortly after application (chapter 5), a finding which was replicated when corticosterone was administered locally into the infralimbic part of the prefrontal cortex (chapter 6). This adds up to a growing body of literature showing the effects of stress and glucocorticoids on individual reward-based decision-making. Especially in stressful jobs, like in the police service, the army, and medical and financial sectors (e.g. Arora et al. 2010; Coates and Herbert 2008; LeBlanc, 2009) making decisions under stress is part of the daily work. Increasingly, studies have appeared which have tried to elucidate the effects of stress on reward-based decision-making (review: Starcke and Brand, 2012). Stress has been repeatedly linked to suboptimal decision-making. Thus, in decision-making tasks stressed subjects consider less options and are faster in making their choices (Chajut and Algom, 2003; Keinan, 1987), are less likely to consider possible contingencies (Johnston et al. 1997), need more time to learn cost-benefit contingencies of different choice options (Preston et al. 2007)and automatic responses are adjusted insufficiently to a variable context (Kassam et al. 2009; Porcelli and Delgado, 2009). Shortly after corticosteroid hormone levels rise, instrumental and habitual behaviour is promoted (Schwabe et al. 2010). In general, men seem to be more risk-taking when stressed, while women become more risk averse or task-focussed (Lighthall et al. 2009; Van den Bos et al. 2009a; Preston et al. 2007) (see section 4 for more in-depth discussion). Thus, men became more risk- taking under acutely stressful conditions in both the IGT (Preston et al. 2007; Van den Bos et al. 2009a) and other decision-making tasks (Balloon Analogue Risk Task, Lighthall et al. 2009; Game of Dice Task, Starcke et al. 2008; Starcke and Brand, 2012). Salivary cortisol levels were inversely related to decision-making performance in the IGT (Van den Bos et al. 2009a). However, it should be noted that whether an altered decision-making strategy or outcome is adaptive or detrimental, depends on the situation or specific task presented. For example, in one task reacting more habitual as a consequence of stress can be rewarding and/or highly adaptive, while in another

136 General Discussion

task it is necessary to stay goal-directed and use higher cognitive functions to solve the problem. The majority of research on reward-based decision-making has examined individual decisions in which the subject has to consider exclusively his/her own values and preferences in order to select an option. However, in reality, decisions are often made in a context of social interactions (Fehr and Fischbacher, 2003). These social decisions can be defined as decisions that affect others as well as one self and are therefore typically informed by both self and other-regarding preferences (Fehr and Camerer, 2007). Social decision-making can induce an emotional response in itself (Sanfey et al. 2003; Takagishi et al. 2009; van ’t Wout et al. 2006), but vice versa stress and emotion are known to alter altruistic punishment strategies and social decision-making (Takahashi, 2005). For instance, sadness induced by a movie clip resulted in increased rejection rates of unfair (but not fair) offers in the Ultimatum Game (Harlé and Sanfey, 2007) and cortisol levels in response to stress were found to correlate positively with egotistic decision-making in emotional moral dilemmas (Starcke et al. 2011). Recently, we found that 75 min after stress compared to directly after stress men showed a lower propensity for altruistic punishment (Vinkers et al. 2013). As reported in this thesis, we did not observe this effect of stress in women on altruistic punishment and trust (chapter 7). The absence of influence of stress on social decision-making in women, in contrast to men, may be explained by the use of hormonal contraceptives, which was associated with a more blunted stress-induced cortisol response, possibly reducing the stress effect on social decision-making. Furthermore, the social decision-making tasks may have represented such a complex situation, as compared to individual reward-based decision-making, that too much individual variations among participants were induced, preventing to observe any treatment differences. As people make the majority of their decisions in a social context, there is a clear need to enhance our understanding of when and how stress affects social decision-making. Stress-induced changes in behaviour may follow a temporal pattern (de Kloet et al. 2005). Shortly after stress, catecholamines and the fast (non-genomic) effects of corticosteroids are likely to be involved in the rapid adjustment of behaviour (Joëls and Baram, 2009), promoting instrumental and habitual behaviour (Schwabe et al. 2010). In contrast, several hours after stress exposure, the slower (genomic) cascade has fully developed, which (among other things) contributes to the overall restoration of homeostasis and allows more flexible behaviour aimed at long-term goals (de Kloet et al. 2005; Diamond et al. 2007; Henckens et al. 2010, 2011, 2012; Williams and Gordon, 2007). Our findings support that these temporal patterns are to some extent also observed with regard to decision-making processes. In male rats, reward- based decision-making was impaired shortly after corticosterone administration but not when tested 3h later (chapter 5). Moreover, in healthy male individuals, a lower propensity for altruistic punishment was found 75 min after stress compared to directly after stress (Vinkers et al. 2013). Notably, no significant correlations were found between these behavioural effects and the cortisol-response. Moreover, we did not observe a temporal effect of stress on social decision-making in women (chapter 7). Here I focused only on the acute effects of stress in humans and a three days corticosterone treatment paradigm in rats. This is different from chronic stress or

137 several weeks of corticosterone administration and thus may affect decision-making differently. Chronic stress biased decision-making strategies in humans toward habits, as choices of stressed subjects became insensitive to changes in outcome value (Soares et al. 2012). Chronically stressed rats became insensitive to changes in outcome value and resistant to changes in action-outcome contingency (Dias- Ferreira et al. 2009). As the burden of chronic stress exposure is increasing in modern society, with potential relevance for stress-related psychopathology (e.g. depression and post-traumatic stress disorder), more research is needed to understand the effect of chronic stress on decision-making.

As is evident from the discussion above, some questions remain. First, I focused here on the effect of stress on fairly straightforward reward-based and social decision- making tasks. However, in real life decision-making is often more complex, involving several aspects at the same time, e.g. social context, long-term versus short-term consequences, private versus professional life, current health status (including chronic stress) et cetera. Some studies have already examined stress in “real life” situations such as the financial stock market (Coates and Herbert 2008), in a medical 8 context (Arora et al. 2010; LeBlanc 2009) and during an assessment-procedure at the police academy (Van den Bos et al. submitted). However, such studies are still incidental. Therefore, they could be expanded to more structured (longitudinal) studies to be able to relate the knowledge generated under laboratory conditions to real life situations. Secondly, in this thesis a stress task as well as application of corticosterone to model real life stress was used. However, it should be noted that corticosteroids alone do not necessarily render the same outcome on behaviour as stress. Some effects of corticosteroids are only present in combination with noradrenergic activation (Roozendaal et al. 2006; Schwabe et al. 2011, 2012b). It is quite likely that together with corticosterone administration to rats in the rIGT (chapters 5 and 6) also (nor) adrenergic activation occurred, due to the (at least slight) arousal related to food deprivation, injection procedure, and possibly expectation of the reward-based task. Therefore, as corticosteroids are not equal to stress, in order to make this translation also the effect of stress on these tasks should be examined. Third, the focus of this thesis was on the effect of stress on decision-making. However, stress has also been shown to affect other cognitive domains, such as memory (review: Schwabe et al. 2012a). As these aspects are vitally important for healthy daily functioning, the effect of stress on these domains should be explored further, also in conjunction with decision-making. Lastly, it should be mentioned that not everybody is affected by stress tothe same extent. Whether, and how much, an individual is affected by stress in later life greatly depends on its genetic background, pre- and neonatal conditions and experiences during development towards adulthood. More and more studies are pointing towards how genetic background and environmental factors interact to lead to a certain phenotype. Particularly the HPA axis has been shown to be susceptible to developmental programming (Champagne et al. 2009). For example, Van Hasselt et al. (2012) showed in rats how individual differences in rearing conditions (level of care received by the mother) affect reward-based decision-making in later life as measured in the rIGT. This, and several other data (e.g. Koolhaas et al. 2010; Oitzl et

138 General Discussion

al. 2010; Pruessner et al. 2004; Rodrigues et al. 2011), point towards the importance of incorporating individual differences in studies into stress and decision-making, e.g. coping mechanisms and how this affects among others decision-making and other daily tasks. To achieve this, both animal studies modelling the individual’s life from the genetic basis to environmental conditions into adulthood as well as longitudinal studies on the development of children in modern society are needed.

4 Gender and decision-making

In the majority of decision-making studies only the behaviour of male subjects has been investigated. However, recently more attention has been paid to gender differences in reward-based decision-making performance (Van den Bos et al. 2013) in both humans (Bolla et al. 2004; Overman et al. 2004, 2006; Reavis and Overman, 2001; Van den Bos et al. 2007, 2013) and animals (Van den Bos et al. 2006a, 2007, 2012b). Whether and how male and female subjects differ behaviourally, depends on the type of task and what aspect has been measured. No differences were observed in risk-based decision-making as measured in the Game of Dice and Balloon Analogue Risk Task (Starcke et al. 2008; Lighthall et al. 2009, 2012), while in the Cambridge Gambling Task gender differences were found in risk-adjustment only, not in risk-taking or impulsivity (Van den Bos et al. 2012a; Deakin et al. 2004). In the Iowa Gambling Task, men learn to differentiate advantageous from disadvantageous options more rapidly than women, who seem to continue to explore the different options (Bolla et al. 2004; Overman et al. 2004, 2006; Reavis and Overman, 2001; Van den Bos et al. 2006a, 2007, 2012b, 2013). It seems that under conditions of ambiguity males acquire information more globally, while females decide on more detailed information (Van den Bos et al. 2013). This can be related to differences in hemispheric activity, with men being more right-oriented, associated with more global information, whereas women are more left-oriented, associated with more detailed information (Andreano and Cahill, 2009; Cahill, 2006). Furthermore, men are more risk-prone than women in a winning situation (Sapienza et al. 2009; Schubert et al. 1999), while women are more risk-seeking than men when tested in a loss-gambling context (Powell and Ansic, 1997; Schubert et al. 1999). Following from this, one cannot claim that one of the genders is performing better or worse than the other, but rather that men and women seem to process information in a different way and use different strategies to solve the same task (Van den Bos et al. 2013). The task context defines whether the expressed behaviour is optimal or less than optimal in that particular situation. In addition when these different strategies escalate, the consequences may differ. For example, excessive focus on details in women can lead to fretting or procrastination of taking decisions, and in men the focus on global aspects can result in taking irresponsible risks due to insufficient information collection (e.g. a factor contributing to gambler’s fallacy; Goodie and Fortune, 2013) A reason why at times gender differences are found and sometimes not, could be dependent on the female population used. Often when gender differences were found, women were naturally cycling (Lighthall et al. 2009, 2012). The use of contraceptives may have been one reason why I did not observe an effect of stress

139 on social decision-making in women, as opposed to what we found in men (Vinkers et al. 2013). Thus, the lack of stress effects on social decision-making in women might be explained by oral contraceptive use, leading to blunted HPA-axis responses and secondary to reduced stress effects on decision-making. Similarly, stress effects on memory performance were observed in men and absent in women (Cornelisse et al. 2011).

In the field of gender differences in decision-making much remains to be explored. The following aspects may need to be considered. First, currently all decision- making tasks used when examining gender differences incorporate monetary rewards, mainly in individual settings, and when in a social setting, involving clear competition. In some sense, these tasks seem to be very “male biased”, focused on either the individual gains or when in a social context involving competition rather than e.g. cooperation. If one wishes to study gender differences in a more “objective” way, more “unisex” (as opposed to sex-specific, even unconsciously) oriented tasks or modulation of existing tasks should be considered, involving features equally important or rewarding to both genders. For example, monetary rewards could be 8 (partially) substituted with food (e.g. Valentin et al. 2007) or social rewards (pictures of positive and negative faces; Lin et al. 2012). Also, studies into social decision- making can be expanded towards a more “unisex” domain, involving e.g. more the social context and hierarchy, rather than (only) linear competition contexts. Second, when studying gender differences the effects of cycle and use of contraceptives should be evaluated, as it seems that hormonal status can have an impact on certain domains. Until now, very often the effect thereof on behaviour has not been tested statistically or information on hormonal status is completely lacking. Especially in stress paradigms this can make a difference in response (e.g. chapter 7). To be able to do so, sufficient attention should be paid to the sample size and distribution, i.e. preferably a sufficient number of both naturally cycling women and females using contraceptives. Finally, studies on decision-making in rodents are still greatly lacking in female animals. Due to the cycle (4-5 days) researchers are often reluctant to use female mice and rats. The potential variation induced by the cycle could be overcome by testing animals in the same phase of the cycle, including a sufficient number of subjects in the cohort or by ovariectomizing them, with or without hormone substitution. Each strategy has its own advantages and drawbacks, e.g. high levels of discomfort for the tested subjects, labour intensive and more expensive, but especially in the light of psychopathology the prevalence of certain disorders in women compared to men, i.e. depression, anxiety and eating disorders (Kendler et al. 1995), should be taken into account when developing a valid animal model to study these disorders. To be able to say something meaningful on the development or progress of a certain disease, both genders should be studied. Note that with discussing gender differences in this thesis, the focus was on biologically founded differences rather than culturally or socio-economically (Andreano and Cahill, 2009). Biological differences do not lead to cultural differences per se, and moreover, being aware of biological differences can prevent one from reacting in an automated way.

140 General Discussion

5 Final remarks

The overall aim of this thesis was to enhance our understanding of the relevance of gender and stressful conditions on decision-making, in rodents and in humans. In mice and rats, gender, internal state, and the level of handling prior to testing could affect the outcome of delay-discounting or reward-based decision-making tasks. Therefore home cage systems were developed to measure choice behaviour in the animal’s own environment. Furthermore, corticosterone disrupted decision-making in rats, which involved the infralimbic part of the prefrontal cortex. Psychosocial stress did affect altruistic punishment or general altruism in men but not in women. Taking important decisions under stressful conditions and in complex social settings is a situation that everyone faces every now and then. A better understanding of how behaviour is affected in men and in women and more insight in the brain networks involved in such decision-making processes under specific circumstances can be of great societal relevance. Apart from improving predictive models of how individuals will respond, it opens up new avenues for knowledge-based behavioural or pharmacological intervention, improving the adaptive capacity of individuals, to the benefit of the society at large.

141 142 REFERENCES

143 A Adriani W, Granstrem O, Shumilina M, Adolphs R, Tranel D, Damasio AR (1998). Ognibene E, Attilia ML, Ceccanti M, et al. The human amygdala in social judgment. (2008). Blood titers of auto-antibodies to Nature 393: 470–474. DAT as biomarkers for risk of alcoholism: Adriani W, Chiarotti F, Laviola G (1998). the case of positive familiarity. In: Needs Elevated novelty seeking and peculiar and challenges for translational medicine, d-amphetamine sensitization in ISS; October 2008. periadolescent mice. Behav Neurosci Adriani W, Boyer F, Gioiosa L, Macrì S, Dreyer 112: 1152-1166. JL, Laviola G (2009). Increased impulsive Adriani W, Canese R, Podo F, Laviola G behavior and risk proneness following (2007). 1H MRS-detectable metabolic lentivirus-mediated dopamine brain changes and reduced impulsive transporter over-expression in rats’ behavior in adult rats exposed to nucleus accumbens. Neuroscience methylphenidate during adolescence. 159: 47-58. Neurotoxicol Teratol 29: 116-125. Adriani W, Boyer F, Leo D, Canese R, Podo Adriani W, Caprioli A, Granstrem O, Carli F, Perrone-Capano C, Dreyer JL, Laviola G M, Laviola G (2003a). The spontaneously (2010). Social withdrawal and gambling- hypertensive-rat as an animal model like profile after lentiviral manipulation of of ADHD: evidence for impulsive and DAT expression in the rat accumbens. Int non-impulsive subpopulations. Neurosci J Neuropsychopharmacol. 13: 1329-1342. Biobehav Rev 27: 639-651. Adriani W, Laviola G (2009). Animal Adriani W, della Seta D, Dessì-Fulgheri models and mechanisms of impulsivity in F, Farabollini F, Laviola G (2003b). Altered adolescence. In T. Palomo, R. Beninger, profiles of spontaneous novelty seeking, T. Archer, & R. Kostrezwa (Eds.), Beyond impulsive behavior, and response to neuropsychiatric diagnostics: Symptoms d-amphetamine in rats perinatally not disorders (Strategies for Studying exposed to bisphenol A. Environ Health Brain Disorders, Vol. 9, p. 385-434) Perspect 111: 395-401. Madrid: Editorial CYM. Adriani W, Laviola G (2003). Elevated Adriani W, Zoratto F, Laviola G (2012). Brain levels of impulsivity and reduced place processes in discounting: consequences conditioning with d-amphetamine: of adolescent methylphenidate exposure. Two behavioral features of adolescence Curr Top Behav Neurosci. 9: 113-143. in mice. Behavioral Neuroscience Alexander JK, Hillier A, Smith RM, Tivarus 117: 695-703. ME, Beversdorf DQ (2007). Beta- Adriani W, Rea M, Baviera M, Invernizzi adrenergic modulation of cognitive W, Carli M, Ghirardi O, Caprioli A, Laviola flexibility during stress. J Cogn Neurosci. G (2004). Acetyl-L-carnitine reduces 19: 468-478. impulsive behaviour in adolescent rats. American Psychiatric Association A. P. A. Psychopharmacology 176: 296-304. (2000). Diagnostic and Statistical Manual Adriani W, Laviola G (2006). Delay aversion of Mental Disorders (fourth ed., text but preference for large and rare rewards revision). Washington, DC, USA. in two choice tasks: implications for the Anderson IM, Richell RA, Bradshaw CM measurement of self-control parameters. (2003). The effect of acute tryptophan BMC Neurosci 7:52 depletion on probabilistic choice. J Psychopharmacol. 17: 3-7.

144 REFERENCES

Andreano JM, Cahill L (2009). Sex influences Bechara A (2004). The role of emotion in on the neurobiology of learning and decision-making: Evidence from memory. Learning and Memory neurological patients with orbitofrontal 16: 248–266. damage. Brain Cogn. 55: 30-40. Arnsten AF (2009). Stress signalling Bechara A (2005). Decision making, impulse pathways that impair prefrontal cortex control and loss of willpower to resist structure and function. Nat Rev Neurosci. drugs: a neurocognitive perspective. 10: 410-422. Nat Neurosci. 8: 1458-1463. Arora S, Sevdalis N, Nestel D, Bechara A, Damasio AR, Damasio H, Woloshynowych M, Darzi A, Kneebone R Anderson SW (1994). Insensitivity to (2010). The impact of stress on surgical future consequences following damage to performance: a systematic review of the human prefrontal cortex. Cognition literature. Surgery 147: 318-330 50: 7-15. Bechara A, Damasio H, Tranel D, Anderson B SW (1998). Dissociation of working Baarendse PJ, Counotte DS, O’Donnell memory from decision making within P, Vanderschuren LJ (2013). Early social the human prefrontal cortex. J Neurosci. experience is critical for the development 18: 428-437. of cognitive control and dopamine Bechara A, Damasio H, Damasio AR, Lee GP modulation of prefrontal cortex function. (1999). Different contributions of the Neuropsychopharmacology 38: 1485- human amygdala and ventromedial 1494. prefrontal cortex to decision-making. Baldwin D, Rudge S (1995). The role of J Neurosci. 19: 5473-5481. serotonin in depression and anxiety. Int Bechara A, Damasio H, Damasio AR, Lee GP Clin Psychopharmacol. 4: 41-45. (1999). Different contributions of the Ball A, Wolf CC, Ocklenburg S, Herrmann human amygdala and ventromedial BL, Pinnow M, Brüne M, Wolf OT, prefrontal cortex to decision-making. Güntürkün O (2013). Variability in ratings J Neurosci. 19: 5473-5481. of trustworthiness across the menstrual Bechara A, Damasio H, Damasio AR cycle. Biol Psychol. 93: 52-57. (2000). Emotion, decision making and Balleine BW, O’Doherty JP (2010). the orbitofrontal cortex. Cereb Cortex Human and rodent homologies in action 10: 295-307. control: corticostriatal determinants Bechara A, Dolan S, Hindes A (2002). of goal-directed and habitual action. Decision-making and addiction (part II): Neuropsychopharmacology 35: 48-69. myopia for the future or hypersensitivity Barsegyan A, Mackenzie SM, Kurose to reward? Neuropsychologia 40: 1690- BD, McGaugh JL, Roozendaal B (2010). 1705. Glucocorticoids in the prefrontal cortex Bechara A, Damasio H, Damasio AR (2003). enhance memory consolidation and Role of the amygdala in decision-making. impair working memory by a common Ann N Y Acad Sci. 985: 356-369. neural mechanism. Proc Natl Acad Sci U S Belda X, Ons S, Carrasco J, Armario A (2005). A. 107: 16655-16660. The effects of chronic food restriction on BeattyWW (1979). Gonadal hormones hypothalamic-pituitary-adrenal activity and sex differences in nonreproductive depend on morning versus evening behaviours in rodents: organizational availability of food. Pharmacol Biochem and activational influences. Horm Behav Behav. 81: 41-46. 12: 112-163.

145 Biggio G, Fadda F, Fanni P, Tagliamonte Brand M, Grabenhorst F, Starcke K, A, Gessa GL (1974). Rapid depletion of Vandekerckhove MM, Markowitsch HJ serum tryptophan, brain tryptophan, (2007). Role of the amygdala in decisions serotonin and 5-hydroxyindoleacetic acid under ambiguity and decisions under risk: by a tryptophan-free diet. Life Sci. 14: evidence from patients with Urbach- 1321-1329. Wiethe disease. Neuropsychologia Birrell JM, Brown VJ (2000). Medial frontal 45: 1305-1317. cortex mediates perceptual attentional Brand M, Kalbe E, Labudda K, Fujiwara set shifting in the rat. J Neurosci. 20: E, Kessler J, Markowitsch HJ (2005). 4320-4324. Decision-making impairment in patients Bizot J, Le Bihan C, Puech AJ, Hamon with pathological gambling. Psychiatry M, Thiébot M (1999). Serotonin and Res. 133: 91-99. tolerance to delay of reward in rats. Brand M, Labudda K, Markowitsch HJ (2006). Psychopharmacology 146: 400–12. Neuropsychological correlates of Bohn I, Giertler C, Hauber W (2003). decision-making in ambiguous and risky Orbital prefrontal cortex and guidance situations. Neural Netw. 19: 1266-1276. of instrumental behaviour in rats under Brand M, Recknor EC, Grabenhorst reversal conditions. Behav Brain Res. F, Bechara A (2007). Decisions under 143: 49-56. ambiguity and decisions under risk: Bolla KI, Eldreth DA, Matochik JA, Cadet JL correlations with executive functions and (2004). Sex-related differences in a comparisons of two different gambling gambling task and its neurological tasks with implicit and explicit rules. J Clin correlates. Cereb Cortex 14: 1226-1232. Exp Neuropsychol. 29: 86-99. Bonini N, Hadjichristidis C, Mazzocco K, Brogan A, Hevey D, Pignatti R (2010). Demattè ML, Zampini M, Sbarbati A, Anorexia, bulimia, and obesity: shared Magon S (2011). Pecunia olet: the role of decision making deficits on the Iowa incidental disgust in the ultimatum game. Gambling Task (IGT). J Int Neuropsychol Emotion 11: 965-969. Soc. 16: 711-715. Bradshaw CM, Szabadi E (1992). Choice Broida J, Svare B (1984). Sex differences in between delayed reinforcers in a discrete- the activity of mice: modulation by trials schedule: the effect of deprivation postnatal gonadal hormones. Horm level. Q J Exp Psychol B 44: 1-6. Behav 18: 65-78. Branchi I (2011). The double edged sword of Brown VJ, Bowman EM (2002). Rodent neural plasticity: increasing serotonin models of prefrontal cortical function. levels leads to greater vulnerability to Trends Neurosci. 25: 340-343. depression and improved capacity to Brown MP, Toptygin D, Lee KB, Animashaun recover. Psychoneuroendocrinology T, Hughes RC, Lee YC, Brand L (1998). 36: 339-351. The tryptophan fluorescence of Brand M, Fujiwara E, Borsutzky S, Kalbe E, Tetracarbidium conophorum agglutinin Kessler J, Markowitsch HJ (2005). II and a solution-based assay for the Decision-making deficits of korsakoff binding of a biantennary glycopeptide. patients in a new gambling task with J Protein Chem. 17: 149-159. explicit rules: associations with executive Burnham TC (2007). High-testosterone men functions. Neuropsychology 19: 267-277. reject low ultimatum game offers. Proc Biol Sci. 274: 2327-2330.

146 REFERENCES

Buser T (2012a). Digit ratios, the menstrual Cardinal RN (2006). Neural systems cycle and social preferences. Games and implicated in delayed and probabilistic Economic Behavior 76: 457-470. reinforcement. Neural Netw 19: 1277- Buser T (2012b). The impact of 1301. the menstrual cycle and hormonal Cardinal RN, Cheung TH (2005). Nucleus contraceptives on competitiveness. accumbens core lesions retard the J Econ Behav Organ. 83: 1-10. instrumental learning and performance Butts KA, Weinberg J, Young AH, Phillips with delayed reinforcement in the rat. AG (2011). Glucocorticoid receptors in BMC Neurosci 2005, 6:9. the prefrontal cortex regulate stress- Cardinal RN, Howes NJ (2005). Effects of evoked dopamine efflux and aspects of lesions of the nucleus accumbens core executive function. Proc Natl Acad Sci U S on choice between small certain rewards A. 108: 18459-18464. and large uncertain rewards in rats. BMC Neurosci 2005, 6:37. C Cardinal RN, Parkinson JA, Marbini HD, Cabib S, Bonaventura N (1997). Parallel Toner AJ, Bussey TJ, Robbins TW, Everitt strain-dependent susceptibility to BJ (2003). Role of the anterior cingulate environmentally-induced stereotypes and cortex in the control over behavior by stress-induced behavioural sensitization Pavlovian conditioned stimuli in rats. in mice. Physiol Behav 61: 499-506. Behav Neurosci. 117: 566-587. Cabib S, Orsini C, Le Moal M, Piazza PV Cardinal RN, Pennicott DR, Sugathapala (2000). Abolition and reversal of strain CL, Robbins TW, Everitt BJ (2001). differences in behavioral responses to Impulsive choice induced in rats by drugs of abuse after a brief experience. lesions of the nucleus accumbens core. Science 289: 463-465. Science 292: 2499-2501. Cahill L (2006). Why sex matter for Cardinal RN, Winstanley CA, Robbins TW, neuroscience. Nature Reviews Everitt BJ (2004). Limbic corticostriatal Neuroscience 7: 477-484. systems and delayed reinforcement. Ann Canese R, Marco EM, De Pasquale F, Podo N Y Acad Sci 1021: 33-50. F, Laviola G, Adriani W (2011). Differential Carli M, Robbins TW, Evenden JL, Everitt response to specific 5-Ht(7) versus whole- BJ (1983). Effects of lesions to ascending serotonergic drugs in rat forebrains: a noradrenergic neurones on performance phMRI study. Neuroimage 58: 885-894. on a 5-choice serial reaction task in Capone F, Adriani W, Shumilina M, Izykenova rats: implications for theories on dorsal G, Granstrem O, Dambinova S, Laviola noradrenergic bundle function based on G (2008). Autoantibodies against opioid selective attention and arousal. Behav or glutamate receptors are associated Brain Res 9: 361-380. with changes in morphine reward Carragher N, McWilliams LA (2011). and physical dependence in mice. A latent class analysis of DSM-IV criteria Psychopharmacology 197: 535-548. for pathological gambling results from the Caprioli D, Celentano M, Dubla A, Lucantonio National Epidemiologic Survey on alcohol F, Nencini P, Badiani A (2009). Ambience and related conditions. Psychiatry Res. and drug choice: Cocaineand heroin- 187, 185-192. taking as a function of environmental Carver CS, Scheier MF, Weintraub JK (1989). context in humans and rats. Biol Assessing coping strategies: a Psychiatry 65: 893-899. theoretically based approach. J Pers Soc Psychol. 56: 267-283.

147 Carver CS, White TL (1994). Behavioral Chudasama Y, Robbins TW (2003). inhibition, behavioral activation, and Dissociable contributions of the affective responses to impending reward orbitofrontal and infralimbic cortex to and punishment: the BIS/BAS scales. pavlovian autoshaping and discrimination J Pers Soc Psychol. 67: 319-333. reversal learning: further evidence for the Cassano T, Gaetani S, Morgese MG, Macheda functional heterogeneity of the rodent T, Laconca L, Dipasquale P, Taltavull frontal cortex. J Neurosci. 23: 8771-8780. J, Shippenberg TS, Cuomo V, Gobbi G Clark L, Bechara A, Damasio H, Aitken MR, (2009). Monoaminergic changes in locus Sahakian BJ, Robbins TW (2008). coeruleus and dorsal raphe nucleus Differential effects of insular and following noradrenaline depletion. ventromedial prefrontal cortex lesions Neurochem Res 34: 1417-1426. on risky decision-making. Brain 131: Cavedini P, Riboldi G, Keller R, D’Annucci 1311-1322. A, Bellodi L (2002). Frontal lobe Clark L, Manes F, Antoun N, Sahakian dysfunction in pathological gambling BJ, Robbins TW (2003). The contributions patients. Biol Psychiatry 51: 334-341. of lesion laterality and lesion volume to Cazakoff BN, Johnson KJ, Howland JG (2010). decision-making impairment following Converging effects of acute stress on frontal lobe damage. Neuropsychologia spatial and recognition memory in 41: 1474-1483. rodents: a review of recent behavioural Clarke HF, Dalley JW, Crofts HS, Robbins and pharmacological findings. Prog TW, Roberts AC (2004). Cognitive Neuropsychopharmacol Biol Psychiatry inflexibility after prefrontal serotonin 34: 733-741. depletion. Science 304: 878-880. Chajut E, Algom D (2003). Selective attention Clarke HF, Robbins TW, Roberts AC (2008). improves under stress: implications for Lesions of the medial striatum in theories of social cognition. J Pers Soc monkeys produce perseverative Psychol 85: 231-248. impairments during reversal learning Champagne DL, de Kloet ER, Joëls M (2009). similar to those produced by lesions Fundamental aspects of the impact of of the orbitofrontal cortex. J Neurosci glucocorticoids on the (immature) brain. 28: 10972-10982. Semin Fetal Neonatal Med. 14: 136-142. Coates JM, Herbert J (2008). Endogenous Chao HM, Choo PH, McEwen BS (1989). steroids and financial risk taking on a Glucocorticoid and mineralocorticoid London trading floor. Proc Natl Acad Sci receptor mRNA expression in rat brain. U S A. 105: 6167-6172. Neuroendocrinology 50: 365-371. Cohen MX, Heller AS, Ranganath C (2005). Christakou A, Robbins TW, Everitt BJ (2001). Functional connectivity with anterior Functional disconnection of a prefrontal cingulate and orbitofrontal cortices cortical-dorsal striatal system disrupts during decision-making. Cogn Brain Res choice reaction time performance: 23: 61-70. implications for attentional function. Conrad CD (2010). A critical review of Behav Neurosci 115: 812-825. chronic stress effects on spatial Christakou A, Robbins TW, Everitt BJ learning and memory. Prog. (2004). Prefrontal cortical-ventral Neuropsychopharmacol. Biol Psychiatry striatal interactions involved in affective 34: 742-755. modulation of attentional performance: implications for corticostriatal circuit function. J Neurosci 24: 773-780.

148 REFERENCES

Contreras M, Ceric F, Torrealba F (2007). D Inactivation of the interoceptive insula d’Acremont M, Van der Linden M (2006). disrupts drug craving and malaise Gender differences in two decision induced by lithium. Science 318: 655-658. making tasks in a community sample Cools R, Nakamura K, Daw ND (2011). of adolescents. International Journal of Serotonin and dopamine: unifying Behavioural Development 30: 352-358. affective, activational, and decision da Rocha FF, Malloy-Diniz L, Lage NV, functions. Neuropsychopharmacology 36: Romano-Silva MA, de Marco LA, Correa 98-113. H (2008). Decision-making impairment is Cools R, Roberts AC, Robbins TW (2008). related to serotonin transporter promoter Serotoninergic regulation of emotional polymorphism in a sample of patients and behavioural control processes. with obsessive compulsive disorder. Trends Cogn Sci 12: 31-40. Behav Brain Res 195: 159-163. Cornelisse S, van Stegeren AH, Joëls M Dalla C, Antoniou K, Drossopoulou (2011). Implications of psychosocial G, Xagoraris M, Kokras N, Sfikakis A, stress on memory formation in a typical Papadopoulou-Daifoti Z. (2005). Chronic male versus female student sample. mild stress impact: are females more Psychoneuroendocrinology 36: 569-578. vulnerable? Neuroscience 135: 703-714. Costall B, Fortune DH, Naylor RJ, Nohria V Dalley JW, Theobald DE, Eagle DM, Passetti (1980). The mesolimbic system, F, Robbins TW (2002). Deficits in impulse denervation and the climbing response in control associated with tonically-elevated the mouse. Eur J Pharmacol 66: 207-215. serotonergic function in rat prefrontal Crabbe JC, Wahlsten D, Dudek BC (1999). cortex. Neuropsychopharmacology Genetics of mouse behavior: Interactions 26: 716-728. with laboratory environment. Science Dallman MF, Akana SF, Bhatnagar S, Bell ME, 284: 1670-1672. Choi S, Chu A, Horsley C, Levin N, Meijer Crockett MJ, Clark L, Robbins TW (2009). O, Soriano LR, Strack AM, Viau V (1999). Reconciling the role of serotonin in Starvation: early signals, sensors, and behavioral inhibition and aversion: sequelae. Endocrinology 140: 4015-4023. acute tryptophan depletion abolishes Daw ND, Kakade S, Dayan P (2002). punishment-induced inhibition in Opponent interactions between serotonin humans. J Neurosci 29: 11993-11999. and dopamine. Neural Netw 15: 603-616. Crook D (1997). Multicenter study of de Kloet ER (1991). Brain corticosteroid endocrine function and plasma lipids receptor balance and homeostatic and lipoproteins in women using oral control. Front Neuroendocrinol contraceptives containing desogestrel 12: 95-164. progestin. UK Desogen Study Group. de Kloet ER, Oitzl MS, Joëls M (1999). Stress Contraception 55: 219-224. and cognition: are corticosteroids good or Cullinan WE, Herman JP, Battaglia DF, bad guys? Trends Neurosci 22: 422–-426. Akil H, Watson SJ (1995). Pattern and de Kloet ER, Joels M, Holsboer F (2005). time course of immediate early gene Stress and the brain: from adaptation to expression in rat brain following acute disease. Nat Rev Neurosci. 6: 463-475. stress. Neuroscience 64: 477-505. de Kloet ER, Karst H, Joëls M (2008). Corticosteroid hormones in the central stress response: quick-and-slow. Front Neuroendocrinol 29: 268-272.

149 de Quervain DJ, Roozendaal B, McGaugh JL Dell’Omo G, Vannoni E, Vyssotski AL, Di Bari (1998). Stress and glucocorticoids impair MA, Nonno R, Agrimi U, Lipp HP (2002). retrieval of long-term spatial memory. Early behavioural changes in mice Nature 394: 787-790. infected with BSE and scrapie: automated de Visser L (2008). Home sweet home: Home home cage monitoring reveals prion cage testing for behavioral phenotyping strain differences. Eur J Neurosci of mice. Unpublished doctoral thesis, 16: 735-742. Utrecht University, The Netherlands. Dellu F, Piazza PV, Mayo W, Le Moal M, de Visser L, van den Bos R, Kuurman WW, Simon H (1996). Novelty-seeking in rats - Kas MJ, Spruijt BM (2006). Novel biobehavioral characteristics and possible approach to the behavioural relationship with the sensation-seeking characterization of inbred mice: trait in man. Neuropsychobiology automated home cage observations. 34: 136-145. Genes Brain Behav 5: 458-466. Di Chiara G, Bassareo V (2007). Reward de Visser L, van der Knaap LJ, van de Loo system and addiction: what dopamine AJ, van der Weerd CM, Ohl F, van den does and doesn’t do. Curr Opin Bos R (2010). Trait anxiety affects Pharmacol 7: 69-76. decision-making differently in healthy Di Giovanni G, Esposito E, Di Matteo men and women: towards gender- V (2010). Role of serotonin in central specific endophenotypes of anxiety. dopamine dysfunction. CNS Neurosci Neuropsychologia 48, 1598-1606. Ther 16: 179-194. de Visser L, Baars AM, Lavrijsen M, van Diamond DM, Campbell AM, Park der Weerd CM, van den Bos R (2011a). CR, Halonen J, Zoladz PR (2007). The Decision-making performance is related temporal dynamics model of emotional to levels of anxiety and differential memory processing: a synthesis on the recruitment of frontostriatal areas in neurobiological basis of stress-induced male rats. Neuroscience 184: 97-106. amnesia, flashbulb and traumatic de Visser L, Baars AM, van ‘t Klooster J, van memories, and the Yerkes-Dodson law. den Bos R (2011b). Transient inactivation Neural Plast. 60803. of the medial prefrontal cortex affects Dias R, Robbins TW, Roberts AC (1996). both anxiety and decision-making in male Dissociation in prefrontal cortex of wistar rats. Front Neurosci. 5: 102. affective and attentional shifts. de Visser L, Homberg JR, Mitsogiannis M, Nature 380: 69-72. Zeeb FD, Rivalan M, Fitoussi A, Galhardo Dias-Ferreira E, Sousa JC, Melo I, Morgado P, V, van den Bos R, Winstanley CA, Dellu- Mesquita AR, Cerqueira JJ, Costa RM, Hagedorn F (2011c). Rodent versions of Sousa N. (2009) Chronic stress causes the iowa gambling task: opportunities frontostriatal reorganization and affects and challenges for the understanding of decision-making. Science 325: 621-625. decision-making. Front Neurosci. 5: 109. Díaz-Muñoz M, Vázquez-Martínez O, Deakin J, Aitken M, Robbins T, Sahakian BJ Aguilar-Roblero R, Escobar C (2000). (2004). Risk taking during decision- Anticipatory changes in liver metabolism making in normal volunteers changes and entrainment of insulin, glucagon, and with age. J Int Neuropsychol Soc. corticosterone in food-restricted rats. 10: 590-598. Am J Physiol Regul Integr Comp Physiol. 279: R2048-56.

150 REFERENCES

Diorio D, Viau V, Meaney MJ (1993). The role Ernst M, Paulus MP. (2005) Neurobiology of of the medial prefrontal cortex (cingulate decision making: a selective review from gyrus) in the regulation of hypothalamic- a neurocognitive and clinical perspective. pituitaryadrenal responses to stress. J Biol Psychiatry 58: 597-604. Neurosci. 13: 3839-3847. Evenden JL (1999). Varieties of impulsivity. Divac I, Rosvold HE, Szwarcbart MK (1967). Psychopharmacology 146: 348–361. Behavioral effects of selective ablation Evenden JL, Ryan CN (1996) The of the caudate nucleus. J Comp Physiol pharmacology of impulsive behaviour Psychol 63: 184-190. in rats: the effects of drugs on response Djordjević J, Jasnić N, Vujović P, Djurašević choice with varying delays S, Djordjević I, Cvijić G (2008). The effect of reinforcement. Psychopharmacology of fasting on the diurnal rhythm of rat 128: 161–170. acth and corticosterone secretion. Evenden JL, Ryan CN (1999). The Arch Biol Sci 60: 541-546. pharmacology of impulsive behaviour Doya K (2008). Modulators of decision in rats VI: the effects of ethanol making. Nat Neurosci 11: 410-416. and selective serotonergic drugs on Dringenberg HC, Hargreaves EL, Baker response choice with varying delays of GB, Cooley RK, Vanderwolf CH (1995). reinforcement. Psychopharmacology p-chlorophenylalanine-induced serotonin 146: 413-421. depletion: reduction in exploratory locomotion but no obvious sensory- F motor deficits. Behav Brain Res Fadda F, Cocco S, Stancampiano R (2000). 68: 229-237. A physiological method to selectively decrease brain serotonin release. Brain E Res Protoc 5: 219-222. Elliott R, Dolan RJ, Frith CD (2000a). Faure A, Richard JM, Berridge KC (2010). Dissociable functions in the medial and Desire and dread from the nucleus lateral orbitofrontal cortex: evidence accumbens: cortical glutamate and from human neuroimaging studies. subcortical GABA differentially generate Cereb Cortex 10: 308-317. motivation and hedonic impact in the rat. Elliott R, Friston KJ, Dolan RJ (2000b). PLoS One 5: e11223. Dissociable neural responses in human Fecteau S, Knoch D, Fregni F, Sultani reward systems. J Neurosci. N, Boggio P, Pascual-Leone A (2007a). 20: 6159-6165. Diminishing risk-taking behavior by Erickson K, Drevets W, Schulkin J (2003). modulation activity in the prefrontal Glucocorticoid regulation of diverse cortex: a direct current stimulation study. cognitive functions in normal and J Neurosci 27: 12500-12505. pathological emotional states. Neurosci Fecteau S, Pascual-Leone A, Zald DH, Biobehav Rev. 27: 233-246. Liguori P, Théoret H, Boggio PS, et al. Eriksson K, Simpson B (2010). Emotional (2007b)Activation of prefrontal cortex by reactions to losing explain gender transcranial direct current stimulation differences in entering a risk lottery. reduces appetite for risk during Judgement and Decision Making ambiguous decision making. 5: 159-163. J Neurosci 27: 6212-6218.

151 Fehr E, Camerer CF (2007). Social Floresco SB, Jentsch JD (2011). neuroeconomics: the neural circuitry Pharmacological enhancement of social preferences. Trends Cogn. of memory and executive Sci 11: 419-427. functioning in laboratory animals. Fehr E, Fischbacher U (2003). The nature of Neuropsychopharmacology 36: 227-250. human altruism. Nature 425: 785-791. Frank MJ, Claus ED (2006). Anatomy of a Fellows LK (2007). The role of orbitofrontal decision: striato-orbitofrontal interactions cortex in decision making: a component in reinforcement learning, decision process account. Ann N Y Acad Sci 1121: making, and reversal. Psychol Rev 421-430. 113: 300-326. Fellows LK, Farah MJ (2005). Different Frankland PW, Bontempi B (2005). The underlying impairments in decision- organization of recent and remote making following ventromedial and memories. Nat Rev Neurosci 6: 119-130. dorsolateral frontal lobe damage in Fuster JM (2001). The prefrontal cortex--an humans. Cereb Cortex 15: 58-63. update: time is of the essence. Neuron Fernandes C, González MI, Wilson CA, File 30: 319-333. SE (1999). Factor analysis shows that G female rat behaviour is characterized primarily by activity, male rats are driven Gallistel CR, Gibbon J (2000). Time, rate, by sex and anxiety. Pharmacol Biochem and conditioning. Psychological Review Behav 64: 731-738. 107: 289-344. Figueiredo HF, Bruestle A, Bodie B, Dolgas Gardner M, Steinberg L (2005). Peer CM, Herman JP (2003). The medial influence on risk taking, risk preference, prefrontal cortex differentially regulates and risky decision making in adolescence stress-induced c-fos expression in the and adulthood: an experimental study. forebrain depending on type of stressor. Dev Psychol 41: 625-635. Eur J Neurosci 18: 2357-2364. Gläscher J, Adolphs R, Damasio H, Bechara Fletcher PJ, Selhi ZF, Azampanah A, Sills TL A, Rudrauf D, Calamia M, Paul LK, Tranel (2001). Reduced brain serotonin D (2012). Lesion mapping of cognitive activity disrupts prepulse inhibition control and value-based decision making of the acoustic startle reflex. in the prefrontal cortex. Proc Natl Acad Effects of 5,7-dihydroxy-tryptamine Sci U S A 109: 14681-14686. and p-chlorophenylalanine. Goldstein JM, Jerram M, Abbs B, Whitfield- Neuropsychopharmacology 24: 399-409. Gabrieli S, Makris N (2010). Sex Floresco SB, Ghods-Sharifi S, Vexelman differences in stress response circuitry C, Magyar O (2006). Dissociable roles for activation dependent on female the nucleus accumbens core and shell hormonal cycle. J Neurosci 30: 431-438. in regulating set shifting. J Neurosci 26: Goodie AS, Fortune EE (2013). Measuring 2449-2457. Cognitive Distortions in Pathological Floresco SB, St Onge JR, Ghods-Sharifi S, Gambling: Review and Meta-Analyses. Winstanley CA (2008). Cortico-limbic- Psychol Addict Behav. doi: 10.1037/ striatal circuits subserving different forms a0031892 of cost-benefit decision making. Cogn Affect Behav Neurosci 8: 375-389.

152 REFERENCES

Goudriaan AE, Oosterlaan J, de Beurs E, Harlé KM, Sanfey AG (2007). Incidental van den Brink W (2005). Decision making sadness biases social economic in pathological gambling: a comparison decisions in the Ultimatum Game. between pathological gamblers, alcohol Emotion 7: 876-881. dependents, persons with Tourette Harrison AA, Everitt BJ, Robbins TW syndrome, and normal controls. Brain Res (1997a). Central 5-HT depletion enhances Cogn Brain Res 23: 137-151. impulsive responding without affecting Graeff FG (2002). On serotonin and the accuracy of attentional performance: experimental anxiety. interactions with dopaminergic Psychopharmacology 163: 467-476. mechanisms. Psychopharmacology Graham LK, Yoon T, Kim JJ (2010). Stress 133: 329-342. impairs optimal behavior in a water Harrison AA, Everitt BJ, Robbins TW (1997b). foraging choice task in rats. Doubly dissociable effects of median- and Learn Mem 17: 1-4. dorsal-raphe lesions on the performance Green L, Myerson J, Ostaszewski P (1999). of the five-choice serial reaction time Amount of reward has opposite effects test of attention in rats. Behav Brain Res on the discounting of delayed and 89: 135-149. probabilistic outcomes. J Exp Psychol Hayden BY, Parikh PC, Deaner RO, Platt ML Learn Mem Cogn 25: 418-427. (2007). Economic principles motivating Green L, Myerson J (2004). A discounting social attention in humans. Proc Biol Sci framework for choice with delayed and 274: 1751-1756. probabilistic rewards. Psychol Bull 130: Hayes DJ, Greenshaw AJ (2011). 5-HT 769-792. receptors and reward-related behaviour: Grimm JW, See RE (2000). Dissociation of a review. Neurosci Biobehav Rev primary and secondary reward-relevant 35: 1419-1449. limbic nuclei in an animal model of Hayes DJ, Hoang J, Greenshaw AJ (2011). relapse. Neuropsychopharmacology The role of nucleus accumbens shell 22: 473-479. GABA receptors on ventral tegmental Groeneweg FL, Karst H, de Kloet ER, Joëls area intracranial self-stimulation and a M (2011). Rapid non-genomic effects potential role for the 5-HT(2C) receptor. of corticosteroids and their role in the J Psychopharmacol 25: 1661-1675. central stress response. J Endocrinol. He Q, Xue G, Chen C, Lu Z, Dong Q, Lei X, 209: 153-167. Ding N, Li J, Li H, Chen C, Li J, Moyzis RK, H Bechara A (2010). Serotonin transporter gene-linked polymorphic region Hall FS, Humby T, Wilkinson LS, Robbins TW (5-HTTLPR) influences decision making (1997). The effects of isolation-rearing under ambiguity and risk in a large on sucrose consumption in rats. Physiol Chinese sample. Neuropharmacology Behav 62: 291-297. 59: 518-526. Hamilton KR, Potenza MN (2012). Relations Heekeren HR, Wartenburger I, Marschner among delay discounting, addictions, and A, Mell T, Villringer A, Reischies money mismanagement: implications FM (2007). Role of ventral striatum and future directions. Am J Drug Alcohol in reward-based decision making. Abuse 38: 30-42. Neuroreport 18: 951-955.

153 Henckens MJ, van Wingen GA, Joels M, Neuropharmacology 55: 80-84. Fernandez G (2010). Time dependent Hsu M, Bhatt M, Adolphs R, Tranel effects of corticosteroids on human D, Camerer CF (2005). Neural systems amygdala processing. J Neurosci responding to degrees of uncertainty in 30: 12725-12732. human decision-making. Science 310: Henckens MJ, van Wingen GA, Joëls M, 1680-1683. Fernández G (2011). Time-dependent I corticosteroid modulation of prefrontal working memory processing. Proc Natl Ibanez A, Blanco C, Perez de Castro Acad Sci U S A 108: 5801-5806. I, Fernandez-Piqueras J, Sáiz-Ruiz J (2003). Henckens MJ, van Wingen GA, Joëls M, Genetics of pathological gambling. J Fernández G (2012). Corticosteroid Gambl Stud 19: 11-22. induced decoupling of the amygdala in J men. Cereb Cortex 22: 2336-2345. Herman JP, Ostrander MM, Mueller Janitzky K, Schwegler H, Kröber A, Roskoden NK, Figueiredo H (2005). Limbic system T, Yanagawa Y, Linke R (2011). Species- mechanisms of stress regulation: relevant inescapable stress differently hypothalamo-pituitaryadrenocortical influences memory consolidation and axis. Prog Neuro-psychopharmacol Biol retrieval of mice in a spatial radial arm Psychiatry 29: 1201-1213. maze. Behav Brain Res 219: 142-148. Herrnstein RJ (1981). Self-control as Jimura K, Chushak MS, Braver TS (2013). response strength. In: Bradshaw Impulsivity and self-control during CM, Szabadi E, Lowe CF, editors. intertemporal decision making linked Quantification of steady-state operant to the neural dynamics of reward value behaviour. Amsterdam: Elsevier; p 3-20. representation. J Neurosci 33: 344-357. Ho MY, Wogar MA, Bradshaw CM, Szabadi E Joëls M, Baram TZ (2009). The neuro- (1997). Choice between delayed symphony of stress. Nat Rev Neurosci. reinforcers: interaction between delay 10: 459-466. and deprivation level. Q J Exp Psychol Joëls M, Fernandez G, Roozendaal B (2011). B: 193-202. Stress and emotional memory: a matter Ho MY, Mobini S, Chiang TJ, Bradshaw of timing. Trends Cogn Sci 15: 280-288. CM, Szabadi E (1999). Theory and Joëls M, Pu Z, Wiegert O, Oitzl MS, Krugers method in the quantitative analysis HJ (2006). Learning under stress: of “impulsive choice” behaviour: how does it work? Trends Cogn Sci implications for psychopharmacology. 10: 152-158. Psychopharmacology 146: 362-372. Joëls M, Sarabdjitsingh A, Karst H (2012). Hobfoll SE, Dunahoo CL, Ben-Porath Y, Time-domains of corticosteroid hormone Monnier J (1994). Gender and coping: influences on brain activity: rapid, slow the dual-axis model of coping. Am J and chronic modes. Pharmacol Rev Community Psychol 22: 49-82. 64: 901-938. Homberg JR (2012). Serotonin and decision Johansen EB, Sagvolden T, Kvande G (2005). making processes. Neurosci Biobehav Rev Effects of delayed reinforcers on the 36: 218-236. behavior of an animal model of attention Homberg JR, van den Bos R, den Heijer deficit/hyperactivity disorder (ADHD). E, Suer R, Cuppen E (2008). Serotonin Behav Brain Res 162: 47-61. transporter dosage modulates long- term decision-making in rat and human.

154 REFERENCES

Johnston AL, File SE (1991). Sex differences Kennerley SW, Walton ME, Behrens TE, in animal tests of anxiety. Physiol Behav Buckley MJ, Rushworth MF (2006). 49: 245-250. Optimal decision making and the anterior Johnston JH, Driskell JE, Salas E (1997). cingulate cortex. Nat Neurosci 9: 940-947. Vigilant and hypervigilant decision Kern S, Oakes TR, Stone CK, McAuliff making. J Appl Psychol 82: 614-622. EM, Kirschbaum C, Davidson RJ (2008). Glucose metabolic changes in the K prefrontal cortex are associated with HPA Kable JW, Glimcher PW (2009). The axis response to a psychosocial stressor. neurobiology of decision: consensus and Psychoneuroendocrinology 33: 517-529. controversy. Neuron 63: 733-745. Kesner RP, Gilbert PE (2007). The role of the Kalenscher T, van Wingerden M (2011). Why agranular insular cortex in anticipation of we should use animals to study economic reward contrast. Neurobiol Learn Mem. decision making - a perspective. Front 88: 82-86. Neurosc 5: 82. Kessler RC, Hwang I, LaBrie R, Petukhova Kantak KM, Mashhoon Y, Silverman DN, M, Sampson NA, Winters KC, Shaffer HJ Janes AC, Goodrich CM (2009). Role (2008). DSM-IV pathological gambling of the orbitofrontal cortex and dorsal in the National Comorbidity Survey striatum in regulating the dose-related Replication. Psychol Med 38: 1351-1360. effects of self-administered cocaine. Kim J, Ragozzino ME (2005). The Behav Brain Res 201: 128-136. involvement of the orbitofrontal Kassam KS, Koslov K, Mendes WB (2009). cortex in learning under changing task Decisions under distress: stress profiles contingencies. Neurobiol Learn Mem influence anchoring and adjustment. 83: 125-133. Psychol Sci 20: 1394-1399. Kirk JM, Logue AW (1997). Effects of Keinan G (1987). Decision making under deprivation level on humans’ self-control stress: scanning of alternatives under for food reinforcers. Appetite 28: controllable and uncontrollable threats. 215-226. J Pers Soc Psychol 52: 639-644. Kirschbaum C, Kudielka BM, Gaab Kelly A, Laroche S, Davis S (2003). Activation J, Schommer NC, Hellhammer DH (1999). of mitogen-activated protein kinase/ Impact of gender, menstrual cycle phase, extracellular signal-regulated kinase in and oral contraceptives on the activity of hippocampal circuitry is required for the hypothalamus-pituitary-adrenal axis. consolidation and reconsolidation of Psychosom Med. 61: 154-162. recognition memory. J Neurosci 23: Knapska E, Walasek G, Nikolaev E, 5354-5360. Neuhäusser-Wespy F, Lipp HP, Kaczmarek Kendler KS, Walters EE, Neale MC, Kessler L, Werka T (2006). Differential RC, Heath AC, Eaves LJ (1995). involvement of the central amygdala in The structure of the genetic and appetitive versus aversive learning. Learn environmental risk factors for six major Mem. 13: 192-200. psychiatric disorders in women. Phobia, Knoch D, Gianotti LR, Pascual-Leone A, generalized anxiety disorder, panic Treyer V, Regard M, Hohmann M, Brugger disorder, bulimia, major depression, P (2006). Disruption of right prefrontal and alcoholism. Arch Gen Psychiatry cortex by low-frequency repetitive 52: 374-383. transcranial magnetic stimulation induces risk-taking behavior. J Neurosci 26: 6469-6472.

155 Koolhaas JM, de Boer SF, Coppens CM, L Buwalda B (2010). Neuroendocrinology Laruelle M (2000). The role of endogenous of coping styles: towards understanding sensitization in the pathophysiology of the biology of individual variation. Front schizophrenia: implications from recent Neuroendocrinol. 31: 307-321. brain imaging studies. Brain Res Brain Res Koot S, Adriani W, Saso L, van den Bos R, Rev 31: 371-384. Laviola G (2009). Home cage testing Laviola G, Adriani W, Terranova ML, Gerra of delay discounting in rats. Behav Res G (1999). Psychobiological risk factors Methods 41: 1169-1176. for vulnerability to psychostimulants in Koot, S., van den Bos, R., Adriani, W., human adolescents and animal models. Laviola, G., August 2010. Home Neurosci Biobehav Rev 23: 993-1010. cage testing of decision-making. In: Laviola G, Macrì S, Morley-Fletcher S, Adriani Proceedings of Measuring Behavior, vol. W (2003). Risk-taking behavior in 25. Eindhoven, The Netherlands. adolescent mice: psycho-biological Koot S, Zoratto F, Cassano T, Colangeli determinants and early epigenetic R, Laviola G, van den Bos R, Adriani W influence. Neurosci Biobehav Rev (2012). Compromised decision-making 27: 19-31. and increased gambling proneness LeBlanc VR (2009). The effects of acute following dietary serotonin depletion in stress on performance: implications for rats. Neuropharmacology 62: 1640-1650. health professions education. Acad Med. Koot S, Baars A, Hesseling P, van den Bos R, 84(10 Suppl): S25-33. Joëls M (2013). Time-dependent effects Leder J, Häusser JA, Mojzisch A of corticosterone on reward-based (2013). Stress and strategic decision- decision-making in a rodent model of the making in the beauty contest game. Iowa Gambling Task. Neuropharmacology Psychoneuroendocrinology. doi: 70: 306-315. 10.1016/j.psyneuen.2012.12.016. Krawczyk DC (2002). Contributions of the Lee DR, Semba R, Kondo H, Goto S, Nakano prefrontal cortex to the neural basis K (1999). Decrease in the levels of of human decision making. Neurosci NGF and BDNF in brains of mice fed Biobehav Rev 26: 631-664. a tryptophan-deficient diet. Biosci Krieger DT (1974). Food and water Biotechnol Biochem 63: 337-340. restriction shifts corticosterone, Lee TMC, Chan CCH, Leung AWS, Fox PT, Gao temperature, activity and brain amine J-H (2009). Sex-related differences in periodicity. Endocrinology 95: 1195-1201. neural activity during risk-taking: an fMRI Kringelbach ML (2005). The human study. Cerebral Cortex 19: 1303-1312. orbitofrontal cortex: linking reward to Lee Y, Kim YT, Seo E, Park O, Jeong SH, Kim hedonic experience. Nat Rev Neurosci SH, Lee SJ (2007). Dissociation of 6: 691-702. emotional decision-making from cognitive Kudielka BM, Kirschbaum C (2005). Sex decision-making in chronic schizophrenia. differences in HPA axis responses to Psychiatry Res 152: 113-120. stress: a review. Biol Psychol 69: 113-132. Lejuez CW, Read JP, Kahler CW, Richards JB, Ramsey SE, Stuart GL, Strong DR, Brown RA (2002). Evaluation of a behavioral measure of risk taking: the Balloon Analogue Risk Task (BART). J Exp Psychol Appl 8: 75-84.

156 REFERENCES

Leung PM, Rogers QR (1973). Effect of Lindová J, Kubena AA, Sturcová H, Krivohlavá amygdaloid lesions on dietry intake of R, Novotná M, Rubesová A, Havlícek disproportionate amounts of amino acids. J, Kodym P, Flegr J (2010). Pattern of Physiol Behav 11: 221-226. money allocation in experimental games Leussis MP, Andersen SL (2008). Is supports the stress hypothesis of gender adolescence a sensitive period differences in Toxoplasma gondii-induced for depression? Behavioral and behavioural changes. Folia Parasitol neuroanatomical findings from a social (Praha) 57: 136-142. stress model. Synapse 62: 22-30. Linnoila M, Virkkunen M, Schenin M, Nutila Levite M, Fleidervish IA, Schwarz A, Pelled A, Rimon R, Goodwin FK. Low D, Futerman AH (1999). Autoantibodies cerebrospinal fluid 5-hydroxyindolacetic to the glutamate receptor kill neurons via acid concentration differentiates activation of the receptor ion channel. impulsive from non-impulsive violent J Autoimmun 13: 61-72. behavior. Life Sci 33: 2609-2614. Leyton M, Okazawa H, Diksic M, Paris J, Rosa Logue AW (1988). Research on self-control: P, Mzengeza S, Young SN, Blier P, an integrated framework. Behav Brain Sci Benkelfat C (2001). Brain regional alpha- 11:665-709. [11C]methyl-L-tryptophan trapping Logue AW, King GR (1991). Self-control and in impulsive subjects with borderline impulsiveness in adult humans when food personality disorder. Am J Psychiatry is the reinforcer. Appetite 17: 105-120. 158: 775-782. Long AB, Kuhn CM, Platt ML (2009). Lieben CK, Blokland A, Westerink B, Deutz Serotonin shapes risky decision making NE (2004). Acute tryptophan and in monkeys. Soc Cogn Affect Neurosci serotonin depletion using an optimized 4: 346-356. tryptophan-free protein-carbohydrate Lucki I (1998). The spectrum of behaviors mixture in the adult rat. Neurochem Int influenced by serotonin. Biol. Psychiatry 44: 9-16. 44,:151-162. Lighthall NR, Mather M, Gorlick MA (2009). Lyon M (1991). Animal models of mania Acute stress increases sex differences in and schizophrenia. In: Willner risk seeking in the balloon analogue risk P, editor. Behavioural models in task. PLoS One 4(7):e6002. psychopharmacology. Cambridge: Lighthall NR, Sakaki M, Vasunilashorn S, Nga Cambridge University Press; p. 253–310. L, Somayajula S, Chen EY, Samii N, Mather M M (2012). Gender differences in reward- related decision processing under stress. Madden GJ, Ewan EE, Lagorio CH (2007). Soc Cogn Affect Neurosci 7: 476-484. Toward an animal model of gambling: Lin A, Adolphs R, Rangel A (2012). Social Delay discounting and the allure of and monetary reward learning engage unpredictable outcomes. J Gambl Stud overlapping neural substrates. Soc Cogn 23: 63-83. Affect Neurosci 7: 274-281. Magno E, Foxe JJ, Molholm S, Robertson IH, Garavan H (2006). The anterior cingulate and error avoidance. J Neurosci 26: 4769-4773. Mak AKY, Hu Z-g, Zhang JXX, Xiao Z, Lee TMC (2009). Sex-related differences in neural activity during emotion regulation. Neuropsychologia 47: 2900-2908.

157 Manes F, Sahakian B, Clark L, Rogers R, McClure SM, Laibson DI, Loewenstein G, Antoun N, Aitken M, Robbins T (2002). Cohen JD (2004). Separate neural systems Decision-making processes following value immediate and delayed monetary damage to the prefrontal cortex. Brain rewards. Science 306: 503-507. 125: 624-639. McClure SM, Ericson KM, Laibson DI, Mantella RC, Vollmer RR, Amico JA (2005). Loewenstein G, Cohen JD (2007). Corticosterone release is heightened in Time discounting for primary rewards. food or water deprived oxytocin deficient J Neurosci 27: 5796-5804. male mice. Brain Res 1058: 56-61. McEwen BS (1979). Influences of Mar AC, Walker AL, Theobald DE, Eagle DM, adrenocortical hormones on pituitary Robbins TW (2011). Dissociable effects of and brain function. Monogr Endocrinol lesions to orbitofrontal cortex subregions 12: 467-492. on impulsive choice in the rat. J Neurosci McEwen BS (2006). Protective and damaging 31: 6398-6404. effects of stress mediators: central role of Marcais H, Protais P, Costentin J, Schwartz the brain. Dialogues Clin Neurosci. JC (1978). A gradual score to 8: 367-381. evaluate the climbing behaviour Meck WH (2006). Neuroanatomical elicited by apomorphine in mice. localization of an internal clock: a Psychopharmacology 56: 233-234. functional link between mesolimbic, Martres MP, Costentin J, Baudry M, Marcais nigrostriatal, and mesocortical H, Protais P, Schwartz JC (1977). Longterm dopaminergic systems. Brain Res changes in the sensitivity of pre-and 1109: 93-107. postsynaptic dopamine receptors in Mendelsohn D, Riedel WJ, Sambeth A mouse striatum evidenced by behavioural (2009). Effects of acute tryptophan and biochemical studies. Brain Res depletion on memory, attention and 136: 319-337. executive functions: a systematic review. Mather M, Lighthall NR (2012). Both risk and Neurosci Biobehav Rev 33: 926-952. reward are processed differently in Milad MR, Quirk GJ (2012). Fear extinction decisions made under stress. Curr Dir as a model for translational neuroscience: Psychol Sci 21: 36-41. ten years of progress. Annu Rev Psychol Mather M, Lighthall NR, Nga L, Gorlick 63: 129-151. MA (2010). Sex differences in how stress Miller EK (2000). The prefrontal cortex: no affects brain activity during face viewing. simple matter. Neuroimage 11: 447-450. NeuroReport 21: 933-937. Miranda MI, Quirarte GL, Rodriguez-Garcia Maurex L, Zaboli G, Wiens S, Asberg M, G, McGaugh JL, Roozendaal B (2008). Leopardi R, Ohman A (2009). Emotionally Glucocorticoids enhance taste aversion controlled decision-making and a gene memory via actions in the insular cortex variant related to serotonin synthesis and basolateral amygdala. Learn Mem in women with borderline personality 15: 468-476. disorder. Scand J Psychol 50: 5-10. Mobini S, Chiang TJ, Al-Ruwaitea AS, Ho MY, McAlonan K, Brown VJ (2003). Orbital Bradshaw CM, Szabadi E (2000a). prefrontal cortex mediates reversal Effect of central 5-hydroxytryptamine learning and not attentional set shifting in depletion on intertemporal the rat. Behav Brain Res 46: 97-103. choice: a quantitative analysis. Psychopharmacology 149: 313-318.

158 REFERENCES

Mobini S, Chiang TJ, Ho MY, Bradshaw 179: 99-107. CM, Szabadi E (2000b) Effects of central Murphy ER, Fernando AB, Urcelay GP, 5-hydroxytryptamine depletion on Robinson ES, Mar AC, Theobald DE, sensitivity to delayed and probabilistic Dalley JW, Robbins TW (2012). Impulsive reinforcement. Psychopharmacology behaviour induced by both NMDA 152: 390-397. receptor antagonism and GABAA receptor Mobini S, Body S, Ho MY, Bradshaw CM, activation in rat ventromedial prefrontal Szabadi E, Deakin JF, Anderson IM cortex. Psychopharmacology (2002). Effects of lesions of the 219: 401-410. orbitofrontal cortex on sensitivity to Murphy SE, Longhitano C, Ayres RE, Cowen delayed and probabilistic reinforcement. PJ, Harmer CJ, Rogers RD (2009). The Psychopharmacology 160: 290-298. role of serotonin in nonnormative Moeller FG, Dougherty DM, Barratt ES, risky choice: the effects of tryptophan Schmitz JM, Swann AC, Grabowski J supplements on the “reflection effect” (2001). The impact of impulsivity on in healthy adult volunteers. J Cogn cocaine use and retention in treatment. Neurosci 21: 1709-1719. J Subst Abuse Treat 21: 193-198. Murray EA, Izquierdo A (2007). Orbitofrontal Mogenson GJ, Jones DL, Yim CY (1980). From cortex and amygdala contributions to motivation to action: functional interface affect and action in primates. Ann N Y between the limbic system and the motor Acad Sci 1121: 273-296. system. Prog Neurobiol 14: 69-97. Must A, Juhász A, Rimanóczy A, Szabó Z, Kéri Monterosso J, Ainslie G (1999). Beyond S, Janka Z (2007). Major depressive discounting: possible experimental disorder, serotonin transporter, and models of impulse control. personality traits: why patients use Psychopharmacology 146: 339-347. suboptimal decision-making strategies? Moreno I, Saiz-Ruiz J, López-Ibor JJ (2004). J Affect Disord 103: 273-276. Serotonin and gambling dependence. N Hum Psychopharmacol Clin Exp 6: S9-S12. Morgan MJ (1972). Fixed-ratio performance Naqvi NH, Bechara A (2009). The hidden under conditions of a delayed island of addiction: the insula. Trends reinforcement J Exp Anal Behav Neurosci. 32: 56-67. 17: 95-98. Nater UM, Rohleder N (2009). Salivary Morón JA, Brockington A, Wise RA, Rocha alpha-amylase as a non-invasive BA, Hope BT (2002). Dopamine uptake biomarker for the sympathetic nervous through the norepinephrine transporter system: current state of research. in brain regions with low levels of the Psychoneuroendocrinology 34: 486-496. dopamine transporter: evidence from Noonan MP, Walton ME, Behrens TE, Sallet J, knock-out mouse lines. J Neurosci Buckley MJ, Rushworth MF (2010). 22: 389-395. Separate value comparison and learning Morrison SE, Salzman CD (2010). Re-valuing mechanisms in macaque medial and the amygdala. Curr Opin Neurobiol 20: lateral orbitofrontal cortex. Proc Natl 221-230. Acad Sci U S A. 107: 20547-20552. Murphy ER, Dalley JW, Robbins TW (2005). Nordin C, Sjodin I (2006). CSF monoamine Local glutamate receptor antagonism patterns in pathological gamblers and in the rat prefrontal cortex disrupts healthy controls. J Psychiatr Res response inhibition in a visuospatial 40: 454-459. attentional task. Psychopharmacology

159 O Overman W, Graham L, Redmond A, O’Carroll RE, Papps BP (2003). Decision Eubank R, Boettcher L, Samplawski O, making in humans: the effect of Walsh K (2006). Contemplation of moral manipulating the central noradrenergic dilemmas eliminates sex differences on system. J Neurol Neurosurg Psychiatry 74: the Iowa gambling task. Behav Neurosci. 376-378. 120: 817–825. O’Doherty J, Kringelbach ML, Rolls ET, Owens MJ, Nemeroff CB (1994). Role Hornak J, Andrews C (2001). Abstract of serotonin in the pathophysiology reward and punishment representations of depression: focus on the serotonin in the human orbitofrontal cortex. transporter. Clin Chem 40: 288-295. Nat Neurosci 4: 95-102. P Oades RD (2002). Dopamine may be ‘hyper’ with respect to noradrenaline Pais-Vieira M, Lima D, Galhardo V (2007). metabolism, but ‘hypo’ with respect to Orbitofrontal cortex lesions disrupt risk serotonin metabolism, in children with assessment in a novel serial decision- attention deficit hyperactivity disorder. making task for rats. Neuroscience Behav Brain Res 130: 97-102. 145: 225-231. Oitzl MS, Champagne DL, van der Veen R, de Palanza P, Morley-Fletcher S, Laviola G Kloet ER (2010). Brain development under (2001). Novelty seeking in periadolescent stress: hypotheses of glucocorticoid mice: sex differences and influence of actions revisited. Neurosci Biobehav intrauterine position. Physiol Behav Rev 34: 853-866. 72: 255-262. Oitzl MS, Reichardt HM, Joëls M, de Kloet Pallanti S, Bernardi S, Quercioli L, DeCaria C, ER (2001). Point mutation in the mouse Hollander E (2006). Serotonin dysfunction glucocorticoid receptor preventing DNA in pathological gamblers: increased binding impairs spatial memory. Proc Natl prolactin response to oral m-CPP versus Acad Sci U S A 98: 12790-12795. placebo. CNS Spectr. 11: 956-964. Ojeda S, Urbanski H (1994). Puberty in Pallanti S, Bernardi S, Allen A, Hollander E the rat. In: Knobil E, Neill JD, editors. (2010). Serotonin function in pathological The physiology of reproduction, cap, vol. gambling: blunted growth hormone 40. New York: Raven Press. response to Sumatriptan. Ossewaarde L, Qin S, Van Marle HJ, J Psychopharmacol 24: 1802-1809. van Wingen GA, Fernández G, Hermans Patel PD, Lopez JF, Lyons DM, Burke EJ (2011). Stress-induced reduction in S, Wallace M, Schatzberg AF (2000). reward-related prefrontal cortex function. Glucocorticoid and mineralocorticoid Neuroimage 55: 345-352. receptor mRNA expression in squirrel Ostlund SB, Balleine BW (2007). monkey brain. J Psychiatr Res. Orbitofrontal cortex mediates 34: 383-392. outcome encoding in Pavlovian but not Patton JH, Stanford MS, Barratt ES (1995). instrumental conditioning. J Neurosci Factor structure of the Barratt 27: 4819-4825. impulsiveness scale. J Clin Psychol Overman WH, Frassrand K, Ansel S, 51: 768-774. Trawalter S, Bies B, Redmond A (2004). Performance on the IOWA card task by adolescents and adults. Neuropsychologia 42: 1838–1851.

160 REFERENCES

Paus T, Petrides M, Evans AC, Meyer E Porcelli AJ, Delgado MR (2009). Acute stress (1993). Role of the human anterior modulates risk taking in financial decision cingulate cortex in the control of making. Psychol Sci 20: 278-283. oculomotor, manual, and speech Potenza MN (2001). The neurobiology responses: a positron emission of pathological gambling. Semin Clin tomography study. J Neurophysiol Neuropsychiatry 6: 217-226. 70: 453-469. Poulin AM, Timofeeva E (2008). The Paxinos G, Watson C (2005). The Rat dynamics of neuronal activation during Brain in Stereotaxic Coordinates, 5th Edn. food anticipation and feeding in the Amsterdam: Elsevier Academic Press. brain of food-entrained rats. Brain Res Perry JL, Nelson SE, Anderson MM, Morgan 1227: 128-141. AD, Carroll ME (2007). Impulsivity (delay- Poulos CX, Le AD, Parker JL (1995). discounting) for food and cocaine in male Impulsivity predicts individual and female rats selectively bred for high susceptibility to high levels of alcohol and low saccharin intake. Pharmacol self-administration. Behav Pharmacol Biochem Behav 86: 822-837. 6: 810-814. Pessoa L (2010). Emotion and cognition and Powell M, Ansic D (1997). Gender the amygdala: from “what is it?” to differences in risk behaviour in financial “what’s to be done?”. Neuropsychologia decision-making: an experimental 2010 analysis. Journal of Economic Psychology Pessoa L, Adolphs R (2010). Emotion 18: 605-628. processing and the amygdala: from a Preston SD, Buchanan TW, Stansfield RB, ‘low road’ to ‘many roads’ of evaluating Bechara A (2007). Effects of anticipatory biological significance. Nat Rev Neurosci stress on decision making in a gambling 11: 773-783. task. Behav Neurosci 121: 257-263. Petry NM (2003). Discounting of money, Preuschoff K, Quartz SR, Bossaerts P health, and freedom in substance abusers (2008). Human insula activation reflects and controls. Drug Alcohol Depend. risk prediction errors as well as risk. 71:133-141. J Neurosci 28: 2745-2752. Piazza PV, Deroche V, Deminière JM, Pruessner JC, Kirschbaum C, Meinlschmid Maccari S, Le Moal M, Simon H (1993). G, Hellhammer DH (2003). Two Corticosterone in the range of stress- formulas for computation of the area induced levels possesses reinforcing under the curve represent measures properties: implications for sensation- of total hormone concentration seeking behaviors. Proc. Natl. Acad. Sci. versus time-dependent change. U S A 90: 11738-11742. Psychoneuroendocrinology 28: 916-931. Piazza PV, Le Moal M (1997). Glucocorticoids Pruessner JC, Champagne F, Meaney MJ, as a biological substrate of reward: Dagher A (2004). Dopamine release in physiological and pathophysiological response to a psychological stress in implications. Brain Res Brain Res Rev humans and its relationship to early 25: 359-372. life maternal care: a positron emission Piazza PV, Le Moal M (1998). The role of tomography study using [11C]raclopride. stress in drug self-administration. Trends J Neurosci. 24: 2825-2831. Pharmacol Sci 19: 67-74. Podlesnik CA, Shahan TA (2008). Response- reinforcer relations and the resistance to change. Behav Processes 77: 109-125.

161 Purgert RJ, Wheeler DS, McDannald MA, Ragozzino ME. (2007) The contribution of Holland PC (2012). Role of amygdala the medial prefrontal cortex, central nucleus in aversive learning orbitofrontal cortex, and dorsomedial produced by shock or by unexpected striatum to behavioral flexibility. Ann N Y omission of food. J Neurosci. 32: 2461- Acad Sci. 1121: 355-375. 2472. Reavis R, Overman WH (2001). Adult sex Putman P, Antypa N, Crysovergi P, van der differences on a decision-making task Does WA (2010). Exogenous cortisol previously shown to depend on the acutely influences motivated decision orbital prefrontal cortex. Behav Neurosci making in healthy young men. 115: 196-206. Psychopharmacology 208: 257-263. Renner M, Bennet A, White J (1992). Age Puumala T, Sirvio J (1998). Changes in and sex as factors influencing activity of dopamine and serotonin spontaneous exploration and object systems in the frontal cortex underlie investigation by preadult rats poor choice accuracy and impulsivity of (Rattus norvegicus). J Comp Psychol rats in an attention task. Neuroscience 106: 217-227. 83: 489-499. Reul JM, de Kloet ER (1985). Two receptor Q systems for corticosterone in rat brain: microdistribution and differential Quilodran R, Rothé M, Procyk E (2008). occupation. Endocrinology Behavioral shifts and action valuation in 117: 2505-2511. the anterior cingulate cortex. Neuron Reynolds B, Schiffbauer R (2004). Measuring 57: 314-325. state changes in human delay Quirarte GL, de la Teja IS, Casillas M, Serafín discounting: an experiential discounting N, Prado-Alcalá RA, Roozendaal B (2009). task. Behav Processes 67: 343-356. Corticosterone infused into the dorsal Reynolds B (2006). A review of delay- striatum selectively enhances memory discounting research with humans: consolidation of cued water-maze Relations to drug use and gambling. training. Learn Mem 16: 586-589. Behavioral Pharmacology 17: 651-667. Quirk GJ, Beer JS (2006). Prefrontal Rilling JK, Sanfey AG (2011). involvement in the regulation of emotion: The neuroscience of social decision- convergence of rat and human studies. making. Annu Rev Psychol 62: 23-48. Curr Opin Neurobiol 16: 723-727. Rivalan M, Ahmed SH, Dellu-Hagedorn F (2009). Risk-prone individuals prefer R the wrong options on a rat version of Ragozzino ME, Detrick S, Kesner RP (1999). the Iowa Gambling Task. Biol Psychiatry Involvement of the prelimbic-infralimbic 66: 743-749. areas of the rodent prefrontal cortex Rivalan M, Coutureau E, Fitoussi A, in behavioral flexibility for place and Dellu-Hagedorn F (2011). Inter-individual response learning. J Neurosci decision-making differences in the effects 19: 4585-4594. of cingulate, orbitofrontal, and prelimbic Ragozzino ME, Kesner RP (2001). The role of cortex lesions in a rat gambling task. rat dorsomedial prefrontal cortex Front Behav Neurosci 5: 22. in working memory for egocentric responses. Neurosci Lett 308: 145-148.

162 REFERENCES

Rivera R, Yacobson I, Grimes D (1999). Rogers RD, Everitt BJ, Baldacchino The mechanism of action of hormonal A, Blackshaw AJ, Swainson R, Wynne contraceptives and intrauterine K, Baker NB, Hunter J, Carthy T, Booker contraceptive devices. Am J Obstet E, London M, Deakin JF, Sahakian BJ, Gynecol 181: 1263-1269. Robbins TW (1999b). Dissociable deficits Robbins TW (2002). The 5-choice serial in the decision-making cognition of reaction time task: behavioral chronic amphetamine abusers, opiate pharmacology and functional abusers, patients with focal damage neurochemistry. Psychopharmacology to prefrontal cortex, and tryptophan- 163: 362-380. depleted normal volunteers: evidence Robbins TW, Arnsten AF (2009). for monoaminergic mechanisms. The neuropsychopharmacology of fronto- Neuropsychopharmacology 20: 322-339. executive function: monoaminergic Rogers RD, Tunbridge EM, Bhagwagar modulation. Annu Rev Neurosci Z, Drevets WC, Sahakian BJ, Carter CS 32: 267-287. (2003). Tryptophan depletion alters the Robinson TE, Becker JB (1986). Enduring decision-making of healthy volunteers changes in brain and behaviour produced through altered processing of reward by chronic amphetamine administration: cues. Neuropsychopharmacology a review and evaluation of animal models 28, 153-162. of amphetamine psychosis. Brain Res Rogers RD, Lancaster M, Wakeley 11: 157-198. J, Bhagwagar Z (2004). Effects of Rodrigues AJ, Leão P, Carvalho M, Almeida beta-adrenoceptor blockade on OF, Sousa N (2011). Potential components of human decision-making. programming of dopaminergic circuits Psychopharmacology 172: 157-164. by early life stress. Psychopharmacology Roozendaal B, McReynolds JR, McGaugh 214: 107-120. JL (2004).The basolateral amygdala Rogers RD (2011). The roles of dopamine interacts with the medial prefrontal and serotonin in decision making: cortex in regulating glucocorticoid evidence from pharmacological effects on working memory impairment. experiments in humans. J Neurosci 24: 1385-1392. Neuropsychopharmacology 36: 114-132. Roozendaal B, Okuda S, de Quervain DJ, Rogers RD, Blackshaw AJ, Middleton HC, McGaugh JL (2006). Glucocorticoids Matthews K, Hawtin K, Crowley C, interact with emotion-induced Hopwood A, Wallace C, Deakin JF, noradrenergic activation in influencing Sahakian BJ, Robbins TW (1999a). different memory functions. Tryptophan depletion impairs stimulus- Neuroscience 138: 901-910. reward learning while methylphenidate Roozendaal B, McReynolds JR, Van der disrupts attentional control in healthy Zee EA, Lee S, McGaugh JL, McIntyre young adults: implications for the CK (2009). Glucocorticoid effects on monoaminergic basis of impulsive memory consolidation depend on behaviour. Psychopharmacology functional interactions between the 146: 482-491. medial prefrontal cortex and basolateral amygdala. J Neurosci 29: 14299-14308. Roozendaal B, McGaugh JL (2011). Memory modulation. Behav Neurosci 125: 797-824.

163 Roper TJ, Nieto J (1979). Schedule-induced Sagvolden T, Sergeant JA (1998). Attention drinking and other behavior in the rat as deficit/hyperactivity disorder: from brain a function of body-weight deficit. Physiol dysfunctions to behaviour. Behav Brain Behav 23: 673-678. Res 94: 1-10. Rosario M, Shinn M, Mørch H, Huckabee CB Sallet J, Quilodran R, Rothé M, Vezoli (1988). Gender differences in coping and J, Joseph JP, Procyk E (2007). social supports: testing socialization and Expectations, gains, and losses in the role constraint theories. J Community anterior cingulate cortex. Cogn Affect Psychol 16: 55-69. Behav Neurosci 7: 327-336. Rougé-Pont F, Piazza PV, Kharouby M, Le Sandi C, Pinelo-Nava MT (2007). Stress Moal M, Simon H (1995). Stress-induced and memory: behavioral effects sensitization and glucocorticoids. and neurobiological mechanisms. II. Sensitization of the increase in Neural Plast: 78970. extracellular dopamine induced by Sanfey AG, Rilling JK, Aronson JA, Nystrom cocaine depends on stress-induced LE, Cohen JD (2003). The neural basis corticosterone secretion. J Neurosci of economic decision-making in 15: 7189-7195. the Ultimatum Game. Rudebeck PH, Walton ME, Smyth AN, Science 300: 1755-1758. Bannerman DM, Rushworth MF (2006). Sapienza P, Zingales L, Maestripieri D Separate neural pathways process (2009). Gender differences in financial different decision costs. Nat Neurosci risk aversion and career choices are 9: 1161-1168. affected by testosterone. Proc Natl Acad Rushworth MF, Behrens TE, Rudebeck Sci U S A 106: 15268-15273. PH, Walton ME (2007). Contrasting roles Sapolsky RM, McEwen BS, Rainbow TC for cingulate and orbitofrontal cortex in (1983). Quantitative autoradiography of decisions and social behaviour. Trends [3H]corticosterone receptors in rat brain. Cogn Sci 11:168-176. Brain Res 271: 331-334. Rushworth MF, Noonan MP, Boorman ED, Sarabdjitsingh RA, Joëls M, de Kloet ER Walton ME, Behrens TE (2011). Frontal (2012). Glucocorticoid pulsatility and cortex and reward-guided learning and rapid corticosteroid actions in the decision-making. Neuron 70: 1054-1069. central stress response. Physiol Behav Rushworth MF, Noonan MP, Boorman ED, 106: 73-80. Walton ME, Behrens TE (2011). Frontal Scheres A, Sanfey AG (2006). Individual cortex and reward-guided learning and differences in decision making: Drive and decision-making. Neuron 70: 1054-1069. Reward Responsiveness affect strategic Russell PA (1973). Sex differences in rats’ bargaining in economic games. Behav stationary-cage activity measured by Brain Funct 2: 35. observation and automatic recording. Schiller L, Donix M, Jähkel M, Oehler J Anim Learn Behav 1: 278-282. (2006a). Serotonin 1A and 2A receptor densities, neurochemical and behavioural S characteristics in two closely related Sagvolden T (2000). Behavioral validation of mice strains after long-term isolation. the spontaneously hypertensive rat (SHR) Prog Neuropsychopharmacol Biol as an animal model of attention deficit/ Psychiatry 30: 492-503. hyperactivity disorder (AD/HD). Neurosci Biobehav Rev 24: 31-39.

164 REFERENCES

Schiller L, Jähkel M, Oehler J (2006b). The Sescousse G, Redouté J, Dreher JC (2010). influence of sex and social isolation The architecture of reward value coding housing on pre- and postsynaptic 5-HT1A in the human orbitofrontal cortex. receptors. Brain Res 1103: 76-87. J Neurosci 30: 13095-13104. Schoenbaum G, Chiba AA, Gallagher M Sescousse G, Caldú X, Segura B, Dreher JC (1999). Neural encoding in orbitofrontal (2013). Processing of primary and cortex and basolateral amygdala during secondary rewards: A quantitative meta- olfactory discrimination learning. analysis and review of human functional J Neurosci. 19: 1876-1884. neuroimaging studies. Neurosci Biobehav Schoenbaum G, Setlow B (2003). Lesions of Rev 37: 681-696. nucleus accumbens disrupt learning Sevy S, Hassoun Y, Bechara A, Yechiam about aversive outcomes. J Neurosci E, Napolitano B, Burdick K, Delman 23: 9833-9841. H, Malhotra A (2006). Emotion based Schoenbaum G, Roesch M (2005). decision-making in healthy subjects: Orbitofrontal cortex, associative learning, short-term effects of reducing dopamine and expectancies. Neuron 47: 633-636. levels. Psychopharmacology 188: Schubert R, Brown M, Gysler M, Brachinger 228-235. HW (1999). Financial decision-making: Sevy S, Burdick KE, Visweswaraiah H, are women really more risk-aversive? The Abdelmessih S, Lukin M, Yechiam E, American Economic Review 89: 381-385. Bechara A (2007). Iowa gambling task Schultz W (2002). Getting formal with in schizophrenia: a review and new dopamine and reward. data in patients with schizophrenia and Neuron 36: 241-263. co-occurring cannabis use disorders. Schwabe L, Tegenthoff M, Höffken O, Wolf Schizophr Res 92: 74-84. OT (2010). Concurrent glucocorticoid and Shaffer HJ, Korn DA (2002). Gambling and noradrenergic activity shifts instrumental related mental disorders: a public behavior from goal-directed to habitual health analysis. Annu. Rev. Public Health control. J Neurosci 30: 8190-8196. 23: 171-212. Schwabe L, Höffken O, Tegenthoff M, Wolf Shafiei N, Gray M, Viau V, Floresco SB OT (2011) Preventing the stress-induced (2012). Acute stress induces selective shift from goal-directed to habit action alterations in cost/benefit decision- with a beta-adrenergic antagonist. making. Neuropsychopharmacology J Neurosci 31: 17317–17325. 37: 2194-2209. Schwabe L, Joëls M, Roozendaal B, Wolf OT, Sierra-Mercado D Jr, Corcoran KA, Lebrón- Oitzl MS (2012a). Stress effects on Milad K, Quirk GJ (2006). Inactivation memory: an update and integration. of the ventromedial prefrontal cortex Neurosci Biobehav Rev 36: 1740-1749. reduces expression of conditioned Schwabe L, Tegenthoff M, Höffken O, Wolf fear and impairs subsequent recall of OT (2012b). Simultaneous glucocorticoid extinction. Eur J Neurosci 24: 1751-1758. and noradrenergic activity disrupts Sierra-Mercado D, Padilla-Coreano N, Quirk the neural basis of goal-directed action GJ (2011). Dissociable roles of in the human brain. J Neurosci. prelimbic and infralimbic cortices, 32: 10146-10155. ventral hippocampus, and basolateral Schwabe L, Wolf OT (2011). Stress-induced amygdala in the expression and modulation of instrumental behavior: extinction of conditioned fear. from goal-directed to habitual control of Neuropsychopharmacology 36: 529-538. action. Behav Brain Res 219: 321-328.

165 Small CM, Manatunga AK, Marcus M. Starcke K, Polzer C, Wolf OT, Brand M (2011). Validity of self-reported menstrual cycle Does stress alter everyday moral decision- length. Ann Epidemiol 17: 163-170. making? Psychoneuroendocrinology Smeets T, Wolf OT, Giesbrecht T, Sijstermans 36: 210-219. K, Telgen S, Joëls M (2009). Stress Starcke K, Brand M (2012). Decision making selectively and lastingly promotes under stress: a selective review. Neurosci learning of context-related high arousing Biobehav Rev 36: 1228-1248. information. Psychoneuroendocrinology Stark R, Wolf OT, Tabbert K, Kagerer S, 34: 1152-1161. Zimmermann M, Kirsch P, Schienle A, Soares JM, Sampaio A, Ferreira LM, Santos Vaitl D (2006). Influence of the stress NC, Marques F, Palha JA, Cerqueira JJ, hormone cortisol on fear conditioning Sousa N (2012). Stress-induced changes in humans: evidence for sex differences in human decision-making are reversible. in the response of the prefrontal cortex. Transl Psychiatry 2: e131. Neuroimage 32: 1290-1298. Solanto MV, Abikoff H, Sonuga-Barke E, Stern CE, Passingham RE (1994). The nucleus Schachar R, Logan GD, Wigal T, Hechtman accumbens in monkeys (Macaca L, Hinshaw S, Turkel E (2001). The fascicularis): I. The organization of ecological validity of delay aversion behaviour. Behav Brain Res 61: 9-21. and response inhibition as measures of Stoltenberg SF, Vandever JM (2010). impulsivity in AD/HD: a supplement to Gender moderates the association the NIMH multi-modal treatment study between 5-HTTLPR and decision-making of AD/HD. J Abnormal Child Psychol under ambiguity but not under risk. 29: 215-228. Neuropharmacology 58: 423-428. Sonuga-Barke EJS, Taylor E, Sembi S, Smith J Stone AA, Neale JM (1984). New measure of (1992). Hyperactivity and delay aversion. daily coping: development and I. The effect of delay on choice. J Child preliminary results. J Pers Soc Psychol Psychol Psychiatry 33: 387-398. 46: 892-906. Soubrié P (1986). Reconciling the role of Strubbe JH, Woods SC (2004). The timing central serotonin neurones in human of meals. Psychological Review and animal behavior. Behav Brain Sci 111: 128-141. 9: 319-364. Sul JH, Kim H, Huh N, Lee D, Jung MW Spielberger C (1989). State-Trait Anxiety (2010). Distinct roles of rodent Inventory: A Comprehensive Bibliography. orbitofrontal and medial prefrontal cortex Consulting Psychologists Press, Palo Alto. in decision making. Neuron 66: 449-460. Stalnaker TA, Takahashi Y, Roesch MR, Sullivan RM, Gratton A (2002). Prefrontal Schoenbaum G (2009). Neural substrates cortical regulation of hypothalamic- of cognitive inflexibility after chronic pituitary-adrenal function in the rat and cocaine exposure. Neuropharmacology implications for psychopathology: side 56 Suppl 1: 63-72. matters. Psychoneuroendocrinology Starcke K, Wolf OT, Markowitsch HJ, Brand 27: 99-114. M (2008). Anticipatory stress influences Swann AC, Bjork JM, Moeller FG, Dougherty decision making under explicit risk DM (2002). Two models of impulsivity: conditions. Behav Neurosci 122: Relationship to personality traits and 1352-1360. psychopathology. Biol Psychiatry 51: 988-994.

166 REFERENCES

Swanson J, Castellanos FX, Murias M (1998). Taylor SE, Klein LC, Lewis BP, Gruenewald Cognitive neuroscience of attention TL, Gurung RA, Updegraff JA (2000). deficit hyperactivity disorder and Biobehavioral responses to stress in hyperkinetic disorder. Curr Opin females: tend-and-befriend, not fight-or- Neurobiol 8: 263-271. flight. Psychol Rev 107: 411-429. T Thiébot MH, Le Bihan C, Soubrié P, Simon P (1985). Benzodiazepines reduce Takagishi H, Fujii T, Kameshima S, Koizumi M, the tolerance to reward delay in rats. Takahashi T (2009). Salivary alpha- Psychopharmacology 86: 147-152. amylase levels and rejection of unfair Tranel D, Damasio H, Denburg NL, Bechara offers in the ultimatum game. Neuro A (2005). Does gender play a role in Endocrinol Lett 30: 643-646. functional asymmetry of ventromedial Takahashi T (2005). Social memory, social prefrontal cortex? Brain 128: 2872-2881. stress, and economic behaviors. Brain Res Tran-Tu-Yen DA, Marchand AR, Pape JR, Di Bull 67: 398-402. Scala G, Coutureau E (2009). Transient Talbot PS, Watson DR, Barrett SL, Cooper SJ role of the rat prelimbic cortex in goal- (2006). Rapid tryptophan depletion directed behaviour. Eur J Neurosci improves decision-making cognition 30: 464-471. in healthy humans without affecting Tucker KA, Potenza MN, Beauvais JE, reversal learning or set shifting. Browndyke JN, Gottschalk PC, Kosten Neuropsychopharmacology TR (2004). Perfusion abnormalities and 31: 1519-1525. decision making in cocaine dependence. Tanaka SC, Doya K, Okada G, Ueda K, Biol Psychiatry 56: 527-530. Okamoto Y, Yamawaki S (2004). Prediction U of immediate and future rewards differentially recruits cortico-basal ganglia Uchida S, Kitamoto A, Umeeda H, Nakagawa loops. Nat Neurosci 7: 887-893. N, Masushige S, Kida S (2005). Chronic Tanaka SC, Schweighofer N, Asahi S, Shishida reduction in dietary tryptophan leads to K, Okamoto Y, Yamawaki S, Doya K (2007). changes in the emotional response to Serotonin differentially regulates short- stress in mice. J Nutr Sci Vitaminol and long-term prediction of rewards in 51: 175-181. the ventral and dorsal striatum. Uylings HB, Groenewegen HJ, Kolb B (2003). PLoS One 2: e1333. Do rats have a prefrontal cortex? Behav Tanaka SC, Shishida K, Schweighofer N, Brain Res 146: 3-17. Okamoto Y, Yamawaki S, Doya K (2009). Uylings HB, van Eden CG (1990). Qualitative Serotonin affects association of aversive and quantitative comparison of the outcomes to past actions. J Neurosci prefrontal cortex in rat and in primates, 29: 15669-15674. including humans. Prog Brain Res Tanke MA, Alserda E, Doornbos B, van der 85: 31-62. Most PJ, Goeman K, Postema F, Korf J V (2008). Low tryptophan diet increases stress-sensitivity, but does not affect Valentin VV, Dickinson A, O’Doherty JP habituation in rats. Neurochem. (2007). Determining the neural substrates Int. 52: 272-281. of goal-directed learning in the human brain. J Neurosci 27: 4019-4026.

167 van den Berg CL, Pijlman FT, Koning HA, van den Bos R, de Visser L, van de Loo AJAE, Diergaarde L, Van Ree JM, Spruijt BM Mets MAJ, van Willigenburg GM, (1999). Isolation changes the incentive Homberg JR, et al (2012a). Sex differences value of sucrose and social behaviour in in decision-making in adult normal juvenile and adult rats. Behav Brain volunteers are related to differences in Res 106: 133-142. the interaction of emotion and cognitive van den Bos R, Lasthuis W, den Heijer E, control. In: Moore KO, Gonzalez NP, van der Harst J, Spruijt B (2006a). Toward editors. Handbook on psychology a rodent model of the Iowa gambling of decision-making: New Research. task. Behav Res Methods 38: 470-478. Hauppage NY (USA): Nova Science van den Bos R, Houx BB, Spruijt BM (2006b). Publisher Inc; p. 179–98. The effect of reward magnitude van den Bos R, Jolles J, van der Knaap L, differences on choosing disadvantageous Baars A, de Visser L (2012b). Male and decks in the Iowa Gambling Task. female Wistar rats differ in decision- Biol Psychol 71: 155-161. making performance in a rodent version van den Bos R, van der Harst J, Jonkman S, of the Iowa Gambling Task. Behav Brain Schilders M, Spruijt B (2006c). Rats assess Res. 234: 375-379. costs and benefits according to van den Bos R, Homberg J, de Visser L an internal standard. Behav Brain Res (2013). A critical review of sex differences 171: 350-354. in decision-making tasks: focus on the van den Bos R, den Heijer E, Vlaar S, Houx Iowa Gambling Task. Behav Brain Res. BB (2007). Exploring gender differences in 238: 95-108. decision-making using the Iowa Gambling van den Bos R, Taris R, Scheppink B, de Haan Task. In: Elsworth JA, editor. Psychology L, Verster JC (submitted). Cortisol affects of decision making in education, behavior risk-taking differently in men and women & high risk situations. Nova Publishers; in the Cambridge Gambling Task. 2007. p. 55–75. van Gaalen MM, van Koten R, van den Bos R, Harteveld M, Stoop H Schoffelmeer AN, Vanderschuren (2009a). Stress and decision- LJ (2006). Critical involvement of making in humans: performance is dopaminergic neurotransmission in related to cortisol reactivity, albeit impulsive decision making. Biol Psychiatry differently in men and women. 60: 66-73. Psychoneuroendocrinology 34: van Hasselt FN, de Visser L, Tieskens JM, 1449-1458. Cornelisse S, Baars AM, Lavrijsen M, van den Bos R, Homberg J, Gijsbers E, den Krugers HJ, van den Bos R, Joëls M Heijer E, Cuppen E (2009b). The effect of (2012). Individual variations in maternal COMT Val 158 Met genotype on decision- care early in life correlate with later life making and preliminary findings on its decision-making and c-fos expression in interaction with the 5-HTTLPR in healthy prefrontal subregions of rats. PLoS One females. Neuropharmacology 56: 7: e37820. 493-498. van Loo PL, Mol JA, Koolhaas JM, Van Zutphen BF, Baumans V (2001). Modulation of aggression in male mice: influence of group size and cage size. Physiol Behav. 72: 675-683.

168 REFERENCES

van Stegeren AH, Wolf OT, Kindt M (2008). Wager TD, Davidson ML, Hughes BL, Salivary alpha amylase and cortisol Lindquist MA, Ochsner KN (2008). Neural responses to different stress tasks: impact mechanisms of emotion regulation: of sex. Int J Psychophysiol. 69: 33-40. evidence for two independent prefrontal- van ‘t Wout M, Kahn RS, Sanfey AG, Aleman subcortical pathways. Neuron 59: A (2006). Affective state and decision- 1037-1050. making in the Ultimatum Game. Exp Brain Wahlsten D, Metten P, Phillips TJ, Boehm SL Res 169: 564-568. 2nd, Burkhart-Kasch S, Dorow J, Doerksen van Wingerden M, Kalenscher T (2012). S, Downing C, Fogarty J, Rodd-Henricks K, Animal decisions - a look across the Hen R, McKinnon CS, Merrill CM, Nolte C, fence. Front Neurosci. 6: 142. Schalomon M, Schlumbohm JP, Sibert JR, Vergnes, M., Kempf, E., 1981. Tryptophan Wenger CD, Dudek BC, Crabbe JC (2003). deprivation: effects on mouse-killing and Different data from different labs: Lessons reactivity in the rat. Pharmacol. Biochem. from studies of gene–environment Behav. 14, 19-23. interaction. J Neurobiol 54: 283-311. Vinkers CH, Zorn JV, Cornelisse S, Koot S, Walters JK, Davis M, Sheard MH (1979). Houtepen LC, Olivier B, Verster JC, Tryptophan-free diet: effects on Kahn RS, Boks MP, Kalenscher T, Joëls the acoustic startle reflex in rats. M (2013). Time-dependent changes in Psychopharmacology 62: 103-109. altruistic punishment following stress. Wang J, Korczykowski M, Rao H, Fan Y, Pluta Psychoneuroendocrinology. doi:pii: J, Gur RC, McEwen BS, Detre JA (2007). S0306-4530(12)00425-8. Gender difference in neural response Voikar V, Colacicco G, Gruber O, Vannoni to psychological stress. Soc Cogn Affect E, Lipp HP, Wolfer DP (2010). Conditioned Neurosci 2: 227-239. response suppression in the IntelliCage: Wayner MJ, Rondeau DB (1976). Schedule assessment of mouse strain differences dependent and schedule-induced and effects of hippocampal and striatal behavior at reduced and recovered body lesions on acquisition and retention of weight. Physiol Behav 17: 325-336. memory. Behav Brain Res 213: 304-312. Weber EU, Johnson EJ (2009). Decisions von Dawans B, Kirschbaum C, Heinrichs under uncertainty: psychological, M (2011). The Trier Social Stress Test economic and neuroeconomic for Groups (TSST-G): A new research explanations of risk preference. In: tool for controlled simultaneous social Glimscher PW, Camerer CF, Fehr E, stress exposure in a group format. Poldrack RA (Eds.), Neuroeconomics: Psychoneuroendocrinology 36: 514-522. Decision Making and the Brain. Elsevier. von Dawans B, Fischbacher U, Kirschbaum Weller JA, Levin IP, Shiv B, Bechara A (2009). C, Fehr E, Heinrichs M (2012). The social The effects of insula damage on decision- dimension of stress reactivity: acute making for risky gains and losses. stress increases prosocial behavior in Soc Neurosci. 4: 347-358. humans. Psychol Sci 23: 651-660. Wet op de Dierproeven (1996). Dutch W Experiments on Animals Act. Staatsblad 500, The Hague. Wager TD, Phan KL, Liberzon I, Taylor SF (2003). Valence, gender, and lateralization of functional brain anatomy in emotion: a meta-analysis of findings from neuroimaging. NeuroImage 19: 513-531.

169 Wiegratz I, Jung-Hoffmann C, Kuhl H Wogar MA, Bradshaw CM, Szabadi E (1992). (1999). Effect of two oral contraceptives Choice between delayed reinforcers in an containing ethinylestradiol and gestodene adjusting-delay schedule: the effects of or norgestimate upon androgen absolute reinforcer size and deprivation parameters and serum binding proteins. level. Q J Exp Psychol B 45: 1-13. Contraception 51: 341-346. Wogar MA, Bradshaw CM, Szabadi E Williams LM, Gordon E (2007). Dynamic (1993). Effect of lesions of the ascending organization of the emotional brain: 5-hydroxytryptaminergic pathways on responsivity, stability, and instability. choice between delayed reinforcers. Neuroscientist 13: 349-370. Psychopharmacology 111: 239-243. Wilson M, Daly M (2004). Do pretty women Wood RM, Rilling JK, Sanfey AG, Bhagwagar inspire men to discount the future? Proc Z, Rogers RD (2006). Effects of tryptophan Biol Sci. 271 Suppl 4: S177-179. depletion on the performance of Windmann S, Kirsch P, Mier D, Stark R, an iterated Prisoner’sDilemma Walter B, Güntürkün O, Vaitl D (2006). On game in healthy adults. framing effects in decision making: linking Neuropsychopharmacology lateral versus medial orbitofrontal cortex 31: 1075-1084. activation to choice outcome processing. Woods PJ (1962). Behavior in a novel J Cogn Neurosci 18: 1198-1211. situation as influenced by the Winstanley C, Dalley J, Theobald D, Robbins immediately preceding environment. T (2004a). Fractionating impulsivity: J Exp Anal Behav 5: 185-190. Contrasting effects of central Wozniak D, Harbaugh WT, Mayr U (2010). 5-HT depletion on different Choices about competition: Differences measures of impulsive behavior. by gender and hormonal fluctuations, Neuropsychopharmacology and the role of relative performance 29: 1331-1343. feedback. Mimeo, University of Oregon. Winstanley CA, Theobald DE, Dalley X JW, Glennon JC, Robbins TW (2004b). 5-HT2A and 2C receptor antagonists Xue G, Lu Z, Levin IP, Bechara A (2010). have opposing effect on measure of The impact of prior risk experiences on impulsivity: interaction with global 5-HT subsequent risky decision-making: depletion. Psychopharmacology the role of the insula. Neuroimage 176: 376-385. 50: 709-716. Winstanley CA, Theobald DE, Dalley JW, Y Robbins TW (2005). Interactions between serotonin and dopamine in the control Yin HH, Knowlton BJ (2006). The role of the of impulsive choice in rats: therapeutic basal ganglia in habit formation. Nat Rev implications for impulse control Neurosci 7: 464-476. disorders. Neuropsychopharmacology Yin HH, Ostlund SB, Balleine BW (2008). 30: 669-682. Reward-guided learning beyond Winstanley CA (2011). The utility of dopamine in the nucleus accumbens: rat models of impulsivity in developing the integrative functions of cortico-basal pharmacotherapies for impulse control ganglia networks. Eur J Neurosci disorders. Br J Pharmacol 164: 28: 1437-1448. 1301-1321.

170 REFERENCES

Young JJ, Shapiro ML (2009). Double dissociation and hierarchical organization of strategy switches and reversals in the rat PFC. Behav Neurosci 123: 1028-1035. Yu AJ, Dayan P (2005). Uncertainty, neuromodulation, and attention. Neuron 46: 681-692. Yuen EY, Liu W, Karatsoreos IN, Feng J, McEwen BS, Yan Z (2009). Acute stress enhances glutamatergic transmission in prefrontal cortex and facilitates working memory. Proc Natl Acad Sci U S A 106: 14075-14079. Z

Zald DH, McHugo M, Ray KL, Glahn DC, Eickhoff SB, Laird AR (2012). Meta- Analytic Connectivity Modeling Reveals Differential Functional Connectivity of the Medial and Lateral Orbitofrontal Cortex. Cereb Cortex. doi: 10.1093/cercor/ bhs308 Zeeb FD, Robbins TW, Winstanley CA (2009). Serotonergic and dopaminergic modulation of gambling behavior as assessed using a novel rat gambling task. Neuropsychopharmacology 34: 2329-2343. Zeeb FD, Winstanley CA (2011). Lesions of the basolateral amygdala and orbitofrontal cortex differentially affect acquisition and performance of a rodent gambling task. J Neurosci 31: 2197-2204. Zhang L, Guadarrama L, Corona-Morales AA, Vega-Gonzalez A, Rocha L, Escobar A (2006). Rats subjected to extended L-tryptophan restriction during early postnatal stage exhibit anxious- depressive features and structural changes. J Neuropathol Exp Neurol 65: 562-570. Zoratto F, Laviola G, Adriani W (2013). Gambling proneness in rats during the transition from adolescence to young adulthood: a home-cage method. Neuropharmacology 67: 444-454.

171 172 NEDERLANDSE SAMENVATTING

173 Beslissen

Zowel mensen als dieren maken gedurende hun leven talrijke keuzes. Voor mensen kunnen dit grote besluiten zijn, zoals hoe je je werk of privéleven inricht, maar ook kleine, meer dagelijkse beslissingen zoals wat je die avond wilt eten. Zulke keuzes, voor mensen en dieren, worden gemaakt voor veel essentiële zaken in het leven, zoals het vinden van voedsel, onderdak en partners of het leggen van sociale contacten in het algemeen. Uiteindelijk komen alle beslissingen op hetzelfde neer: ze hebben betrekking op het afwegen van kosten en baten van – mogelijk tegenstrijdige – handelingen, waarvan de uitkomst al dan niet van tevoren duidelijk is. Keuzegedrag wordt bepaald door verschillende neurale processen, zoals de gevoeligheid voor beloning en straf, impulsiviteit en het onderdrukken van acute reacties, zodat de voors en tegens op zowel korte als lange termijn goed tegen elkaar afgewogen kunnen worden. Daarnaast is een continue evaluatie van risico’s en voordelen van verschillende mogelijkheden belangrijk, evenals flexibiliteit. Keuzegedrag komt tot stand op basis van activiteit in verschillende delen van de hersenen. Onderzoek heeft laten zien dat gecoördineerde activiteit in verschillende prefrontale en subcorticale hersengebieden nodig is om optimaal keuzes te kunnen maken. De uitkomst van een keuzeproces, dat wil zeggen de actie die uiteindelijk ondernomen wordt, wordt grofweg bepaald door de interactie tussen twee circuits in de hersenen. Enerzijds is er het limbische (affectieve/motivationele) circuit, bestaande uit de ventromediale prefrontale cortex, de orbitofrontale cortex, de amygdala en het ventrale striatum. Dit circuit is betrokken bij het direct reageren op (mogelijke) beloningen, verliezen of bedreigingen, evenals bij de regulatie van emoties. Anderzijds is er het cognitieve (uitvoerende/motorische) circuit, dat belangrijk is bij het vormen van lange termijn of toekomstperspectieven, dus bij cognitieve controle over gedrag. Dit circuit bestaat uit de dorsolaterale prefrontale cortex, de anteriore cingulate cortex en het dorsale striatum. Keuzegedrag op het niveau van deze hersengebieden wordt onder andere gereguleerd door neurotransmitters (signaalstoffen) als serotonine en dopamine. Deze neurotransmitters worden onder meer in verband gebracht met het onderdrukken van impulsieve reacties, associatief leren en gevoeligheid voor straf en beloning. In dit proefschrift heb ik twee factoren onderzocht die besluitvorming kunnen beïnvloeden bij mensen en dieren, namelijk stress en sekse.

Beslissen en stress In de huidige (Westerse) samenleving worden vaak keuzes gemaakt onder stressvolle omstandigheden, zoals bij de politie, in het leger en in de medische en financiële sector. Bestudering van de invloed van stress op keuzegedrag is dan ook zeer relevant. Verscheidene studies hebben laten zien dat stress keuzegedrag kan beïnvloeden: gestreste proefpersonen overwegen minder mogelijkheden, maken keuzes sneller, zijn cognitief minder flexibel en reageren vaker automatisch in plaats van weloverwogen. Effecten van stress op keuzegedrag worden mede veroorzaakt door neurotransmitters en hormonen die vrijkomen in reactie op een (mogelijk) bedreigende situatie. Zo komt (nor)adrenaline vrij direct na activatie van het autonome zenuwstelsel, waardoor het lichaam extra alert is, en in verhoogde staat van paraatheid wordt gebracht. Corticosteroïden (cortisol bij mensen, corticosteron

174 Nederlandse samenvatting

bij knaagdieren) komen wat later vrij, na activatie van de hypothalamus-hypofyse- bijnier-as. Door corticosteroïden wordt het metabolisme verhoogd en worden ook neurale netwerken beïnvloed, wat bijdraagt aan de regulatie van de stressrespons. De effecten van corticosteroïden komen tot stand door binding ervan aan twee verschillende receptoren: de mineralocorticoïd receptor (MR) en de glucocorticoïd receptor (GR), die in de hersenen en ook in de rest van het lichaam voorkomen. Corticosteroïden blijken neuraal zowel een snelle als een langzame uitwerking te hebben. Via membraangebonden receptoren lijkt de snelle (non-genomische) werking van corticosteroïden bij te dragen aan het snelle optimaliseren van informatieverwerking en adaptief gedrag. Daarentegen lijken langzame (genomische) veranderingen veroorzaakt door corticosteroïden te zorgen voor de regulatie van de stressrespons en bij te dragen aan de normalisatie van het brein wanneer de stressvolle gebeurtenis voorbij is.

Beslissen en sekse Vaak wordt bij de bestudering van keuzegedrag alleen het gedrag van mannelijke proefpersonen of dieren onderzocht. Er blijken echter sekseverschillen voor te komen in keuzegedrag, zowel bij mensen als bij dieren. Zo zijn er verschillen gevonden tussen mannen en vrouwen op het gebied van risico-inschatting, gevoeligheid voor straf en beloning en cognitieve sturing van emotionele processen in het algemeen. Deze verschillen kunnen bijdragen aan sekseafhankelijke uitkomsten bij keuzetaken. Het lijkt erop dat mannelijke en vrouwelijke proefpersonen niet zozeer beter of slechter zijn in bepaalde taken, maar dat ze informatie anders verwerken: over het algemeen verzamelen mannen informatie vrij globaal, terwijl vrouwen meer gedetailleerd informatie verzamelen.

Beslissen en experimenten in dit proefschrift Onderzoek naar stress, sekseverschillen en besluitvorming staat nog in de kinderschoenen. Daarom heb ik in dit proefschrift onderzocht hoe stress en sekse keuzegedrag beïnvloeden bij mensen, ratten en muizen in opstellingen waarbij mensen en dieren keuzes moeten maken tussen opties waarvan de beloning op een korte termijn en lange termijn een verschillende waarde heeft. Ook heb ik bij mensen experimenten gedaan om de effecten van stress op (aspecten van) sociaal gedrag te onderzoeken. De bevindingen beschreven in hoofdstuk 2 tot en met 7 worden hieronder kort besproken.

Sekse, testcondities en serotonine In hoofdstuk 2 heb ik impulsiviteit, één aspect van keuzegedrag, onderzocht door middel van het testen van muizen in een delay-discounting taak. Delay-discounting is het verschijnsel dat de subjectieve waarde van zaken daalt met uitstel (de afstand in de tijd). Hierdoor kan het op een gegeven moment aantrekkelijker worden om onmiddellijk een kleine beloning te verkiezen boven een grotere beloning die pas na een lange tijd komt. In deze taak kregen muizen de keuze voorgelegd tussen een grote (vijf voedselpellets, soort snoepjes) en een kleine (één voedselpellet) beloning. Tegen de tijd dat alle muizen betrouwbaar genoeg voor de grote beloning kozen, werd langzaamaan een wachttijd ingevoerd, van 5 seconden in het begin naar 150 seconden aan het eind van het experiment. Nu moesten de muizen kiezen tussen

175 een kleine beloning die ze meteen kregen, of een grote beloning die ze kregen nadat ze enige tijd hadden gewacht. De veronderstelling is dat “ongeduldige” (impulsieve) dieren minder lang kunnen wachten op hun beloning en daardoor vaker voor de kleine directe dan voor de grote uitgestelde beloning zullen kiezen. Aangezien er slechts enkele studies over de verschillen tussen mannelijke en vrouwelijke muizen op deze taak zijn gepubliceerd, werden in dit hoofdstuk beide seksen getest. Mannelijke en vrouwelijke muizen maakten gedeeltelijk dezelfde keuzes. Echter, de vrouwelijke meer ‘ongeduldige’ muizen, die al bij relatief korte wachttijden de kleine beloning boven de grote beloning verkozen, leken vaker de kleine snelle beloning te verkiezen dan de mannelijke meer ‘ongeduldige’ muizen. Deze resultaten zijn besproken tegen de achtergrond van impulsiviteit, exploratief gedrag en het effect van andere testafhankelijke omstandigheden die wellicht het gedrag hebben beïnvloed, zoals het niveau van voedseldeprivatie dat nodig is om de dieren te motiveren om de test uit te voeren. Behalve door factoren zoals sekse en interne (honger)toestand, kan de uitkomst van de keuzetaak ook beïnvloed worden door hoeveel de dieren voorafgaand aan het testen zijn gehanteerd (in de hand genomen). Om deze reden is het testen van dieren in hun thuiskooi steeds populairder geworden. Het idee is dat dieren dan minder stress ervaren tijdens het experiment. Om toe te werken naar een minder stressvolle methode om knaagdieren op keuzegedrag te testen, is in hoofdstuk 3 een pilotstudie beschreven naar de mogelijkheid van het testen van delay- discounting in de thuiskooi bij ratten. Het belangrijkste resultaat van deze studie was dat bij het testen van delay-discounting in de thuiskooi vergelijkbare patronen werden gevonden als in de gebruikelijke opstelling (Skinner box). Verder duurde dit thuiskooi-protocol korter dan het traditionele protocol, wat een voordeel is wanneer dieren bijvoorbeeld tijdens de adolescentie worden getest. Door deze resultaten was er de mogelijkheid om te kijken of knaagdieren ook op meer complex keuzegedrag in de thuiskooi te testen zijn. Een veel gebruikte taak om complex keuzegedrag bij mensen te kunnen testen is de Iowa Gambling Task (IGT), waarbij proefpersonen geld kunnen verdienen door een keuze te maken uit vier verschillende opties. Twee opties geven winst van relatief grote geldbedragen (beloning), maar ook verlies van grote geldbedragen (straf), waardoor keuzes voor die opties op lange termijn nadelig zijn. Twee andere opties leiden tot winst van kleinere geldbedragen, maar ook tot verlies van minder grote geldbedragen, waardoor deze opties uiteindelijk voordelig uitpakken. Er zijn verschillende varianten van deze taak voor knaagdieren ontwikkeld. In hoofdstuk 4 is een protocol geïntroduceerd om een knaagdierenvariant van de Iowa Gambling Task (rIGT) in een thuiskooi-setting te testen. In de originele (niet- thuiskooi) versie van de rIGT kunnen ratten uit vier verschillende opties kiezen: in een eenvoudig doolhof staat aan het eind van iedere gang een bakje met een beloning (suikerpellets), een straf (suikerpellets behandeld met de bittere stof kinine; deze zijn niet lekker, maar wel eetbaar), of niets (leeg bakje). In een van de gangen vindt de rat vaak een kleine beloning (één suikerpellet) en soms een straf (suikerpellet behandeld met kinine), waardoor keuzes voor deze gang uiteindelijk leiden tot een behoorlijk aantal suikerpellets en dus voordelig zijn. In een andere gang vindt de rat meestal een straf en soms een grote beloning (drie suikerpellets), wat op de lange termijn ongunstig is (lager aantal suikerpellets). In de thuiskooivariant van de rIGT zijn er geen gangen, maar kunnen ratten hun keuze maken door hun neus te

176 Nederlandse samenvatting

steken in een van de twee gaten van een (operant) paneel in hun eigen kooi. Na deze (operante) respons krijgt de rat vervolgens suikerpellets of kininepellets. Ook dit thuiskooiprotocol duurde korter dan het originele protocol waarop deze variant is gebaseerd. Om de validiteit van deze benadering te testen werd in dit hoofdstuk bij ratten onderzocht welk effect manipulatie van de serotonineniveaus in de hersenen had op hun prestaties in operante taken op zowel keuzegedrag (rIGT) als geneigdheid om te gokken. In de totstandkoming van keuzegedrag speelt ook (on)zekerheid op beloning en bereidheid tot het nemen van bepaalde risico’s een rol, wat omschreven kan worden als gokneiging. Op basis van eerder onderzoek werd verwacht dat verlaging van serotonineniveaus leidt tot een verslechtering van besluitvorming. Neiging tot gokken werd getest in een traditionele Skinner box, waarbij de ratten de keuze hadden tussen een kleine beloning (twee pellets) die met zekerheid werd gegeven, en een grote beloning (zes pellets) die soms wel en soms niet werd gegeven en daarmee de keuze hiervoor een ‘gok’ maakten. Verlaging van serotonineniveaus leidde tot slechtere prestaties op keuzegedrag en een grotere neiging om te gaan gokken. Bovendien was in controleratten met normale serotonineniveaus het slechter presteren in de rIGT gerelateerd aan hun neiging tot gokken, maar dit was niet aan elkaar gerelateerd in ratten met verlaagde serotonineniveaus.

Stress, sekse en besluitvorming In eerder onderzoek was aangetoond in mensen dat een verhoging van cortisol, veroorzaakt door stress, bij mannen leidt tot een verslechtering van besluitvorming in de IGT. Studies in mensen en ratten hebben aangetoond dat corticosteroïden in verschillende tijdsdomeinen actief zijn. Daarom heb ik in hoofdstuk 5 de tijdsafhankelijke effecten van het stresshormoon corticosteron op keuzegedrag bij mannelijke ratten onderzocht. Ratten werden op twee verschillende tijdstippen voorafgaand aan het testen in de rIGT behandeld met corticosteron. Deze keer werd de rIGT in de traditionele opstelling met gangen gebruikt. Ratten kregen ofwel 30 minuten ofwel 3 uur voorafgaand aan de test een injectie met corticosteron of met placebo. Toediening van corticosteron 3 uur voorafgaand aan het testen, lang genoeg voor mogelijke ontwikkeling van langzame (genomische) effecten, had geen effect op het keuzegedrag. Daarentegen leidde toediening van corticosteron 30 minuten voorafgaand aan het testen (snelle werking) tot verstoord beloningsgericht keuzegedrag: controleratten leerden gaandeweg dat de optie met de kleine beloning uiteindelijk (op de lange termijn) meer oplevert dan de optie met de grote beloning; ratten die corticosteron toegediend hadden gekregen leerden dit niet en kozen even vaak voor de kleine als de grote beloning. Dit gedrag werd geassocieerd met een verhoogde c-Fos expressie, een maat voor neurale activiteit, in een aantal gebieden in de prefrontale schors, namelijk de laterale orbitofrontale cortex, de infralimbic cortex en de insula. Het belang van deze hersengebieden in beloningsgericht keuzegedrag onder stressvolle omstandigheden is verder onderzocht in hoofdstuk 6. Hierin is beschreven dat wanneer corticosteron direct toegediend werd in de infralimbic cortex het beloningsgerichte keuzegedrag wederom werd verstoord, net zoals in hoofdstuk 5 is beschreven. De resultaten voor de orbitofrontale cortex waren minder eenduidig, wellicht door (onbedoelde) weefselschade in het brein, waardoor eventuele behandelingseffecten van corticosteron niet waargenomen konden worden.

177 In hoofdstuk 7 is gekeken naar tijdsafhankelijke effecten van stress op sociaal keuzegedrag bij vrouwen. Sociaal keuzegedrag werd gemeten door proefpersonen te vragen om op een computer in verschillende situaties al dan niet een bepaald geldbedrag te doneren aan of te delen met een anonieme tegenspeler. Drie aspecten van sociaal keuzegedrag werden onderzocht. Zo werd altruïsme (onbaatzuchtigheid) getest, in dit geval door te vragen hoeveel de proefpersoon wilde doneren aan het goede doel Unicef. Bij het testen van vertrouwen en wederkerige uitwisseling bepaalde de proefpersoon hoeveel geld zij wilde geven aan een anonieme tegenspeler. Deze tegenspeler mocht, maar hoefde niet, vervolgens weer een bedrag teruggeven aan de proefpersoon, wat daarna werd verdrievoudigd. Bij het testen van (on)rechtvaardigheid verdeelde een anonieme tegenspeler een bedrag tussen zichzelf en de proefpersoon. De proefpersoon kon vervolgens dit bedrag aannemen, of juist niet, en in dat laatste geval ontving geen van beide spelers iets. Wanneer de proefpersoon het bod onrechtvaardig achtte, kon zij de anonieme tegenspeler zodoende altruïstisch straffen door te zorgen dat ook de tegenspeler niets ontving. In eerder onderzoek was aangetoond dat experimenteel geïnduceerde stress bij mannen leidt tot minder vrijgevigheid en een andere neiging tot altruïstisch straffen op een tijdsafhankelijke manier. Specifieker gezegd, 75 minuten nadat mannen werden onderworpen aan een stresstaak waren deze proefpersonen minder geneigd om altruïstisch te straffen dan mannen die werden getest direct na de stresstaak. In hoofdstuk 7 onderzochten we of vrouwen vergelijkbaar reageerden na experimenteel geïnduceerde stress. Het bleek dat, in tegenstelling tot de uitkomsten bij de mannen, vrouwen niet werden beïnvloed door stress in hun neiging tot altruïstisch straffen of altruïsme in het algemeen. Aangezien vrouwen die hormonale anticonceptiva gebruikten een verlaagde stressrespons lieten zien na de stresstaak, zijn de bevindingen onder andere in deze context besproken. Geconcludeerd werd dat ook in sociaal keuzegedrag onder stress sekseverschillen lijken voor te komen.

Beslissen en conclusies De bevindingen van dit proefschrift worden besproken en in de context van de huidige literatuur geplaatst in hoofdstuk 8. Verder heb ik een aantal vragen voor vervolgonderzoek geformuleerd. We kunnen uit de studies in dit proefschrift een aantal voorzichtige conclusies trekken. Ten eerste, stress, in het bijzonder het stressgerelateerde hormoon corticosteron, heeft een effect op besluitvorming, onder meer via prefrontale hersenstructuren. Deze conclusie volgt uit de experimenten met ratten in de rIGT. Ten tweede, stress heeft mogelijk een verschillend effect bij mannelijke en vrouwelijke proefpersonen op besluitvorming. Mogelijk is dit ook het geval bij proefdieren. Deze conclusie kan worden getrokken uit de combinatie van de experimenten met mannelijke en vrouwelijke muizen in delay-discounting, en het experiment naar het effect van stress op aspecten van sociaal keuzegedrag bij mensen. Ten derde, thuiskooivarianten van testopstellingen zijn een valide en snelle manier om experimenten naar keuzegedrag uit te voeren. Bovendien kunnen deze de impact van verstoringen (potentiële stressoren, zoals hanteren en nieuwe omgeving) op de uitkomsten verminderen. Deze conclusie kan worden getrokken uit de experimenten met ratten in delay-discounting en beloningsgericht keuzegedrag in de rIGT.

178 Nederlandse samenvatting

Beslissen en implicaties Iedereen krijgt weleens te maken met het nemen van belangrijke beslissingen onder stressvolle omstandigheden en in complexe sociale situaties. Door onderzoek, zoals beschreven in dit proefschrift, leren we beter begrijpen wat het verschil is in keuzegedrag tussen mannen en vrouwen, hoe dit onderscheiden en beïnvloed wordt en welke hersengebieden hierbij betrokken zijn. Deze kennis geeft de mogelijkheid om bijvoorbeeld voorspellingen te doen over hoe individuen zullen handelen onder invloed van stress. Dit is van belang voor beroepen waarin stress besluitvorming kan beïnvloeden, met grote persoonlijke, financiële of maatschappelijke gevolgen indien onjuiste beslissingen worden genomen.

179 180 CURRICULUM VITAE

181 Curriculum Vitae

Susanne Koot was born on June 12th 1983 in Nijmegen. In 2001 she graduated from secondary school (Johan de Witt-gymnasium, Dordrecht) and obtained her Bachelor of Science degree in Animal Husbandry in 2005 at HAS Den Bosch, University of Applied Sciences. After working as a teacher at the same university, she continued her studies with the special track Ethology and Animal Welfare within the master programme Bioveterinary Sciences, part of Biomedical Sciences at Utrecht University. During her research master she first performed an internship on welfare monitoring in chimpanzees at Stichting AAP, Almere, under supervisor of dr. Francien de Jonge and drs. Eva Schippers. After that, she went to Italy for an internship on the neurobiology of impulsivity in rodents at the Section of Behavioral Neuroscience, Department of Cell Biology and Neurosciences, at Istituto Superiore di Sanità, Rome, under supervision of dr. Walter Adriani, dr. Gianni Laviola and dr. Ruud van den Bos, which resulted in several publications. During her master, she worked as interviewer at the Faculty of Social Sciences (Utrecht University) on the project Research Adolescent Development and Relationships, and as research assistant at the Faculty of Psychology & Education (VU Free University Amsterdam) on the project Spelregels. After obtaining her Master of Science degree in 2008, she worked with dr. ir. Kees van Reenen at the Animal Sciences Group, part of Wageningen UR Livestock Research, on the welfare of veal calves and circus animals. Subsequently, she moved to Costa Rica to study the behaviour of white faced capuchin monkeys at the field site of dr. Susan Perry, Department of Anthropology, UCLA, Los Angeles (USA). In October 2009, Susanne started working on her PhD project under supervision of Prof. dr. Marian Joëls, Prof. dr. Louk Vanderschuren and dr. Ruud van den Bos. She performed her experiments at the Section of Behavioral Neuroscience, Department of Cell Biology and Neurosciences (Istituto Superiore di Sanità, Rome), and at the Department of Animals in Science and Society at the Faculty of Veterinary Medicine (Utrecht University), to continue her work at the Rudolf Magnus Institute of Neuroscience (University Medical Center Utrecht). The results of this research project are presented in this thesis.

182 Curriculum Vitae

List of publications

Koot, S., Baars, A., Hesseling, P., Van den Bos, R., Joëls, M. (2013) Time-dependent effects of corticosterone in a rodent model of the Iowa Gambling Task? Neuropharmacology 70C: 306-315.

Vinkers, C., Zorn, J., Cornelisse, S., Koot, S., Houtepen, L., Olivier, B., Verster, J., Kahn, R., Boks, M., Kalenscher, T., Joëls, M. (2013). Time-dependent changes in altruistic punishment following stress. Psychoneuroendocrinology. doi:pii: S0306- 4530(12)00425-8.

Adriani, W., Koot, S., Columba-Cabezas, S., Romano, E., Van den Bos, R., Granstrem, O., Ali, S.F., Laviola, G. (2012). Immunization with DAT fragments is associated with long-term striatal impairment, hyperactivity and reduced cognitive flexibility in mice. Behavioral and Brain Functions 8:54

Koot, S., Zoratto, F., Cassano, T., Colangeli, R., Laviola, G., Van den Bos, R., Adriani, W. (2012). Compromised decision-making and increased gambling proneness following dietary serotonin depletion in rats. Neuropharmacology 62: 1640-1650

Koot, S., Adriani, W., Saso, L., Van den Bos, R., and Laviola, G. (2009) Home-cage testing of delay-discounting in rats. Behavior Research Methods 4: 1169-1176

Koot, S., Van den Bos, R., Adriani, W., and Laviola, G. (2009) Gender differences in delay-discounting under mild food restriction. Behavioural Brain Research 1: 134-143

Publications in preparation

Koot, S., Vinkers, C., Zorn, J., Olivier, B., Boks, M., Van den Bos, R., Kalenscher, T., Joëls, M. (in preparation). Temporal domains in social decision-making following stress in women.

Koot, S., Koukou, M., Baars, A., Hesseling, P., Van ’t Klooster, J., Van den Bos, R., Joëls, M. (in preparation) Corticosterone and decision-making in male Wistar rats: the effect of corticosterone application in the infralimbic and orbitofrontal cortex.

183 184 DANKWOORD ACKNOWLEDGEMENTS

185 Daar is het dan: mijn proefschrift! De weg ernaartoe is intrigerend, spannend, vol met stressvolle en ook euforische momenten. Het is in ieder geval niet gemakkelijk. Gelukkig leg je deze weg niet alleen af, dat zou niet eens lukken. Veel mensen hebben mij hierin begeleid, geholpen en mentaal bijgestaan. Hen wil ik hierbij graag bedanken.

Marian Joëls, mijn eerste promotor, hartelijk dank dat je me de kans hebt gegeven bij je te mogen promoveren. Jij bracht het nodige inzicht en vertrouwen, en het zo belangrijke pragmatisme. Je zorgvuldige efficiëntie werkt aanstekelijk en inspirerend. Dank je wel voor je helderheid, je motivatie en de kansen die je me hebt geboden!

Ook wil ik Louk Vanderschuren, mijn tweede promotor, van harte bedanken. Je kritische noten leken in eerste instantie vaak pittig, maar bleken uiteindelijk zeer waardevol. Dank voor je support.

Ruud van den Bos, mijn co-promotor, jij maakte het voor mij als HBO-er mogelijk dat ik kon beginnen aan een research master. Vervolgens gaf je me de kans mijn major stage in Italië uit te voeren, wat tot drie publicaties heeft geleid. Tot slot vroeg je me na wat omzwervingen terug te keren in het lab om te werken aan dit proefschrift. Kortom, jouw betrokkenheid is essentieel geweest in mijn academische carrière tot nu toe. Heel erg bedankt voor je vertrouwen en alles wat je me al die jaren hebt bijgebracht, van academische vaardigheden tot zelfinzicht.

Walter Adriani, thank you for adopting me at the Istituto Superiore di Sanità. Being a master student, you were the first to introduce me into the world of rodent labwork. Thank you for your patience, clear explanations and confidence in my abilities. I am proud of the four papers we wrote together! Gianni Laviola, thank you for welcoming me in your lab and giving me the opportunity to run my experiments in Rome.

Naast de inhoudelijke begeleiding ben ik een aantal personen op het lab veel dank verschuldigd voor de hulp bij het uitvoeren van mijn experimenten bij het departement Dier in Wetenschap en Maatschappij (DWM), faculteit Diergeneeskunde (Universiteit Utrecht). Annemarie, natuurlijk had je groot gelijk: het komt altijd allemaal goed! Heel erg bedankt voor je ervaring, hulp bij opzet en uitvoering van experimenten, alle technieken die je me hebt bijgebracht. Je mentale steun is onmisbaar geweest, zowel in het lab als daarbuiten. Ons wekelijkse uurtje yoga was daarbij de kers op de taart. Wat heb ik veel van je geleerd! Peter, altijd positief ingesteld en nimmer te beroerd om nog een tandje bij te zetten. Je bent een houvast geweest waarop ik telkens blind kon vertrouwen tijdens experimenten. Dank je wel voor alle steun en de lol die we hebben gehad! Dank je wel, José, dat je hebt willen bijspringen wanneer het nodig was, met het tropenrooster rondom het laatste experiment als hoogtepunt! Susanne, dank voor je ondersteuning in het vinden van juiste stoffen en producten voor mijn experimenten. Marla, naast jou heb ik me altijd iemand met twee linkerhanden gevoeld, dus wat ben ik blij dat je hebt geholpen bij het labwerk zodra er een pipet bij aan te pas kwam!

186 Dankwoord | Acknowledgements

A couple of students have been closely involved during my final two experiments. Magda, thank you for being always precise and hard working, I am glad I could trust you blindly with the rats. At last, all the hard work resulted in a nice chapter! Remi, Robin, Simeon en Chantal, hartelijk dank voor jullie onmisbare hulp en inzet bij de CHOICE. I learned a lot from you and wish you all the best in your careers!

Hoewel ik het onderzoek beschreven in dit proefschrift uitgevoerd heb bij drie verschillende instituten, is DWM telkens mijn basis geweest. Zo waren hier mijn kamergenootjes, de andere AIO’s en postdocs mijn dagelijkse steun en toeverlaat. Hetty, hoewel we al die jaren nooit op dezelfde projecten hebben gewerkt, liepen onze paden parallel. Dank je wel dat je er was op moeilijke momenten, voor de broodnodige lol en ontspanning, ook in Sevilla en Barcelona! Marijke en Janneke, jullie waren hele gezellige kamergenootjes! Leonie en Amber, jullie gaven als ”ouderejaars” het goede voorbeeld hoe zaken aan te pakken. Mede-AIO’s Manon, Maaike, Marcia en Joost, dank voor jullie inspirerende houding, ideeën en voor alle hulp onderling! Alle anderen bij DWM, hartelijk dank voor alle jaren van gezelligheid, koffiepraatjes en borrels, jullie waren ook mijn thuis!

Op het Rudolf Magnus Instituut voor Neurowetenschappen (Universitair Medisch Centrum Utrecht) heb ik bij de ‘stressgroep’ de inhoudelijke inspiratie en uitdaging gevonden. Sandra, jouw organisatietalent en wetenschappelijke scherpheid zijn een voorbeeld voor mij, maar je bent toch vooral ook heel gezellig tijdens congressen en buiten het werk! Natasha, your positivism is inspiring, but I am glad we also understand each other during less pleasant moments. Keep up the good work! Christiaan, dank je wel dat je me de mogelijkheid hebt gegeven om met de CHOICE aan de slag te gaan. Ik heb je hulp en doortastendheid enorm gewaardeerd! Lotte en Jelle, dank voor alle hulp bij de CHOICE en plezier onder het genot van een ‘Sjoefje’! Henk, Angela, Anup, Marloes, Manila and all others of the ‘stress group’: thank you for the inspiring meetings!

The experiments described in three chapters of this thesis were performed at the Department of Cell Biology and Neurosciences, at the Istituto Seriore di Sanità, Rome, Italy. Francesca, I admire your attention for details and hardworking nature. Thank you for our periods of mutual support! Eva, your serious but joyful scientific approach was a great example to me! Pietro and Alessio, thank you for your tireless help with the home cage panels, even during weekends! Alessandra, Augusto, Arianna, Emilia, Fiorenza, Valentina and all others, thank you for the lovely time I had in Rome, with lots of fun and (almost) always good food!

Maar er is natuurlijk meer dan werk alleen. Gelukkig was er ook een hele fijne achterban om me te steunen en soms af te leiden. Veel vrienden en familie hebben gezorgd voor de nodige ontspanning en relativering.

Pauline, we kennen elkaar al zo lang dat het niet nodig is om elkaar heel vaak te zien, maar als het dan zo ver is, is het altijd direct hilarisch als vanouds! Dank je wel voor de gezellige dagen en etentjes. Wat fijn om mijn Utrechtse vriendinnen dicht in de buurt te hebben, ook wanneer

187 ik zelf ver weg was! Maaike, Sylvia en Carolina, afgelopen jaren hebben we elkaar soms heel veel maar vaak ook te weinig gezien. Gelukkig heb ik nu weer alle tijd voor eindeloze kletssessies bij koppen thee, etentjes, barbecues en relaxen in het park. Dank jullie wel dat jullie er voor me zijn! In de periode dat ik werkte aan dit proefschrift heb ik verschillende huisgenootjes gehad. Nadine, het zal vast niet altijd makkelijk zijn geweest om met een gestresste AIO samen te wonen, maar ik ben je enorm dankbaar voor alle goede gesprekken, heerlijke toetjes en filmavonden samen. Paola, thank you for your mental support and making me truly feel home again in Rome! HAS-vriendinnetjes Mirella, Sandra, Myrthe, Carian en Naline, dank jullie wel voor de gezellige avonden en jullie ontnuchterende kijk op de wereld buiten de academische bubbel! Voor mij blijkt capoeira, een Braziliaans vechtsport, de ideale manier om na een lange stressvolle dag het hoofd helemaal leeg te maken. Bram en Mathijs, jullie inspirerende lessen zorgen altijd voor hernieuwde energie en een opgefleurd humeur, veel dank daarvoor! Saskia, Dorothee, Jackson, Maud, Dieuwke, Debby, Marieke, Mirella, Grande, Cigana, Alex, Luis, alle anderen van Grupo Capoeira Brasil Utrecht, tutti gli amici di Soluna Capoeira Roma, dank voor de gezelligheid en het incasseren van alle trappen. Jullie waren mijn perfecte uitlaatklep na lange dagen van experimenten en schrijfdeadlines!

Cari Rita, Antonio, Carmen e Maria, grazie per essere la mia famiglia italiana: avere la possibilitá di ricaricarmi durante le vacanze con voi mi è sempre di grande aiuto. Gerard en Liesbeth, waar familie vroeger een “gegeven” leek, wordt het voor mij steeds belangrijker. Ik geniet enorm van de gezellige dagen met jullie en de kids. Charlotte, wat geweldig dat wij zoveel delen, qua verleden, familieband, werk en andere interesses. Onze sterke band betekent heel veel voor me! Zusjes, wat geweldig dat jullie mijn paranimfen willen zijn, ik voel me trots en vereerd! Dank voor alle hulp en feedback. En dat Koot, Koot, Koot and Koot paper komt er! Lieve papa en mama, dank voor het veilige nest waarin we zijn opgegroeid en nog steeds graag neerstrijken. Jullie zijn de basis van het zelfvertrouwen dat zo nodig was bij de diverse paden die ik heb willen bewandelen. Dank jullie wel voor alle ruimte en de onvoorwaardelijke steun hierbij. Mam, jij ziet altijd mijn meest innerlijke zelf, waar ter wereld ik me ook bevind en wat ik ook doe. Dank voor de spiegel die je me daarmee voorhoudt. Pap, de appel valt niet ver van de boom. Veel dank voor het grote voorbeeld dat je voor me bent, met immer weldoordachte raad en altijd direct in de bres.

Prisco, without the work described in this thesis we would not have met each other. However, we clearly don’t need this work to stay together. I am so happy we share our lives together! Thank you for being my buddy, my counterweight, my love, my all. Thank you for being who you are!

188