Defensive Avoidance in Paranoid Delusions: Experimental and Compu- Tational Approaches

Home , Cognitive dissonance, Paranoid anxiety

DEFENSIVE AVOIDANCEIN PARANOID DELUSIONS: EXPERIMENTAL AND COMPUTATIONAL APPROACHES

A THESIS SUBMITTEDTO THE UNIVERSITYOF MANCHESTER FORTHEDEGREEOF DOCTOROFPHILOSOPHY (PHD) IN THE FACULTY OF MEDICALAND HUMAN SCIENCES

MICHAEL MOUTOUSSIS

SCHOOLOF PSYCHOLOGICAL SCIENCES 2011 2 CONTENTS

Abbreviations & Key Symbols 11

Abstract 15

Declaration 17

Submission in Alternative Format 21

Acknowledgements 23

The Author 25

1 Introduction 27 1.1 Importance...... 27 1.2 Schools of thought on paranoid delusions...... 29 1.2.1 Paranoid delusions in psychiatry...... 29 1.3 The psychological understanding of paranoia...... 30 1.3.1 Psychodynamics of paranoia...... 30 1.3.2 Cognitive-behavioural approaches to paranoia...... 34 1.4 Thesis outline...... 36

2 Paranoia and Conditioned Avoidance 37 2.1 Summary...... 37 2.2 The importance of threat-related content in delusions...... 38 2.3 Threats & Attribution theory...... 39 2.3.1 ‘Poor-me’ and ‘bad-me’ paranoia...... 41 2.4 The Conditioned Avoidance paradigm...... 42 2.4.1 Linking paranoid delusions and the CAR...... 43 2.4.2 The experience of threat in the CAR and in paranoia...... 44 2.4.3 Safety behaviours may help maintain paranoia...... 44 2.4.4 The avoidance of internal states in Poor-me paranoia...... 45

3 CONTENTS

2.5 CAR and paranoia: Conclusions...... 46

3 Computational methods for cost-based decision making 49 3.1 Summary...... 49 3.2 A modelling framework: ‘Cached’ & ‘tree-search’ models...... 50 3.3 Temporal-difference models...... 52 3.3.1 Fixed parameters used in the Advantage-learning model of avoidance...... 56 3.4 Tree-search & related models...... 57 3.4.1 Costed Bayesian model...... 57 3.4.2 Sequential probability ratio test model...... 59 3.5 Fitting models to data...... 60 3.5.1 Full expectation-maximisation ﬁtting...... 61 3.5.2 Experimental Bayesian distribution...... 65 3.6 Model Evaluation...... 65 3.6.1 Bootstrap methods...... 66 3.6.2 Model comparison and the Bayesian information criterion.... 71 3.7 Computational methods: Conclusions...... 71

4 A Temporal Difference Account of Avoidance 73 4.1 Summary...... 73 4.2 Conditioned avoidance (CAR) and reinforcement learning...... 74 4.2.1 The CAR experimental paradigm...... 75 4.3 The Advantage-learning model...... 78 4.4 Results...... 80 4.4.1 Simulation of escape-from-fear learning...... 80 4.4.2 Simulation of normal CAR learning...... 84 4.4.3 Simulation of dopaminergic manipulations in the CAR...... 86 4.5 Discussion...... 86 4.5.1 Neurobiological substrate...... 90 4.5.2 Relevance of the Avoidance model...... 92 4.6 Temporal-difference modelling of avoidance: Conclusions...... 94

5 Avoidance and Jumping-to-Conclusions in Paranoia 95 5.1 Summary...... 95 5.2 Jumping-to-Conclusions: an introduction...... 96 5.2.1 Delusions & Bayesian reasoning...... 96 5.2.2 Explanations of the Jumping-to-Conclusions (JTC) bias..... 97 5.2.3 The ideal-observer Bayesian approach...... 101 5.2.4 Comparison with the Sequential Probability Ratio Test...... 103

4 CONTENTS

5.3 Methods...... 105 5.4 Results...... 106 5.5 Discussion...... 111 5.5.1 High-noise processing vs. the high-sampling-cost hypothesis... 111 5.5.2 Paranoid decision-making and noise...... 112 5.5.3 Bayesian vs. threshold-driven decisions in paranoia...... 113 5.5.4 Methodological advances...... 114 5.5.5 Limitations of the present study...... 116 5.6 Modelling the JTC bias: Conclusions...... 117

6 Empirical study of defensive avoidance 119 6.1 Summary...... 119 6.2 Background & importance...... 120 6.3 Ethical issues...... 122 6.3.1 The possibility of distressing or harming participants...... 122 6.3.2 Vulnerable groups & informed consent issues...... 123 6.4 Empirical study: Design & research Instruments...... 123 6.4.1 Participant groups...... 123 6.4.2 Inclusion & exclusion criteria...... 124 6.4.3 Data-gathering procedures and instruments...... 125 6.4.4 Outcome measures & power calculations...... 129 6.4.5 Analysis strategy...... 130 6.5 Interview protocol...... 130 6.5.1 Identiﬁcation & recruitment of participants...... 130 6.5.2 The research interview...... 131 6.6 Results...... 131 6.6.1 Summary...... 131 6.6.2 Participant welfare...... 132 6.6.3 Key hypotheses...... 133 6.6.4 Regression analyses...... 146 6.7 Discussion...... 153 6.7.1 Poor-me paranoia is not due to increased defensive avoidance.. 153 6.7.2 Interpretation of the results...... 153 6.7.3 Limitations of the present empirical study...... 158 6.7.4 Future directions...... 158 6.8 Empirical investigation of defensive avoidance: Conclusions...... 159

7 Defensive avoidance in paranoia: Synthesis and general discussion 161 7.1 Summary...... 161

5 CONTENTS

7.2 An integrative approach to paranoid delusions...... 162 7.2.1 Avoidance and self-discrepancies in paranoia...... 163 7.2.2 The role of self-esteem in paranoia...... 167 7.3 Limitations...... 168 7.4 Implications for future research & therapy...... 170 7.4.1 Clinical implications...... 170 7.4.2 Research implications...... 171

Contribution 175

References 177

Final word count: 52927

6 LIST OF FIGURES

1.1 Thoughts and feelings related to paranoia are common and distressing.. 28

3.1 State space for the TD model of the CAR...... 53 3.2 Probability to decide at each step of the beads task (CB model)...... 62 3.3 How typical an example of model output would the data be?...... 67 3.4 Monte-Carlo simulations of experimental summary statistics...... 69 3.5 Monte-Carlo simulations of model macroparameters...... 70

4.1 Conditioned avoidance: Experimental interventions & response...... 75 4.2 CAR: Key experimental results...... 77 4.3 Information ﬂow in the actor-critic model...... 80 4.4 Escape-From-Fear simulation results...... 82 4.5 Escape-From-Fear – Value Learning...... 83 4.6 Escape-From-Fear – Advantage Learning...... 83 4.7 Simulation of normal CAR learning...... 85 4.8 CAR learning under DA block...... 87 4.9 CAR learning under reversal of DA block...... 88

5.1 Workings of the SPRT model...... 104 5.2 Best-ﬁt parameters and BIC values for different participant groups.... 109 5.3 Illustration of the effect of noise in the healthy and paranoid case..... 110 5.4 Differences between Costed-Bayesian and SPRT models revealed by urgency plot...... 115 5.5 Marginal posterior density plot reveals possible ﬁne structure of data... 116

6.1 Request for ’Feared Self’ attributes...... 127 6.2 Persecution and deservedness according to clinical group...... 134 6.3 Self-Discrepancies and paranoia...... 138 6.4 Depression and self-discrepancies...... 140 6.5 AAQ-2 and MC scores in paranoia...... 142 6.6 Distribution of change in response time with IA discrepancy...... 144

7 8 LIST OF TABLES

2.1 Positive symptoms of patients recruited to the SoCRATES study..... 39

5.1 Best-ﬁt parameters and BIC values for JTC tasks...... 107

6.1 Key descriptive statistics...... 135 6.2 Paranoia grouping vs. Self-Discrepancies...... 137 6.3 Paranoia grouping vs. AAQ-2 and social desirability...... 141 6.4 Depression × paranoia analysis...... 141 6.5 Interactive measures – descriptive statistics...... 145 6.6 Overall change in median response times...... 145 6.7 ANOVA of response times...... 145 6.8 ANOVA of Fraction of high-engagement responses...... 146 6.9 Regression analysis for auxiliary variables...... 149 6.10 Regression analysis - PADS-P...... 149 6.11 Regression analysis - undeserved paranoia...... 150 6.12 Exploratory regression analysis - PADS-P...... 152 6.13 Exploratory regression analysis - AAQ-2...... 152

9 10 ABBREVIATIONS & KEY SYMBOLS

Abbreviations 5HT 5 hydroxytryptamine (Serotonin; if followed by additional symbols, e.g. 5HT2a, it denotes a serotonin receptor type), 43 AAQ-2 ‘Acceptance and action questionnaire’, second version, 131 ACT Acceptance and Commitment Therapy, 162 AID Actual–Ideal discrepancy, 121 ANOVA Analysis of variance, 130 AR Avoidance response (or responding), 42 AS Aberrant salience (theory of psychosis), 29 bCI Bootstrap confidence interval, 106 BIC Bayesian Information Criterion, 71 BMP Bad-Me, ‘deserved’ paranoia, 35 BOLD Blood-Oxygen-level dependent [changes in the magnetic properties of blood reflecting changes in tissue haemodynamics], 163 BPRS Brief Psychiatric Rating Scale, 124 CAR Conditioned Avoidance Response (or Responding), 15 CB Costed-bayesian model, 57 CBT Cognitive behaviour theory (or therapy), 34 CC Clinical control participant, 131 CMHT Community Mental Health Team, 133 CPA Care Program Approach, 133 CPZ eq. Chlorpromazine equivalent dose, 147 CS Conditioned stimulus, 42 D2 Dopamine type 2 receptor, 43 DI Overall Discrepancy-from-Ideal measure, 136 DSM Diagnostic and statistical manual (usually followed by a version indicator, e.g. DSM-III-R), 39 DtD Draws-to-decision, 69 EA Experiential avoidance, 35 EM Expectation - maximization model-fitting method, 49

11 ABBREVIATIONS & KEY SYMBOLS

GNU GPL This is the GNU General Public Licence (‘GNU’ itself is a software project with a somewhat esoteric recursive acronym), 106 GSR Galvanic skin response, 163 HADS Hospital Anxiety and Depression scale, 132 HADS-A Anxiety score from the HADS, 139 HADS-D Depression score from the HADS, 139 HC Healthy control participant, 131 HDSC High-discrepancy self-characteristics, 143 IAT Implicit association test, 168 ICD-10 International Classiﬁcation of Diseases, 10th version, 125 JTC Jumping-to-Conclusions bias, 15 LDSC Low-discrepancy self-characteristics, 143 MC Marlowe-Crowne social desirability scale (or score), 131 MHSU Mental health services usage (or user), 125 NAS Nucleus accumbens septi, 43 NCS Need for closure scale, 99 OAD Other–Actual discrepancy, 121 ORES Observer-rated engagement scale, 144 ORT Object relations theory, 31 PADS-D Deservedness score from the ‘paranoia and deservedness scale’, 133 PADS-P Paranoia score of the ‘paranoia and deservedness scale’, 131 PANSS Positive and Negative Syndrome Scale, 38

PE Prediction error (see also symbol δV ), 53 PMP Poor-Me, ‘undeserved’ paranoia, 35 RL Reinforcement-learning, 50 SoCRATES Study of cognitive realignment therapy in early schizophrenia, 38 SPRT Sequential probability ratio test, 49 SWLSTG South West London & St. George’s NHS Mental Health Trust, 130 TD Temporal difference, 50 US Unconditioned stimulus, 42 VIF Variance Inﬂation Factor, 147 Key Symbols θ Threshold (unless it refers to the standard parametrisation of the gamma distribution, when it is simply the scale parameter), 59 F ∆ (tmed) Fractional change in median times spent talking about non-aversive versus aversive self-characteristics, 143

δV Error estimated in the prediction of state values (Prediction error), 53 π(a ; s) Probability of taking action a in state s (policy probability), 52 τ Temperature, or noise, or exploration parameter, 56 CK Cost of outcome K, 58

12 ABBREVIATIONS & KEY SYMBOLS

CImin Condition Index associated with the smallest eigenvalue, 147 dj Data resulting from jth sample from a population or distribution, 61

DK Deciding on option K, 57

H0 Null hypothesis, 106 L(M, D) Likelihood of model M when the data D has been observed, 66 l(t) log-likelihood ratio at time t, 59

PU Undeserved paranoia, 131 Q(a ; s) Value of taking action a in state s (action value), 52

Q[v; dj] Approximate recognition distribution of causes v when data dj is known, 63 tSC Time spent talking about a self-characteristic, 143 V (s) Value of state s, 52 F Negative-free-energy, 63

13 14 ABSTRACT

This abstract summarises the thesis entitled Defensive Avoidance in Paranoid Delu- sions: Experimental and Computational Approaches, submitted by Michael Moutous- sis to The University of Manchester for the degree of Doctor of Philosophy (PhD) in the faculty of Medical and Human Sciences, in 2011.

The possible aetiological role of defensive avoidance in paranoia was investigated in this work. First the psychological significance of the Conditioned Avoidance Response (CAR) was reappraised. The CAR activates normal threat-processing mechanisms that may be pathologically over-activated in the anticipation of threats in paranoia. This may apply both to external threats and also to threats to the self-esteem. A temporal-difference computational model of the CAR suggested that a dopamine- independent process may signal that a particular state has led to a worse-than-expected outcome. On the contrary, learning about actions is likely to involve dopamine in signalling both worse-than-expected and better-than-expected outcomes. The psychological mode of action of dopamine blocking drugs may involve dampening (1) the vigour of the avoidance response and (2) the prediction-error signals that drive action learning. Excessive anticipation of negative events might lead to inappropriately perceived high costs of delaying decisions. Efforts to avoid such costs might explain the Jumping- to-Conclusions (JTC) bias found in paranoid patients. Two decision-theoretical models were used to analyse data from the ‘beads-in-a-jar’ task. One model employed an ideal-observer Bayesian approach; a control model made decisions by weighing evidence against a fixed threshold of certainty. We found no support for our ‘high cost’ hypothesis. According to both models the JTC bias was better explained by higher levels of ‘cognitive noise’ (relative to motivation) in paranoid patients. This ‘noise’ appears to limit the ability of paranoid patients to be influenced by cognitively distant possibilities. It was further hypothesised that excessive avoidance of negative aspects of the self may fuel paranoia. This was investigated empirically. Important self-attributes were elicited in paranoid patients and controls. Conscious and non-conscious avoidance were assessed while negative thoughts about the self were presented. Both ‘deserved’ and ‘undeserved’ persecutory beliefs were associated with high avoidance/control strategies in general, but not with increased of avoidance of negative thoughts about the self. On the basis of the present studies the former is therefore considerably more likely than the latter to play an aetiological role in paranoia. This work has introduced novel computational methods, especially useful in the study of ‘hidden’ psychological variables. It supported and deepened some key hypotheses about paranoia and provided consistent evidence against other important aetiological hypotheses. These contributions have substantial implications for research and for some aspects of clinical practice.

15 16 DECLARATION

The University of Manchester PhD Candidate Declaration

Candidate Name: Michael Moutoussis

Faculty: Medical and Human Sciences

Thesis Title: Defensive Avoidance in Paranoid Delusions: Experimental and Compu- tational Approaches

Declaration to be completed by the candidate:

I declare that no portion of this work referred to in this thesis has been submitted in support of an application for another degree or qualiﬁcation of this or any other university or other institute of learning.

Signed: Date: March 8, 2011

17 18 COPYRIGHT

i Copies of this thesis, either in full or in extracts and whether in hard or electronic copy, may be made only in accordance with the Copyright, Designs and Patents Act 1988 (as amended) and regulations issued under it or, where appropriate, in accordance with licensing agreements which the University has from time to time. This page must form part of any such copies made. ii The ownership of certain Copyright, patents, designs, trade marks and other intellectual property (the “Intellectual Property”) and any reproductions of copyright works in the thesis, for example graphs and tables (“Reproductions”), which may be described in this thesis, may not be owned by the author and may be owned by third parties. Such Intellectual Property and Reproductions cannot and must not be made available for use without the prior written permission of the owner(s) of the relevant Intellectual Property and/or Reproductions. iii The author of this thesis (including any appendices and/or schedules to this thesis) owns certain copyright or related rights in it (the “Copyright”) and s/he has given The University of Manchester certain rights to use such Copyright, including for adminis- trative purposes. iv Further information on the conditions under which disclosure, publication and com- mercialisation of this thesis, the Copyright and any Intellectual Property and/or Re- productions described in it may take place is available in the University IP Policy (see http://www.campus.manchester.ac.uk/medialibrary/policies/ intellectual-property.pdf), in any relevant Thesis restriction declarations deposited in the University Library, The University Library’s regulations (see http: //www.manchester.ac.uk/library/aboutus/regulations) and in The University’s policy on presentation of Theses.

19 20 SUBMISSION IN ALTERNATIVE FORMAT

Author’s and Collaborators’ Contributions to this work

This thesis contains updated versions of three papers that have already been published, namely chapters2,4 and5, as well as an expanded version of a paper submitted for publication, chapter6. In all of these works Michael Moutoussis contributed the key ideas to be researched, derived the versions of the equations used in the mathematical models involved, carried out the relevant programming and simulations, and drafted all the papers for publication. The only aspect of computer programming that Michael Moutoussis did not carry out himself was the computerised ‘self-lines’ and ‘engagement with negative attributes’ instrument described in section 6.4.3. Instead the author designed the tasks involved and supervised their implementation. Mr Yu Li, staff of the University of Manch- ester, did the programming. Michael Moutoussis, under the supervision of Professor Richard Bentall, designed and carried out all aspects of the empirical study described in chapter6.

The roles of the co-authors of work presented here have as follows. Dr El-Deredy retained a supervisory role throughout this work and contributed to the papers through speciﬁc, circumscribed editing. Dr Jonathan Williams co-authored chapter4 through pro- vision of similar, editorial input. Professor Peter Dayan provided advice in the selection of mathematical methods, checked the mathematics described in chapters3–5 and provided editorial input to the paper drafts. Professor Richard Bentall provided extensive supervision of all aspects of the present work and edited all the drafts that the author prepared for publication.

21 SUBMISSION IN ALTERNATIVE FORMAT

Permission to submit in Alternative Format

22 ACKNOWLEDGEMENTS

I would like to thank the following people for supporting me in this work: • All the patients that trusted me with their worries and fears through the years, and especially those that participated in the present research project.

• The clinicians, and especially the psychiatric nurses and doctors, that made me welcome in their wards and team bases. I would like to mention here the staff of Lilacs and Orchids wards in Tolworth Hospital, Ward 2 in Springﬁeld hospital, the Community Forensic Team, Richmond Community Mental health team and Kingston & Richmond Assertive Outreach Teams. Thank you to Chris Ken, Mireia Pujol, Jale Punter, Sean Whyte, Maria Alonso, John Murphy, Antonio Corniello, Hayley Ponsford, Tracy Burrows and several others who referred potential study participants to me.

• Ann Moutoussi & Quentin Huys, for some very energizing discussions and insights; and Jonathan Williams, one of the few people in the UK to excel both in clinical psychiatry and in computational neuroscience.

• Pam, Peter, Eireni and Alex Moutoussi who bore signiﬁcant costs so that I could carry out this work.

• Wael El-Deredy, my main supervisor who helped me so much to keep focused.

• Richard Bentall and his admin and research team. He encouraged me to embark on this work and his belief that I would be able to do good scientiﬁc work was truly touching.

• Peter Dayan, who advised me in a superb manner on the computational aspects of this project and encouraged and boosted my morale at crucial points.

• Diana Menzies, from the Henderson Personality Disorder team, who arranged an honorary post for me and was so supportive with NHS bureaucracy. There were many times in the last few years that the stresses which constant NHS restructuring visited upon us, its staff, were almost unbearable. I owe a debt of gratitude to Peter Dayan, Richard Bentall, Wael El-Deredy and Diana Menzies for helping my own health and wellbeing to stay intact.

23 24 THE AUTHOR

This is a brief summary of the author’s relevant research experience, as well as training relevant to the current work. I hold the following Degrees:

1. B.Sc. in Physics (First class; Imperial College, University of London, 1987) 2. M.Sc. in Physiology (University College London, University of London, 1990) 3. M.B.B.S. (Honours in Medicine & Pathology; Charing Cross & Westminster Med- ical School, University of London, 1993) 4. M.Sc. in Psychiatric Theory & Research Methods (University College London, University of London, 1990)

Other relevant qualifications include: Membership of the Royal College of Psychiatrists (MRCPsych: 1999); and membership of the Specialist Register as a Psychotherapist. I joined the latter in 2009. In terms of Research Experience, I have gradually worked towards applying the mathematical modelling approach to basic mental health research. Mathematical modelling permits a more rigorous integration of information across psychology, physiology, pharmacology and other disciplines. I first worked in cardiology research, using a nonlin- ear dynamical approach. A relevant publication was: Murphy, C.F., Lab, M.J, Harrison, F.G., Horner, S.M., Dick, D.J. & Moutoussis, M., (1993) Characterisation of regional myocardial dynamics during mechanical alternans in heart of anaesthetised pig, Cardio- vascular Research 27, 1639–1644. I brought this experience first to the study of animal epicortical EEG and eventually to the study of “executive” dysfunction in Alzheimer’s disease. The main relevant publication was: Moutoussis, M., Orrell, M.W. & Morris, R. (2004) Modelling Discoordination of Cortical Neuroactivity: Relevance for the Executive Control of Attention in Alzheimer’s Disease, Journal of Integrative Neuroscience 3, 85–104. I have also performed a limited amount of applied clinical research (e.g.: Moutous- sis, M., Gilmour, F., Orrell, M.W. & Baker, D. (2000) Quality of Care in a Psychiatric Outpatient Department, Journal of Mental Health 9, 409–420. The work detailed in this thesis has so far resulted in three peer-reviewed published articles and a contribution to a book chapter (Chapters2,4 and5). The material presented in Chapters6–7 has been submitted for publication.

25 26 CHAPTER ONE

INTRODUCTION

1.1 Importance

Paranoid ideation is very common in serious psychiatric disorders (Moutoussis, Williams, Dayan, & Bentall, 2007). It is distressing for patients and carers and is considered to el- evate the risk of violence (Swanson et al., 2006). When paranoid ideation affects the patient-professional relationship it often renders treatment very difficult. Several clinical and theoretical models of paranoia exist but they are incomplete, often contradictory, and of limited therapeutic potency. It is therefore most important to clarify our understanding of paranoid ideas. In the long term this is likely to improve treatments and especially the psychosocial aspects of the management of paranoia. This work concerns itself with aspects of the aetiology of paranoid delusions. Different meanings have been attached to the term “paranoid” over the years, so a brief clarification is in order. We will call “paranoia” the exaggerated, dual belief that: (a) the individual is at risk of substantial harm, and that (b) this risk is due to the deliberate intention of one or more persecutors, as defined by Freeman and Garety (2004a). This definition of the term “paranoia” is narrower than the one traditionally used in psychiatric training, which is based on Descriptive Phenomenology (Sims, Mundt, Berner, & Barocka, 2000). The latter terms paranoid all those ideas that are excessively self-referent, whatever their precise content. Part of the motivation for the definition used here is that on the one hand, persecutory paranoia is a very important delusional idea but, on the other, it is exists in a continuum between healthy and delusional states (Kendler, Glazer, & Morgenstern, 1983; Bentall, 2003; Freeman et al., 2010). Figure 1.1 illustrates the importance of paranoia in folk psychology; these issues will be discussed in more depth in section 2.2. I shall now discuss how some important medical and human sciences have attempted to describe paranoia. This will set the scene for the present set of studies which examine the role of just one factor – defensive avoidance – in the aetiology of paranoia.

27 CHAPTER 1. INTRODUCTION

Figure 1.1: Struggling with the possibility that others are out to deceive or harm us is quite a distressing, though common, preoccupation. In this popular press article the cor- respondent wonders if she is ‘paranoid’, the adviser suggests that if so it could be about her own ‘insecurity’, while readers’ letters are published that offer either reassurance or advice about drastic action.

28 1.2. SCHOOLS OF THOUGHT ON PARANOID DELUSIONS

1.2 Schools of thought on paranoid delusions

1.2.1 Paranoid delusions in psychiatry

The paranoid beliefs addressed in this study are the ones that clinical psychiatry terms “overvalued persecutory ideas and all types of delusions of persecution”. Although the term ‘paranoid’ is extensively used in everyday psychiatric practise, formally it is retained as a mere label qualifying diagnostic subcategories. The International Classification of Mental and Behavioural Disorders (World Health Organisation, 1992), for example, does not define the term ‘paranoid’, but states, under the ‘paranoid’ subcategory of Schizophre- nia: “Examples of . . . paranoid symptoms are: (a) delusions of persecution, reference, exalted birth, special mission, bodily change, or jealousy; (b) hallucinatory voices . . . or auditory hallucinations without verbal form . . . ” Descriptive phenomenology, on which psychiatric diagnosis is historically based, holds that there are certain classes of experience, especially hallucinations and delusions, which are clearly outside the norm (Hoenig, 1982). Within classic, Jasperian descriptive psychopathology (Jaspers, 1913/1959) trully pathological beliefs are characterised by abnormal inference, as judged by an empathic, healthy observer. A pathological belief is one which an empathic observer would be certain to find inconsistent and alien to the premises from which the patient holding it has derived it, had the observer been in the patient’s position. Descriptive psychopathology considers the break from normal inference to be qualitative rather than quantitative, so that the pathological belief is un-understandable to the healthy individual. The pathological beliefs defined in this manner are called “primary delusions”. Honouring this tradition the dictionaries of modern mental health practise define delusions as beliefs that appear fixed but are unwarranted on the basis of the available evidence (American Psychiatric Association Task Force on DSM-IV, 2000). Clinical psychiatry has taken an interest in pathological beliefs, including paranoid ones, over and above abnormal inference. As an important example, Maher and Ross (1984) described delusions as normally derived beliefs based on abnormal experiences. A very important contemporary account of delusions – and therefore paranoid delusions – is included in the aberrant-salience (AS) theory of psychosis (Kapur, 2003). This states that psychosis is the end-result of attributing a very high salience to percepts that do not deserve it1. The theory suggested a pathophysiological mechanism accounting for the abnormal inferences. It is hypothesised that the mesolimbocortical dopamine system normally mediates the attribution of salience to percepts, something necessary for learning what is important. It is further hypothesised that in the prodrome to acute psychosis dopamine is released in a chaotic manner, thus attaching salience to various irrelevant perceptions. Kapur (2003) hypothesised that delusions are the result of a cognitive effort to give meaning to these disparate, apparently highly salient data2. This theory has im-

1This paragraph ﬁrst appeared in the author’s contribution to a book chapter (Bentall, Kinderman & Moutoussis, 2008) 2The AS theory can thus be seen as a descendant of both the Jasperian view, in that the inference from

29 CHAPTER 1. INTRODUCTION mediate implications for the mode of action of antipsychotic drugs. It claims that they do not directly alter “positive” psychotic symptoms, but that they reduce the sense of salience that has been inappropriately attached to specific contexts; this in turn allows for reality-testing and re-learning, which is what reduces the positive symptoms. The theory makes predictions as to the time-course of the effect of these drugs on animal models of antipsychotic action, and on the timing of therapeutic action of antipsychotic agents. As far as paranoid ideas are concerned, the AS theory could therefore be applied as follows: Events in the human environment whose cause is not obvious can find a possible explanation in the action of others, especially powerful others (such as social groups, the government etc.). Chaotic dopamine activity might attach special salience to such events – in other words, make them of the greatest personal significance. One might go from “I have this unusual feeling today ... oh look, there’s a broken bottle on the pavement, some kid might get hurt” to “This broken bottle looks tremendously significant; Someone is saying YOU WILL GET HURT; THEY ARE SIGNALLING I’M IN DANGER” – and so on. Despite its substantial merits, the aberrant salience theory also has some important problems which make it, in our opinion, an inadequate explanation of paranoid states. We shall return to it in chapters2,4 and5; here we note that it follows the descriptive- phenomenological school whereby the content of delusions is essentially unrelated to their genesis – it is only a convenient explanation for hyper-salient experiences.

1.3 The psychological understanding of paranoia

Psychological thinking has focused on information processing in the mind, such as may be gleaned in thoughts, feelings and behaviour, which may lead to an exaggerated conviction that others are out to hurt us. Several schools have put psychological defensiveness at the centre of such processes; that is, they posit that paranoid ideas are the product of information processing whose goal is to speciﬁcally avoid the subject’s distress, to guard against a speciﬁc psychological threat. The nature of this threat or distress varies in the different hypotheses, as does the mechanism by which paranoid ideas are produced. Most importantly, the information-processing context in which these processes are hypothesised to operate also differs greatly. We now turn – selectively – to different psychological explanations of paranoia.

1.3.1 Psychodynamics of paranoia

Object Relations Theory

Psychodynamic thinking lies at the opposite extreme from descriptive psychopathology, which in its purest form considers the content of paranoid delusions irrelevant to percept to salience is un-understandable, and the Maherian view in the sense that the subsequent meaning- making effort is “normal”.

30 1.3. THE PSYCHOLOGICAL UNDERSTANDING OF PARANOIA what is interesting about them. An early and well-known psychodynamic theory suggested that paranoia is the result of repressed homosexual wishes (Freud, 1911), but the Object Relations Theory (ORT) of psychoanalytic thinking is much more relevant to the understanding of paranoid delusions – especially in the UK. In ORT, paranoia is so important that the fundamental mode employed by human babies in the first few months of their lives to deal with distress is termed the paranoid-schizoid position. In order to discuss it, however, some definitions are in order. This is important as some terms have been adopted from everyday language and given technical meaning which is not obvious. Thus an object here refers to a person, or part of a person, that is important with respect to the infant’s needs at a particular moment. It can be either external (e.g. the infant’s actual mother1), or internal. An internal object is a representation of a person or part thereof which forms part of the infant’s own mind. An object relationship is the relationship between objects and entails important actions. An example would be the feeding relationship between the hungry infant and the mother’s breast. In the context of object relationships, the ego is the part of the personality that includes the internal objects, processes object relationships and which functions as if it were the self (without necessarily this being phenomenologically evident, i.e. conscious) (Rycroft, 1995). When it comes to paranoia, the basic ideas of ORT are simple and elegant. First, it is postulated that the few-weeks-old infant, and specifically his ego, is unable manage extremely distressing experiences to which he is nevertheless quite prone. Klein, a founder of this theory, writes: “The baby reacts to unpleasant stimuli, and to the frustration of his pleasure, with feelings of hatred and aggression. These feelings are directed towards . . . the breasts of the mother . . . ” (Klein, 1975/1936). Klein and her followers believe that the feelings of frustration and hatred are simply too confusing, overwhelming, dangerous for the infant to associate with his beloved mother. Second, as the baby has no language and is unable to think in anything but the most concrete experiential images, the baby is held to simply treat his relationship with his mother as if it were two separate relationships (Ogden, 1983). One mother is the one who feeds him and whom he loves, the other one who frustrates him (nay, tortures him) and whom he hates2. The latter relationship is the essence of the experience of paranoia: the person has constructed in phantasy a hated figure that persecutes him. Thus in ORT paranoia involves, as in the everyday meaning of the term, the conviction that there is a persecutor out there who is out to harm the person. It is a fundamentally defense-based account. According to ORT what drives the infant towards the paranoid way of organising his perceptions is not a simple frustration or discomfort as an adult would consciously perceive them, but a state of overwhelming distress, and violent ag-

1As the infant’s mother will play a major role in the discussion that follows, I will refer to the infant as ‘he’ to aid the distinction from ‘her’, the mother 2The term ‘schizoid’ in ‘paranoid-schizoid position’ refers in large degree to the mechanisms by which this separation into two object-relationships is said to take place. The discussion here is not cast in terms ‘splitting’ and ‘projection’ as these terms often confuse the postulated concreteness of unconscious phantasy with a concreteness of psychological mechanism.

31 CHAPTER 1. INTRODUCTION gression. What is being defensively avoided is that one has hateful and attacking feelings towards the beloved other, that he risks damaging (or losing altogether) the good object. Feelings of guilt and confusion are thus defended against. ORT claims that normal infants overcome this paranoid way of perceiving, but that it remains an available modus operandi throughout life. Thus it can form the basis for persecutory symptoms in mental illness. Of course adults do have language, and a detailed knowledge of reality; therefore in order to perceive themselves and others in the original, infantile manner further drastic measures are required. These have been most eloquently described, from an ORT point of view, by Segal (1994). In short, there has to be a breakdown of the ability to use ordinary reality to test the consistency of one’s perceptions. Still, in the paranoid psychotic state we expect to ﬁnd -

• A denial of either the fact or the consequences of one’s own hatred and aggression, especially towards people one is consciously fond of.

• Retaining ‘good’ qualities for oneself while attributing ‘bad’ qualities to persecutors.

This having been a very brief exposition of ORT, an important clarification must be made. Paranoid-position functioning does not defend people from feeling depressed, i.e. low in mood. It ‘defends’ them against the distress that would be caused by being in touch with a specific set of unpleasant thoughts and feelings: allowing oneself to feel guilty, confused, anxious about having damaged a good, nurturing relationship, loving and hating someone at the same time, being sad. This constellation was termed by Klein the depressive position. The relevant distressing emotions (guilt etc.) were termed depressive anxieties. These are unfortunate terms, as the more severe clinical depression is, the more it is a paranoid-schizoid phenomenon, not a depressive-position one. A much better term for the ‘sadder and wiser’ normal stance is D.W. Winnicott’s position of concern (e.g. Carveth, 1994), but the Kleinian terms still predominate. Emotional functioning in severe depression is best seen as paranoid-schizoid because patients very much see the world and their relationships in separated, inflexibly ‘good’ and ‘bad’ terms. Hence they pass harsh judgement on themselves and often the world around them. Admittedly the mental organisation here is considerably more complex than the one attributed to the infant; the persecution is not simply ‘the Other is plain nasty’ but more ‘the Other is merciless’. In terms of object-relations, this implies a more complex mind imputed both to the persecutor and the persecuted, with moral judgement often preceding a persecutory attack. The ego can identify with either the persecutory pole (for example, the harshly judging) and/or the persecuted pole (the harshly judged)1. A letter of a patient of Emil Kraepelin illustrates how far we are from a position of concern: “I wish to inform

1That both are simultaneously possible is illustrated by Isaiah Ch.10 verses 5-6, where God uses the evil Assyrians to punish His sinning people: “Ah, Assyria is the rod of my anger, and the staff in their hand is my indignation. I will send him against a hypocritical nation, and against the people of my wrath. I gave him charge to take spoil, and to take prey, and to tread them down like the mire of the streets”.

32 1.3. THE PSYCHOLOGICAL UNDERSTANDING OF PARANOIA you that I have received the cake. Many thanks, but I am not worthy. You sent it on the anniversary of the child’s death, for I am not worthy of my birthday; I must weep myself to death; I cannot live and I cannot die, because I have failed so much, I shall bring my husband and children to hell”. From an ORT point of view, we still have -

• A denial of the lack of compassion inherent in one’s own hatred and aggression, especially towards the self, as well as towards ﬁgures that one is consciously fond of.

• Attributing ‘bad’ qualities to persecutors (such as God), such as harshness and mer- cilessness, but using elaborate constructs to deny the ‘badness’ of the judge or the judgement.

There is in fact empirical support for the statement that suicide attempters are characterised by more ‘primitive’ / paranoid-schizoid object relationships as compared with psychiatric controls. Using the Thematic Appreciation Test and other measures, Kaslow et al. (1998) found that suicide attempters displayed impaired affect, emotional investment and complexity of representation in their relationships compared to other psychiatric inpatients. Finally, we note that in the psychoanalytic framework, pathological depression has a powerful defensive function. This again has to do with denial of aggression, hostility and badness (for example, of the righteous self who sits in judgement against the wrongdoing self) as well as turning away from the suffering so inﬂicted (lack of genuine concern).

Other psychoanalytic theories of paranoia

Psychoanalytic thinkers do not shy away from positing many causes for one phenomenon. In the case of paranoia, fragile cognitive functions that would in health belong to the autonomous functions of the ego (Hartmann, 1958) are hypothesised to be vulnerable to emotional conﬂicts. The latter recruit a variety of defensive constellations (H. Blum, 1980; H. Blum, 1994):

• Managing the narcissistic hurt and humiliation of actual traumatic reality and abuse.

• Identifying with familial strategies of scapegoating.

• Defending against object loss that threatens complete breakdown through the lesser evil of establishing a persecutory object.

• Expressing masochistic -but unconscious- wishes to be attacked.

Such over-determinism undermines the feasibility of testing these theories. In practise, it reduces theory testing to weighing-up by expert opinion; but experts from different schools arrive at quite different formulations. In seeking a central tenet of contemporary psychoanalytic thought that may be worth putting to empirical scrutiny we can do worse

33 CHAPTER 1. INTRODUCTION than adopting Harold Blum’s summary: “In paranoia, murderous hostility is now considered far more important than [the Freudian theories e.g. of] repressed homosexual love1”. We still need to keep in mind that this summary represents a parochial viewpoint.

1.3.2 Cognitive-behavioural approaches to paranoia

Cognitive-behavioural theory and therapy (CBT) hinges on the idea that how we ap- praise an experience – not the experience itself – is what determines how we shall feel, and largely what we shall do2. Within the broad community of CBT there are many approaches addressing paranoia (Chadwick, Birchwood, & Trower, 1996; Bentall, 2003; Freeman, Garety, Kuipers, Fowler, & Bebbington, 2002; Moritz & Woodward, 2007; Kingdon & Turkington, 2005; Pretzer, 2007). These have much in common; they agree that perception of threat is clinically important, that paranoia has many facets and that it is important to address it specifically. There are, however, important differences between Cognitive schools. The aetiological importance they attach to defensive avoidance, and any recommended techniques to alter it, are key examples. This is, potentially, of clinical importance. After all CBT is strongly recommended as an intervention in psychosis but it is neither curative (National Collaborating Centre for Mental Health, 2010) nor even as effective as at one time hoped (e.g. Garety et al., 2008). We will now turn to the differences between two specific cognitive theories as to the possible role of psychological- defensiveness in paranoia, as they serve well to focus discussion. The ‘search-for-meaning theory’ sees the formation of paranoid delusions as a subjective explanation rendering anomalous experiences meaningful in the light of pre-existing beliefs about the self, others and the world. Neurocognitive biases associated with psychosis also contribute (Freeman et al., 2002). According to this theory beliefs about the self (e.g. vulnerable) and others (e.g. hostile) help turn a paranoid idea into a convincing explanation for odd internal feelings experienced in social situations (Freeman et al., 2008) and for subjectively felt anxiety (which inherently signifies threat). Self-schemas are intimately involved and self-esteem is affected, but psychological defensiveness is held to be un-necessary to explain the phenomena. ‘Attribution theory’, on the other hand, starts from the premise that people prone to paranoia harbour negative beliefs about themselves. These, however, are not always activated. However the advent of adverse life events can activate negative self-schemas: shortcomings of the self are a readily accessible and convincing explanation for the adverse event. The attribution theory then postulates that an alternative cause for the adverse event is sought, and an easy solution is to attribute the adverse event to the ‘badness’ of others. The theory posits that blaming others helps avoid the activation of negative thoughts about the self, at least to some extent, thus defending self-esteem (Bentall, Cor-

1Quote from http://www.enotes.com/psychoanalysis-encyclopedia/paranoia, as of 25 May 2010. 2The Stoic Epictetus described this idea clearly and taught how to use it to relieve suffering and aid decision-making (preserved in Arrian’s The Manual, second century AD).

34 1.3. THE PSYCHOLOGICAL UNDERSTANDING OF PARANOIA coran, Howard, Blackwood, & Kinderman, 2001). What about people who feel that the persecution they suffer is well-deserved? In their case the attribution of an adverse event to a persecutor is compatible with a powerfully activated negative self-image. In fact Bentall (2003) hypothesised a “dynamic relationship between attributions, different kinds of self-representations, mood and paranoid delusions, as if the paranoid patient is constantly fighting to maintain a positive view of the self, sometimes winning but more often losing” (emphasis added). In terms of observable predictions, therefore, relatively successful defensiveness should be more evident when paranoia is expressed but it is felt to be undeserved (Fornells-Ambrojo & Garety, 2009; Udachina, Varese, Oorschot, Myin-Germeys, & Bentall, in submission). People with undeserved paranoia are referred to as ‘Poor-Me paranoid’ (PMP) whereas those who feel they deserve persecution are called ‘Bad-Me paranoid’ (BMP) . The search-for-meaning theory seeks recourse to a variety of influences to justify why a paranoid explanation – and not another type – is so powerfully chosen. However, the attribution theory also appeals to other influences to justify the firm choice of a paranoid self-serving attribution, as opposed to a self-serving attribution of another type. Information-processing biases, such as the tendency of paranoid patients to “Jump to conclusions” (JTC), are important to both theories. The JTC bias is the tendency of people with delusions to arrive at conclusions based on substantially less information than healthy people. We shall return to this issue in section 3.2, section 3.4 and in chapter5. It has been thought for thousands of years that people turn away from contemplating their own faults1, yet only recently has the avoidance of subjective experiences been formulated in a way conducive to empirical research. Experiential avoidance (EA) is defined as the process whereby “a person is unwilling to remain in contact with particular private experiences (e.g., bodily sensations, emotions, thoughts, memories, images etc.) and takes steps to alter the form or frequency of these experiences, even when these forms of avoidance cause behavioural harm” (Hayes, Wilson, Gifford, Follette, & Strosahl, 1996). Avoiding the activation of negative self-schemas in individuals prone to paranoia, as attribution theory describes, can therefore be seen as a very specific example of EA. Tools to assess the degree of EA employed in a specific context have been developed and it has been claimed that EA plays an important role in mental health difficulties (e.g. Hayes et al., 2004; Pankey & Hayes, 2003). The opportunity thus presents itself to assess whether negative self-schemas are actively avoided in paranoid patients, especially in PMP where an ameliorated self-representation may well result.

1“Each man is born into the world carrying two bags. The one hanging on his back carries his own faults; the one on his front, his neighbour’s. Thus it is that men are quick to see others’ faults but are blind to their own”. Myth attributed to Aesop, c. 500 BC.

35 CHAPTER 1. INTRODUCTION

1.4 Defensiveness and paranoia: Research directions and thesis outline

There is thus vigorous debate in the area of paranoia research, and in the wider Mental Health community, as to the importance of defensive psychological processes. A consen- sus exists, however, that the avoidance of unpleasant private experiences is central to psychological defensiveness. Theories of ‘psychological defence’ in paranoia have only recently started to be tested and refined experimentally. There is, therefore, an urgent need to establish if defensive avoidance is indeed increased in paranoid ideation. As we have seen there are grounds to expect that the avoidance of unpleasant states may be increased in conditions where paranoia is most prominent. In the first part of this thesis the neuropharmacological and animal-behavioural basis for this expectation, that avoidance may be increased in paranoia, will first be reviewed. It will be argued that the Conditioned Avoidance Response is an animal paradigm that captures some aspects of this increased avoidance as well as its blunting by antipsychotic drugs. It will be noted that there is a seeming paradox as Dopamine, the target of most antipsychotic medications, is thought to be largely peripheral to the direct responding to aversive stimuli. In order to help resolve this paradox, and thus better understand the role of avoidance in paranoia, a detailed model of the mechanisms that may underlie aversive learning will be built. This model explains most existing experimental findings and suggests some important differences between appetitive and aversive learning. In the second part of the thesis, explicitly thinking about alternatives in paranoia will be considered (‘tree-searching’). It was hypothesised that increased avoidance might underlie some well-established biases, especially the ‘Jumping to Conclusions’ bias, found in paranoid patients. Subjectively perceived high costs of delaying a decision, possibly based on a heightened ‘need for closure’ or on fears of harm to the self-esteem, might lead patients to hasty decisions. An ideal-observer information processing model, based on Bayesian reasoning, was built and applied to existing data to test this hypothesis. In the final part of the thesis an experimental study will be presented. Here the role of defensive avoidance of a specific kind, namely avoidance of negative thoughts about the self, will be examined. It was hypothesised that paranoid patients would show increased self-serving avoidance on behavioural, self-report and observer-rated measures of the avoidance of negative thoughts about the self. Healthy control participants, non- paranoid clinical controls, paranoid patients who believed that they do not deserve persecution and paranoid patients feeling that they do deserve persecution were compared. The thesis will end with an overview of the evidence for the proposition that defensive avoidance has an important role in paranoia. It will also make recommendations for future research, both in terms of the methodology and in terms of key questions that future research should examine in the area of defensive avoidance in paranoia.

36 CHAPTER TWO

PARANOIA AND CONDITIONED AVOIDANCE

This chapter is an updated version of the published article: Moutoussis, M., Williams, J., Dayan, P., & Bentall, R. P. (2007). Persecutory delusions and the conditioned avoidance paradigm: towards an integration of the psychology and biology of paranoia. Cognitive Neuropsychiatry, 12 (6), 495–510.

2.1 Summary

Introduction: Theories of delusions often underplay the role of their content. With respect to persecutory delusions, taking threat as fundamental suggests that models of threat-related, aversive learning, such as the Conditioned Avoidance Response (CAR) task, might offer valid insights into the underlying normal and abnormal processes. In this study we reappraise the psychological significance of the CAR model of antipsychotic drug action; and we relate this to contemporary psychological theories of paranoia. Methods: Review and synthesis of literature. Results: Anticipation and recall of aversive events are abnormally accentuated in paranoia. Safety (avoidance) behaviours may help perpetuate and fix persecutory ideas by preventing their disconfirmation. In addition, patients may explain negative events in a paranoid way instead of making negative self-attributions (i.e. in an attempt to maintain self-esteem). This defensive function only predominates in the overtly psychotic patients. The ’safety behaviours’ of paranoid patients, their avoidance of negative self-attributions and the anti-paranoid effect of antipsychotic medication all resonate with aspects of the CAR. Conclusions: The CAR appears to activate some normal psychological and biological processes that are pathologically activated in paranoid psychosis. Paranoid psychological defences may be a result of basic aversive learning mechanisms which are accentuated during acute psychosis.

Keywords: Persecutory ideas; Conditioned Avoidance; Attribution Theory.

37 CHAPTER 2. PARANOIA AND CONDITIONED AVOIDANCE

2.2 The importance of threat-related content in delusions

Following Schneider (1949/1974), psychiatrists have traditionally distinguished between the form of psychopathology and its content cf. Hoenig, 1982. The phenomenological tradition of the twentieth century therefore focused on the abnormal formal inferences found in delusions, declaring that their content is culturally determined, rather arbitrary and thus of little interest (Berrios, 1991). This position is now considered extreme for two reasons. First, it is difficult to make a clear distinction between ordinary beliefs and delusions either in terms of form or content (David, 1999) and delusions must therefore be seen as lying at the end of a dimension or series of dimensions of belief attributes (Kendler et al., 1983); this principle is especially important in the case of paranoid delusions, as sub-clinical beliefs about persecution appear to be fairly common in non-psychiatric samples (Freeman et al., 2005). Second, it is now clear that the affective processes associated with the content of abnormal ideas play important roles in their genesis and maintenance (Bentall, 2003; Freeman & Garety, 2005; Raune, Bebbington, Dunn, & Kuipers, 2006). In this chapter we will argue that the content of persecutory delusions can be explained by brain processes that process threat-related emotional information, and that these processes may also account for the fixity of these kinds of beliefs. The content of abnormal beliefs typically reflects a small range of core themes, such as persecution, grandiosity and jealousy, which reflect concerns about the individual’s place in the social universe (Bentall, 1994). Research in many cultures has consistently found that the most common type of delusion involves the belief that the self is being threatened by malevolent others (Garety & Hemsley, 1987; Jorgensen & Jensen, 1994; Ndetei & Vadher, 1984; Stompe et al., 1999). By way of illustrating this point, Table 2.1 shows previously unreported symptom data from a cohort of 255 first-episode schizophrenia spectrum patients recruited to the SoCRATES trial of cognitive-behaviour therapy for early psychosis (Tarrier et al., 2004). The symptoms were assessed with the Positive and Negative Syndrome Scale (PANSS; Kay & Opler, 1987) within fourteen days of admission. If scores of 3 or more on both subscale P1 (delusions) and subscale P6 (suspiciousness) are taken as evidence of persecutory delusions it seems that more than 90% of this highly representative sample experienced significant paranoid ideation. It is apparent from these observations that an adequate understanding of delusions must include an account of how the specific content (e.g. persecution, or perception of threat) is related to form (unwarranted derivation, fixity). Since threat is key we must study the perception of real and delusional unpleasant events, and the responses of both healthy control participants and of patients in the face of these perceptions. This approach implies that models of aversive processing in healthy humans, and indeed animals, might capture important aspects of paranoia. The psychological and neurobiological aspects particularly of animal models are experimentally and theoretically quite tractable. They therefore offer considerable opportunities to identify important components of the elementary normal and abnormal processes that might be

38 2.3. THREATS & ATTRIBUTION THEORY

N (of 255) % Mean Median Symptom P ANSS ≥ 3 P ANSS ≥ 3 Score Score Delusions (P1) 250 98.0 5.26 5.0 Suspicion (P6) 235 91.8 4.53 5.0 Delusions & Suspicion (P1 and P6) 230 90.2 Hallucinations (P3) 177 69.1 3.41 4.0 Formal thought disorder (P2) 144 56.5 2.70 3.0 Agitation (P4) 179 70.2 3.03 3.0 Hostility (P7) 97 37.9 2.30 2.0 Grandiosity (P5) 98 38.6 2.25 1.0

Table 2.1: Positive symptoms of first-episode DSM-III-R diagnosed schizophrenia spectrum patients (schizophrenia, schizophreniform disorder, schizoaffective disorder, delusional disorder or psychotic disorder not otherwise specified; total N = 255) recruited to the SoCRATES (Study of Cognitive Realignment Therapy in Early Schizophrenia; Tar- rier et al. 2004) study. Patients were recruited from over 26 months from 11 mental health units serving three geographically defined English catchment areas: Liverpool, Manchester and Salford, and North Nottinghamshire. Assessments were conducted by trained psychiatrists within 14 days of admission using the Positive and Negative Syn- drome Schedule. Symptoms are ranked in order of frequency and mean severity. involved in paranoid thinking. In the following sections, we first highlight the central role played by threat and aversion in persecutory delusions. We then describe the CAR from both psychological and neural perspectives. Finally, we discuss three aspects of the CAR as a model of persecutory delusions, and then indicate directions for future research.

2.3 Threat perception and the attributional model of paranoia

The perception of threat is a central feature of paranoia almost by deﬁnition. How- ever, several studies have explored this issue empirically by asking paranoid patients to estimate the past frequency with which they had experienced positive, negative and neutral events, and also the probability that they will experience these events in the future (Cor- coran et al., 2006; Kaney, Bowen-Jones, Dewey, & Bentall, 1997; Bentall et al., 2008). In these studies, patients have reported high estimates for both past and future negative events, a phenomenon that can be resolved into three separate components. First, there is considerable evidence that paranoid patients have indeed actually experienced an abnormal frequency of adverse events such as discrimination and victimisation (Fuchs, 1992; Janssen et al., 2003; Mirowsky & Ross, 1983). Not only does this affect evaluations of the past, but also, because there is a tendency to rely on recollection of past events when making predictions about the future (called the availability heuristic; Kahneman, Slovic,

39 CHAPTER 2. PARANOIA AND CONDITIONED AVOIDANCE

& Tversky, 1982), it also tends to inflate estimates of future negative events. Second, patients also preferentially recall threat-related information (Bentall, Kaney, & Bowen- Jones, 1995; Kaney, Wolfenden, Dewey, & Bentall, 1992), thus further biasing future estimates via the availability heuristic. Third, it is found that paranoid patients make inflated estimates of future negative events even after controlling for the above effects, as well as for the effects of comorbid anxiety and depression (Bentall et al., 2008). This third component suggests that there is a specific abnormality in the mechanism responsible for aversive processing; and we will argue that this makes a large contribution to the formation of paranoid delusions.

Given perceived and potential aversive outcomes, which are exaggerated in paranoid patients, a second question concerns appropriate cognitive responses. A universal human response when faced with salient events is to construct an explanation for them, and attribution theory is the ﬁeld of psychology that deals with how individuals construct such explanations (or attributions); it has been estimated that ordinary people generate a statement that either includes or implies the word "because" in every few hundred words of speech (Zullow, Oettingen, Peterson, & Seligman, 1988). Building on early psychodynamic and social-psychological work, proponents of attributional models of psychopathology have suggested that people appeal to two main classes of explanation for negative events. One is to attribute these events to something they themselves did (an internal explanation). The other is to attribute them to factors external to the self (an external attribution), and this latter kind of explanation can be further subdivided into other-blaming (external-personal) and circumstance (external-situational) attributions (Kinderman & Bentall, 1997). Most people err towards attributing negative events to external causes, which is thought to buffer against self-esteem loss in the face of failure or other threats to the self (Campbell & Sedikides, 1999; Mezulis, Abramson, Hyde, & Hankin, 2004), a phenomenon known as the self-serving bias.

It is known that the kinds of attributions people make have important implications for psychopathology. Numerous studies have shown that depressed patients tend to make abnormally internal attributions for negative events (Mezulis et al., 2004). However, a number of studies have shown that paranoid patients, by contrast, tend to attribute negative events to excessively external causes (e.g. Kaney & Bentall, 1989; Fear, Sharp, & Healy, 1996) and especially external personal causes (Kinderman & Bentall, 1997). These observations have led to attempts to explain paranoid delusions in terms of these attributional processes. According to an early model (Bentall, 1994), paranoid patients have implicit negative self-schemas, which would ordinarily be readily activated to provoke conscious discrepancies between the individual’s ideal self-concept and actual perception of the self. In an attempt to avoid this discrepancy, the individual attributes the cause of the schema-activating event to an external-personal cause (the actions of other people) but this leads to the belief that other people have malevolent intentions towards the self. The model proposed that persecutory delusions arise as the consequence of the iterative use of

40 2.3. THREATS & ATTRIBUTION THEORY this defensive strategy in the face of repeated threats. A common criticism of this model is that self-esteem is often low in paranoid patients (Freeman et al., 1998). In fact, research on self-esteem in paranoid patients has revealed a complex picture, with some studies showing either a close association between negative self-esteem and paranoia e.g. Freeman et al., 1998; Bentall et al., 2008, relatively preserved self-esteem on explicit measures but low self-esteem on implicit measures e.g. Lyon, Kaney, and Bentall, 1994; Moritz, Woodward, and Hausmann, 2006; McKay, Lang- don, and Coltheart, 2007, or no relationship between self-esteem and paranoia (Drake et al., 2004). Partly in response to this criticism, and also in the light of evidence that attributional judgements are inﬂuenced by current self-esteem e.g. Kinderman and Bentall, 2000 and are highly labile in paranoid patients (Bentall & Kaney, 2005), a more recent version of the attributional model was proposed, in which a cyclic relationship between attributions and self-esteem was hypothesised to lead to highly unstable self-esteem in paranoid patients (Bentall et al., 2001).

2.3.1 ‘Poor-me’ and ‘bad-me’ paranoia

A further complication so far not explicitly addressed by the attributional models is that the defensive function of paranoid attributions appears to predominate only for a specific type of paranoia, or perhaps at specific stages in the development of persecutory delusions. Trower and Chadwick (1995) have distinguished between two types of paranoid beliefs: poor-me (in which persecution is believed to be undeserved) and bad-me (in which it is believed to be deserved), and have argued that defensive processes operate only in the first of these types (Trower & Chadwick, 1995) 1. Chadwick, Trower, Juusti- Butler, and Maguire (2005) reported that self-esteem is relatively preserved in poor-me patients compared to bad-me patients. A subsequent study reported that acutely ill patients often switch between poor-me and bad-me beliefs but that, consistent with Trower and Chadwick’s predictions, abnormal attributions are only present when patients hold poor-me beliefs (Melo, Taylor, & Bentall, 2006). Abnormal attributions also appear to be absent in non-psychotic individuals with paranoid beliefs (McKay, Langdon, & Coltheart, 2005; Janssen et al., 2006). The implication of these observations is that defensive processes are evident only in the acutely ill poor- me phase, and that bad-me paranoia is probably more evident during the prodromal phase before an acute crisis. Consistent with this account, a recent study of prodromal patients reported that this group has very marked discrepancies between their ideal self and their perceived self, but that the presence of actual psychotic symptoms was associated with a lack of such discrepancies (Morrison et al., 2006).

1As discussed in the introduction, this is a rather different use of the term “defensive processes” than is made in the psychodynamic tradition. In the latter attributing negative events to the self can be just as “defensive”, i.e. stemming from a desire to avoid intense psychological discomfort - except that a different trade-off is made. Please see page 32 for details.

41 CHAPTER 2. PARANOIA AND CONDITIONED AVOIDANCE

In the following account, we will argue that the conditioned avoidance paradigm helps us to understand the defensive processes operating in the poor-me phase.

2.4 The Conditioned Avoidance paradigm

The Conditioned Avoidance Response (CAR) paradigm was designed to assess learning and performance of behaviours motivated by aversion. Since the discovery of chlorpromazine (Swazey, 1979) it has been known that antipsychotic drugs selectively suppress avoidance responding, leaving escape responding relatively intact; the CAR is thus routinely used to help assess if a new compound is likely to be active against psychosis (Wadenberg & Hicks, 1999). Despite some important modelling of the role of dopamine in the CAR and psychosis (Kapur, Mizrahi, & Li, 2005; Smith, Becker, & Kapur, 2005), the threat inherent in the CAR has not yet been directly linked to the role of threat perception in persecutory delusions. In the animal CAR paradigm the subject is placed in a shuttle box with two compartments, for example one black and one grey. An animal placed in one of the compartments learns that a neutral warning stimulus (e.g. a light) is followed after some seconds by an unconditioned aversive stimulus (US: an electric shock). After the onset of the warning stimulus the subject can avoid the US by moving (shuttling) to the other compartment of the apparatus. Shuttling before the onset of the US avoids the shock and also interrupts the warning stimulus (in typical experiments the duration of the light would otherwise overlap with the start of the shock). This is termed an avoidance response (AR). Shut- tling after US onset (an escape response, or ER) also aborts the shock. Thus shuttling becomes a conditioned response, and we will refer to the warning stimulus as a “conditioned stimulus” (CS). CAR tasks are also performed with human participants. In human experiments shock administration is typically by cutaneous electrodes while the AR/ER involves pulling a lever (Unger, Evans, Rourke, & Levis, 2003); other experiments use a burst of white noise as a US. Naive animals typically first display freezing in response to the US and subsequently show increased locomotion. After a few shocks, they perform the ER, presumably by chance. From then on they quickly learn to shuttle before US onset (AR). One main theory of learning in the CAR (Mowrer, 1947; Schmajuk & Zanutto, 1997) is that subjects first learn to fear (i.e. to predict the aversive values of) the states leading from the CS to the US in the absence of avoidance. They also learn the neutral value of the ’other’ compartment, and then have the avoidance response reinforced by the appetitive affective change experienced in going from the aversive states to the neutral (safety) states. For a detailed summary of experimental findings see the review by Schmajuk and Zanutto (1997). Once successful avoidance has been reliably achieved, the AR becomes resistant to extinction. Factors that confer resistance to extinction include increased magnitude of

42 2.4. THE CONDITIONED AVOIDANCE PARADIGM the US, factors relating to the timing of the US, and others whose details are peripheral to the issues in question here. These factors may all strengthen the CS-US association (Schmajuk & Zanutto, 1997). In some cases the latency between the warning signal and the avoidance response continues decreasing long after a reliable avoidance CR has been achieved (Solomon, Kamin, & Wynne, 1953). In these cases not only is there no extinction, but learning seems to continue to occur in the absence of shocks. One important way in which this can be reversed is by blocking shuttling while a CS - No-US contingency is presented. Some of the neurobiological substrates of the CAR are well-established. All known antipsychotics (unlike other psychotropic compounds) disrupt performance of the well- learnt avoidance response at doses much lower than needed to affect the escape response (Wadenberg & Hicks, 1999). Almost all antipsychotic compounds with selective action on the CAR block dopamine D2 receptors. D2 blockade disrupts performance of well- learnt avoidance responses, but also the acquisition of the AR (reviewed by Smith et al., 2005). It was realised at an early stage (Beninger, Mason, Phillips, & Fibiger, 1980b) that D2 blockade does not disrupt the development of the association between warning and aversive stimuli but the development of the AR itself. This was shown by the fact that when both the US and the D2 blocker were eliminated, presentation of the CS on its own led to the gradual acquisition of the AR. Anatomically, one of the most important sites of action of D2 receptors with respect to the CAR is the shell of the nucleus accumbens septi (NAS-shell). Drugs that affect other neurotransmitters can also affect the CAR, but mostly in synergy with dopaminergic modulation. The role of serotonin is particularly interesting, as NAS-shell 5HT2 blockade greatly potentiates the effect of D2 blockade on the CAR (Wadenberg & Hicks, 1999; Wadenberg, 2010).

2.4.1 Linking paranoid delusions and the CAR

Besides the CAR’s well-recognised predictive validity for antipsychotic effects, we suggest that it is also a valid and revealing model for fundamental aspects of paranoid delusions. Further biological links do exist. For example drugs that enhance dopamine function often cause paranoid syndromes in humans e.g. Satel, Southwick, and Gawin, 1991. Moreover, neuroimaging studies indicate that it is during the acute stage of psychosis, when poor-me delusions are most evident, that abnormal functioning of the midbrain dopamine system is most evident (Laruelle, Abi-Dargham, Gil, Kegeles, & Innis, 1999). In this chapter, however, we shall concentrate on the psychological / functional links. We draw three key psychological / functional parallels. First, that both CAR and paranoia involve threat-perception mechanisms. We thus relate the taking of defensive action in the CAR to defensive avoidance in paranoia. Second, we note that avoidance responses in the CAR, like paranoid delusions, are markedly resistant to extinction. Finally, a more subtle point perhaps, we consider that the defensive function of poor-me beliefs may represent a form of covert avoidance.

43 CHAPTER 2. PARANOIA AND CONDITIONED AVOIDANCE

According to this hypothesis threat-perception mechanisms are linked not only to normal aversive learning in the CAR but also to unwarranted associations in paranoia. Mod- ern models of affectively charged adaptive behaviour have made important inroads in understanding both reward- and threat-motivated learning. These models already include accounts of functional aspects of neuromodulators and especially of dopamine (Mon- tague, Dayan, & Sejnowski, 1996; Schultz, Dayan, & Montague, 1997; Daw, Kakade, & Dayan, 2002; Seymour et al., 2004). If our hypothesis is valid a whole new ﬁeld of investigation opens up - relating models of speciﬁcally aversive learning to the abnormal psychology of paranoia.

2.4.2 The experience of threat in the CAR and in paranoia

The first parallel, namely that both the CAR and paranoia involve the perception of substantial threat seems straightforward. In the CAR the shock is of course quite real while paranoid delusions are, by definition, unrealistic. The initial establishment of threat- perception in paranoia is thus clearly important. Indeed, one main aim of our research programme is to understand what might be going wrong in paranoia by considering what might be going on in the CAR. One hypothesis is that abnormalities of aversive processing, perhaps in a prodromal phase of psychosis, create fictitious, internal aversive states, and then lead to bad-me and then poor-me delusions through evaluative and defensive mechanisms. The content of persecutory delusions is specifically about social/interpersonal threat. This is not surprising given that, in highly social animals, emotional systems are naturally responsive to the harms (and benefits) that may come from conspecifics. In this context it is interesting to note that animal studies show that repeated exposure to social defeat leads to sensitisation of the mesolimbic dopamine system (Tidey & Miczeck, 1996; Selten & Cantor-Graae, 2005). This is analogous, perhaps, to the experiences of discrimination and victimisation that seem to confer a high risk of paranoia, (Fuchs, 1992; Janssen et al., 2006; Mirowsky & Ross, 1983). Another interesting and relevant observation is that oversensitivity of the dopamine system is important in the emotional sensitivity of people at high risk of psychosis (Myin-Germeys, Marcelis, Krabbendam, Delespaul, & van Os, 2005). Increased expectation of socially mediated harm as well as low self-esteem are likely psychological sequelae of repeated social defeat accompanied by increased dopamine reactivity. Hence these observations are consistent with the hypothesis that fragile self-esteem plays a role in the onset of paranoia, as proposed by attribution theorists.

2.4.3 Safety behaviours may help maintain paranoia

The second parallel concerns overt avoidance behaviours. The notion that safety behaviours help maintain paranoid ideas was put forward by Morrison (1998) on the basis of

44 2.4. THE CONDITIONED AVOIDANCE PARADIGM case studies. Paranoid patients perceive serious threat in the social environment and take efficient action to neutralise the threat, mainly by avoiding circumstances in which the expected threat might be encountered. As a consequence, they miss opportunities to find out that their threat-beliefs are unfounded (a simple example is when a paranoid patient stays indoors to avoid meeting imagined persecutors, thereby failing to learn that people outside the home are benign). Freeman, Garety, and Kuipers (2001) formally investigated safety behaviours in a sample of 25 psychotic patients and found that 92% of participants reported overt avoidance. As avoidance behaviours appear to reduce exposure to disconfirmatory evidence and hence prevent modification of threat-beliefs, cognitive-behaviour therapists often find it helpful to use behavioural experiments to help test persecutory beliefs (Morrison, Ren- ton, Dunn, Williams, & Bentall, 2003). From this point of view, avoidance behaviours are extremely common in paranoia. They reduce the experience of perceived warning stimuli not leading to feared consequences, thus reducing opportunities to modify inappropriate predictions. The CAR provides a close parallel in that response-blocking is often required to achieve extinction of avoidance responses.

2.4.4 The avoidance of internal states in Poor-me paranoia

The third parallel has to do with avoidance of internal aversive states in paranoia (i.e. experiential rather than overt avoidance). We suggest a mapping between avoidance behaviours in CAR and the possible defensive role of paranoid ideas. As already noted, there has been an intensive debate in the literature about whether persecutory beliefs enable the individual to avoid feelings of low self-esteem (Garety & Freeman, 1999) as initially suggested by attribution researchers (Bentall, Kinderman, & Kaney, 1994). This suggestion has been challenged on the basis of ﬁndings of low self-esteem in paranoid patients (Freeman et al., 1998). As we have already indicated, a possible resolution to this problem can be found in the distinction between poor-me and bad-me paranoia, and the observation that abnormal attributions and relatively preserved self-esteem are only found when acutely ill psychotic patients hold poor-me beliefs (Chadwick et al., 2005; Melo et al., 2006). As we have also already seen, abnormal attributions are only found in acutely psychotic patients (McKay et al., 2005; Janssen et al., 2006) who are nearly always poor-me (Bentall et al., 2008; Fornells-Ambrojo & Garety, 2005). Moreover, in prodromal patients scores on self-esteem related measures appear to improve with increasing psychosis (Mor- rison et al., 2006). Together these observations suggest a developmental pathway leading to clinical paranoia, in which experiences of victimisation and social defeat lead to poor self-esteem and the growing conviction that others also hold negative views about the self, and hence to bad-me beliefs. These beliefs, maintained and ampliﬁed by safety behaviours, are eventually transformed into poor-me beliefs in acute psychosis, when

45 CHAPTER 2. PARANOIA AND CONDITIONED AVOIDANCE avoidance is extended to attempts to avoid negative thoughts about the self. Note that, in this hypothesized progression, the ﬁnal defensive response (including the generation of an explanation for a negative event that implicates external-personal causes, a poor-me delusion) can be understood within the CAR framework as a form of covert avoidance behaviour, in which negative thoughts about the self are a covert CS which would elicit a strong internal US (a negative emotional state) which is avoided by means of an external-personal attribution. This account assumes that negative thoughts about the self have strong emotional effects, can be regarded as response-provoking stimuli, and that individuals sometimes respond so as to avoid them successfully. Clearly, the ﬁrst two of these assumptions are concordant with everyday observation (for example, of the surge of negative affect that follows the realisation that one has been seen to do something shameful) and the last is consistent with many recent accounts of psychopathology which have emphasized the role of experiential avoidance processes e.g. Hayes, Strosahl, and Wilson, 1999; Rassin, Merckelbach, and Muris, 2000. In this context, it is interesting to consider the possibility that the tendency to jump to conclusions when reasoning about probabilistic information found in many deluded individuals (Garety and Freeman, 1999) may also be related to the avoidance mechanisms involved in the CAR. Exaggerated avoidance of the discomfort associated with uncertainty could contribute to the cognitive biases of jumping-to-conclusions and increased need-for-closure found in delusions (McKay, Langdon, & Coltheart, 2006). This possibility is certainly worthy of further investigation, as we shall see in chapter5.

2.5 CAR and paranoia: Conclusions

In this paper we have outlined a number of parallels between the CAR and evidence regarding the psychological mechanisms in paranoia. The CAR has previously been studied as a model of anxiety disorders (Lovibond, 2006) hence our account suggests some overlap between the processes involved in paranoia and anxiety. Freeman and co-workers (Freeman & Garety, 2005) have previously argued for a close relationship between paranoia and anxiety. However in their account it is assumed that the subjective experience of anxiety prompts paranoid verbalisations (i.e. the latter are expressive of the sense of impending danger that is central to anxiety). In our account, by contrast, anxiety is the consequence of the perception of either external or internal threat. The potential relationship between CAR and paranoia opens up a number of lines of inquiry. From a neuroscience perspective, the role of dopamine is both central and enigmatic. Dopamine is substantially implicated in many aspects of learning predictions (Montague et al., 1996) and optimizing actions. These aspects have been the target of substantial computational modelling that links neural, psychological and ethological ideas. Such modelling is relevant to the CAR, but is unlikely to apply directly as these roles of dopamine have been established in the context of reward stimuli, not threats. Analysis of

46 2.5. CAR AND PARANOIA: CONCLUSIONS dopamine’s role in signalling with respect to aversive events is rather less clear (Ungless, Magill, & Bolam, 2004). It may have little role in the signalling of the aversive events themselves, as evidenced by the intact formation of CS-US associations under dopamine blockade (Beninger et al., 1980b). This finding is also a challenge to models that require a dopaminergic ’teacher’ signal for the formation of internal representations of the environment cf. Smith, Li, Becker, and Kapur, 2006. Dopamine may instead be involved in the reward (relief) brought on by evasive actions. Opponency between dopamine and other neuromodulators, particularly serotonin, may turn out to be key in this case (Daw et al., 2002; Ungless, 2004). Most importantly, the suggested relationship between the CAR and paranoia permits theory-based psychological and neurobiological investigation, guided by quantitative computational models. Detailed mathematical modelling of the basic psychological and biological mechanisms involved in the CAR is therefore desirable, and will be pursued in chapter4. Investigations should in due course include neuroimaging studies, of both healthy individuals and paranoid patients using paradigms analogous to the CAR paradigms used in animal studies extended to socially threatening stimuli and threats to self-esteem. Further research into the CAR and its relation to human psychological processes would thus help develop an integrated psychobiological understanding of the threat beliefs that are one of the most common symptoms of severe mental illness. The psychological effects of antipsychotic drugs are yet to be integrated with these potential roles of neuromodulators, especially with respect to threat-based learning. How- ever, if our account is correct, it follows that these drugs may function psychologically by reducing patients’ experiential avoidance, which may help maintain their symptoms. An interesting corollary is that bad-me delusions, which we hypothesize not to involve this avoidance, should be less responsive to antipsychotic drugs than poor-me delusions. Although this possibility has never been tested empirically, it is interesting to note that a recent Cochrane review of the treatment of psychotic depression found no evidence that antipsychotics are effective in this condition (Wijkstra, Lijmer, Balk, Geddes, & Nolen, 2005). If our suggestion is valid, and flight from states of high negative self-esteem plays an important contributory factor in poor-me persecutory delusions, then individuals prone to such delusions must differ from healthy individuals in the related parameters. That is, our account suggests that individuals vulnerable to paranoia may make excessive estimations of internal threat, or use excessively avoidant cognitive strategies to deal with it. A detailed empirical study investigating such possibly increased avoidance will be described in chapter6. Finally, the suggested avoidance mechanisms may play an important role in the transition from subclinical to clinical paranoia. This progression needs to be verified in a longitudinal investigation of prodromal patients. It would also be profitable to directly compare avoidance mechanisms in healthy individuals and those prone to delusions. This

47 CHAPTER 2. PARANOIA AND CONDITIONED AVOIDANCE could take place in CAR-like experimental learning and the extent to which antipsychotic drugs blunt such learning, as they do in animals, would be important to investigate.

48 CHAPTER THREE

COMPUTATIONAL METHODS FOR COST-BASED DECISION MAKING

3.1 Summary

Introduction: A set of systematic methods is needed to build models of aversive processing and to relate them to actual experiments and clinical observations. The models that simulate the phenomena (generative models) then need to be aided by methods that use experimental data to estimate the model parameters. Further methods are required to establish how well the models, complete with the best-fit parameters, describe the data; and whether some models are significantly better than others. Methods: Equations were first formulated to describe Conditioned-Avoidance responding (CAR). Key constraints were brought to bear and a cached, or habit-based, model was built. Another key task used to investigate decision making in paranoia, namely the ‘beads-in-a-jar’ task, was described in terms of two very different models: a bayesian reasoning model and a sequential - probability - ratio - test (SPRT) model. Expectation - maximization (EM) equations were derived to fit models to experimental data. Finally, parametric bootstrap and likelihood-based methods were formulated to evaluate model fit. Results: A choice of the advantage-learning temporal-difference model, as well as its fixed parameters, was made for the CAR. This will be used in chapter4. The bayesian reasoning model required consideration of states far into the cognitive future, whereas the SPRT was structurally simpler. These will be used in chapter5. Equations were derived for the EM methods. Bootstrap confidence intervals and other evaluation methods were also formulated and piloted. Conclusions: The key models chosen to investigate cost-based decision making, and hence avoidance, were tractable and their mathematical descriptions were successfully derived. The implementation of some (especially bootstrap confidence intervals) was, however, quite demanding of computer resources.

Keywords: Cached model; Tree-search model; Bayesian reasoning; Expectation-Maximisation;

49 CHAPTER 3. COMPUTATIONAL METHODS FOR COST-BASED DECISION MAKING

Parametric bootstrap.

3.2 A modelling framework: ‘Cached’ & ‘tree-search’ models

A range of powerful theoretical approaches is available in order to analyse how threat is perceived and how decisions are taken under threat. These not only have precise mathematical form but also give rise to tractable computational models. Thus they can greatly aid statistical analysis of data. The Conditioned Avoidance Response may be useful to further the understanding of paranoia, as we saw in Chapter2. A good starting point to understand the CAR is, in turn, the two-factor theory (Mowrer, 1947) which was briefly presented on page 42. The two- factor theory is a reinforcement-learning (RL) theory and the CAR involves delivery of behavioural reinforcement (shock). In recent years there have been great advances in the understanding of RL (Sutton & Barto, 1998). Essential to this modern approach is the idea that the organism perceives a sequence of states as time passes, and that s/he attaches an affective value to each state s/he discerns. This affective value summarises how good the future outcomes are that are expected to follow each state. We can regard these affective values as an analogue, or even a substrate, of the affects that point to the future, such as expectation of rewards (looking forward, wanting) and of harm or punishment (fearing, feeling threatened)1. We will consider two important types of mathematical models, in both of which affective values are attached to each state experienced or discerned by the organism. In the first type of model, the cached or habit model (Daw, Niv, & Dayan, 2005), the organism has no knowledge of how different states follow each other. If, for example, they experience the state “I am in room A ...” they may also experience “... and this is bad news” but they have no knowledge such as “in room A electric shocks are likely to take place”. We will refer to these models as cached, in that the affective value of the state summarises future outcomes which are not explicitly represented. When outcomes are not explicitly represented, the value of states is gradually learnt through cumulative experience. Therefore, if outcomes suddenly change the model’s behaviour only changes gradually – hence the terminology associating this type of model with habits. We will focus on a specific type of cached model, the Temporal Difference (TD) model. In the second type of model the organism carries an explicit representation of the structure of their experiences. In the example above, they explicitly represent “being in room A → shock often follows”. We will refer to this type of representation as a tree-search. This is because the agent explicitly stores knowledge of the possible state transitions branching

1It has to be stressed that the theory of RL does not concern itself with phenomenal correlates of such values, as a mental health professional might. It concerns itself with the predictive and motivational aspects of these values, as will be detailed below.

50 3.2. A MODELLING FRAMEWORK: ‘CACHED’ & ‘TREE-SEARCH’ MODELS forth from each state. An organism carrying out a tree-search is also referred to in the literature as being in possession of a ‘forward model’ (Daw et al., 2005). We will avoid the latter term to prevent confusion of the ‘model’ of the world that the organism uses with the mathematical models that we will use to study these phenomena. Organisms that tree- search in order to predict outcomes and choose actions are also called ‘goal-directed’, as they explicitly represent the states they want to achieve and avoid (goals). Tree-searches are often difficult to use as they require explicit consideration of complicated patterns of future outcomes. Branches may have multiple sub-branches, and goals may be many steps ahead, far into the (cognitive) future. Given adequate computational resources, however, a tree-search is an optimal way to calculate outcomes. We will use a tree-search model to calculate optimal solutions to the ‘beads-in-a-jar task’. This is important in the assessment of the Jumping-to-Conclusions bias found in paranoia, as was mentioned above (page 35); mathematical details of this model will be provided in this chapter. Details of another, simpler model of decision-making in the ‘beads-in-a-jar task’ will also be provided. This simpler model, based on the Sequential Probability Ratio Test (SPRT; Green & Swets, 1966), will also be used to analyse decisions of paranoid patients in Chapter5. It will be described together with the optimal model for convenience, although it is not a tree-search model. Once mathematical models are build they often have to be fitted to experimental data. This can be done by maximising the likelihood that a specific model, including specific values of its parameters, might give rise to the distribution of real data observed. When considering groups of individuals such fitting can take place either at the level of the group or the individual, ignoring constraints pertaining to the groups. We will touch on likelihood-maximisation for each individual experimental participant, but we will concentrate on a method that takes into account group constraints; this is the Expectation- Maximisation (EM) approach (Dempster, Laird, & Rubin, 1977). Measures that evaluate how successful the final model is in describing the experimental data will also be described. The actual value of the likelihood that the experimental data might have arisen given the model is one such measure. The Bayesian information criterion is an improvement on the raw likelihood (Raftery, 1995). Sometimes we may not be so concerned that the whole dataset be accurately described. In some cases, for example, simple measures of central tendency (usually the mean) and spread (usually the standard deviation or the variance) of a measured variable are more intuitive than the log- likelihood of a model. In such a case we may be interested in some group-level statistical measures derived from the experimental data. A model would then have to describe with confidence such key statistical measures. All the techniques described in the following sections of this chapter were programmed in C++ by the author. Use was made of the GNU Scientific Library (www.gnu.org/ software/gsl/) and the Blitz++ matrix manipulation library (www.oonumerics. org/blitz/). Key code can be found at www.eireni.gn.apc.org/avoidance_

51 CHAPTER 3. COMPUTATIONAL METHODS FOR COST-BASED DECISION MAKING code/avoidance_index.htm; all code and compiled programs are freely available from the author under the GNU Public License (email: [email protected]).

3.3 Temporal-difference models

Given a particular environment, an agent encounters a sequence of states: s1...sk, sk+1, ...slast. Moving from one state to another is associated with receiving returns which can be appetitive (‘rewards’) or aversive (‘punishments’). The agent going through these states keeps account of the value of each state, V (sk). This reflects the total expected return following sk, for all the states that are likely to follow the present one. In order for these quantities to be well-defined we should add that returns are summed over the probability that the various future states will be visited under the current policy; we will only need to consider cases where this long-term sum is well-defined. ‘Policy’ is the set of probabilities π(ai; sk) that any action ai may be chosen in a particular state sk (Sutton &

Barto, 1998; Dayan & Abbot, 2001). As an action ai will lead to a well-defined probability distribution of next-states sk+1, the value of each action Q(ai, sk) is also well-defined – it is simply the total future return following taking action ai in state sk. We will consider agents that can keep explicit account of the action-values Q(ai; sk), or of a related set of policy parameters m(ai; sk) from which the probabilities π(ai; sk) can be derived. It is not a priori obvious that such an elaboration is necessary; RL schemes that keep track of only action-values (Q learning: Watkins, 1989) or only state-values e.g. Todorov, 2006 certainly exist. Rather, it is inspired by some basic experimental findings that we will discuss in detail in chapter4. Our aim will not be to simulate in detail one specific experimental setup and to provide a close statistical fit to it. Rather, we will aim to provide a more general model, implementing key psychobiological hypotheses, which can be tested by a wide range of semi-quantitative experimental results. Semi-quantitative here means that a successful model should replicate the patterns of results in several variants of the experimental setup in question, each of the variants providing a pattern of results that the model might in principle violate and be falsified. Figure 3.1 shows a sequence of possible states that can be used to simulate the conditioned avoidance response and related tasks. In this paradigm, as we saw see in detail in chapter2 (page 42), an agent perceives a conditioned stimulus which warns that a fixed time later an unconditioned aversive stimulus (US) will be delivered; unless, that is, the agent takes successful avoiding action. This sequence is repeated for a substantial number of trials, so the agent has the opportunity to learn first to escape and then to avoid the US. Each trial starts with the agent in s1. There follows a ‘tapped delay line’, that is, the animal (or other agent) perceives the passage of time as an obligatory move from one state to the other, from left to right in figure 3.1. At each distinct state certain actions a(sk) are available. An animal would have available a number of actions, such as exploring,

52 3.3. TEMPORAL-DIFFERENCE MODELS

Figure 3.1: State space used to simulate the Conditioned Avoidance response, including the actions available in each state. The CS starts at state labelled 1 (s1). Action ‘stay’ results in horizontal arrows and has no intrinsic cost. Action ‘shuttle’ moves from the unsafe side (top row) to the safe side of the apparatus (bottom row) and incurs a motoric cost of jumping the barrier. The transition s5 → s6 results in shock. Once in state 6 or 11 the trial ends, i.e. these states are terminal. They can be considered to have a value of zero, although, of course, ‘landing’ on s6 delivers a large aversive return. States 7-11 could have been condensed to one safety state – they are shown separately just for added clarity of timing. Note that we have disallowed jumping back to the unsafe side. grooming or resting, which have their own intrinsic values. In order to represent this, in our models several ‘stay’ actions can thus be available from each state. ‘Temporal difference’ refers to the difference between returns that were expected and returns that were received as a step in time was taken. In TD learning, rewards anticipated in a particular state of the animal are compared to rewards actually received during the immediately subsequent state. The simplest formulation of this difference, the prediction error, is

δV = [Rk+1 + V (sk+1)] − V (sk) (3.3.1)

Where R(k + 1) is the return (reward or cost) experienced in going from s(k) to s(k + 1).

The difference between expectation and return, δV , can be used by the learner to improve its estimate of what Value to attach to the original state1. It is a function of state values

Vk, k ∈ {s1...s11} as above:

V (sk)new = V (sk)old + αδV (3.3.2)

Where α is a learning rate parameter. We wanted to investigate the action of antipsychotic drugs on the CAR. We thus took into consideration the ﬁnding that when agents encounter better-than-expected states, i.e. when δV > 0, phasic dopamine release appears to report the prediction error in key structures of the basal ganglia (Schultz, Apicella, & Ljungberg, 1993; Schultz et al., 1997;

1In most accounts of TD learning an additional factor γ ≤ 1 multiplies the value of the future state in this formula to denote that returns which are distant in time appear less valuable to an agent than proximal ones. In our case, however, the whole of each trial only lasts a few seconds and hence temporal discounting is not prima facie of great importance. Key simulations of the CAR experiment, especially of the normal CAR, were performed that included a temporal discounting factor. This did not, however, enrich the behaviour of the model and it was decided not to include temporal discounting unless forced by experimental data. Over the whole set of studies that will be reported here this did not prove necessary.

53 CHAPTER 3. COMPUTATIONAL METHODS FOR COST-BASED DECISION MAKING

Smith et al., 2006, 2007). At the same time learning the basic association between a predicting stimulus and an aversive unconditioned stimulus is not dopamine dependent (Beninger, Mason, Phillips, & Fibiger, 1980a, 1980b). We thus introduced an additional + factor D to simulate dopamine modulation, for example by drugs, but only if δV > 0..

+ V (sk)new = V (sk)old + αD δV (3.3.3)

For the purpose of displaying simulations of aversive learning we also reversed the sign of punishments and aversive values so that they display as positive quantities. This is simply a matter of convenience which does not affect the dynamics of the model. How can policy parameters and policy probabilities be determined? Given a set of policy parameters that encode the degree to which each choice is preferable to the agent1, a rule such as ‘Herrnstein matching’ (Herrnstein, 1961) or ‘Gibbs softmax’ (Sutton & Barto, 1998). Daw and Doya (2006) reviewed relevant data from animal experiments and concluded that the Gibbs softmax rule is somewhat more accurate in describing behavioural choices. For the present purposes the exact choice of rule is not crucial, at least in the ﬁrst instance, as a broad pattern of agreement with experimental results, rather than precise simulation, is the aim. We therefore adopted the Gibbs rule, which, given a set of policy parameters m(ak, s) and an additional scaling parameter τ (to be discussed below) determines the probability to choose the action ai as follows:

em(ai,s)/τ π(ai; s) = X (3.3.4) em(ak,s)/τ all k

The next step is to provide a mechanism for the agent to learn the policy parameters m(ai, s). This could be done by a rule whereby policy corrections are proportionate to the prediction error (Sutton & Barto, 1998):

m(ai, s)new = m(ai, s)old − δV (3.3.5)

Where is the policy learning rate2. The negative sign in front of serves to conform to the convention above, whereby aversive values and δV s are positive. The set of equations 3.3.1– 3.3.5 are an example of Actor-Critic learning (Sutton & Barto, 1998). The policy part is termed ‘Actor’, as it decides how often each action is taken, whereas the value structure produces the prediction error signal which ‘criticises’ (evaluates and cor- rects) the actions. We now consider that, when it comes to policy, dopamine antagonists impair learning of both better-than-expected and worse-than-expected actions. In the ﬁrst instance we can simulate this by replacing the δV term with DδV . The dopamine modulation term

1A set of action-values could, of course, stand for such a set of preferences directly. 2 might itself be a function of the current policy but again this would not improve our models in the current context.

54 3.3. TEMPORAL-DIFFERENCE MODELS

D now applies irrespective of whether the PE is better or worse than expected.

m(ai, s)new = m(ai, s)old − DδV (3.3.6)

However, this simple change in the policy learning rule does not sufﬁce to describe the psychopharmacological data. We can immediately see that introducing D here is simply equivalent to modulating the learning rate. Consider an equilibrium solution of the above equations, i.e. the case where stable probabilities π(shuttle; s5), π(stay; s5) and correspondingly stable policy parameters m(ai, s) have been learnt. If m(ai, s)new = 1 m(ai, s)old then δV = 0 by equation 3.3.6. Let us for the purposes of this demonstration take Rsafety = 0 Rshock = 1. Hence for a = shuttle,

δV = Rsafety + V (s11) − V (s5) =⇒ V (s5) = 0. However at equilibrium we also have that

V (s5) = π(shuttle; s5) × Rsafety + π(stay; s5) × Rshock =⇒ π(stay; s5) = 0. Note that this derivation does not depend on the value of D, as long as D > 0. D = 0.2 blockade, for example, would slow learning down (reduce the learning rate by 80%) but would not change the asymptotic probability of taking the avoidance action. This is contradicted by the pharmacological data, which shows that the steady-state probability of stay dramatically increases under dopamine blockade e.g. Beninger et al., 1980a; Smith et al., 2007. What we need to do is modify equation 3.3.5 so that it can have a ﬁxed point for nonzero D × δV . In addition, the asymptotic probability of the avoidance response should decrease as blockade increases for a given level of shock, or cost; or, similarly, the same equilibrium m(a, s) should obtain for increasing shocks Rshock as D becomes smaller. Possibly the simplest way to modify 3.3.5 to achieve this is:

m(ai, s)new = m(ai, s)old − (DδV − m(ai, s)old) (3.3.7)

Although we arrived at this equation by considering the dose-dependence of dopamine blockade in the CAR, it actually happens to be a recognised variant of TD learning, namely Advantage learning (Dayan & Balleine, 2002). D apart, it can be shown that the policy-correction term in equation 3.3.7 is equivalent to basing the policy m(ai, s) on the advantage estimated for action ai in state s relative to the outcome expected from state s overall. Let us also consider the behaviour of the advantage-learning rule without dopamine modulation, D = 1. Note that for a suboptimal policy, one that does not maximally exploit the available returns, the action amax that leads to the optimal reward from state s will not always be chosen. Therefore the δV calculated in the trials where amax is chosen will not be zero according to equation 3.3.1. In contrast, equation 3.3.7 will converge

1We will use a similar, but not identical, pattern of returns for the simulations that follow; this does not affect the argument presented here.

55 CHAPTER 3. COMPUTATIONAL METHODS FOR COST-BASED DECISION MAKING for non-optimal policies to m(a, s) equalling the average (non-zero) δV for the particular action a. In this way the model is naturally consistent with the tendency of animals to choose actions with probabilities depending on their return, rather than maximising returns by choosing simply the action associated with highest return. In other words the advantage-learning model does not ‘try to beat’ the Gibbs (or Herrnstein) rule.

3.3.1 Fixed parameters used in the Advantage-learning model of avoidance

The basic structure used for the simulations shown in the figures is shown in figure 3.1. There are 5 time steps between CS onset and the end of each trial, but as long as there is meaningful time resolution compared to the biological scales involved this is not important; we could have used, for example, 10 time steps instead. In addition, six identical stay actions and one shuttle action were available from each state. Again the precise degeneracy of the stay action does not matter; it serves to capture the idea that in the absence of any reward / punishment structure the agent will not choose stay vs. shuttle equiprobably. Stay actions had a cost of zero (a reference value), while the cost of shuttling was Rshuttle = 0.2. The cost of the shock was set at or above 4.0. This 20- fold or greater ratio was meant to simulate the traumatic nature of the traditional animal CAR (and, possibly, the dire anticipations of persecutory ideation). It is motivated by the review of Schmajuk and Zanutto (1997) which describes that the characteristic pattern of CAR results (resistance to extinction etc.) is reliably seen for powerfully aversive stimuli. In our advantage-learning models exploration and asymptotic behaviour depend on a ‘temperature’ parameter τ, as in equation 3.3.4. 1/τ is also called ‘brittleness’, to use the term adopted by Williams and Dayan (2005). The more brittle the model, the more a difference in costs (or rewards) between actions translates to a difference in probability of their selection. 1/τ can therefore be thought of as an index of how important for action selection the typical costs and benefits involved in the experiment are. Increasing τ leads to more exploration of alternatives before the final policy probabilities are reached. This affects, in turn, whether the learner has a propensity to modify the policy followed once a way of avoiding shock (by shuttling from a particular state) has been discovered. τ should be large enough to allow exploration around the cost of the non-traumatic shuttle action but not to dampen the effect of the high cost of shock. In the simulations shown here τ was hence set to 0.2 (brittleness 1/τ = 5.0), the same as Rshuttle. The learning rate for state values was 0.5. This was dictated by the finding that the CS-US association is normally learnt very rapidly, within a few trials. The learning rate for policies it was 0.075; as is typical in actor-critic schemes, the learning rate of the critic must be substantially greater than that of the actor for performance to be appropriate. Each run of trials started with 100 unshocked trials, so that by the time shocks started the ‘stay’ vs. ‘shuttle’ policies were determined by their relative costs rather than any initial values. The results of this model’s application to the CAR will be presented in

56 3.4. TREE-SEARCH & RELATED MODELS chapter4.

3.4 Tree-search & related models

In the ‘beads task’ (Volans, 1976; Huq, Garety, & Hemsley, 1988) the agent is presented with a sequence of binary items of information. These are randomly selected with replacement from one of two stationary distributions. A cover story is given, which is usually that beads are randomly drawn from one of two jars, each with very many beads of just two colours - say green and blue. One jar has a majority of green beads; let’s call it jar G; the other, has the same majority of the opposite colour. Let’s call it jar B. Agents have three actions (or decisions) available to them. They can:

1. decide that the sampled information comes from the one distribution i.e. beads

come from G (action DG)

2. decide that the sampled information comes from the other distribution B (action

DB); Or they can

3. decide to collect another item of information, i.e. ‘sample again’ (action DS).

The objective is to decide correctly which distribution the data comes from. In the version of the task that we will model here the first two actions terminate the task. A maximum of nmax items of information are allowed, at which point only the first two actions are available as the task terminates anyway. At each point of this task the state that the agent finds themselves in is uniquely defined simply by the number of b and g items drawn up to the point in question; the order that they have been drawn in is immaterial. We will consider an ideal tree-search strategy which we will analyse. An ideal observer will be able to calculate the probability that each state may arise at each point of the task, down to the maximum number of draws nmax. They will then use explicitly calculated state-values and action-values to make their decisions. We will refer to this as the Costed Bayesian (CB) model. Here we present its mathematical specification, leaving its application for chapter5 (cf. page 101).

3.4.1 Costed Bayesian model

Let q > 0.5 be the proportion of blue beads in jar B. That is, the probability of drawing a blue ball given that the jar of origin is B is P (b|B) = q. Similarly for the ‘Green’ jar, P (g|G) = q. The prior probabilities that the beads come from either jar before any beads are drawn are the same, P (B|0, 0) = P (G|0, 0) = 0.5. We use the notation ‘|0, 0’ as follows: the ﬁrst 0 means that nd = 0 draws have taken place, the second 0 that ng = 0 g[reen] balls have been drawn. In other words, we deﬁne a state s by the total number of items information drawn so far, nd, and the number of those that are g, ng. Our ideal observer will make use of Bayes theorem to calculate the probabilities that the different

57 CHAPTER 3. COMPUTATIONAL METHODS FOR COST-BASED DECISION MAKING possible states may arise. By Bayes theorem, the posterior probability of jar G being the underlying cause if nd beads have been drawn, of which ng were of type g, is:

P (nd, ng|G)P (G|0, 0) P (G|nd, ng) = (3.4.1) P (nd, ng|G)P (G|0, 0) + P (nd, ng|B)P (B|0, 0)

Given the notation and conventions above, this becomes

1 P (G|nd, ng) = (3.4.2) q (nd−2ng) 1 + ( 1−q )

We will use the symbol ‘|’ to denote conditional probability whereas expressions following ‘;’ will denote a known state. The set of all π(a; s) deﬁnes the current policy that the agent pursues. If we take the return, i.e. the reward or cost, of choosing the correct decision to be zero and that of deciding erroneously to be CW , we have for the Action Values Q(a; s) :

W Q(DB; nd, ng) = C × P (G|nd, ng) (3.4.3) W Q(DG; nd, ng) = C × (1 − P (G|nd, ng)) (3.4.4)

The immediate cost of taking a sample is CS. However, since sampling leads the agent to further choices at further states, the full cost is a function of the values, V (s), of the states s weighted by the probabilities of getting to those states. If sampling one more bead ‘lands’ the agent at a next state s0, with a value V (s0), the ideal observer can 0 calculate Q(DS; s) by considering the probabilities that the various subsequent states s may arise given the possible underlying causes / jars J ∈ {G, B} and marginalising:

S X X 0 0 Q(DS; s) = C + P (J|s) V (s )P (s |J) (3.4.5) J∈{G,B} all s0

The possible outcomes of sampling are that either b or g will turn up. In case g turns up, the participant will ﬁnd themselves in state (nd + 1, ng + 1). Otherwise the state will be (nd + 1, ng). Let V (nd + 1, ng + 1) and V (nd + 1, ng) be the (as yet unknown) values of these states. If the true underlying cause is G, the latter value will obtain with probability P (b|G) = 1 − q; the former value, with probability P (g|G) = q. In addition, the underlying cause is G with probability P (G|nd, ng). We can thus add all the contributions together and expand the equation above to obtain:

S Q(DS; nd, ng) = C +

P (G|nd, ng) × [V (nd + 1, ng + 1)P (g|G) + V (nd + 1, ng)P (b|G)] +

P (B|nd, ng) × [V (nd + 1, ng + 1)P (g|B) + V (nd + 1, ng)P (b|B)] (3.4.6)

58 3.4. TREE-SEARCH & RELATED MODELS

A participant might always choose the decision with the lowest cost in each state, i.e. perform noise-free decision making. This, however, is not necessary; people do not choose so deterministically and a measure of noise might be unavoidable. To introduce behavioural uncertainty we can use the softmax function (equation 3.3.4), including its noise parameter τ. Equation 3.3.4 now becomes:

eQ(a,s)/τ π(a; s) = (3.4.7) P eQ(b,s)/τ b∈{DG,DB ,DS }

(except for nd = 20, when a, b ∈ {DG,DB}). The Value of each state then is:

V (nd, ng) = π(DS; nd, ng) × Q(DS; nd, ng)+

π(DG; nd, ng) × Q(DG; nd, ng) +

π(DB; nd, ng) × Q(DB; nd, ng) (3.4.8)

At the last step, when nd = nmax = 20, the action ‘Sample again’ is unavailable and equation 3.4.8 has no term associated with DS . Once all the state values

V (nmax, nmax − ng) can been calculated (with the help of eq. 3.4.4, 3.4.7 and 3.4.8), these can be used in to obtain all action values for nd = 19, and so on back to nd = 1.

For each possible sequence of beads, therefore, a number of draws ndec can be found where DS ceases to be the most rewarding action. If the same action-values were kept but behavioural noise was eliminated, this would become the step where the model would ‘declare’.

3.4.2 Sequential probability ratio test model

In the SPRT, the key computed quantity at time t is the log-likelihood ratio of the sequence of data that have been produced given one possible cause, over the equivalent expression given the other cause:

P (d , d ...d |cause = G) l(t) = ln 1 2 t (3.4.9) P (d1, d2...dt|cause = B)

Decisions are taken by comparing l(t) two thresholds, θG and θB :

DG if l(nd, ng) > θG

D = DB if l(nd, ng) < θB

DS otherwise (3.4.10)

We used θG = −θB for consistency with using a single cost of making the wrong decision (CW above). Since the draws are independent (which amounts to assuming that the jars contain many more beads than the participant is allowed to draw), l(t) accumu-

59 CHAPTER 3. COMPUTATIONAL METHODS FOR COST-BASED DECISION MAKING lates additively. Following the rationale described for the Bayesian model, we also used an uncertainty or noise factor ε:

P (d |G) l(t) = l(t − 1) + ln t + ε (3.4.11) P (dt|B)

We assumed that ε is normally distributed with mean zero and standard deviation τ. In our simulations we also included a very small amount of noise at the starting point t=0. In the deterministic case it would be easy to increment l(t) by adding the penultimate term of eq. 3.4.11. In the case involving noise we have to add to the random variable l(t − 1) the last two terms of eq. 3.4.11, which also form a random variable. We thus ﬁnd the probability distribution of l by convolution:

Z ∞ pt(l) = pt−1(v) pt(l − v) dv (3.4.12) −∞

Where pt−1 is the prior pdf of l and pt is the probability distribution of the increment for the current step, t (i.e. the pdf of the last two terms in eq. 3.4.11). At each step, the values of l that fall outside the thresholds result in a decision, so that the probability distribution is truncated at the thresholds. It is then renormalised to compute the distribution of l for which the decision was ‘sample again’. The latter distribution forms the starting point for the next step (ﬁg. 5.1). The case involving noise or error is more demanding to implement efﬁciently on a computer than the noise-free case.

3.5 Fitting models to data

In the study of the ‘beads task’ of Corcoran et al. (2008) each participant provided six items of data, three for each of two slightly different versions of the task (experimental details will be presented in chapter5). The models that we consider, such as the CB or the SPRT, should characterise the information processes of each particular experimental participant i, for example their decision threshold θi and the cognitive noise level τi.

Let gi be a vector containing parameters for a specific participant i under the information processing model M. We will refer to g as ‘microparameters’ to distinguish them from parameters that may characterise the specific group, such as healthy control, actively paranoid etc. to which each participant belongs. These groups will be described by their own, group-level, parameters – for example the mean and variance of the distribution of decision thresholds for a specific group. We will call these group-level descriptors ‘macroparameters’. It is important to be able to obtain good quality estimates of the model parameters that might give rise to the data di. This is often done through Maximum Like- ML lihood estimation, i.e. choosing gi that maximises the likelihood that the gi has given rise to the data di :

60 3.5. FITTING MODELS TO DATA

ML Y gi = maxg P (dik ; g) (3.5.1) all trials k However in the case of the ‘beads task’ this likelihood function was not well-peaked. Many participants made decisions after only a few samples, and simulations showed that many combinations of microparameters could give rise to these with high probability. Consider the sub-space of parameters shown in figure 3.2. As an example, any of the parameter values in the ‘ridge’ of the surface labelled ‘1’ would explain a choice of ‘1,1,1’ in three trials with probability approaching 1.0. Such individual experimental results were not uncommon. A standard alternative is to build a so-called random effects model (Penny, Holmes, & Friston, 2003), in which parameters characterising participants are required to come from distributions specific to their diagnostic group. We can then ask directly what macroparameters are most likely to underlie the whole set of experimental results of each group. Of course the form of the distribution of microparameters is not known; however, costs are unlikely to assume positive values (i.e. mistakes or slowness will not be positively rewarding) and uncertainty cannot be negative. We therefore assumed that the microparameters came from independent gamma distributions. We used the Expectation-Maximisation approach (EM; Dempster et al., 1977) to find the best-fit group-level parameters. We first attempted a shortcut to using the EM method. We attempted to directly derive iterative estimates of macroparameters (rather like equations 10.4 of Dayan & Abbot, 2001). We tested this on artificial data based on the product of independent gamma distributions. The shortcut appeared to provide good estimates of the group-level means of the gamma distributions but failed to provide good estimates for their variances1. We solved this problem by using the full EM equations, resulting in algorithms that worked well on surrogate data. We now turn to the derivation of these equations.

3.5.1 Full expectation-maximisation ﬁtting

Let us consider a two-level model G. At the upper level causes v are sampled, and at the lower level these give rise to observations (data) dj . In our case the upper level is the one of the macroparameters; these parametrise the distributions from which the speciﬁc microparameters v are to be sampled. As an example we take v to be the microparameters of the CB model – though an equivalent derivation would obtain for the macroparameters of the SPRT. The EM algorithm aims to maximise the log-likelihood that the experimental data have been created by the model G. It does this by minimising the discrepancy between the observed distribution of the experimental data and the distribution produced by the model (Dayan & Abbot, 2001, Section 10.2). In order to ﬁnd the most likely causes v

1We are indebted to Nathaniel Daw for personal communications conﬁrming the difﬁculty using such shortcuts.

61 CHAPTER 3. COMPUTATIONAL METHODS FOR COST-BASED DECISION MAKING

Figure 3.2: Each surface is the probability of deciding (i.e. not-calling-for-another sample) at a specific step: for example, the hump labelled ‘5’ is the probability of exactly 5 draws-to-decision. The sequence presented is 01000100101000011001 (b=0, g=1). The proportion of the predominant beads in the jar is .75. The first 11 components are plotted. The noise parameter τ is fixed at 0.5. In this simulation, unlike the rest of the present study, deciding before seeing any draws has been allowed. In this and subsequent figures both cost and noise parameters are measured in arbitrary units (as long as all costs are measured in the same ‘currency’, it doesn’t matter what this currency is).

62 3.5. FITTING MODELS TO DATA

under the model G, given the experimental data di, EM makes use of the recognition distribution P [v|dj; G], i.e. the probability that the cause v was associated with the data dj. It also makes use of the joint distribution over causes and data, p[v, dj; G], which describes the probability that both the cause v and the data dj have been produced by the model G.

EM minimises the discrepancy mentioned above, between the observed distribution of the experimental data and the distribution produced by the model, by maximising a quantity called the negative-free-energy, F . We start with a ﬁrst approximation to the best model parameters. These give rise to an approximate recognition distribution, Q[v; dj]. The negative-free-energy F then is (Dayan & Abbot, 2001): P F (Q, G) = h v Q[v; dj](ln(p[v, dj; G]/Q[v; dj]))ij P = h v Q[v; dj](ln P [dj|v; G] + ln(p[v; G]) − ln(Q[v; dj]))ij The ‘upper level’ of the model G consists of the product of two gamma distributions: p[v; G] = f[CS; κCS, θCS] × f[τ; κτ , θτ ]. These give rise to parameters {CS, τ} ≡ v.

At the ‘lower level’ P [dj|v; G] is in turn specified. In our case P [dj|v; G] has no further dependence on the model G, once a specific ‘cause’ {CS, τ} has been specified. In this case,

X F (Q, G) = h Q[CS, τ; dj](ln P [dj|CS, τ]+ CS,τ

ln(f[CS; κCS, θCS]) + ln(f[τ; κτ , θτ ])

− ln(Q[CS, τ; dj]))ij (3.5.2)

During the M phase we aim to maximise F with respect to the model G, keeping Q constant. Therefore, as each ‘upper level’ parameter (κCS, θCS, κτ , θτ ) only affects few terms, if we differentiate e.g. with respect to κτ we get:

∂F (Q, G(κ , θ )) τ τ = ∂κτ X ∂ = h Q[CS, τ; dj]( ln(f[τ; κτ , θτ ]))ij ∂κτ CS,τ

X ∂ κτ −1 −τ/θτ −κτ = h Q[CS, τ; dj]( ln(τ e θτ /Γ(κτ )))ij ∂κτ CS,τ X ∂ = h Q[CS, τ; dj]( ((κτ − 1) ln(τ) − τ/θτ − κτ ln(θτ ) − ln(Γ(κτ ))))ij ∂κτ CS,τ X = h Q[CS, τ; dj](ln(τ) − ln(θτ ) − Ψ0(κτ ))ij (3.5.3) CS,τ

Where Ψ0 is the digamma function; similarly

63 CHAPTER 3. COMPUTATIONAL METHODS FOR COST-BASED DECISION MAKING

∂F (Q, G(κ , θ )) τ τ = ∂θτ X ∂ = h Q[CS, τ; dj]( ((κτ − 1) ln(τ) − τ/θτ − κτ ln(θτ ) − ln(Γ(κτ ))))ij ∂θτ CS,τ X 2 = h Q[CS, τ; dj](τ/θτ − κτ /θτ )ij (3.5.4) CS,τ

Setting this expression equal to zero, taking into account that the mean of the gamma distribution is the product κ × θ and that

X Q[CS, τ; dj] = 1 for all dj (3.5.5) CS,τ

we get: ZZ κτ θτ = τµ = h Q[CS, τ; dj] τ dCS dτij := I1 (3.5.6) CS,τ Or, in other words, the (grand) mean value of the causal parameter using the approximate recognition distributions is the one that minimises the free energy (maximises F ). I1, the mean value in question, is relatively straightforward to estimate numerically. Now if we set expression 3.5.3 equal to 0, we get

X X I2 := h Q[CS, τ; dj] ln(τ)ij = (ln(θτ + Ψ0(κτ ))h Q[CS, τ; dj]ij CS,τ CS,τ

I1 ⇒ I2 = ln( ) + Ψ0(κτ ) κ τ ⇒ ln(I1) − I2 = ln(κτ ) − Ψ0(κτ ) (3.5.7)

An equation of the form S = ln(κτ ) − Ψ0(κτ ) has an approximate solution κτ = (3 − S + p(S − 3)2 + 24S)/12S. This can be improved iteratively by the Newton- Raphson method,

κτ (n+1) = κτ (n) − (ln(κτ (n)) − Ψ0(κτ (n)) − S)/(1/κτ (n) − Ψ1(κτ (n)), 0 where Ψ1 is the trigamma function, Ψ1 = Ψ0. This estimates the new shape parameter of the gamma distribution. Once the estimate of κτ has converged, we can calculate 2 2 θτ = I1/κτ . The new variance of the distribution for τ then is στ = I1 /κτ . We note that whereas the expression for the new mean is rather intuitive, the other parameters interact in a complex way with the approximate recognition distribution to give the new measures of spread.

Next, the Expectation step can proceed by updating the approximate recognition dis-

64 3.6. MODEL EVALUATION tribution, i.e. setting

P [dj|CS, τ]f(CS; κCS, θCS)f(τ; κτ , θτ ) Q[CS, τ; dj] = P [CS, τ|dj; G] = RR CS,τ P [dj|CS, τ]f(CS; κCS, θCS)f(τ; κτ , θτ ) dCS dτ (3.5.8)

3.5.2 Experimental Bayesian distribution

Once a particular model M is chosen1, including macroparameters that describe a spe- ciﬁc experimental group j, the recognition distribution density p(g|di ; M) describes how likely it that the participant i, who has furnished data di, is characterised by microparameters g. We can now average the recognition distribution over all participants i in group j to obtain an estimate of the probability density pexp(gj) of microparameters for a random member of this group. This is sometimes called a marginal posterior density (Gelman, 2002); but for clarity, we call it an experimental (Bayesian) distribution:

Nj 1 X P (di|gj ; M)p(gj ; M) p (g ) = (3.5.9) exp j N R P (d |g ; M)p(g ; M) dg j i=1 all g i 3.6 How good is a model? Methods for model evaluation

There were three types of comparisons that were of interest in this study. First, we were interested in how well our models described the actual experimental data. Could it be, for example, that a model might produce the experimental data with a reasonable likelihood, but its typical output might actually deviate significantly from the data? In that case the fact that the observed data would not be included in these typical outputs would be evidence against the model. Could it be, on the other hand, that the fitting procedure might make such compromises as to render it likely that the model output data would be characterised by key statistics quite different to the ones observed in the real data? An example would be the grand mean of draws-to-decision in the different tasks: we should expect a model to produce such a global measure similar to the one observed with high likelihood. Second, we were interested to know if there were statistically significant differences between the macroparameters that best described the different experimental groups. Third, we were interested to know if one of the mathematical models we used, the CB or the SPRT, fitted the data better than the other. The first two types of model evaluation can be carried out by using bootstrap, or resampling, methods. The last comparison was carried out by using the Bayesian Infor- mation Criterion (Raftery, 1995). We now describe these methods in more detail.

1Here we change the notation slightly to emphasise that we refer to the general case rather than the example of the CB model that we used above.

65 CHAPTER 3. COMPUTATIONAL METHODS FOR COST-BASED DECISION MAKING

3.6.1 Bootstrap methods

We adopted a Monte-Carlo approach. That is, we used the best-ﬁt macroparameters for each model to produce an ensemble of generated data. We then asked, as mentioned above, whether the experimental data were typical of this ensemble. We can use the log-likelihood for the model in question to evaluate this. To do this we compared the likelihood estimated for the actual experimental data with the distribution of likelihoods that can be calculated for the generated data. As the data for each participant arise independently within each group, the likelihood of model M for a speciﬁc group is:

Y L(M, D) = P (D ; M) = P (di ; M) (3.6.1) all i

Where D is the matrix of all the data of the group in question and di is the vector of data for subject i. Taking the natural logarithm,

N X ln(L) = ln(P [di ; M]) (3.6.2) i=1

Figure 3.3 shows an example of this approach for the CB generative model. The example uses the best-fit (by EM) parameters to the data from the actively-paranoid group performing a specific version of the beads-task. This version used two categories of words as stimuli instead of two colours of beads. We can see that the likelihood of the experimental data is not untypical of those produced by the model, which is reassuring. This is in fact a conservative example, as for this specific data there was some evidence that the Costed-Bayesian model was somewhat inferior to the SPRT. This will be discussed in more detail in chapter5 (page 106).

Summary properties (statistics) produced by models

If we are interested in a measure of the quality of fit that has a straightforward meaning we may want to consider the difference in the average number of draws-to-decision between control and index groups of participants in the beads-task. This would reflect the question ‘roughly how precipitous are paranoid patients’ decisions?’. Although such grand averages are often reported in the literature, it might be objected that averaging over a small, heterogeneous set of stimuli such as those used by Corcoran et al. (2008) makes these measures too specific to each study in question. On the other hand the proximity of such measures to the actual observations may make them less vulnerable to assumptions that may have creeped unnoticed into more sophisticated comparisons. In addition, such direct measures have the advantage that they can be very easily computed from the generative model, which means that vast generated-data sample sizes can be used. This is in contrast to the bootstrap confidence interval technique discussed below (section 3.6.1)

66 3.6. MODEL EVALUATION

Figure 3.3: Example of the likelihood of the model to produce the experimental data (dashed vertical line) compared to the distribution of generated data that the model produces (histogram). Here the Costed Bayesian model has been ﬁtted to the data from actively paranoid participants performing the probabilistic reasoning task. Fifty generated datasets have been produced, the log-likelihood of the model to have produced them has been calculated and its distribution is displayed. The model often produces data of similar likelihood to the experimental dataset.

67 CHAPTER 3. COMPUTATIONAL METHODS FOR COST-BASED DECISION MAKING which are much more expensive computationally. The variance of a distribution is also a useful, intuitive measure. In the case of our models of draws-to-decision it offers a very interesting cross-check of the quality the models. Preliminary analyses indicated that precipitous decisions in the paranoid groups might be because of increased mean cognitive noise, codiﬁed by the macroparemeter τµ in our models. The preliminary analyses also suggested an increased standard deviation

τσ for these groups. We were concerned that the fitting procedure might gravitate towards such a fit in order to account for the early decisions but at the expense of overestimating the variability of draws-to-decision in this group. In such a case we might expect an accurate fit of the difference of draws-to-decision between control and paranoid groups, but an inaccurate reproduction of the variance of the paranoid group. Two examples of the application of this method are shown in figure 3.4. Reassuringly the model did not overestimate outcome variability. In fact the procedure described led to the identification of an outlying individual whose choices are poorly predicted by either one of the models considered here (See figure 3.4 b. and section 5.4 for further details). This method can be straightforwardly extended to aid hypothesis testing. In order to test the null hypothesis that the decisions of two or more experimental groups derive from the same underlying cognitive mechanisms (same model and same parameters), the data of the two groups can be merged and EM applied to the merged data to derive best-fit ‘hull hypothesis parameters’. A large sample of simulated experiments can be generated, and the likelihood of the experimentally observed summary statistics to arise can be determined, for example by calculating the 95% confidence interval for the simulated statistic in question. The most important statistic here is the grand-mean difference between the actively paranoid group and the controls; the latter can consist of the participants in all control groups (a more agnostic null hypothesis, in the spirit of ANOVA) or a specific control group (to address a narrower question of psychological significance: e.g., do previously-paranoid, now-remitted subjects choose more like healthy ones, or more like actively paranoid ones?). The normal distribution of the difference-of-means aids the determination of the confidence intervals in question. In our case the Student MATLAB package was used to fit Gaussian curves to the simulated distributions and confidence intervals were fitted from these fitted Gaussians. MATLAB also provides error measures on the confidence intervals so determined; the results reported in Chapter5 take these into account conservatively.

Bootstrap conﬁdence intervals for model parameters

A Monte-Carlo approach similar to the one above (section 3.6.1) was used to estimate the uncertainty of the model parameter fits produced by EM. An ensemble of generated data was again produced based on the best-fit estimates for the experimental group in question. The EM algorithm was then applied again to the generated data, producing a set of estimated best-fit parameters. The distributions of these simulated best-fit parameters

68 3.6. MODEL EVALUATION

0 Fitted Gaussian Experimental 1 value including

0 outlier

r F Experimental

0 value without

5 outlier

0.0 2.0 4.0 6.0 0.0 4.0 8.0 12.0 Grand-mean difference, Averaged variance in draws-to- a. paranoid – healthy control b. decision, paranoid group.

Figure 3.4: Distributions of simulated summary statistics for the beads task. In this case the experiment of Corcoran et al. (2008) has been simulated 50,000 times using the CB model with best-ﬁt parameters. a. Difference between the mean draws-to-decision (DtD) for paranoid vs. healthy control groups. The DtD has been averaged over trials and participants (but not over task variants: only the original ‘beads’ version of the task is considered here). The central limit theorem and the linear combination of the variables guarantee that the distribution is Gaussian. The experimental value of the quantity simulated here is 2.50, near the mode of the distribution shown. b. The variance of DtD for the paranoid group (averaged over stimuli) in the same simulation. The distribution is not normal. The two vertical lines show the experimental values. If all the experimental data is taken into account, the model considerably underestimates this mean variance. How- ever this is because of the variance being particularly sensitive to a single outlier; once this outlier is excluded, the model provides a good ﬁt. The mean difference of panel (a.) is quite insensitive to the presence of this outlier.

69 CHAPTER 3. COMPUTATIONAL METHODS FOR COST-BASED DECISION MAKING

Figure 3.5: A sample of fifty simulated experiments was generated with the CB model with best-fit parameters for the paranoid and healthy-control groups. The EM algorithm was then used to reanalyse the generated data. This figure shows the distributions of the mean cognitive noise τµ and standard deviations τσ retrieved in this way. a. The retrieved means τµ form well separated distributions. If they are log-transformed they became approximately normal, and the log-transformed versions were used for statistical comparisons. b. The standard deviationτσ in this example are not so well separated. We cannot exclude the possibility that the healthy-control best-fit τσ may have arisen from the paranoid-parameter distribution. were then compared across groups. They cannot actually be compared to the parameters underlying the experimental data in the same way as the summary statistics can, as we had no independent way of deriving ‘true’ macroparameters for the experimental data. Two examples of this are shown in figure 3.5. The advantage of this method is that the random errors inherent in the computer implementation of the EM algorithm itself are included in the production of these distributions. Disadvantages include first, that the procedure is very laborious computationally and hence only small samples were used, compared to the Monte-Carlo simulations for summary statistics. Secondly, it would be preferable to calculate those intervals in parameter space which, were they to contain the true values of the best-fit macroparameters, would not be too unlikely to give rise to the best-fitted values actually observed. If we were to consider the example in figure 3.5a, we could ‘scan’ the τµ say from 0 to 10 and discover those values that would render the actually fitted value for the healthy group (τµ = 5.09; see section 5.4) unlikely, for example at the 5% level. The resulting interval in τµ would be a more rigorous measure of a confident interval, as it would allow for the change in shape of the distribution of fitted values as the putative population value would change. Unfortunately such a more rigorous procedure was not possible to carry out within the available resources.

70 3.7. COMPUTATIONAL METHODS: CONCLUSIONS

3.6.2 Model comparison and the Bayesian information criterion

No single way of selecting a preferred model is always best. One of our comparisons asked which model gave a better account of the data of a particular group for a particular condition. Another comparison involved different groupings of the data. A basic measure was therfore needed which would be applicable across models of different structure. Model likelihood is such a measure: this is the probability that the experimental data would arise if the model in question was the mechanism giving rise to them. However, a model with more free parameters, i.e. one which is less parsimonious, should ﬁt the data better. In essence we should consider the model evidence P (D ; M) prior to using the maximum-likelihood values for the parameters of the model M (Daw, in press). The Bayesian Information Criterion (BIC; Schwarz, 1978) is an approximation (in the limit of large datasets) for this model evidence. The BIC combines the model likelihood L at its maximised value with a penalty for the number of parameters k used, taking account also of the number of data points N that are to be explained:

BIC = −2 ln L + k ln N (3.6.3)

Given two models applied to the same data, the one with the lower value of BIC is to be preferred. The BIC is very useful as it can be readily computed and errs on the conservative side, i.e. tends to over-penalize increasing numbers of explanatory parameters. On the other hand it has been criticised (Daw, in press) as it treats all explanatory parameters on equal footing, that is, it does not take into account the uncertainty in the posterior parameter estimates. For these reasons we used the BIC in conjunction with the methods above when considering model validity.

3.7 Computational methods: Conclusions

In this chapter mathematical models were formulated to simulate conditioned avoidance responding and the ‘beads-in-a-jar’ probabilistic decision-making task. Conditioned responding was modelled so as to explore a range of different variants of conditioned-avoidance learning, with an emphasis on dopaminergic manipulations. This will be pursued in chapter4 and the outcome combined with the earlier, qualitative discussion of conditioned avoidance and defensiveness (section 2.5). An empirical test of some interesting predictions that follow will then be designed (chapter6). Models for the ‘beads-in-a-jar’ task were also derived, together with a ‘toolbox’ of methods for detailed ﬁtting to experimental results and for model evaluation. In chap- ter5 these will be applied to an empirical dataset in order to test two key hypothesis: (i) Whether avoidance of perceived high costs of gathering information leads to precipitous decisions in paranoia; and (ii) whether people with delusions deviate from the ideal Bayesian reasoning more than non-deluded controls.

71 72 CHAPTER FOUR

A TEMPORAL DIFFERENCE ACCOUNT OF AVOIDANCE LEARNING

This chapter is an updated version of the published paper: Moutoussis, M., Bentall, R. P., Williams, J., & Dayan, P. (2008). A temporal difference account of avoidance learning. Network, 19(2), 137–60.

4.1 Summary

Introduction: We have argued that processing aversive information may be important in paranoid symptoms of psychosis. We developed a temporal-difference model of the Conditioned Avoidance Response, an important experimental model for aversive learning and a central pharmacological model of psychosis, to explore whether mechanisms involved in the CAR explain some paranoid symptoms. Methods: In the model, dopamine neurons reported outcomes that were better than the learner expected, typically coming from reaching safety states, and thus controlled the acquisition of a suitable policy. Results: The model accounted for normal Conditioned Avoidance learning, the persistence of responding in extinction, and critical effects of dopamine blockade, notably that subjects experiencing shocks under dopamine blockade, and hence failing to avoid them, nevertheless develop avoidance responses when both shocks and dopamine blockade are subsequently removed. Conclusions: These postulated roles of dopamine in aversive learning can thus account for many of the effects of dopaminergic modulation seen in laboratory models of psychopathological processes. The model mechanisms do not explain delusional ﬁxity, but they do explain the powerful persistence of avoidance.

Keywords: Temporal-difference learning; Conditioned Avoidance Response; Dopamine; Serotonin; Psychosis.

73 CHAPTER 4. A TEMPORAL DIFFERENCE ACCOUNT OF AVOIDANCE

4.2 Conditioned avoidance (CAR) and reinforcement learning

As was discussed in chapter2, the CAR is likely to involve key psychobiological mechanisms that may be activated to an exaggerated degree in clinically paranoid anticipation of threats (Moutoussis et al., 2007), although the role of specifically aversive learning mechanisms in psychopathology has yet to be completely understood. It is important to explore whether mechanisms involved in the CAR may be responsible persistence of avoidance in paranoia and if an over-activation of such mechanisms might underlie impaired learning, as may be involved in delusional fixity. There is a wide disparity between the sophistication of our understanding of learning and choice in appetitive versus aversive contexts. Appetitive learning has attracted theories of the acquisition and expression of habits based on temporal difference learning, which somewhat seamlessly link statistical, psychological and neural ideas and data (Montague et al., 1996; Schultz et al., 1993). There is also a developing understanding at all these levels of the relationship between habits and motivationally-sophisticated goal- directed actions. By contrast, despite some notable studies (Daw et al., 2002; Grossberg, 1972; Johnson, Li, Li, & Klopf, 2002; Schmajuk & Zanutto, 1997; Seymour et al., 2004), the functional basis of aversive learning remains more obscure. The Conditioned Avoidance Response (CAR) has particular significance for psychiatry, having inspired the development of behavioural therapy techniques (e.g. response prevention) by psychologists. It is also a standard test-bed for assessing antipsychotic drugs by psychopharmacologists e.g. Anisman, 1978; Bardin et al., 2007; Siuciak et al., 2007 It has therefore been very important to understand the CAR in detail. Despite this, efforts towards such understanding waned as it was realised that classical, operant, cognitive- expectancy and possibly other brain mechanisms were all involved. In more recent years interest in the CAR resumed as both theoretical e.g. Smith et al., 2005; Smith et al., 2006, 2007 and psychopharmacological e.g. Wadenberg and Hicks, 1999; Samaha, Seeman, Stewart, Rajabi, and Kapur, 2007 progress was made. In this chapter we explore three hypotheses. First, we suggest that a temporal-difference learning model of the CAR can capture its key qualitative, empirical psychological findings. Second, we explore the hypothesis that Dopamine is involved in the learning of aversive associations to stimuli, as held by the main theories so far suggested for its role in the CAR. The latter include important theories based on temporal difference ideas (notably Smith et al., 2005; Smith et al., 2006, 2007). We show that a temporal-difference model integrating a wider range of pharmaco-behavioural findings implies that dopamine is unlikely to be involved in the learning of aversive associations between stimuli. How- ever, it most likely does have an important role in the learning of responses. Third, we show that our model helps transcend the difficulties of the classical, qualitative psychological accounts of aversive learning even without appealing to special, additional features of

74 4.2. CONDITIONED AVOIDANCE (CAR) AND REINFORCEMENT LEARNING

Figure 4.1: The onset of the Warning Stimulus signiﬁes the start of a conditioned avoidance trial. (i) The aversive (Shock) stimulus follows after a standard interval Ts.(ii) Performing a speciﬁc safety behaviour after US onset (ER) stops both CS and US. (iii) If the safety behaviour is performed with a latency Tl < Ts, (AR) the CS is aborted and no US is delivered. (iv) After acquisition, response may be blocked while no US is given (response prevention). the mechanisms involved. We end this chapter by drawing out some predictions that our analysis implies. These predictions are relevant to the study of human appraisal of threat, including paranoia.

4.2.1 The CAR experimental paradigm

In a typical rodent version of the CAR, a subject learns that a neutral warning conditioned stimulus (CS) will be followed by an unconditioned aversive stimulus (US) – usually an electric shock (figure 4.1). Here we briefly summarize the basic experimental protocol detailed in chapter2 (page 42), we describe additional behavioural and pharmacological manipulations that we will analyse and we also describe some variants of the protocol. In the basic CAR protocol the subject can escape the US after its onset, or can avoid the US altogether by performing an experimenter-determined skeletal response within a specific time interval after the onset of the CS. The latter is termed the avoidance response (AR). In the case of rats or dogs it usually consists of shuttling to a different part of the experimental enclosure. Generally, the AR interrupts the CS and aborts the US. Shuttling after US onset interrupts exposure to the US and is termed an escape response (ER). In the human version of the CAR, the US is often a burst of loud white noise, and the AR/ER generally involves pulling a lever rather than shuttling (Unger et al., 2003). Under normal circumstances, animals take only a few shocks to discover that shuttling interrupts the shock (ER). From then on they quickly learn to perform the AR. Sub- jects achieve a high percentage of ARs in a few tens of trials (cf. Fig. 4.2a (Beninger et al., 1980a)). As learning proceeds, their latencies of responding decrease (cf. Fig. 4.2b (Solomon & Wynne, 1953), and they lose any sign of overt fear to the CS. Once

75 CHAPTER 4. A TEMPORAL DIFFERENCE ACCOUNT OF AVOIDANCE the AR is well-learnt, providing the shocks have been of sufficient magnitude, responding continues for many trials even if no shocks are, or would be, delivered (Fig. 4.2b and c (McAllister, McAllister, Scoles, & Hampton, 1986; Solomon & Wynne, 1953)) Ex- tinction can be accelerated by physically preventing the animal from shuttling (response prevention Fig. 4.1-iv). This initially leads to an increase in signs of anxiety, which successful avoidance-responding had eliminated. If, however, after a few trials the animal is allowed to shuttle, the frequency of AR is much reduced – even in the presence of residual signs of anxiety (Mineka, 1979). A variant of the CAR designed to separate a phase of Pavlovian-like aversive learning from a phase of instrumental-like learning is the escape-from-fear (EFF) paradigm (McAllister, McAllister, Hampton, & Scoles, 1980). Here again a warning stimulus is followed by a shock for a set number of training trials and, during these trials, the animal has no way of terminating the shocks. In the immediately subsequent trials, however, shuttling becomes available as a response. In most experiments, shocks also stop being delivered following the warning stimulus. Animals are observed to acquire a shuttling response quickly, but not immediately; its latency then decreases. Shuttling may again persist for dozens of trials before gradually extinguishing (figure 4.2c). Critically, the CAR is suppressed by blocking Dopamine receptor type 2 (D2) function with antipsychotic drugs (figure 4.2a). Although at high doses, motor output itself is compromised, at lower doses the escape response is unaffected, suggesting that antipsychotics affect acquisition during training (Smith et al., 2007). Equally, DA blockers reduce expression of the AR, in a way that resembles the extinction-like effect that administering such drugs has on reward-motivated behaviour (Wise, Spindler, deWit, & Gerberg, 1978). One might think that dopamine is involved in pathways reporting aversive events in a way analogous to its role in reporting better-than-expected outcomes in reward learning (Schultz et al., 1993). Indeed, microdialysis and other studies showed that dopamine is released in response to aversive stimuli (Horvitz, 2000). In addition, imaging studies showed activation in response to aversive stimuli in areas innervated by the monoaminer- gic systems (Jensen et al., 2003; Menon et al., 2007). There are, however, reasons to believe that dopamine is not directly involved in reporting negative outcomes. First, the ventral-tegmental neurons that are excited (at least under some conditions) by aversive stimuli are likely not dopaminergic, as originally thought (Ungless, 2004)1. Secondly, dopaminergic neurons that do predict outcomes only code better-than-expected ones (Bayer & Glimcher, 2005) with any substantial fast fidelity. Third, in a recent human imaging study (Menon et al., 2007) it was noted that dopaminergic enhancement or blockade did not affect the subjectively reported anxiety experienced in response to a conditioned stimulus predicting pain; neither did dopaminergic manipulations affect the subjects’ ability to learn which conditioned stimulus predicted the painful one.

1However since the present chapter was published dopaminergic VTA neurons excited by noxious stimuli have been more reliably identiﬁed (Brischoux, Chakraborty, Brierley, & Ungless, 2009).

76 4.2. CONDITIONED AVOIDANCE (CAR) AND REINFORCEMENT LEARNING

2a 2b

2c 2d

Figure 4.2: a. Pooled avoidance probability data from rats. In normal CAR learning, near-perfect avoidance is achieved within a few trials (from (Beninger et al., 1980b)). DA block shows a powerful dose-dependent effect (Pimozide doses in mg/kg). b. Sample of data from a dog (from (Solomon & Wynne, 1953)) where response latencies decrease for many trials after achievement of 100% avoidance. c. Pooled latency data from animals subject to Escape-From-Fear training. Following the CS they were first given inescapable shocks; then they were given the opportunity to shuttle with the shocks turned off. Per- formance increases (i.e. latency decreases) for about 20 trials, remains stable for another 50 trials, then declines slowly first, then faster; from McAllister and McAllister (1991). d. Latency data from rats (similar to a.). Unfilled diamonds: unmedicated rats performing the standard CAR in training and testing. Unfilled squares: shocks turned off (normal extinction) during testing. 10 seconds is indicated, being the delay with which the US was given in training trials. Filled symbols are data from rats that received pimozide during training only; from Beninger et al. (1980b).

77 CHAPTER 4. A TEMPORAL DIFFERENCE ACCOUNT OF AVOIDANCE

However, perhaps the most signiﬁcant challenge to existing dopaminergic (Smith et al., 2005; Smith et al., 2006, 2007) and indeed some non-dopaminergic (Schmajuk & Zanutto, 1997) models of the CAR is from a condition that turns out to resemble escape from fear. In this, the animals are trained with a blockade of dopamine D2 receptors, and thus fail to acquire the AR. However, if the dopamine blockade and the shocks are removed after the subjects have experienced a few shocks, subjects actually acquire the AR (Beninger et al., 1980b), following a learning curve that resembles that obtained in the EFF paradigm (Fig. 4.2c). The obvious interpretation of this is that dopamine is more likely to be involved in learning to respond based on the Pavlovian CS-US association, rather than in the formation of the association itself. In the present study, we built a temporal-difference learning (Sutton & Barto, 1998) model of the CAR and investigated the consequences of manipulating dopamine in the model. The model successfully reproduced a broad range of CAR phenomena. We argue that this bolsters the hypothesis that DA is involved in boosting predictions and actions when outcomes are better than expected. Other systems, putatively (though controver- sially) serotonin (Daw et al., 2002) could play a similar role for aversive prediction learning when outcomes are worse than expected, but appear not to be able to effect action learning by itself, at least in contexts like CAR. The model does not seek to account for slow (tonic-like) timescale dopamine effects, which are important for understanding some results of pharmacological manipulations and some microdialysis ﬁndings. We shall refer to these separately.

4.3 The Advantage-learning Temporal Difference model

In the temporal difference (TD) variant of reinforcement learning (RL), subjects come to expect particular gains or losses (collectively value) to accrue from each situation or state they encounter. The change in these expectations should stochastically match the immediate gains and losses they experience; if it does not, then there is a prediction error PE that can be used to improve the estimates of the returns. Based on the substantial evidence about the phasic activity of dopamine cells (Bayer & Glimcher, 2005; Schultz et al., 1993), we modelled the appetitive portion of the PE as being dopaminergic. In the CAR context, appetitive prediction errors arise when the subject performs the avoidance response, and so changes from being in a state of fear, anticipating the shock, to being in a state of safety, when the shock has been averted. It is this transition that is reported by phasic dopamine. On the other hand, given our TD architecture, the post-dopamine- blockade avoidance acquisition data (Beninger et al., 1980b) suggest that aversive Value- learning can proceed even in the presence of dopamine blockade. This conclusion is also consistent with theoretical suggestions about appetitive-aversive opponency (Daw et al., 2002; Solomon & Corbit, 1974). Values are only one part of RL; and are normally acquired in the service of learning

78 4.3. THE ADVANTAGE-LEARNING MODEL policies, which are a systematic (though possibly stochastic) ways of assigning actions to states. In several variants of TD, the aversive Values and PEs can be directly used to learn actions that minimize the Values. This can be seen as a form of Mowrer’s two-factor theory (Mowrer, 1947), with the conditioned fear (or anxiety (Gray & McNaughton, 1996)) arising from the Value predictions; and with the (dopaminergically-reported) reduction in conditioned fear acting like an appetitive reinforcer, boosting the subsequent selection of the associated action. There are several ways that policies may be represented, notably indirect methods, in which they are derived from predictions of long run values, and direct methods, in which they have their own parameters. For appetitive learning, about which rather more is known in this respect, there may even be functional and indeed structural transitions between different forms of policy over the course of learning (Belin & Everitt, 2008). More precisely, the class of models that the present work belongs to is termed actor- critic (Sutton & Barto, 1998) models. They have two key components :

• A critic which learns affective expectations. This is the part of the model which associates with each distinct state that the animal perceives the affective Value which summarises how good or bad this state is (a measure of the total return to be expected to follow this state).

• An actor which learns to make appropriate decisions in the light of these expectations. The (usually probabilistic) rules of taking these decisions is what the actor learns and is termed the "behavioural policy". In our case, the optimal policy is the one that minimises long-term costs.

This is summarized in figure 4.3. In such a model long-term costs, and thus Values themselves, will depend in turn on the actions that the animal takes (e.g., whether or not it avoids the shock). Note that the output of the critic embodies the expectations and predictions that appeared so puzzling to some in the purely behaviourist era (Lovibond, 2006). Advantage learning (Dayan & Balleine, 2002) is a form of the actor-critic in which action choice depends on a particular aspect of the value of an action. The advantage m(a, s) of action a in state s quantifies how much better this action is compared to the policy followed on average, i.e. it is defined as the difference between the value of the particular action Q(a, s) and the value of the state V (s). A major spur to our use of it is that O’Doherty and co-workers (O’Doherty et al., 2004) showed that advantage learning provided a good model for the BOLD signal in the dorsal striatum during the acquisition of a simple (appetitive) instrumental task). The experimental data strongly constrained the architecture of our model. First, a model that used only one set of values (i.e. action values) to learn could be discounted in favour of one that had both state-values and action-related (e.g. advantage) values. This is because the action-value models under dopamine blockade or EFF would only have these

79 CHAPTER 4. A TEMPORAL DIFFERENCE ACCOUNT OF AVOIDANCE

Figure 4.3: Flow of information in the actor-critic model. action-values to remember, and hence would have no basis for preferring the avoidance response once that becomes available (as per ﬁg. 4.2). Once we adopted an actor-critic architecture, we were forced to interpret the PE differently in the critic (value) vs. actor (policy) limbs. Worse-than-expected PEs could not depend on dopamine in the critic part, as aversive value learning survives dopamine blockade as discussed above. However, in the actor limb learning signals should depend on dopamine, as avoidance action learning does not survive dopamine blockade (Beninger et al., 1980b). An additional experimental constraint determined our choice of the advantage-learning variant of the actor-critic method. This is that under dopaminergic blockade the asymptotic frequency of avoidance responding appears to change in a quantitative, dose-dependent manner (Smith et al., 2007). In other common variants of the actor-critic formalism (e.g. Sutton and Barto (1998), chapter 6) the rate of policy learning would change, but not the asymptotic policy preference. An algebraic description of the advantage-learning model of avoidance is presented in the chapter3, section 3.3.

4.4 Results

4.4.1 Simulation of escape-from-fear learning

The inescapable-shock phase of the EFF paradigm is conceptually simpler than the escapable-shock CAR. This phase also serves as a useful comparison for the shocked phase of the CAR that takes place under dopamine block. Fig. 4.4 shows a simulation of

80 4.4. RESULTS

EFF. Before the onset of shocks, the subject explores the available actions and occasion- ally shuttles. The Values of states 1 to 5 stay around zero, as the only slightly aversive outcome is the small motoric cost of the occasional shuttling. When shocks start the “safe” states (S7 to S11) become unavailable. The states temporally near to the shock (e.g. S5, diamonds in Fig. 4.4a) first acquire aversive Values. These they gradually feed back to earlier predictive states (cf. S3, triangles, and S1, crosses, in Fig. 4.4a). Shuttling is then allowed. Most learners soon try shuttling again, and hence experience a large positive PE – from high Value (S1-S5) to zero (S7-S11). This gradually reduces the Values of S1 to S5, but at the same time teaches that shuttling is quite advantageous. The probability of avoidance rises rapidly and persists at an elevated level for tens of trials (Fig. 4.4b; cf. Fig. 4.2c). Features of this latter, unshocked phase of the EFF, which resembles closely the corresponding phase of the standard CAR, will be presented in the context of the latter. The underlying mechanics of learning are presented in the fig. 4.5- 4.6). During the first phase of the EFF, from trials 100 on in fig. 4.4, subjects have learnt that all states 1-5 predict shock for all actions. Within about 20 trials, all states have Values about equal to the return of the shock. During the second phase ’shuttle’ actions lead to safety states of Value zero. This takes place from trial 150 onwards in the example of fig. 4.4b (same example in fig. 4.5 and 4.6). Therefore when these shuttle actions are taken, the Values of the originating states reduce according to equation 3.3.1 (page 53). In our example, the Value of state 4 decays from 4 (= return of shock) towards 0.2 (= return of shuttle) each time shuttling occurs from that state, as happens in trials 152, 154 etc. The reduction in Value of state 5 reduces for “stay” too, as there is now no shock (return = 0). If we take the first non-shocked trial, trial 151, as an example, and using eq. 3.3.1, we get:

δV (5) = Rnon−shocked(5 → 6) + V (6)old − V (5)old = 0 + 0 − 4 = −4 (4.4.1)

Using equation 3.3.3, this gives:

V (5)new = V (5)old + 0.5 × δV (5) = 4 + 0.5 × (−4) = 2 (4.4.2)

This is shown in ﬁg. 4.5. Note the unusual dynamics for the Value of state 1. When shuttling occurs, V (s1) reduces as expected. However, since s2 is little visited, its Value (V (s2)) is not greatly reduced (in fact, mostly what was inherited from S4), Thus, when ’stay’ is chosen in s1, and so s2 is visited, the value of s1 usually increases sharply (trials 153, 155, 169 etc.) because of on-going learning. These phenomena have a very interesting effect on policy, shown in ﬁg. 4.6. The advantages for the action ’shuttle’ (upper curve) and ’stay’ (lower curve). While V(S1) is high, each ’shuttle’ leads to learning that this action is more advantageous, according to

81 CHAPTER 4. A TEMPORAL DIFFERENCE ACCOUNT OF AVOIDANCE

Figure 4.4: a. First part of Escape-from-fear trial for one subject. In this and subsequent ﬁgures, shocks are turned on at trial 101. Lower plot: response latency. Each black diamond corresponds to the last state of the ’unsafe’ side visited during the trial (e.g. a value of 1 means shuttling from state 1 while 6 means no shuttling during that trial). Top panel: Values of states 1 (crosses), 3 (triangles) and 5 (diamonds), showing how they converge in turn to the cost-of-shock when the latter is inescapable. b. Second part of EFF. Bottom panel: same as a. Top - overall probability of avoidance calculated from model variables (See text for further details). Avoidance probability increases quickly; then decays slowly, but is boosted when later states are visited. c. Pooled Latency for 100 simulated identical subjects.

82 4.4. RESULTS

Figure 4.5: Escape-From-Fear – Value Learning

Figure 4.6: Escape-From-Fear – Advantage Learning

83 CHAPTER 4. A TEMPORAL DIFFERENCE ACCOUNT OF AVOIDANCE eq. 3.3.7. Let us take trial 153 as an example. Here shuttling occurs for the ﬁrst time from state 1 (to the ’safe’ state 7). We now have, as above:

δV (1) = Rshuttle(1 → 7) + V (7) − V (1)old = 0.2 + 0 − 4 = −3.8 (4.4.3)

and, substituting in eq. 3.3.7 while reversing the sign of the value of δV to respect the convention of the model description above, the new value of the policy m for action a = shuttle at state s = s1 is

m(shuttle, s1)new =

= m(shuttle, s1)old + 0.075 × (δV − m(shuttle, s1)old) = 0 + 0.075 × (3.8 + 0) = 0.285 (4.4.4)

In addition, the occasional ’stay’ actions in S1 lead to visiting the unextinguished s2, and hence teach that ’stay’ is disadvantageous. This happens, in trials 154, 169 etc. in our example. The policy of shuttling from this early state is therefore boosted (eq. 3.3.4). This is similar to the psychological explanation that has been given for the tendency of the CAR to persist. Conversely, delaying the AR exposes the animal to later, still unextinguished, parts of the CS, increasing momentary fear, and thus delaying extinction.

4.4.2 Simulation of normal CAR learning

Fig. 4.7 shows a typical example of a learning curve from our model. First, as is true of control subjects, after receiving a small number of shocks, the model learns to favour the avoidance action, with a probability approaching one. If adequate exploration occurs, responses tend to move to earlier states quickly, reducing response latency. The steep learning curve is followed by consistent avoidance, which is quite persistent (though not wholly immune from being extinguished) even after programmed shocks have stopped. This is similar to the animal data shown in ﬁgures 4.2b and 4.2c (McAllister et al., 1986; Solomon & Wynne, 1953).

If the learner is forced to follow action ’stay’ rather than ’shuttle’ during extinction, then the model learns not-to-avoid more rapidly. It is thus sensitive to response- prevention. At the onset of this phase, the Values of early states show a transient increase (Fig. 4.7c, ﬁrst 5-10 trials of RP). If we assume that visiting a high-Value state is psychologically alarming to the animal, we have a situation analogous to the alarm initially experienced by animals or humans subject to response-prevention (this is similar to transient increases during response persistence – cf. Fig 4.5). At the same time, Values of later states decay to zero. With more learning, all Values decay to near-zero levels.

84 4.4. RESULTS

Figure 4.7: a. Unmodulated CAR simulation. Shocks are turned on between trials 101 and 150 (dark grey), but they are escapable. Response prevention (forced staying) occurs between trials 250 and 280 (light grey). Top plot – overall probability of avoidance calculated from model variables (based on Equation 4). Probabilities are shown on the left ordinate. Roman numerals mark eras discussed below. Bottom (“Latency”): as in Figure 4.4. b. Avoidance probabilities averaged over 300 ’subjects’ × 380 trials each, protocol as per Figure 5(a). Before training (I) most learners reach state 6, where shock is delivered. There is relatively little degradation in performance 30 trials after shocks stop (IV), but some degradation is evident 65 trials later (V). Avoidance is reduced faster after response prevention (VI). 100 trials later, responding is much like pre-training (VII). c. Values of state 3, from which much shuttling takes place, for example in panel (a.). Note that during the late parts of shocked-learning, but also the early parts of ‘persistence’ trials, choosing action ‘stay’ results in a dramatic increase of the value (i.e. the aversiveness) of state 3. Trials 250 to 280 show details of response-prevention. Note the transient increase in state 3 value.

85 CHAPTER 4. A TEMPORAL DIFFERENCE ACCOUNT OF AVOIDANCE

4.4.3 Simulation of dopaminergic manipulations in the CAR

DA manipulations can be initiated and removed at various times during learning. Most straightforwardly, if D2 blockade is effected after acquisition, simulations show that persistent responding in the absence of shocks is more likely to spontaneously extinguish earlier (data not shown). If D2 receptors are blocked during initial learning, then the model is much slower to acquire the avoidance response (Fig. 4.8. This is similar to the animal ﬁnding that D2 blockade results in a dramatic decrease in AR learning. In Fig. 4.2d, for example, rats treated with pimozide during training showed an average latency of responding of 12 sec, as opposed to 5 sec for the control rats (Beninger et al., 1980b). In our models, peak responding (and shortest latency) takes much longer to achieve, and furthermore, the peak probability of response is also signiﬁcantly reduced. Empirically, if DA blockade is removed when shocks are also turned off, avoidance dramatically strengthens in extinction (as in Fig. 4.2b, ’Pimozide Extinction’). In our model removing the blockade affects both the probability of response and the latency for responding (Fig. 4.9a, b). Note that ARs are even more persistent, and average latencies even shorter, than in the normal case shown in Figure 4.7. This is largely due to these learners having being exposed to more shocks during acquisition of the aversive state- value structure, on which the subsequent response acquisition and persistence depends. Finally, we simulated the effects of boosting rather than suppressing dopamine. This increases the rate of learning and subsequently the persistence of behaviour in the absence of further shocks, but does not result in true ’resistance to extinction’, i.e. an impairment of the effect of response prevention. The experimental literature on this topic is limited but suggests that low doses of DA agonists enhance the effect of response prevention (Christy & Reid, 1975; Cooper, Coon, Mejta, & Reid, 1974), consistent with our models (data not shown).

4.5 Discussion

The temporal-difference model of avoidance successfully describes a large range of qualitative experimental results. It suggests specific computational roles for the elements of two-factor theory, linking it directly to dopaminergic mechanisms. The model resolves the apparent paradox that dopamine receptor antagonists dramatically suppress avoidance responding, and yet appear not to be involved in reporting worse-than-expected, aversive outcomes. It also sheds light on other findings that appear puzzling or counterintuitive from the point of view of qualitative two-factor theory. These include the relatively persistent, efficient shuttling in the absence of fear, which has been subject to much debate. In our models, this naturally follows from the fact that it is the cumulative experience of differences between outcomes, not the outcomes themselves, which result in policy learning.

86 4.5. DISCUSSION

Figure 4.8: Example of simulation of 80% DA blockade affecting positive corrections to values and also all corrections to policy parameters. Otherwise the trials are conducted as in Figure 4.7. Grey-scale coding also as per Figure 4.7. a. Example of avoidance probability and latency. A much lower level of responding is established (cf. Figure 4.2a ’Pimozide’). Response prevention has little effect. b. Latencies averaged over 100 ’subjects’. DA blockade has caused much less avoidance of state 6, where shocks were delivered in training. Some reduction in latency took place during the ’persistence’ trials (IV – V).

87 CHAPTER 4. A TEMPORAL DIFFERENCE ACCOUNT OF AVOIDANCE

Figure 4.9: Advantage-learning model that has been DA-blocked (85% blockade) during learning is then freed both from DA block and from shocks from trial 131 on. a. Example of avoidance probability and latency. Spontaneous learning occurs to a high level, with a dramatic increase upon release from DA blockade. b. Averaged latency plot, showing the dramatic response to withdrawal of DA blockade (black curve). Light grey curve is the same as Figure 4.8b, persisting DA blockade, for comparison. Note effect of response prevention.

88 4.5. DISCUSSION

There have been other models of the Conditioned Avoidance Response. In the category of two-factor models (like ours), we note the work of e.g. Grossberg (1972), who has carefully analysed opponency in non-linear dynamical settings, and Schmajuk and Zanutto and Johnson and co-workers (Johnson et al., 2002; Schmajuk & Zanutto, 1997). Despite their many attractive features, these models are more psychological in nature. They pay less attention to the substantial data on the involvement of dopamine in appetitive learning, together with its apparent opponent-based role in the CAR. They therefore do not set out to address the key experimental data that constrains our model. Note that in our model the opponent dopamine signals (due to “relief” that the aversive stimulus has been avoided) are on the same footing with all others, and that signals due to different psychological sources (CS, AR) sum linearly. It was not necessary to appeal to any special properties of the ’safety signals’ (for instance as extinction-resistant conditioned inhibitors of fear (Gray, 1987)) that accompany the avoidance response.

A second category of model of avoidance involves versions of Expectancy theories, the prototype of which is that of Seligman and Johnson (Seligman & Johnston, 1973). Ex- pectancy theories emphasise that, subsequent to the initial associative fear-conditioning, animals explicitly learn which events are likely to follow each action in each state. Sub- jects can then decide which action to take by comparing the ultimate expected outcomes of sequences of actions. The learner is said to use a ’forward model’ (Wolpert & Miall, 1996), or, in a conditioning context, goal-directed actions (Dickinson & Balleine, 2002; Daw et al., 2005). Two of Smith and co-workers’ influential models (Smith et al., 2005; Smith et al., 2006) are perhaps most correctly seen in expectancy terms. Dopamine plays two roles in these particular models. One, putatively ascribed to tonic levels of this neuro- modulator, controls the course of inference through the forward model. The second role, identified with phasic DA, is to report a product of surprise and significance which is used to learn transition coefficients connecting the states comprising the forward model. These models have trouble accounting for the gradual acquisition seen in Beninger’s blockade study (Beninger et al., 1980b), since the expectancy of fear that is clearly established does not immediately lead to appropriate responses even when the blockade is removed. Their contact with existing data on DA’s involvement in appetitive learning is also somewhat distant.

In the context of appetitive conditioning it has been suggested (following (Coutureau & Killcross, 2003)) that model free, cached, reinforcement learning (RL) methods would coexist with model-based, goal-directed, RL (Daw et al., 2005). Expectancy theories involve just the sort of forward models that are implicated in the goal-directed system; while our two-factor model is a form of cached controller. Thus it would be natural to suppose that both sorts of model may coexist in this aversive case too; a possibility that opens up various lines of experimental enquiry. Daw and co-workers suggested that arbitra- tion between the two different controllers should depend on their relative uncertainties, a suggestion that would also be suitable for this aversive case (Daw et al., 2005).

89 CHAPTER 4. A TEMPORAL DIFFERENCE ACCOUNT OF AVOIDANCE

Rather more sui generis is the third model of CAR suggested by Smith and colleagues (Smith et al., 2007). In this work, affective values translate directly to probabilities of action and hence it is again difﬁcult to account for the development of responding in extinction (Beninger et al., 1980a). Compared with this, we placed particular emphasis on the aversive framing of the CAR, and the apparent opponent interaction between dopamine and its putative aversive opponent.

4.5.1 Neurobiological substrate

A rich nexus of neural areas appears to be involved in the habitual system’s instan- tiation of the CAR (noting that different paradigms of aversive learning recruit different such groupings, e.g. (Reis, Masson, de Oliveira, & Brandao, 2004)). First, the basolateral amygdala (BLA) appears necessary for the acquisition of instrumental avoidance (Poremba & Gabriel, 1995) whereas different amygdalar regions appear to be involved in other types of fear learning, such as conditioned suppression (Killcross, Robbins, & Everitt, 1997). The human amygdala may be involved in the evaluation of unexpected costs (Yacubian et al., 2006) although some studies did not detect such activity (Seymour et al., 2004). The amygdala appears only to be involved in the initial stages of aversive learning and even the passage of time alone (not learning trials) reduces its involvement (Poremba & Gabriel, 1999). This may contribute to the heterogeneity of imaging ﬁndings (Jensen et al., 2003). BLA projection neurons receive glutamatergic projections from sensory association cortices. They also receive inhibitory input from prefrontal cortex, via inhibitory interneurons. It is here that mesolimbic dopaminergic inputs intervene. Both D1 and D2 receptors on BLA neurons serve to suppress medial prefrontal inhibition and enhance responses to sensory inputs (Rosenkranz & Grace, 2001, 2002). The orbitorfrontal cortex also plays an important role in avoidance, at least in human subjects. There is evidence that this region encodes not-incurring the ’cost’ or ’loss’ associated with an aversive stimulus during avoidance tasks as if it were a reward value (Kim, Shimojo, & O’Doherty, 2006). Such a “reward” value could serve to reinforce safety behaviours. Imaging studies also implicate two other areas in encoding the discrepancy between how aversive a person estimates a situation to be, and how aversive this situation actually turns out to be (what we termed the PE). The areas involved are the insular cortex and the corpus striatum (Seymour, Daw, Dayan, Singer, & Dolan, 2007; Seymour et al., 2004). The corpus striatum appears to contain different functional subregions, which are preferentially activated by rewarding PE (near the nucleus accumbens) and aversive PE (slightly more posterior, in the putamen). Finally, the most critical question for our model is the neural substrate for the representation of the aversive prediction error to control the learning of the aversive Val- ues. Based on various lines of evidence, it has been suggested that serotonin may play a critical role, perhaps as an opponent (Grossberg, 1972; Solomon & Corbit, 1974) to

90 4.5. DISCUSSION dopamine (Daw et al., 2002; Deakin & Graeff, 1991). Indeed, serotonin has been shown to play an important role in learning in the CAR (Ma & Yu, 1993; Titov, Shamakina IIu, & Ashmarin, 1983; Wadenberg, Soliman, VanderSpek, & Kapur, 2001; Wadenberg & Hicks, 1999). However, there are known synergies between DA and 5-HT as well as this opponency (for instance, 5-HT2A receptors in the striatum appear to boost the effects of dopamine), making this prediction complicated. Further, systemic opioids and ACh- muscarinic agonists reportedly have less pronounced neuroleptic-like effects on the CAR (Aguilar, Minarro, & Simon, 2004; Shannon et al., 1999). As currently constituted, our model incorporates an important asymmetry between dopamine and its putative opponent. (a) Value learning depends on PEs from both dopamine and its opponent. However, (b) action learning and extinction depend exclusively on dopaminergic activity being greater than and less than its baseline respectively. In the latter case, note that a PE whose net affective value is aversive is represented by a net negative dopamine signal. This is partly a placeholder for what must surely be a more complicated relationship underpinning other phenomena too such as the nature (Tobler, Dickinson, & Schultz, 2003) and non-extinction of conditioned inhibitors. We postulate this asymmetry because of the effect of dopamine blockade, with aversive Values being well learned when DA is blocked (a), unlike avoidance (b). Unlike current theories (Smith et al., 2006), a forward-model based account that would respect the DA-blockade data would also have to differentiate between the roles of DA in appetitive vs. aversive learning and in forward-model structure vs. action-preference learning. One possible rationale for this asymmetry is that, at least in this particular context, there is much more information in the one avoidance action than in the many actions that do not prevent the oncoming shock, a fact that the subjects could learn whilst ﬂailing. However, it may be instead that a more fundamental role for tonic levels of dopamine in controlling action vigour (Niv, 2007; Niv, Daw, Joel, & Dayan, 2007) could include an effect of completely blocking the impetus to learn about effortful actions such as shuttling. Testing this possibility would require dissociating tonic and phasic dopamine signalling, something of active interest in the literature (Cagniard et al., 2006; Goto, Otani, & Grace, 2007; Grace, 2000). Our model only treats part of the involvement of dopamine in CAR, leaving out effects of tonic or sustained concentrations or release, and also involvement in freezing. The release of DA in aversive situations has been amply demonstrated using microdialysis, and is known not to be simply due to the offset of punishment, either through avoidance or otherwise (Young, 2004). One interpretation of this again comes back to tonic dopamine signalling, arguing that this release is associated with an expectation that an effortful, vigorous, avoidance or escape action will be required (Niv et al., 2007). The main effect of this in the model would be to enhance the sloth of acquisition and the speed of extinction, the latter simply by reducing exposure to the consequences of the “shuttle” action (data not shown). Freezing behaviour in response to shock is usually thought of

91 CHAPTER 4. A TEMPORAL DIFFERENCE ACCOUNT OF AVOIDANCE as a Pavlovian, simpler paradigm than the CAR. Pavlovian responses to aversive events are, in fact, modulated in complicated and controversial ways by different dopaminergic drugs (Miyamoto et al., 2004; Reis et al., 2004; de Oliveira, Reimer, & Brandao, 2006) and there are interesting avenues for investigation as to the best interpretation of this with respect to prediction errors (Iordanova, Westbrook, & Killcross, 2006).

4.5.2 Relevance for human psychopathology and future research

Future research in aversive information processing

Our models make some predictions that would be important to put to experimental test.

1. The basic patterns of avoidance behaviour considered here in the CAR, as well as their modulation by dopaminergic drugs, should correspond to analogous patterns in healthy humans. However, unfortunately, we have been able to ﬁnd little evidence on the effect of psychotropic drugs on human aversive learning.

2. The “better than expected” signal that follows perceived avoidance of an aversive outcome should be reported by dopamine, both within value-learning and within action-learning circuitries. This is consistent a large body of psychological research stressing the reinforcing property of reaching ’safety states’ and of some instances of CS offset e.g. Gray, 1987. It is also consistent with recent fMRI evidence (Kim et al., 2006). We note that within our framework it is not the offset of an aversive stimulus per se that would be associated with positive dopamine activity, but its better-than-expected signiﬁcance. How closely, however, dopamine is involved and whether it has similar roles in value vs. action learning remains to be tested directly.

3. If the dopaminergic-opponent process relies on serotonin, we would expect 5HT blockade signiﬁcantly to delay acquisition but not extinction.

4. We would expect much less ’rebound’ action-learning on removal of 5HT blockade, as compared to DA.

The activity of the goal-directed system could complicate outcomes and therefore results would be clearer in experiments where the goal-directed system is inhibited (Blundell, Hall, & Killcross, 2003; Killcross & Coutureau, 2003). The present models help set the scene for an investigation of the cooperative and com- petitive interactions of the habitual, forward-model and Pavlovian controllers in aversive processing. That the goal-directed controller favoured by expectancy theory does not appear to control performance on at least some of the existing collection of animal CAR studies, does not imply that it will not trump our habitual controller under any circumstance. Further, there also appears to be a third, Pavlovian, controller, which directly couples predictions of aversion to stereotypical defensive actions (Dayan, Niv, Seymour,

92 4.5. DISCUSSION

& Daw, 2006; Gray & McNaughton, 1996) in a way that can be synergistic or opponent to the instrumental choices of either the habitual or the goal-directed controller. Indeed, perhaps a very important future direction for this work is towards understanding the integration of these three mechanisms in terms of learning and performance, and thus, most optimistically, the potential combination of pharmacological and learning-based interventions in the treatment of psychotic psychopathology.

Implications for aversive information processing in psychopathology

There are good psychopathological grounds for focusing on aversive processing as an essential component of psychosis. The literature has traditionally concentrated on neu- rotic disorders (Forsyth, Eifert, & Barrios, 2006), but recent research highlights the importance of the aversive learning system in conditions where dopamine plays an important role. People suffering from paranoid psychosis often have a background of victimisation (Janssen et al., 2003; Mirowsky & Ross, 1983) and thus exposure to aversive learning. They tend to make exaggerated predictions of aversive events, even considering this background (Kaney et al., 1997; Corcoran et al., 2006). Their use of safety-behaviours (strategies for avoiding situations associated with perceived threat) seems to perpetuate their problems (Freeman et al., 2001). We have argued in Chapter2 that efforts to avoid both external threats and, crucially, threatening private experiences (Bach & Hayes, 2002) contribute to the aetiology of persecutory syndromes (Moutoussis et al., 2007). The current study provides a detailed basis for linking the psychology of anticipation of threat and the biology of psychosis. It permits, for example, calculation of regression coefficients relevant to avoidance-learning, so as to help locate corresponding anatomical areas by functional imaging (as per (Seymour et al., 2004)). In considering the psychobiological mechanisms involved in the CAR a clinically important corollary immediately follows which lies outside the scope of the present study, but is highly relevant for future research. The psychological processes examined here, especially vigour, detection of threat and the anticipation of rewards of actions, are very important in manic disorders. These are characterised by pathological anticipation of positive returns. TD modelling may help clarify the neurobiological mechanisms involved and link them to emerging psychological research, for example to the precipitation of manic episodes by positive goal attainment events (Johnson et al., 2000). This study suggests a set of functional mechanisms whose activity may be exaggerated in persecutory syndromes: the Dopamine-dependent reinforcement of escape and avoidance actions and the dopamine-independent prediction error about aversive states. We note, however, that the present study provides no support for the hypothesis that increased dopamine activity might underlie the fixity of paranoid fears. To the extend, that is, that the fixity of exaggerated aversive predictions about the world in paranoia could have been reflected in resistance to response-prevention in our models.

93 CHAPTER 4. A TEMPORAL DIFFERENCE ACCOUNT OF AVOIDANCE

We may therefore expect unmedicated, psychosis-prone individuals to show biased learning in CAR-like situations. As paranoia is intimately related to increased prediction of threat (Kaney et al., 1997; Corcoran et al., 2006), increased avoidance should be detectable in actively paranoid individuals, whether medicated or not. In the former, however, over-avoidance may rely on dopamine-independent mechanisms.

4.6 Temporal-difference modelling of avoidance: Conclusions

This study explored advantage-learning temporal-difference models of avoidance, inspired from two-factor theory. The models successfully replicated many phenomena seen in the conditioned-avoidance and related paradigms. These phenomena include the rapid acquisition of the avoidance response, its persistence in the absence of reinfocement and the facilitation of its extinction by response-prevention. Of great relevance to the understanding of threat-anticipation in psychosis are the ﬁndings concerning the effects of dopamine antagonists. Here the models successfully replicate the suppression of acquisition of the avoidance response, the ongoing suppression of avoidance in the presence of the drug, but also the acquisition of avoidance in the absence of shocks once the drugs are withdrawn (rather like the ’escape from fear’ paradigm). On the othe hand a hyperdopaminergic state appears to exaggerate the acquisition of the avoidance response, and thus aid its persistence, but no evidence was found for its involvement in true ﬁxity of responding. In the models dopamine reported positive rewards associated with successful avoidance. It appeared involved in learning about actions following both better-than-expected and worse-than-expected actions. It also appears involved in reporting positive prediction errors about states of the world. However a dopamine-independent mechanism, possibly serotonergic, appears to predominate in reporting negative prediction errors about these states. The present study thus forms a good basis to study mechanisms of threat anticipation in paranoia. It offers a number of novel predictions in the biological domain, that may be explored in imaging, animal and psychopharmacology research. In the psychological domain the modelling results largely support and augment the analysis presented in chapter2, although their application to the domain of private experiences such as aversive thoughts remains to be explored. We will investigate two such instances in which avoidance mechanisms may be overactive in paranoia in chapters5 and6.

94 CHAPTER FIVE

DOES AVOIDANCE OF PERCEIVED NEGATIVE CONSEQUENCES CONTRIBUTE TO JUMPING-TO-CONCLUSIONS IN PARANOIA?

This chapter is an adapted version of the published paper: Moutoussis, M., Bentall, R. P., El-Deredy, W., & Dayan, P. (2011). Bayesian modelling of Jumping-to-Conclusions Bias in delusional patients. Cognitive Neuropsychiatry, in press.

5.1 Summary

Introduction: When asked to make decisions about the cause of serially-presented events, patients with delusions utilize fewer events than healthy controls, i.e. they show a ’Jumping-to-Conclusions’ bias. This has been widely hypothesized to be due to patients expecting to incur higher costs if they sample more information (a type of motivational bias), but this hypothesis is unconfirmed. Aims: We aimed to test whether (1) higher perceived costs in paranoid patients account for JTC bias (2) Paranoid patients deviate more from ideal Bayesian reasoning, compared with healthy and clinical controls. Methods: We analysed patient and control data using two parametrised models that provide explicitly quantifiable variables characterizing decision-making. One model was based on calculating the potential costs of making a decision; the other compared a measure of certainty to a fixed threshold. Results: (1) Differences between patients and controls were found, but not in the way that was previously hypothesized: Differences in the ‘noise’ of decision-making accounted for group differences more robustly than differences in perceived costs. (2) Paranoid participants deviated from the ideal Bayesian estimation more than healthy controls but not more than remitted, previously-deluded patients. Conclusions: The Jumping-to-conclusions bias is unlikely to be due to an overestima- tion of the cost of gathering more information. The analytic approach we used, involving a Bayesian model to estimate the parameters characterizing different participant popu-

95 CHAPTER 5. AVOIDANCE AND JUMPING-TO-CONCLUSIONS IN PARANOIA lations, is well suited to testing hypotheses regarding ‘hidden’ variables underpinning observed behaviours.

Keywords: Jumping-to-Conclusions; Paranoia; Psychosis; Bayesian reasoning; Sequen- tial probability ratio test.

5.2 Jumping-to-Conclusions: an introduction

5.2.1 Probabilistic reasoning, Delusions and Bayesian inference

As we saw in the introduction of the concept of a delusion on page 29, delusions have been considered to be beliefs that appear fixed but are unwarranted on the basis of the available evidence (American Psychiatric Association Task Force on DSM-IV, 2000). The coherence, validity, applicability and even the exact meaning of this rather elegant definition has been much debated, giving rise to much fruitful work. In a seminal paper Hemsley and Garety (1986) proposed that the abnormality or otherwise of how beliefs are kept and maintained should be judged against a normative framework. That is, we can define a yardstick by which people should change their belief in a proposition if they reasoned in the best possible way. Building on the work of Fischhoff and Beyth-Marom (1983) they suggested that the Bayesian understanding of probabilistic reasoning provides just the needed yardstick for this purpose. This represents a powerful assumption, i.e. that the strength of belief in a proposition by a real live human being is quite comparable to the mathematical interpretation of probability as strength-of-belief which is subject to Bayes’ rule. Let us rewrite Bayes’ rule in terms of (human) beliefs to stress the point:

Belief in a hypothesis H having observed an event E = Belief in H before E was observed × Belief that E might happen if H were true (5.2.1) Belief that E might happen whether H were true or not

Thus reasoning under uncertainty can be quantiﬁed. It was therefore hypothesized that bayesian probabilistic reasoning is defective in deluded patients (Hemsley & Garety, 1986) and the ‘beads in a jar’ test was adopted to assess such reasoning (Huq et al., 1988). This task, described in more detail below, was used by Volans (1976) to examine the basic machinery of probabilistic reasoning in obsessive-compulsive and phobic people while abstracting away the speciﬁc content of their morbid ideas. Similarly, the task allows testing the hypothesis that the basic machinery implementing bayesian-like reasoning may be impaired in deluded people, independent of the personal (including threat-related) content of delusions. In the most common version of the task, which we will refer to for convenience as the ‘beads task’, participants are told that a sequence of coloured beads, say

96 5.2. JUMPING-TO-CONCLUSIONS: AN INTRODUCTION blue (b) and green (g), will be drawn from one of two jars. One jar, B, has a majority of blue beads; the other jar, G, has the same majority of green beads. Study participants are presented with beads one by one. They are asked to think whether B or G is the underlying cause of the bead sequence and asked to declare which it is when they themselves are sure. The main outcome is the number of draws that participants take to decide, nDTD.

Some studies also examine the proportion of participants that decide for nDTD ≤ nJTC where nJTC has usually been arbitrarily chosen to be 1 or 2. This has been reported as the proportion of participants who ‘jump to conclusions’. Recently, however, an important study has attempted to use a normative rationale to determine nJTC (Lincoln, Ziegler, Mehl, & Rief, 2010). A robust finding of such studies is that participants with paranoid beliefs or other kinds of delusions take a smaller number of draws to decide than controls (Garety, Hemsley, & Wessely, 1991; Fear & Healy, 1997; Garety et al., 2005; Corcoran et al., 2008). This test result is usually interpreted as a ‘Jumping to Conclusions (JTC) bias’, thus claiming face validity in reflecting an aspect of delusion formation. Further studies, however, showed that paranoid reasoning in the ‘beads task’ shows modest abnormalities compared to what one might expect if delusions were largely due to jumping to conclusions (Fine, Gardner, Craigie, & Gold, 2007). In one variant of the task, for example, participants are asked to give serial estimates of the chance that the beads are drawn from one specific jar, rather than to decide when to stop drawing. In this version of the task paranoid subjects appear to shift their certainty estimates more than controls when presented with beads favouring the jar opposite to their currently preferred one (Fear & Healy, 1997; Young & Bentall, 1997). This would be surprising if developing a preference for a jar involved jumping to a delusion-like conclusion, as delusions are by definition resistant to contrary evidence (American Psychiatric Association Task Force on DSM-IV, 2000). Studies like these challenged the validity of the standard interpretation of precipitous decisions in the ‘beads task’ in terms of the cognitive processes involved in paranoid inference.

5.2.2 Explanations proposed for the Jumping-to-Conclusions bias

A number of theories have been advanced to account for the jumping-to-conclusions bias and to explain its occurrence in delusions.

Erroneously increased certainty

First, the possibility has been explored that people with delusions make erroneous estimates of probability. In the light of the deﬁnition of delusions, it might be expected that deluded participants might be over-certain of their preferred cause for an event. This has been explored experimentally by asking participants to report their estimated probability

97 CHAPTER 5. AVOIDANCE AND JUMPING-TO-CONCLUSIONS IN PARANOIA or subjective certainty about the cause underlying a certain sequence of beads, without asking them to declare a preference and hence terminate the sequence. Fear and Healy (1997) compared the exact Bayesian likelihoods with estimates that patients and healthy controls made in this serial-estimates version of the task. In this and several other experiments the same participants also performed the sequence-terminating task (Young & Bentall, 1997; Dudley, John, Young, & Over, 1997a). These studies have found that people with delusions do not show signiﬁcant abnormalities in their certainty or probability estimates in the beads task. In their extensive qualitative review and meta-analysis of the ﬁeld Fine et al. (2007) concluded that hasty decisions in deluded patients do not stem from a bias in reasoning about probabilities but from a ‘lower threshold for the amount of information required’. Here we note that the widespread, and often implicit, assumption that decisions are arrived at by comparing certainty to a threshold.

Difﬁculty in processing sequential information

It has then been argued that if basic probabilistic reasoning is intact, the JTC bias can be best be seen as a data-gathering bias (Garety & Freeman, 1999). The second possibility that has been put forward is that of difficulty in processing sequential information. The literature offers different variants of this explanation. On the one hand, it could be that this is simply a nonspecific cognitive deficit, such as due to a generally reduced in- telligence found in paranoid groups; or due to a cognitive difficulty best characterised as a negative symptom of psychosis (Lincoln et al., 2010). On the other hand, difficulty in processing sequential information could be related to a motivational effect. Paranoid participants actively avoid facing a long sequence of beads that they can’t analyse, as Fine et al. (2007) interpret the difficulty in processing sequential information, or they may have working memory difficulties leading them to take inadequate account of previous draws as compared to the latest ones (Menon, Pomarol-Clotet, McKenna, & McCarthy, 2006). The later claim stems from a study where a memory aid abolished rendered the difference in nDTD between paranoid and control groups nonsignificant. This abolition effect is not, however, well replicated (Dudley et al., 1997a).

Increased motivation to draw conclusions

The third possibility that has been put forward is that the JTC bias arises from an increased motivation to arrive at a conclusion. Such increased motivation has also been put forward as a mechanism contributing to the unwarranted derivation of beliefs in paranoid thinking, which, if conﬁrmed, would make it of great clinical relevance (Bentall, 2003). The distinguishing feature of all these theories is, however, that evaluation of the different outcomes, broadly speaking consideration of their potential costs and beneﬁts, makes people with delusions jump to conclusions. Several different (and often overlapping) formulations of such motivational factors have been put forward:

98 5.2. JUMPING-TO-CONCLUSIONS: AN INTRODUCTION

i. Hypersalience of each piece of information: Menon, Mizrahi, and Kapur (2008) suggested that if each piece of information is taken to be as more salient than it actually is, an otherwise normal ‘decision threshold’ would be crossed with less information, giving rise to JTC. They thus related the JTC phenomenon to the aberrant salience theory of psychosis. The contrasted this explanation to another possibility, the liberal acceptance hypothesis of Moritz et al. (2006) where the ‘decision threshold’ itself is thought to be lower1.

ii. Need for Closure: Uncertainty in the outcome of a choice or decision, also referred to as economic risk, can paradoxically be treated by people as a cost separate to the average return of the choice in question (Rushworth & Behrens, 2008). Kruglanski, Webster, and Klem (1993) sought to assess the propensity of individuals to avoid am- biguity and developed the Need for Closure scale (NCS) to measure this. Bentall and Swarbrick (2003) hypothesized that the JTC bias, as well as delusions themselves, may be explained by a high Need for Closure. According to this theory, not-deciding at each particular step of the beads task is associated with a hidden cost. These authors found evidence that both acutely ill and remitted paranoid patients scored high in the NCS. In their sample this was not explained by anxiety and depression, but this was not replicated by Freeman et al. (2006). The latter study found that high NCS scores did not explain JTC.

iii. Nonspecific anxiety: Many studies have investigated affective variables when investigating the relationship between JTC and delusions. Bentall et al. (2009), for example, found that affective variables like anxiety and cognitive variables like JTC independently contributed to paranoid delusions. However most of these studies did not specifically look for the interaction between the two, i.e. at the effect that anxiety specifically applicable to the context of the decision task itself could have on JTC and on delusional thinking. In an elegant experimental study, Lincoln, Peter, Schafer, and Moritz (2010) induced anxiety in healthy people with varying levels of vulnerability to psychotic symptoms and measured both paranoid ideation and JTC. They found that anxiety induction resulted in higher paranoia scores only in people with higher baseline vulnerability and that a part of this effect was mediated by the tendency to JTC. The authors theorized that stress regulation demands cognitive resources based on common or interacting systems with those required for cognitive control. Para- noid ideation might thus be due to limited information-processing capacity induced by the anxiety condition. This explanation appears to downplay the physiological

1These two explanatory hypotheses, hypersalience-of-evidence vs. liberal-acceptance, appear to be rather different, their authors relating the ﬁrst, but not the second, to a motivationally important construct (hypersalience). However distinguishing between the two assumes a third, reliable, independent measure by which the salience of each piece of information and the threshold can be independently gauged. This is a non-trivial assumption, not least because salience and decision threshold are likely to be highly individual. It is safer to consider these two explanatory hypotheses together in the ﬁrst instance, in that they both put forward a decreased decision threshold relative to the salience of each piece of information

99 CHAPTER 5. AVOIDANCE AND JUMPING-TO-CONCLUSIONS IN PARANOIA

role of anxiety to ‘tag’ a context characterized by a higher probability of aversive outcomes (including harm by others), i.e. that there is a normative function to the bias in question.

iv. Confirmatory reasoning style: Dudley and Over (2003) provided a contribution to the understanding of motivational factors that may be important both for JTC and for delusions. They observed, first, that danger scenarios invoke a confirmatory reasoning style in healthy people. If one were faced with the possibility ‘if there is smoke, there is fire’, one might in theory look for instances where a fire-less cause led to the production of smoke, as this would disprove the proposition. However, these authors write, “believing this claim might protect us from threat ... it would therefore be sensible to focus on confirming instances of this conditional [and be] less worried about the few cases in which we [end up falsely believing] that smoke is associated with fire”. They argued, furthermore, that the possible presence of threat would lead to increased desire for a definite answer (Bentall & Swarbrick, 2003) and that the confirmatory reasoning style would be a form of seeking closure. The argument is subtle but the authors should be credited for introducing the idea that threat-related reasoning has special properties and that paranoid people may misapply essentially normal threat-related (high-cost) reasoning to low-threat situations.

v. Vulnerability of Self: The idea that rational thought can be distorted by affective considerations also has a long tradition amongst psychological theorists who think that psychological defensiveness (preserving a more positive image of the self and those important to the self) is important in paranoia. From the psychodynamic point of view, as we saw on page 30, people prone to paranoia are strongly believed to ‘project’ danger into many types of stressful situation in order not to face their own underlying unacceptable feelings. One might therefore expect a demanding situation involving uncertainty, such as being tested in the beads task, to involve such projections. The person in question might be expected to try to avoid the task and its context as if its continuation was in itself somewhat ominous (or, in the terms of the present discussion, potentially costly). In terms of preserving self-esteem, which has been hypothesized to be vulnerable in paranoia (Bentall, 2003), one might construe the task instructions as implying ‘make the correct choice as soon as possible’. This could, for example, be based on the social stereotype that clever people are ‘quick’ to make good decisions, whereas slowness to decide carries many negative conno- tations (stupid, prevaricating, indecisive etc.). Talking many draws to decide might be injurious to one’s self-esteem, especially if one had little faith that postponing the decision would enable one to make a better choice. Related considerations led to the idea that paranoid patients would show even more JTC in versions of the task that would force them to think about positive and negative characteristics of people. Young and Bentall (1997) developed a task with good and bad personality characteristics pertaining to a person, as opposed to different colour beads pertaining to a

100 5.2. JUMPING-TO-CONCLUSIONS: AN INTRODUCTION

jar, arguing that “paranoid patients are preoccupied by the intentions of others, and therefore any difference between their performance for meaningful [personality] and non-meaningful [coloured beads] materials is especially likely to be evident with a task of this sort”. Several groups took up variants of this “social characteristics” task but the hypothesis that paranoid patients would be particularly sensitive to it was not clearly conﬁrmed (Fine et al., 2007), a matter to which we will return.

In recent years great strides have been made to address quantitatively many of the ideas that have been put forward to explain the JTC and its relation to delusions. The beads task can be examined from an optimal decision making approach (Green & Swets, 2008). We can ask, what is the problem that the human mind is posing itself when faced with the task? This may be the one that the experimenter has in mind, or strategies from a relevant - but different - problem may be ‘imported’ by the participants to the lab (Oaks- ford & Chater, 2001). Is there an optimal solution to the problem people are trying to solve? Considerations such as whether people seek to optimize avoidance of danger or threats to self-esteem, and how they weigh up the likelihood of the relevant events, can be quantified. If people seek to optimize specific outcomes, does their behaviour (not necessarily self-reports) approximate the optimal solution e.g. Oaksford and Chater, 1995? In what ways do people’s behaviour deviate from this optimal solution? and finally, how can such deviations be described in terms of psychological (and, ideally, also social and biological) processes? We sought to take advantage of advances in the theory of decision-making to develop a more refined view of the processes and mechanisms associated with the beads task. Our intent was to realize the constructs underlying decision-making, especially cost-benefit considerations, in a rigorous and testable Bayesian framework. This could potentially reveal even subtle biases and inferential flaws in performance on the task.

5.2.3 The ideal-observer Bayesian approach

Models of optimal decision-making explicitly parametrise all the factors, including prior expectations, costs and noise, which should control choice (Green & Swets, 2008; Kording, 2007; Dayan & Daw, 2008). They can be used to ask the key questions of what problem the brain is trying to solve, what is its optimal solution and how people’s reasoning may deviate from optimality. Decision-making in ideal observer models is based on two considerations: the posterior probabilities of various scenarios, given what has been observed, and the values of different decisions (action values) for each scenario. In tasks such as ‘beads-in-a-jar’, the likelihoods can be estimated exactly using Bayes theorem (see eq. 3.4.1- 3.4.2). As we saw on page 97, no evidence has been found to support the hypothesis that delusion-formation is based on a deﬁcit of estimating such probabilities. Analysing the factors contributing to action values may therefore be quite important.

101 CHAPTER 5. AVOIDANCE AND JUMPING-TO-CONCLUSIONS IN PARANOIA

In the version of task where paranoid patients do differ from controls, actual choices or commitments must be made, rather than estimates of probabilities. After each sample, three choices are available: DB and DG , which decide on jars B and G respectively, and DS i.e. ‘sample again’. In the decision-theory literature, these problems are called optimal stopping problems (Puterman, 1994; Bertsekas, 1995). The task instructions do not completely specify the costs or benefits of different actions. Participants are only told to sample beads until they are sure which jar they come from, up to a maximum nmax = 20. These instructions may suggest that the subjects should weigh heavily the cost of deciding wrongly (CW ). The cost that normally afflicts decision-making problems under uncertainty, namely the cost of sampling a further bead (CS), is not mentioned. However, CS may nevertheless be important. One factor that has often been put forward to explain delusional thinking is inflated personal cost (or value) associated with the collection of information under uncertainty. Different instances of this ‘high-cost hypothesis’ in the psychology literature include the ‘need for closure’ reflecting a high subjective cost of uncertainty (Bentall & Swarbrick, 2003), the cost to self-esteem (Bentall, 2003) or the cost of cognitive dissonance experienced when intensely salient experiences strain the patient’s explanatory theories about the world (Kapur, 2003; Freeman & Garety, 2004b). A common theme is that delusional patients may experience sampling costs to be greater than healthy people, and adopt cognitive strategies to minimize them. Our primary hypothesis was that paranoid participants tend to assume higher costs of gathering more data in serial probabilistic inferencing tasks, which explains their early decisions (the high-sampling-cost hypothesis). Our secondary hypothesis rendered more rigorous and testable the long-standing psychological hypothesis that paranoid participants may show deficiencies in Bayesian reasoning (Hemsley & Garety, 1986). We hypothesized that paranoid probabilistic reasoning may deviate away from the Bayesian ideal (towards a simpler model) more than that of healthy subjects. We thus built two models for the task. The first was an implementation of ideal observer Bayesian analysis, including Costs (the CB model). It is detailed in page 57; here we just note that its calculation of the cost of gathering more information (sampling again) involves a broad and deep consideration of all future outcomes (eq. 3.4.6- 3.4.8). We expect such calculations to be highly challenging for patients, and indeed controls, so that in vivo approximations may be used. Our simpler model was based on the Sequential Probability Ratio Test (SPRT, see below). This treats costs in a less direct way, and is much simpler in practice. The SPRT is also an operationalized, quantitative version of the model of “comparing certainty to a threshold” which is often referred to, or implied, in the psychological literature. It is thus an excellent way to approach the hypersalience and liberal-acceptance explanations of the JTC bias (Menon et al., 2008; Moritz et al., 2006). We estimated all model parameters by using the Expectation-Minimization (EM) algorithm (Dempster et al., 1977). One critical deviation from ideal that we can expect from both controls and patients is behavioural noise e.g. Pleskac, Dougherty, Rivadeneira, and Wallsten, 2009. Consider

102 5.2. JUMPING-TO-CONCLUSIONS: AN INTRODUCTION data from 99 trials of 33 healthy people performing the beads task that we will analyse below (Corcoran et al., 2008). For seven trials (from five participants), a decision was taken after the second sample, which was always discordant with the first. Thus at that stage participants had no information at all as to the underlying jar, and yet they were so far from the maximum possible number of sampled beads (20, in this case), that this limit would be unlikely to be exerting an effect. Some type of process error or extraneous influence that we can subsume in the concept of noise must be having an effect. A standard manoeuver to encompass such choices is to introduce behavioural noise by having subjects choose randomly between the three possible actions with probabilities depending on their relative action-values. We assume that the impact of the ‘noise’ is controlled by a temperature-like parameter, τ, via the sort of softmax or Luce choice rule employed by a bulk of other models of human and animal decision-making e.g. O’Doherty et al., 2004. The CB model would then have three parameters, τ, CS and CW . However if τ, CS and CW are all scaled by the same factor, the same probabilities will ensue. In other words, we need to choose the ‘currency’ by which to measure these three parameters arbitrarily. We thus set the cost CW = 100 for all participants; CW can thus be seen as the ‘unit of internal cost’ for each person, relative to which all other quantities are measured.

5.2.4 Comparison with the Sequential Probability Ratio Test

Optimal stopping problems frequently arise in psychological studies of decision-making (Laming, 1968; Link, 1992; Usher & McClelland, 2001; Gold & Shadlen, 2001; Ratcliff & Smith, 2004; Smith & Ratcliff, 2004). Much of this work has been organized around the

SPRT. In the SPRT one maintains the log-likelihood ratio, l(nd, ng), that the beads come from jar G rather than B, if ng green beads have been drawn out of nd samples. Decisions are taken when l(nd, ng) crosses one of two thresholds, θG or θB, as per equation 3.4.10.

We used thresholds equidistant from l(nd, ng) = 0, the point of maximum uncertainty:

θG = −θB = θ. The thresholds encode the level of certainty (excess beads of one colour) that a participant may demand to decide. In keeping with the observations above, we also used an additive noise term (parameters: mean 0, variance τ 2) to account for behavioural noise (see ﬁg. 5.1 & eq. 3.4.9- 3.4.12). Given appropriate thresholds, the (noise-free) SPRT would produce exactly the same choices as the (noise-free) CB model under the assumption that there is no limit on the number of possible samples (Wald, 1945). This is a remarkable result, since the SPRT is computationally a vastly more straightforward implementation of an optimal policy than is known for almost any other case for CB, not requiring the tree of possible future states to be explicitly enumerated. However, it is not quite statistically optimal for the beads task, since nmax is not inﬁnite. Noise may also perturb it in a different way. We used the SPRT to test whether some subjects use a decision-making strategy considerably simpler

103 CHAPTER 5. AVOIDANCE AND JUMPING-TO-CONCLUSIONS IN PARANOIA

Figure 5.1: a. SPRT simulation, draw 7 of sequence 1011110... (sequence 3). In this case, by draw 6 most participants have declared; only the lower tail of a gaussian-like distribution remains, truncated at the high threshold θG = +1.25 and renormalised before considering draw 7. This draw is discordant with the ones before (B), so that the increment in l(t) is distributed as a gaussian with a negative mean. The sum of the two random variables gives a left-skewed probability density function (pdf) relatively removed from both thresholds. b. Probability of deciding at draw 4 as a function of Threshold and Uncertainty / Error in the estimation of the increment of l(t). The distribution peaks at near-zero error. There it still has a ﬁnite width with respect to Threshold values as the initial condition l(0) has been set to contain a small error.

104 5.3. METHODS than the optimal.

5.3 Methods

In order to test our hypotheses we applied the CB and SPRT models to reanalyse experimental results previously obtained by Corcoran et al. (2008). Participants. The study of Corcoran et al. (2008) included three diagnostic groups of participants aged over 65, and five groups under 65. Our hypotheses pertained to the group with persecutory delusions who were under 65 (original N = 39; we excluded 3 participants with very limited beads-task data, giving N = 36). We therefore analysed their data together with those of the under-65 healthy controls (N = 33). Perse- cutory delusions were judged to be present on the basis of endorsement of the question ‘Does anyone seem to be trying to harm you (trying to poison you or kill you?)’ (World Health Organization, 1997), examination of case notes, and the answer to the question ‘Do you ever feel as if you are being persecuted in some way?’ from the Peters et al Delusions Inventory (Peters, Joseph, & Garety, 1999). We also performed limited analyses on depressed-without-delusions (N = 26), depressed-with-delusions (N = 20) and remitted-paranoid (N = 29) under-65 groups. The reader is referred to the original study for further details of the participants. Stimuli Participants were tested using the original version of the beads task (Garety et al., 1991), and a formally equivalent version of the task developed by Dudley, John, Young, and Over (1997b) which uses valenced words rather than coloured beads. In the latter, participants had to choose between two surveys, each containing good and bad comments about an individual. They were told that each jar (or survey) contained 60% of the dominant colour (comment type). Each version used three particular sequences, 1: 01000010001011110111; 2: 01000100101000011001; and 3: 10111101110100001000 (using ‘0’ and ‘1’ to stand in for a particular colour or valence). Each participant thus provided six values of the number of beads or social words viewed before deciding, also called ‘draws-to-decision’ (2 task versions ×3 sequences). Analysis. We considered a pair of statistical models: one generative, and its statistical inverse, the recognition model. The generative model parametrises the process by which the experimental data are considered to have been generated. The recognition model takes the actual data from the participants and infers the parameters of the generative model that are likely to be responsible. We used the EM algorithm to fit model parameters to the experimental data. It is the ‘E’ phase of the EM algorithm that involves the recognition model (Dempster et al., 1977). In detail, for the generative model, we assumed that each experimental group was described by its own parametrised statistical prior distribution. We called these group- S level descriptive statistics ‘macroparameters’. These are the mean Cµ , standard deviation S S Cσ etc. The specific ‘microparameters’ Ci (or, for the SPRT, θi) and τi characterizing

105 CHAPTER 5. AVOIDANCE AND JUMPING-TO-CONCLUSIONS IN PARANOIA participant i are considered to be sampled from the distributions associated with the group of that participant. These microparameters act through the CB or SPRT model of the task to determine the distribution over possible experimental choices of participant i. As costs are unlikely to assume positive values (i.e. mistakes or slowness will not be positively rewarding) and uncertainty cannot be negative, we assumed that the ‘microparameters’ are sampled from independent gamma distributions (in the case of costs with the sign ‘flipped’ to negative values). We used the EM algorithm to find the values of the macroparameters that maximize the log-likelihood that the experimental data for each group could have been created by each model (CB or SPRT). We assessed how well the models accounted for the data in several ways. First we used a parametric bootstrap re-sampling technique (Efron & Tibshirani, 1993). Here we used the best-fit parameter values to simulate the experiment of Corcoran et al. (2008) many times. The bootstrap tested if different models of decision-making produced outcomes resembling the real data. The main outcome we examined was the difference between the mean draws-to-decision in the two participant groups. Secondly, in order to examine the null hypothesis (H0) that the different groups could be described equally well by a single set of parameters, we merged the group data and fitted parameters to the combined set. We then compared the ‘merged’ versus ‘separate’ models using the Bayesian Informa- tion Criterion (BIC; Schwarz, 1978; Raftery, 1995). This is a measure based on model likelihood that takes into account the number of parameters a model uses (see page 71). Thirdly, we derived distributions of key parameter models through parametric bootstrap re-sampling. We thus estimated ‘bootstrap’ confidence intervals (bCI) for the parameters fitted to each group. All programs used in the analyses are freely available from the authors on request & under General Public Licence (GNU GPL)

5.4 Results

In general, both the SPRT and CB models produced good fits to the data. That is, when the fitted parameters were used to generate artificial data sets and these were re- analysed with the methods that we used for the original data, the sum-of-log-likelihoods for the actual data fell well within the distribution of values of the artificial data (example in figure 3.3). Therefore the observed data could be a typical output of our generative model. Most critically, the results were contrary to our high-sampling-cost hypothesis. In the beads task analysed with the CB model, the mean and variance of the cost of sampling converged to near-zero values for both healthy and paranoid groups (table 5.1), consistent with the experimental instructions for the task. This finding challenged somewhat our use of a gamma probability distribution to fit the population distribution of the sampling-cost parameter, as the range of this distribution does not include zero itself. More importantly, it implied that we could simplify our CB models by removing the sampling cost variable

106 5.4. RESULTS

Table 5.1: Best-ﬁt parameters and BIC values for JTC tasks

Beads task version BIC S S CB (k = 4) τµ τσ Cµ Cσ Healthy 5.09 3.63 -0.06 0.05 468.4 Remitted 5.75 4.06 -0.01 0.01 464.0 Paranoid 12.07 7.18 -0.05 0.03 423.9 CB (k = 2) τµ τσ Healthy 5.29 3.82 - - 459.3 Remitted 5.74 4.06 - - 457.3 Paranoid 12.09 7.09 - - 416.0 SPRT (k = 4) τµ τσ θµ θσ Healthy 0.76 0.60 1.34 0.18 492.6 Remitted 0.74 0.64 1.49 0.76 401.1 Paranoid 1.53 0.90 1.20 0.38 393.0 Words task version S S CB (k = 4) τµ τσ Cµ Cσ Healthy 5.91 3.83 -0.04 0.03 461.2 Remitted 8.05 7.45 -0.11 0.16 418.1 Paranoid 9.83 5.36 -0.04 0.03 423.8 CB (k = 2) τµ τσ Healthy 5.92 3.83 - - 454.8 Remitted 8.25 7.51 - - 423.2 Paranoid 9.83 5.36 - - 416.6 SPRT (k = 4) τµ τσ θµ θσ Healthy 0.81 0.68 1.35 0.44 459.8 Remitted 0.87 0.53 1.41 0.81 375.2 Paranoid 1.39 0.62 1.4 0.73 410.9

107 CHAPTER 5. AVOIDANCE AND JUMPING-TO-CONCLUSIONS IN PARANOIA

(or by always setting CS = 0), with a negligible reduction in model fit. The BIC value for this simplified model improved, as the model has two fewer parameters. Note that as the SPRT has no strictly equivalent or separate cost parameter, removing or fusing parameters to obtain a simpler model would be arbitrary. The second important finding was that the paranoid group had a higher cognitive noise

(parameter τµ) than both the healthy and clinical controls and for both versions of the task (ﬁg. 5.2). This ﬁnding was statistically robust (see below). In the SPRT model the noise parameter was again larger for the paranoid than the healthy group (table 5.1).

The effect of the higher noise parameter τµ on decisions in the CB model is illustrated in fig. 5.3. This shows how action values for the three possible actions change through the experiment. Initially the ‘Sample again’ action is quite advantageous for the control group, but less so for the paranoid group. Two samples later only the average control participant (but not the paranoid one) still perceives an advantage in sampling again. We used three different approaches to judge the statistical significance of the increased cognitive noise for paranoid group. First, we assumed that the all five groups were sampled from populations with similar noise structures (a null hypothesis, H0). Hence we merged the data-sets for all groups and fitted a merged-group set of CB parameters. We compared the ability of these merged-group-parameters to describe the data, as compared to the separate-group parameter fit shown in table 5.1( H1). We therefore simulated the experiments using merged vs. separate parameters, thus creating large sets of simulated data. We compared three key descriptive statistics for the beads task under H0 vs. H1: the grand mean difference in draws-to-decision between groups, averaged over the three experimental sequences used; and the within-sequence variances in draws-to-decisions in each group. We found that the merged-data model was quite unlikely (p < 0.02 for words, p < 0.001 for beads, two-tailed) to give rise to the experimentally observed difference in draws-to-decision. The latter was near the modes of the simulated distribution under H1. Therefore H0 is rejected in favour of H1. Similarly, if the remitted and paranoid groups are merged and best-fit parameters derived, 0 hypothesis H0 that the difference in draws-to-decision between them arose by chance is again rejected for the beads task (p ≈ 0.0002) and the words task (p ≈ 0.02). We repeated this last analysis using the SPRT rather than the CB as generative model, and got essentially identical results. The beads-task results remain significant for the beads task under Bonferroni correction for multiple comparisons (p < 0.001), but the words- task results are reduced to (just) trend significance (both p ≈ 0.06). If the same analysis is repeated for the difference between control and remitted group, the null hypothesis that they are sampled from the same distribution cannot be rejected. The experimentally observed variance in draws-to-decision of the control group was accounted for equally well under H0 or H1. The variance of the paranoid group was also consistent between the H1 model and experiment, if a single outlier result was excluded. The decisions of this participant (2,20 and 20 draws-to-decision) are indeed extremely unlikely to be

108 5.4. RESULTS

Figure 5.2: Group comparisons: a. Best-ﬁt noise parameters for the Bayesian model. The analogous noise parameter plot for the SPRT is almost identical (not shown). The paranoid group has larger mean noise, especially for the beads task. b. Model ﬁt according to the BIC. Positive values favour the Bayesian model, negative the SPRT. A difference of 10 is conventionally considered ‘very strong’ evidence in favour of a model (Raftery, 1995). In the beads task, never-psychotic groups are closer to the Bayesian norm whereas schizophrenia-spectrum-disorder groups are closer to the SPRT. The words task brings out differences less clearly, probably due to the complex social thinking it invites.

109 CHAPTER 5. AVOIDANCE AND JUMPING-TO-CONCLUSIONS IN PARANOIA

Figure 5.3: Action values and resulting probabilities-to-declare corresponding to the best- ﬁt mean noise for the control and paranoid groups (the effect of only the mean for each group is illustrated for reasons of clarity). The sequence presented is bgbbbbgb... a. Action Values curves for ‘Sample again’ and ‘Decide on Blue’ for the two groups. The ‘Decide on Blue’ curve is the same for both groups. b. Resulting probabilities of deciding at speciﬁc stages. The peak is at 1 for the paranoid group but much later, at 4, for the controls.

110 5.5. DISCUSSION produced by a stable Bayesian or SPRT model. Exclusion of this individual resulted in similar parameter estimates for the paranoid group but improved the model fit as might be expected. The other analyses reported here were not materially affected by excluding this outlier. Secondly, we used the BIC to compare a CB model of the paranoid and control groups fitted separately, with one where these two groups were merged. The BIC penalizes extra parameters substantially, but it still slightly favoured fitting separate parameters to each S S group (BIC = 898 vs. 902 when τµ, τσ , Cµ & Cσ are fitted). Thirdly, we applied EM to obtain bootstrap-confidence-intervals (bCI) for the parameters. We found that the noise parameter τµ estimate for the healthy-beads group as well as the corresponding estimate for the remitted group fell outside the 0.01 bCI for the control-beads group (correcting for multiple comparisons). The 0.01 bCI for τµ for the beads-version of the task for each group, however, each included the estimate for the words version for the paranoid group. Compared with the beads-version, the words-version looks as if it makes levels of cognitive noise more similar across groups, rendering differences non-significant by the measure of bCI. Finally, we sought to compare which model, CB vs. SPRT, best fitted the data of the normal, paranoid and remitted groups. Our hypothesis was that paranoid participants deviated from the Bayesian model more than healthy ones, and that this difference would be related to the paranoid state itself. For the healthy controls, the BIC favoured the τµ

, τσ CB model over the τµ, τσ , θµ , θσ SPRT model for both (but especially the beads) versions of the task. For the paranoid group the SPRT did better (beads) or somewhat better (words) than the CB model (table 5.1), in support of our hypothesis. However the remitted group was clearly nearer to the SPRT (ﬁg. 5.2), contrary to expectation.

5.5 Discussion

We used a Bayesian approach to analyse the ‘beads in a jar’ task, a popular tool used to assess probabilistic reasoning in patients with psychiatric disorders (Garety et al., 1991; Fear & Healy, 1997). It is important to elucidate the mechanisms that the task assesses, as the ‘jumping to conclusions’ bias seen in this task has been postulated to have aetiological importance in delusions (Freeman et al., 1998). Such is its perceived importance that speciﬁc therapeutic procedures have been designed to correct this bias (Moritz & Wood- ward, 2007). Our work is the ﬁrst to quantify subjective motivational factors (perceived cost) in this task, and to test the common assumption that cost considerations account for jumping-to-conclusions.

5.5.1 High-noise processing vs. the high-sampling-cost hypothesis

Contrary to the ‘high sampling cost hypothesis’, we found that increased noise in decision-making accounts robustly for the JTC bias. We measured noise relative to the

111 CHAPTER 5. AVOIDANCE AND JUMPING-TO-CONCLUSIONS IN PARANOIA subjective cost of making a wrong decision; an alternative interpretation of this result might be that paranoid patients utilise reduced effective costs of making wrong decisions. Distorted effective salience (Kapur, 2003) might conceivably make it difficult for paranoid patients to put cost estimates to good use. However we consider this interpretation unlikely as paranoid participants tend be highly avoidant (Freeman et al., 2001) and sensitive to failure experiences (Bentall & Kaney, 2005). In addition, patients with marked ‘negative’ symptoms were excluded from this study. While the possibility of ‘reduced effective motivation’ cannot be ruled out on the basis of this study, it can be expected that paranoid patients would be highly motivated to avoid failure experiences and thus would be unlikely to have a reduced cost of making the wrong decision. Our interpretation implies that this task should be compared with a control one where paranoid participants demon- strate equal motivation not to ‘get it wrong’ as controls, irrespective of ability. Most importantly, experimentally manipulated (e.g. monetary) cost-of-sampling and cost-of- wrong-decision should be examined. We predict that increasing CS relative to CW would not, as might be expected from the high sampling cost hypothesis, make healthy control data delusion-like, but inducing ‘noise’ in selecting one of the three actions would. Mod- elling could allow the influence of control tasks and externally manipulated costs to be used to infer the relative value of the ‘personal’ cost of error that we used as a comparator here. We found no evidence that paranoid participants perceive increased costs in this task when given socially salient stimuli. This is consistent with other work (Warman, Lysaker, Martin, Davis, & Haudenschield, 2007). It may still be that the anticipation of high personal costs specifically contributes to the fixity of the self-referent ideas that paranoid participants hold. Future research should therefore examine probabilistic reasoning relevant to specific delusional beliefs. It could test how such beliefs may (or may not) shift in the face of different types of personally salient evidence. Applying a Bayesian approach would allow estimation of (i) prior probabilities of harm (ii) accuracy of derivation of posterior probabilities (iii) ‘internal/social’ costs such as ‘if this belief is false, I must be mad’ and (iv) ‘external’ costs such as ‘if I get it wrong, my persecutors will get me’. A related direction for future research is the examination of asymmetric costs. In the case of paranoia, deciding that someone is trustworthy when they are not may incur a much greater immediate cost than the opposite error. Deciding that people are ill-disposed when they are not may be more costly in the long run.

5.5.2 Paranoid decision-making and noise

Decisions may be affected by noise in two key ways which our Bayesian modelling helps to clarify. First, noise directly reduces the impact of a given difference between the values of the actions on choice. ‘Noisy’ participants would declare more often even if faced with similar differences in action-values favouring sampling again (eq. 3.4.7). Secondly, early decisions in the beads task reﬂect smaller differences in action-values

112 5.5. DISCUSSION favouring different actions, as per fig. 5.3. This is because the calculation of the action value for sampling relies on values of future states being taken into account accurately (eq. 3.4.7 feeds into eq. 3.4.8; this feeds into eq 3.4.6 for the previous step). Note the assumption under the CB model that paranoid participants still perform optimal Bayesian reasoning given their view of future outcomes. Of course, calculating the values of actions based on a search through a forward model is challenging. Humans probably carry out such searches to solve simpler tasks such the “towers of London” (Marczewski, Van der Linden, & Laroi, 2001) while rats may engage in forward searching in the course of goal-directed decision making (Dickinson & Balleine, 2002; Daw et al., 2005). In both cases, there is a critical role for areas of prefrontal cortex, and specific regions of the striatum (Unterrainer & Owen, 2006; Balleine, Liljeholm, & Ostlund, 2009). The schizophrenia-spectrum diagnoses associated with our paranoid group are thought to involve a relative hypofrontality, with a predisposing and/or consequent limbic hyperdopaminergia (Langdon, McKay, & Coltheart, 2008; Laruelle, 2008) and such pathological processes may contribute to our findings. Affect could also contribute to the process substantially, if it involves a sense of greater proximity of threat. There is evidence that the latter shifts information processing away from frontal areas (Mobbs et al., 2007). Here the research implication of the present study is that the psychological mechanisms causing higher ‘noise’, including perception of threat, need to be elucidated. Fur- thermore, a beads-in-a-jar task could be used to separate deluded patients with respect to their level of cognitive ‘noise’. This would require more trials-per-participant so as to enable accurate determination of each participant’s individual noise level. Cognitive mechanisms underlying delusions in the presence of low noise may differ from those in the high-noise case. High noise in itself makes cognition inefficient, or even biased, as cognitively more distant alternatives cannot be taken into account well. Therefore factors eventually found to increase this noise may be a target of therapeutic interventions. The high-noise explanation of the JTC phenomenon also suggests that ‘high-noise’ paranoid subjects should be compared to non-paranoid participants with similar cognitive impair- ments in probabilistic reasoning.

5.5.3 Bayesian vs. threshold-driven decisions in paranoia

We found some support for the hypothesis that paranoid inference deviates from the Bayesian ideal more than that of healthy subjects. People with paranoia may employ more often non-Bayesian reasoning, where the estimated likelihood of the cause of an event is simply compared to a threshold. Overall, the costed-Bayesian model ﬁt the healthy subjects better, whereas the data from paranoid subjects (and remitted) were better explained by the simpler sequential-probability-ratio-test model which does not involve consideration of possible future outcomes. It could be that SPRT-type-reasoning is a lowest-common-denominator mechanism, on which people improve by using a more

113 CHAPTER 5. AVOIDANCE AND JUMPING-TO-CONCLUSIONS IN PARANOIA

Bayesian-like approximation. This would be consistent with evidence that paranoid subjects tend to revert more easily to simple heuristics (Glockner & Moritz, 2008). Bayesian reasoning, however, requires considerable cognitive resources. The structure of decision-making for the two models can be compared by using an ‘urgency plot’. This shows an effective threshold for the Bayesian model, allowing clear comparison with the fixed threshold of the SPRT (fig. 5.4). The Bayesian and SPRT methods of estimation diverge most markedly for the last five draws, when the Bayesian model decides with greater urgency (fewer excess beads of one colour). We therefore suggest that future studies seeking to differentiate the types of human reasoning may employ shorter sequences, of only about ten pieces of information, so as to bring the last few draws within the range actually chosen by participants.

5.5.4 Methodological advances

Assessment procedures should ideally measure accurately those factors which contribute substantially to pathological processes. Such assessment procedures would highlight in each individual patient causal factors that would make good targets for therapeutic intervention. Unfortunately assessment of paranoid ideation has not yet reached this stage. The specific model-based analysis that we have developed here is not as yet intended for clinical practice but for research into the cognitive biases and deficits contributing to paranoia. Our study sits comfortably within the current trend in studies of decision-making. These studies utilize models that quantitatively capture observable behaviour by postulating hidden psychological variables such as subjective beliefs and values that obey near- normative dynamics. Such variables are frequently the target of functional neuro-imaging studies (Doya, 2002; O’Doherty et al., 2004) and offer accounts of neural activity in animals e.g. Schultz et al., 1997; Morris, Nevet, Arkadir, Vaadia, and Bergman, 2006. These models are sufficiently precise to test and rule out important hypotheses such as the standard view of the motivational factors in JTC presented here. Similar approaches have been used in other psychiatric and neurological patient populations (Busemeyer & Stout, 2002; Frank, 2005; Batchelder & Riefer, 2007; Dayan & Huys, 2008; Kumar et al., 2008). We note how the effort to build a model-based theory forces a closer examination of the assumptions underlying various theories, and particularly of areas where these theories are underspecified. It is interesting that the vast majority of studies on the beads task fail to ask what should count as a premature conclusion. One important study that attempts to quantify and manipulate this experimentally (Lincoln et al., 2010) fails to take into account that in following the task instructions, people are motivated to make the right decision (here quantified by CW ). Again, although everybody knows that people make errors no study has attempted to include this in the understanding of JTC. Some studies have specifically excluded from analysis participants that made decisions against the currently most-probable cause, assuming that such participants surely did not understand the

114 5.5. DISCUSSION

Figure 5.4: Comparison of SPRT and Bayesian models using the mean parameters for the healthy group doing the beads task. The SPRT model gives a decision when the estimated log-likelihood ratio crosses one of the two constant, symmetric boundaries. The Bayesian model does not have such ﬁxed boundaries; an effective threshold, i.e. the log-likelihood ratio corresponding to a probability of deciding of 0.5, is plotted for comparison. This curve is not deﬁned for 4 or fewer draws, as the probability of sampling again is always greater than 0.5 for these states. The Bayesian model estimates backwards, starting from the last draw, while the SPRT does not take account of the approaching end and this allows the greater ‘urgency’ of the Bayesian model in the last few draws.

115 CHAPTER 5. AVOIDANCE AND JUMPING-TO-CONCLUSIONS IN PARANOIA

Figure 5.5: Best-fit gamma distributions (grey) and experimental Bayesian distributions (black) for the Beads data, for the control and paranoid groups. Bringing the data to bear does not alter the curve for the paranoid group much, but the control group appears to contain at least one subgroup characterised by a very small error parameter. task rules. Building normative models allows for examination of different types of null hypothesis pertaining to the underlying variables in silico. It also allows checking that the ‘best fit’ model is likely, in absolute terms, to have produced the experimental results. This process showed that our model clearly separated the experimental groups. One further useful property of this form of modelling is that it provides a signature that the groups under study may not be well described by the unimodal distributions over model microparameters that we assumed as a starting point. Once the best-fit parameters for a group have been found, the posterior parameter distributions for each of the participants can be calculated and accumulated to produce what we called the ‘experimental Bayesian’ distribution. This is sometimes called the marginal posterior density (Gelman, 2002)(eq. 3.5.9). The form of this distribution provides hints as to extra structure in the data that is missed in the current model. As an illustration, we found evidence (fig. 5.5) that there is a subgroup of healthy subjects with very low error rates (and, on inspection, late decisions) who may have applied different heuristics, or indeed interpreted the rather vague instructions in a different way. It would be interesting to test task variants that probe these characteristics.

5.5.5 Limitations of the present study

Explicit, sequential Bayesian calculations are a competence rather than a performance model (Marr, 1982), and we have only been able to speculate in rather coarse terms about the (prefrontal) processes involved. The same is true for the many Bayesian models in

116 5.6. MODELLING THE JTC BIAS: CONCLUSIONS modern computational cognitive science (Xu & Tenenbaum, 2007; Chater, Tenenbaum, & Yuille, 2006). There is a pressing need to study the approximations that biological systems may use to estimate posterior probabilities and related variables (Yu & Dayan, 2005). Like other ideal observer accounts, the costed Bayesian model thus serves first and foremost as a point of reference. It may be argued that the high-noise explanation is inconsistent with the finding that paranoid subjects underestimate their own uncertainties (Fine et al., 2007; Warman et al., 2007). Such a metacognitive deficit may, however, be quite consistent with schizophrenia being characterized by a poor perception of one’s own mental function (Fletcher & Frith, 2009). Our study did not allow for explicit comparison with some systematic deviations from the Bayesian norm, such as the primacy effect. Future studies could include specific, psychologically motivated models of human heuristics as comparators to our noisy-Bayesian model. Our ‘high-noise’ model predicts that delusional participants would make choices discordant with the most likely cause of sequences of information more often than controls. This is consistent with the literature e.g. Fear and Healy, 1997 but the Corcoran et al. (2008) data did not allow a relevant analysis. Methodologically, we note that the ‘bootstrap confidence intervals’ that we used are not true confidence intervals, i.e. intervals such that if the true value of the parameter falls within the interval then the parameter estimate actually obtained would not be too improbable. We also note that the gamma distribution may not be optimal for describing cost parameters, as it can be poorly behaved very near zero. It will be important to replicate the current analysis with other datasets and test our ‘high noise interpretation of JTC’ with new data. Analyses could also include other groups, such as older participants, participants suffering from OCD etc.

5.6 Modelling the JTC bias: Conclusions

We have introduced explicit models of how cost considerations and noise may skew probabilistic judgments, and we compared two types of model, the (optimal) costed- Bayesian and the sequential-probability-ratio test models. We applied an expectation- minimization algorithm and analysis of synthetic data to estimate best-fit model parameters and choose the preferred model. We compared healthy and paranoid people as to their probabilistic reasoning. We found that the costed-Bayesian model gave a better account overall of the performance of healthy participants, while the SPRT fitted paranoid (and remitted) participants better. The commonly held hypothesis that paranoid people make early decisions through assuming a higher cost of gathering information was rejected. The most striking finding in both models was the much higher noise parameter for the paranoid group. Therefore the ‘beads task’ may best be seen as assessing not ‘jumping to conclusions’ but executive functions subserving probabilistic reasoning. We suggest sev-

117 CHAPTER 5. AVOIDANCE AND JUMPING-TO-CONCLUSIONS IN PARANOIA eral new directions for methodological, computational and experimental research. Based on modelling, we suggest that the ‘beads task’ should be used with shorter sequences (e.g. maximum of 10 draws), more trials-per-participant and experimentally manipulated rewarding and aversive returns. Interpretation of task results in terms of underlying decision mechanisms (costs, noise and decision thresholds) has the potential to increase the construct validity of the task and eventually even to render it more relevant to assessing decision-making in clinical situations.

118 CHAPTER SIX

EXPERIMENTAL STUDY OF DEFENSIVE AVOIDANCE

Dis aliter visum The gods saw it otherwise (Virgil, unsourced.)

6.1 Summary

Introduction: The contention that psychological defensive avoidance contributes to the aetiology of paranoia has a sophisticated clinical-theoretical basis but empirical tests of this contention have been inconclusive. A detailed empirical study was therefore carried out to test it. The main hypothesis was that poor-me paranoid patients, those who believe that the persecution they perceive is undeserved, would show prominent defensive avoidance. Bad-me patients, who blame themselves for the persecution, might tend to engage in ruminations about negative aspects of the self. Poor-me patients were also expected to show a relatively preserved overt view of the self. Methods: Healthy people and patients with serious mental health difﬁculties, with either poor-me, bad-me or little paranoia, were recruited. The act of avoidance of aversive mental contents was assessed, rather than the ultimate success or failure of such avoidance as a defence. Participants were assessed for mood, social desirability, paranoia, perceived deservedness and self-discrepancies (how close they were to their ideals, both in their own eyes and in the eyes of others). In addition, questionnaire, clinical and laboratory ratings of defensive avoidance were carried out. The laboratory task was based on earlier theoretical (chapter2) and computational (chapter4) studies. Results: It was found that negative thoughts about the self were not excessively avoided by paranoid patients – neither according to laboratory nor according to clinical - psychotherapeutic criteria. In addition, neither paranoid patients in general nor the poor-me subgroup displayed any relative preservation of self-image. Only non-depressed paranoid participants showed relatively normal self-discrepancies. However, both poor-me and bad-me patients showed very low self-reported tolerance of negative mental contents, consistent with high levels of experiential avoidance. Conclusions: Defensive avoidance of negative aspects of the self may have little role in the aetiology of paranoia even if, in some of its forms, it may ameliorate the self-image.

119 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE

Keywords: Poor-me paranoia; bad-me paranoia; avoidance of negative aspects of the self; self-discrepancy; experiential avoidance; defensiveness.

6.2 Background & importance

The idea that defensive avoidance critically contributes to the aetiology of paranoia has a very sophisticated clinical-theoretical background, as we saw in sections 1.3.1, 1.3.2 and 2.4.4. However it has received inadequate empirical attention. This lack of empirical attention first concerns the contention that the psychological defensiveness contributes to paranoia . The scarcity of empirical studies also applies to the set of hypotheses postulating that the threat-information-processing system is co-opted in an exaggerated manner in paranoia (sections 2.4.2– 2.4.3). It is the former hypothesis, that of psychological defensiveness, which the present study sought to address. The attributional theory of paranoia holds that paranoid patients attribute to others responsibility for negative circumstances. They attempt to avoid blaming themselves, and hence feel somewhat better. The theory initially received some experimental support (Kinderman & Bentall, 1997). It subsequently also received serious challenges. One of the most straightforward is the observation that paranoid patients tend to have low self-esteem - hence paranoia seemed not to help people feel better about themselves. A refinement of the original hypothesis followed, namely that only people that feel they do not deserve to be persecuted (as opposed to those who feel that they do deserve persecution) benefit from a self-esteem-preserving effect of paranoid ideation. There is some evidence supporting this refined hypothesis (Bentall, Kinderman, & Moutoussis, 2008); however self-esteem is an overall measure affected by many variables complicating the picture (such as the social consequences of the psychological disturbance, patient status, developmental confounders like low IQ etc.). Self-esteem may thus be removed from the detailed mechanisms that sustain paranoid thinking. At the same time, as we saw in chapter2, one of the key effects of the ‘antipsychotic’ drugs used to treat paranoid syndromes on animals is to severely blunt avoidance responses. We thus reduced the ‘defence’ hypotheses of paranoia to their most basic and general sine qua non. While it can be hotly debated whether such defensive mechanisms are consciously accessible or part of a dynamic unconscious, whether they succeed in making the person feel less depressed, or just help them feel better in a particular area of how they view themselves, whether they improve overall self-esteem, or whatever their ultimate effects may be, the basic hypothesis is as follows: Paranoid thinking is fuelled partly by overactive avoidance of negative ideas about the self.

• In the case of those who believe that they don’t deserve to be persecuted, this avoidance would not be just of external circumstances (as in the animal experiments) but of negative thoughts about the self.

120 6.2. BACKGROUND & IMPORTANCE

• In the case of people who are in a ‘bad-me paranoid’ state, in contrast, negative thoughts about the self, thoughts consciously recognised as important, would be less avoided than in healthy individuals1.

If such an effect were to be established, it would be both clinically and scientifically important. Scientifically it would link, for the first time, the psychopharmacology of avoidance to the psychological processing of aversive thoughts. Clinically it would mean that attention has to be paid to the activation and avoidance of specific negative self-attributes in the achievement of remission and the prevention of relapse of paranoid syndromes. Were we to understand exactly what motivates paranoid ideas, this would help clinicians to establish a better therapeutic alliance with paranoid patients. This is is often very difficult to achieve. We expected paranoid patients to display the pattern previously found, of perceiving others seeing them in more negative colours than they see themselves. We will refer to this as an Other–Actual discrepancy (OAD). We expected them to score high on measures of experiential avoidance. We expected ‘poor-me’ paranoid patients to honestly present themselves in a more positive light that might be expected, thus showing high social desirability scores. In contrast, we expected ‘bad–me’ paranoid patients to see themselves negatively (greater Actual–Ideal discrepancy, AID) , and to see others seeing them in a similar light (smaller OAD). In that respect we would expect ‘bad–me’ patients to be similar to depressed non-paranoid patients, although theoretically there should be differences between the latter two categories. We sought to find out whether paranoid patients are more prone to avoid negative subjective states (experiential avoidance; Hayes et al., 1999). We sought to elicit the specific self-attributes that each individual participant might be motivated to avoid, and then to present them with a form of these negative stimuli2. It appeared plausible – especially from a psychodynamic point of view – that potentially exposing people to important avoided ideas about themselves, and potentially probing the ‘defences’ employed to ward such ideas off, might be distressing. It raises important ethical considerations, to which we now turn before describing in detail the methodological structure of the study.

1whether this is explained by attenuated defences to the self-esteem, as per the cognitive model, or by recruitment of defences such ‘turning against the self’ or ‘identifying with the aggressor’ of psychodynamic theory. 2From a psychodynamic standpoint one would also expect characteristics referring to aggression to be particularly avoided, when they are mentioned. However there would be no clear prediction as to whether such characteristics would be mentioned at all; people might avoid them defensively either by acknowledg- ing their importance, but then denying that they apply to themselves – “it is important not to be hostile with people, and I’m never hostile with people” – or to deny their importance altogether, not including them in a list of idiographically important attributes.

121 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE

6.3 Ethical issues

The ﬁrst ethical issues to be addressed are those encountered during any assessment of psychiatric patients:

i. The possibility that taking part will somehow distress the participants. Past experience with similar studies carried out by Prof. R.P. Bentall’s research team suggested that this would be a rare occurrence. Nevertheless this could not be assumed to always be the case.

ii. The possibility that capacity to consent will be compromised in some patients (in which case they will not be asked to take part)

With regards to distressing the participants, it was planned for signs of discomfort or distress to be monitored by the (clinically qualiﬁed) experimenter during the study. Appropriate clinical management would take place such as explanation, de-escalation and appropriate reassurance. The interview would, if necessary, be terminated.

6.3.1 The possibility of distressing or harming participants

We considered that topics might be raised that could be sensitive, embarrassing or upsetting. In order to study negative thoughts about the self, as well as their avoidance, participants would be asked what sort of characteristics they might approve, and what they might disapprove of in a person such as themselves. They would then be asked to rate the degree to which these attribute applies to themselves. Later they would also be asked to imagine what it would be like to possess a variety of disapproved-of (as well as some approved-of) attributes. This would constitute an aversive stimulus. The study, however, would in no way suggest that that these negative attributes actually applied to the participants. In order to give the participants control over the situation, they would first receive warning that a potentially unpleasant term would be presented to them, and then have opportunity to skip this unpleasant question altogether (we expected the percentage of such ‘skips’ to be a useful measure of avoidance). The protocol was first discussed with a group of psychiatrists in the Trust from which clinical participants were recruited, South-West London & St. George’s mental health NHS Trust. On the whole they felt that the aversive stimulus might even be too mild for its effects to be clearly visible. Psychodynamic psychotherapists tended to be more cautious, some expressing the opinion that participants may get angry with the experimenter. It was also possible that disclosures requiring further action might arise during the study, for example related to abuse that the participant may have suffered. Participants were informed that if a disclosure with clinical implications was made this would first be discussed with the participant themselves and then, if appropriate, with their Care Coordinator, Responsible Clinician or General Practitioner.

122 6.4. EMPIRICAL STUDY: DESIGN & RESEARCH INSTRUMENTS

6.3.2 Vulnerable groups & informed consent issues

Only participants capable of informed consent were recruited. Participants were reassured that taking part would have no impact on their life outside the study (with the exception of serious clinical issues or incidents, as discussed above). Refusing to participate would equally have no repercussions. Consent was taken by the author (the experimenter), who was already experienced in discussing informed consent in clinical settings; for the research purposes of this study, he was closely supervised and instructed by Prof. Bentall. A small monetary payment (ten pounds plus expenses, or ﬁfteen pounds ﬂat) was made upon completion of participation. The study results will be disseminated in a summarised form and care will be taken for this dissemination to be empowering and therapeutic if at all possible. Recruitment, participant information, consent and study protocol procedures were submitted to the Integrated Research Application System1 and were approved by the Riverside Research Ethics Committee on behalf of the National Research Ethics Service.

6.4 Design and instruments for the empirical study of defensive avoidance

We therefore designed a study to directly explore avoidance in paranoid patients, both ‘poor-me’ and ‘bad-me’, and compare it with healthy volunteers and with people with mental health problems of a similar severity who were not signiﬁcantly paranoid.

6.4.1 Participant groups

We recruited a group of clinically paranoid patients, a group of mental-health service users with little paranoia, and a group of healthy control participants. In the spirit of honest communication with the participants, we recruited the clinical participants on the basis of whether they themselves felt threatened and whether this was a problem that they recognised. We called our ﬁrst clinical group ‘Threatened’ participants, and the brief title of the study was ‘Thinking about the self when feeling under threat’. The second group of clinical participants were those that felt, and were thought to be by the mental- health care staff, as predominantly ‘Low mood’ rather than ‘Threatened’. We expected the ﬁrst group to be mostly paranoid (and mostly ‘poor–me’), and the second group to be mostly depressed (with less paranoia, which would be mostly ‘bad–me’). The group- based analysis of the data would, however, take place on the basis of the psychopathology as assessed, rather than as per recruitment group. We purposefully sought to match paranoid and non-paranoid clinical participants with respect to their psychiatric status and severity of problems. We therefore required that all

1https://www.myresearchproject.org.uk/Help/Contact.aspx

123 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE participants should be at least on ‘Enhanced Care Program Approach’ status, either receiving care from a Community team or being inpatients. The Community Teams included: Community Mental Health Teams, an Assertive Intervention Team, a Community Foren- sic Team and an Early Intervention Team. In the case of the latter, participants were required to have an established diagnosis of psychotic, non-bipolar disorder. Alternatively, participants were inpatients in Acute Adult or Medium Security wards. Participants who were detained under the English Mental Health Act were not excluded. The non-clinical (healthy control) group was a convenience sample, recruited to approximately match the clinical groups for age and sex. We also aimed to match participants for years of education as a proxy of intellectual ability, but statistical adjustment turned out to be necessary here. The healthy sample consisted of individuals who had registered an interest by placing their names on the University Bangor School of Psycho- logical Sciences subject panel.

6.4.2 Inclusion & exclusion criteria

Inclusion criteria

Threatened / Paranoia group: Participants had to suffer from a mental disorder (other than a primary personality or substance abuse disorder) that included, during the present episode, suspiciousness and delusions or clear overvalued ideas, as determined by a score of 3 or more on item 11 of the Brief Psychiatric Rating Scale (BPRS). The severity of the patient’s disorder (as opposed to other problems or needs) should be as described above (at least ‘Enhanced CPA’). These inclusion criteria should ensure that patients have a signiﬁcant current tendency to hold persecutory ideas. Previous research showed that such subjects primarily display ‘poor-me’ paranoia. Low-mood / depression group: Participants had to suffer from a serious psychiatric disorder, other than a primary personality or substance abuse disorder, of comparable severity to the above and with depressive symptoms being most prominent. We aimed to recruit participants without persecutory ideas to this group, but this proved impractical to determine at the time of recruitment. Healthy control group: These participants did not suffer from any psychiatric disorder at the time of testing. We expect them to show less experiential avoidance on either the questionnaire or the computer (behavioural) task.

Exclusion criteria

1. A primary diagnosis of substance abuse disorder or of Personality Disorder. Both these may show a very marked variability in the content of their ideation; the theory underlying our approach is not tailored to these disorders; and there is a lack of empirical studies on the interaction of ideas about the self and paranoid ideation in these disorders, that would make results difﬁcult to interpret. Separate, speciﬁc studies would be

124 6.4. EMPIRICAL STUDY: DESIGN & RESEARCH INSTRUMENTS

required to compare persecutory ideas in such disorders and in the groups of primary psychotic disorders that have received most attention so far.

2. Clinically signiﬁcant learning disability or cognitive impairment: There is evidence that cognitive ability is inversely correlated to persecutory ideation. We would not like effects related to low cognitive ability to dominate our results.

3. Major medical illness: This has major and unpredictable effects on both affective and cognitive function.

4. Current or past history of manic episodes: Mania can include persecutory ideas and may involve ‘defensive’ thinking that overlaps, but is not identical, to the hypothesis of the study. As manic patients may have reasons other than those related to the study hypothesis to engage in such ‘defensive’ thinking, mania would be a powerful confounder.

5. Poor use of English: Regrettably not all of the instruments used have been validated in non-English-speaking subjects and therefore results would be of questionable validity.

6.4.3 Data-gathering procedures and instruments

Clinical & demographic information

Key clinical and demographic information was ﬁrst gleaned from the notes. The ‘Pre- senting Problems’ precipitating mental health services usage (MHSU) , the history of those presenting problems and the most recent assessment of the patient’s clinical state were summarised. Key background features from the personal history were sought and noted if relevant. Also recorded were: age, gender, whether the patient received a therapeutic dose of antidepressants, any anxiolytics or hypnotics, the total chlorpromazine- equivalent daily antipsychotic dosage, whether the patient received Clozapine, years in full-time education and socioeconomic achievement. The clinical diagnoses were also noted. If however at the end of the structured meeting there were important gaps in the account of the current problems or psychiatric diagnostic evaluation, additional brief psychiatric interviewing was carried. The ﬁnal diagnoses recorded were the ones arrived at by the experimenter, Dr Moutoussis, according to ICD- 10 (World Health Organisation, 1992).

Paper questionnaires & interview measures

These included the following measures:

• Persecution and Deservedness Scale (Melo et al., 2006). This is a ten-item, psychometrically robust measure of paranoia and beliefs about whether persecution is deserved.

125 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE

• Acceptance and Action Questionnaire, second version. This is a ten-item, validated questionnaire measure of Experiential Avoidance. It is thought to measure the propensity to avoid or control certain internal states (Hayes et al., 2004).

• Marlowe-Crowne Social Desirability scale (Crowne & Marlowe, 1960). This is a validated questionnaire measure of the propensity to give an overly positive self- representation.

• Selected items of the Brief Psychiatric Rating Scale. Items: 4. Conceptual disorganisation - thought processes confused, disconnected, disor- ganised, disrupted; 8. Grandiosity - exaggerated self-opinion, arrogance, conviction of unusual power or abilities; 11. Suspiciousness; and 12. Hallucinatory behaviour - perceptions without normal external stimulus.

• The Hospital Anxiety and Depression Scale.

• The Abbreviated Mental Test, a simple, validated, widely used screening test for cognitive impairment.

The ‘Computerised self-lines questionnaire’ & ‘Engagement with negative attributes task’

The Computerised Self-Lines Questionnaire was an adaptation of the ‘Self Lines Mea- sure’ (Francis, Boldero, & Sambell, 2006). It was transferred to an interactive computer format which followed closely the original paper version. Participants were asked to list five attributes in each of three domains of self-representation – their ideals (ideal- attributes), their duties (ought-attributes) and the attributes they would most desire to avoid (feared-attributes; figure 6.1). For each of these fifteen attributes they were asked to provide what they thought was the opposite term. They were then presented with a line stretching from the one attribute to the opposite and asked to rate where on that line they considered to be themselves. One line was presented with the question “How would you rate yourself in this area, where do you think you actually are?”. Participants would rate themselves with a slider on the screen. Some of the older participants were not familiar with computers and pointed with their finger where on the line the experimenter should place the slider. A second line concerned the same pair of attributes, but asked “How do you think other people see you in this area?”. Diverging slightly from the approach by Kinderman and Bentall (1996), here participants were not asked the specific opinions of their parents about them. Instead they were asked to make the best guess they could as to what people who know them “reasonably well” were likely to think about them. The computer calculated and recorded three discrepancy scores for each item: the AI and OI discrepancies and their difference, the Actual-Other

126 6.4. EMPIRICAL STUDY: DESIGN & RESEARCH INSTRUMENTS

Figure 6.1: Request for ’Feared Self’ attributes discrepancy. These discrepancy scores formed the basis of the subsequent part of the study. A pilot version also enquired where people would like to be (desired-self) with respect to the same attributes, as in the original ‘Self-lines’ measure. Pilot subjects almost always described each domain of self-representation in terms of a desirable and an undesirable pole. When marking where their ‘desired self’ would be, they just marked the desired- pole word. We therefore dropped the ‘desired self’ line and considered the desired self to lie at the relevant end of the dyad. The Engagement with Negative Attributes task was based on the idiographically determined self-discrepancies. Here participants were asked to imagine and explain how some undesirable (and some desirable) characteristics might apply to themselves. Each of thirty questions exposed them to an undesirable (or, in about 5 questions out of 30, desirable) characteristic, putatively attributed to themselves. Each trial was preceded by a simple shape that gave an indication of the Other–Actual discrepancy that the participant rated for the pair to which the upcoming negative characteristic belonged. This was designed to be an experiential analogue of the conditioned-avoidance setup (chapter2). That is, warning stimuli predicted the aversiveness of the following stimuli. Participants could learn to perform an avoidance response (before the stimulus appeared) or an escape response (after exposure to the stimulus started). We expected that in the course of a few trials people would associate the correct shape with the most aversive attributes. They were told to say ‘skip’ if at any point they wanted to move straight to the next trial. This experiential avoidance task was explained to the participants in detail, but terms such as engagement with negative attributes or avoidance were not used. The following explanation was given, checking after each part of it that the participant understood and

127 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE giving them the opportunity to ask questions: “The computer will now ask you some questions, based on the words or phrases you entered in the last part of the experiment. It will ask you, for example, what other people might think if you appeared to have a particular characteristic. Before each question a shape will appear on the screen; the kind of shape that appears is a hint as to how you rated yourself about the characteristic that will appear afterwards, in the question. You can think about which shape goes with the more negative characteristics, and tell me to skip that question if you want. However the most important thing is to answer the questions. If a question appears that you don’t want to answer, again please tell me and we will skip it. The computer makes the questions up by choosing words randomly, so it may repeat itself. If a question repeats itself please answer it again for me, giving me another example if possible, as this would help my carrying out the study. If a question is however unpleasant in any way please feel free to tell me to skip it.” We also asked the participants to read out loud each question, so that we knew that they had attended to it. In all, four types of question followed the statement “tell us about being (characteristic)”. These were:

• Are there people you know that might care if you were like that, for example your parents? Please describe what they might think if you appeared (characteristic).

• Please tell us what other people might think about you if you appeared (characteristic).

• Please describe a situation where you might be (characteristic).

• Please describe what would it mean for you if you were (characteristic).

We collected the following data: Whether the participant avoided or escaped an item; the latencies for doing so; The amount of time they spent actually answering the question; and a sound recording of their actual answer. The median times for low-discrepancy, medium-discrepancy and high-discrepancy items were obtained for each participant. The median time for all the items per participant was also obtained. The recorded answers were transcribed off-line and rated on a 4-point Engagement / Avoidance in thinking about the self scale. This scale had as follows:

0. No engagement: e.g. Prompt: “Describe a situation where you might be selﬁsh” Response: “I wouldn’t be selﬁsh deliberately ... I’m trying to think of one where I might be by mistake ... no, I can’t think of one, next question”. ‘0’ also included reversal of the valence with which the term was used, as participants were instructed that these questions were about elaborating on the previous part of the experiment; e.g. if anger previously described as undesirable now became a positive attribute.

1. Low engagement: here the description only ﬂeetingly made contact with the potentially negative (or positive) experiences; it was easy for the rater to see how in

128 6.4. EMPIRICAL STUDY: DESIGN & RESEARCH INSTRUMENTS

the situation presented the pt. avoided or ‘distanced’, justiﬁed or ameliorated the experience. ‘1’ included most one-word answers, or “I suppose I would lie to prevent somebody else’s suffering”. This rating also included sophisticated ‘balanced’ accounts, especially of the yes-but variety, that left the speaker and listener with an anodyne experience. It also included answers which circumvented the key point of the question e.g. what others think or a situation.

2. Moderate engagement: This described situations or consequences in a way that lacked affective tone; It might be just factual e.g. “if I was unreliable I wouldn’t get the letters out in time”. It might also describe aversive (or desirable) experiences / situations in vague, nonspeciﬁc, nondescript, conventional terms.

3. High engagement: Here a palpable sense of threat (or support) to self-esteem or esteem from others (or the self within a situation) was described e.g. “if I was unreliable, other people would think badly of me / avoid me”. The rater could easily imagine that what was described corresponded to an aversive (or desirable) situation.

6.4.4 Outcome measures & power calculations

Primary outcome measure: Experiential avoidance as measured by our novel computerised task. Secondary outcome measures: Self-discrepancy scores as determined by the computerised task, Experiential Avoidance scores as measured by the AAQ, Self-deceptive positivity as measured by the Marlowe–Crowne social desirability scale and Observer-rated engagement with (vs. avoidance of) thinking about the self.

Power calculations

To compute sample sizes, we used data from the following sources. For AAQ scores we used pilot data collected by Prof. Bentall, indicating that mean scores for paranoid, depressed and control participants were likely to be 33, 42 and 25 respectively1, with a sample SD of 9. Expected actual-ideal discrepancy data was available from Kinderman and Bentall (1996). Actual-Ideal normalised discrepancies were found to be for paranoid subjects 0.182, for depressed subjects -0.022 and for Control subjects 0.25. The sample SD was 0.276. From the same source, a measure of actual-other discrepancies was available. In that study, Parent-Actual normalised discrepancies were found to be for paranoid subjects 0.101, for depressed subjects 0.174 and for Control subjects expected to be 0.303. The sample SD was 0.161. Setting alpha at 0.01 and power at 0.7, these data suggest a total sample size require- ment of 8 per group for the AAQ, 23 per group for actual-ideal discrepancies and 16 per

1These scores are not directly comparable to the ones obtained in this study, as they pertain to the previous version of the AAQ.

129 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE group for actual-other discrepancies. As our computerised avoidance task has not previously been tried with psychiatric patients, we were not able to perform a sample size estimation for it. Thus a sample size of 23 per group would provide adequate power, and aiming for 25 per group would allow for a small safety margin above that.

6.4.5 Analysis strategy

We planned to use two-way ANOVA (group × sex) to examine group differences in scores on the different tests. In the event of us being unable to adequately balance the groups for age and years of education, we planned to include these variables as covariates. When ANOVA reveals significant differences between the groups, appropriate post- hoc tests can be used to detect which groups differ from each other. As the study is motivated by specific hypotheses, Simple Contrast tests would be used in the first instance to examine whether the paranoid groups, and especially the poor-me paranoid group about which we make the most central hypotheses, significantly differed from the healthy control group. In addition, we would use linear regression analyses to examine how the ‘psychological’ variables, the ones most closely related to the psychological mechanisms which – we postulate – contribute to symptoms, are related to the symptoms themselves. We would perform two-stage linear regression analyses. In the first stage variables widely believed to contribute to symptoms but which are weakly related to our central hypotheses would be entered. In a second stage, variables related to our hypotheses would be entered. We would examine whether the augmented model resulted in a significant change in the F statistic, and whether the significant regression (β) coefficients were consistent with the study hypotheses. Finally we would employ standard diagnostic statistics, especially assessing collinearity, to ensure the quality of the regression analysis and to introduce appropriate corrections.

6.5 Interview protocol

6.5.1 Identiﬁcation & recruitment of participants

Control participants were recruited through the ‘University of Bangor School of Psy- chology subject panel’. Clinical participants were found by asking senior clinicians in South West London & St. George’s NHS mental health Trust (SWLSTG) as to whether they had suitable people under their care. The study was performed in London at a time where each person using specialist mental health services had a ‘Care Coordinator’, an experienced clinician responsible for their overall care. Team leaders were ﬁrst approached, were told about the study and gave permission to proceed. Care Coordinators or senior inpatient staff made preliminary suggestions regarding possible participants. These were then approached by Dr Moutoussis according to Ethics Committee regulations.

130 6.6. RESULTS

6.5.2 The research interview

There was just one test session, which lasted between one and two hours including breaks, depending on the participant. Exclusion criteria were enquired about first, then inclusion criteria. All eligible participants were then administered the Persecution and Deservedness Scale and the Acceptance and Action Questionnaire. Clinical participants were administered the Hospital Anxiety and Depression Scale at this point. All participants were then asked to complete the computerised Self-Lines Questionnaire and En- gagement with Negative Attributes task. Participants were finally given the Marlowe– Crowne social desirability scale and items of the SCAN rating for ‘grandiosity’, ‘hallucinations’ and ‘conceptual disorganisation’ as required. All participants were thanked for their participation and given the 10-pound payment either on completion of the study session or when they decided (or, exceptionally, agreed to the experimenter’s suggestion) to stop. If any participants wished to complain or had any concerns, further meetings with the author and University authorities were available. Finally, once the study results are analysed, all participants will be sent a brief summary of the findings, without indication of how they personally compared with other participants.

6.6 Results

6.6.1 Summary

This study assessed defensive avoidance of negative thoughts about the self in a healthy control (HC) group, in poor-me paranoid patients (PMP), bad-me paranoid patients (BMP) and in a nonparanoid clinical control (CC) group. The study was in general well tolerated, though roughly half the patients who were invited to participate did not do so. Two participants out of the 75 showed distress while contemplating the aversive thoughts, and another two experienced distress associated with general mistrust towards the experimenters. Self-discrepancies of three types were ﬁrst assessed. Actual-Ideal discrepancies (AID) were statistically predictive of paranoia, as measured by the ‘persecution and deservedness scale’ (paranoia subscale; PADS-P) . Other-Ideal discrepancies (OID) showed a similar pattern but post-hoc tests showed higher OID for the BMP group only. Contrary to our hypotheses, (i) Other-Actual discrepancy (OAD) did not discriminate between groups and (ii) Levels of undeserved paranoia (PU ) did not correlate with relatively preserved AID. The hypothesis that PMP would be associated with high defensive avoidance of negative thoughts about the self was assessed with two questionnaire measures, the ‘acceptance and action questionnaire 2’ (AAQ-2) and the Marlowe-Crowne social desirability Scale (MC) . AAQ-2 scores correlated highly with paranoia but did not distinguish PMP from BMP. There was only trend-level signiﬁcance of MC scores, and that was in a direc-

131 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE tion inconsistent with the study hypothesis. Behavioural and interactive measures used to assess defensive avoidance included time spent engaging with more aversive questions, and observer-rated engagement (vs. avoidance) of negative thoughts about the self. Participants showed a statistically sig- niﬁcant elongation of time spent engaging with negative (vs. low-aversive or positive) items, by about 25% overall. In this they varied widely within each group, but showed no statistically signiﬁcant differences between groups. Neither time measures nor observer ratings of avoidance discriminated between any of the groups. In regression analyses, years-in-education predicted PADS-P scores, but the most powerful predictor was AAQ-2 score. HADS scores also correlated with paranoia, but HADS was collinear with AAQ-2, r = −0.85, P < 0.001. Grandiosity was the only other symptom predicting paranoia in forced-variable-entry analysis. Following Lattin, Carroll, and Green (2003), exploratory stepwise regression analysis was used to examine variables for which no a priori hypotheses were made. Here OID also emerged as a predictor of PADS-P. PU was predicted by MC score (in the direction opposite to that expected), as well as AAQ-2. Stepwise analysis revealed no further potential predictors. The best regression models predicted only half as much of the PU variance (33%) compared with the PADS-P variance (67%). Finally, we note that 80% of the variance in AAQ-2 scores was predicted by a linear combination of PADS-P and HADS.

6.6.2 Participant welfare

Very few participants found the study distressing. One patient in the ‘low mood’ group found the Avoidance task irritating, as different aspects of the same attributes were asked about; the interview had to be terminated. In the ’feeling threatened’ recruitment group two participants did not ﬁnd the study straightforward. The ﬁrst was troubled by marked conceptual disorganisation. He asked for two breaks in the interview and when he enquired about a third the experimenter suggested that the interview be terminated. The participant took this up. Interestingly this participant subsequently expressed satisfaction with the study. A second ‘threatened group’ participant was extremely concerned that the experimenter might be somehow ‘in league’ with staff providing healthcare for him. He required several appointments to be reassured. One further ‘low mood’ participant carried out most of the Self-lines and Computerised Avoidance tasks in tears. She insisted, however, to complete the study and there was no doubt that she was competent to make this decision. The last two participants granted permission for their responsible clinicians to be contacted, which proved quite helpful. All these participants were happy for data contributed to the study to be used for research purposes. About half of the MHSUs that were introduced to the experimenter by clinical staff declined to participate. As they did not consent, no clinical or demographic details were recorded for them. The fact that participants were not only selected by healthcare staff

132 6.6. RESULTS but also powerfully self-selected may limit the generalizability of the ﬁndings of this study. There was, however, no a priori reason to think that either selection by staff or self-selection would relate speciﬁcally to the role of defensive psychological avoidance in paranoid ideation.

6.6.3 Examination of key hypotheses

Valid data was collected from 25 threatened / paranoid participants, 23 low mood / depressed participants and 27 healthy control participants. Both regression and group-based analyses were carried out on the data. However the groups into which the participants were recruited were not useful for testing our key hypotheses. There were two reasons for this. The first was that it was not possible to directly recruit participants into different paranoia subgroups, especially with respect to how ‘deserved’ persecution was felt to be, as the relevant information was simply not available in advance. The second reason was that, contrary to expectation, many people invited to participate in the ‘low-mood’ group experienced significant levels of persecution (figure 6.2). It is likely that this was a consequence of matching the ‘low-mood’ and ‘threatened’ groups for severity of psychiatric problems, operationalized in the first instance on the basis of service utilisation. It appears that people with severe depression without persecutory ideas or psychotic symptoms are unlikely either to be admitted to inpatient wards or to spend much time in the ‘enhanced CPA’ CMHT caseload. Note also that the way in which persecutory ideas were examined (by using high-scored PADS items to open the subject) elicited that high PADS scores in the ‘low mood’ group reflected a range of life experiences and beliefs, from bizarre depressive delusions to plausible persecutory experiences leading to refugee status. The distribution of both persecution and deservedness scores is shown in figure 6.2. In this and subsequent figures questionnaire scores are all normalised to a value of 1.0, i.e. the fraction of the maximum possible score on the questionnaire is displayed. The distributions shown in figure 6.2 suggest that Persecution scores of up to 0.3, which include 85% of healthy participants, can be considered ‘non-persecuted’. Each clinical group appears to separate into a large low deservedness subgroup and a less populous high deservedness subgroup (bad-me), separated by a score of PADS-D = 0.3 . In the presence of significant paranoia (PADS-P > 0.3) we can call the former ‘poor-me paranoid’ and the latter ‘bad-me paranoid’. These thresholds separated the data into four paranoia-groups: Healthy non-threatened (N = 23), Mental health service user - non-threatened (N = 9), bad-me paranoid (N = 14) and poor-me paranoid (N = 29). The key descriptive statistics about paranoia & deservedness, depression & anxiety, self-discrepancies, ‘avoidance & action’ and social desirability are shown in table 6.1, p. 135.

133 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE

Figure 6.2: Persecution and deservedness scores according to clinical participant group. a. Persecution scores according to the PADS. Note that the healthy control group has very low persecution scores: 23 of 27 healthy participants have a score of 0.3 or less (12 out of a maximum of 40 in un-normalised scores). However most ‘low mood’ participants have paranoia scores comparable to those of the ‘threatened’ group (upper two panels). b. Deservedness scores. The value of 0.3 seems to separate a large low-scoring subgroup from a sparser high-scoring subgroup in all clinical groups.

134 6.6. RESULTS

Table 6.1: Means (& standard deviations) for the questionnaire measures pertaining to the four groups of participants.

Healthy Clin. Control Bad-Me Poor-Me M (SD) M (SD) M (SD) M (SD) Paranoia Paranoia, PADS-P 0.12 (0.10) 0.22 (0.049) 0.63 (0.21) 0.63 (0.16) Deservedness, PADS-D 0.11 (0.11) 0.06 (0.17) 0.57 (0.20) 0.15 (0.094) Undeserved Paran., PU 0.14 (0.11) 0.21 (0.067) 0.25 (0.13) 0.53 (0.13) Affect HADS-Depression 0.12 (0.096) 0.38 (0.21) 0.47 (0.26) 0.40 (0.24) HADS-Anxiety 0.21 (0.14) 0.38 (0.14) 0.55 (0.20) 0.55 (0.24) HADS total 0.17 (0.085) 0.38 (0.16) 0.50 (0.22) 0.48 (0.23) Self-Discrepancies Actual-Ideal Discr. -24.9 (13.8) -27.4 (14.6) -42.6 (23.0) -42.0 (24.6) Other-Ideal Discr. -26.0 (14.2) -25.0 (14.5) -46.4 (21.0) -38.4 (23.0) Combined Discr. -25.5 (13.3) -26.2 (12.4) -44.5 (16.1) -40.2 (23.4) Other-Actual Discr. 1.08 (8.64) -2.30 (15.1) 3.79 (28.5) -0.62 (16.3) ‘Avoidance & Action’ and social desirability AAQ-2 0.81 (0.14) 0.58 (0.21) 0.40 (0.19) 0.39 (0.21) Marlowe-Crowne 0.57 (0.15) 0.67 (0.090) 0.55 (0.15) 0.49 (0.17)

Other-Actual self discrepancies are not increased in paranoia

The first key hypothesis was that in comparison to both non-persecuted patients and controls, persecuted patients would report greater other-actual discrepancies. This hypothesis was not supported by the data. Two-way ANOVA (paranoia group × gender; age & years-in-education as covariates) revealed no significant main effects or interaction (table 6.2, p. 137). Similarly, the partial correlation between PADS-Paranoia score and Other-Actual Discrepancy, controlling for age, gender and years-in-education was also far from significant (r = −0.09,P = 0.52).

Discrepancies from the ideal are predictive of symptom scores

Other discrepancy measures were, however, predictive of symptom variables. The overall relationship between paranoia and self-discrepancies is presented here; the role of affect in this relationship, which is very important, will be considered later (page 139). Actual-Ideal discrepancy was highly correlated with paranoia scores (r = −0.50, P < 0.001; corrected for age, gender and years-in-education). A similar pattern emerged for the Other-Ideal discrepancy (r = −0.46, P < 0.001; corrected for the same three variables). The more one deviated from one’s ideals, either in one’s own eyes or as perceived to do in the eyes of others, the higher the sense of threat (ﬁgure 6.3).

135 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE

The scores for the two types of discrepancy, AID and OID, were highly correlated with each other (partial r = −0.70, P < 0.001; corrected for age, gender and years-in- education). We carried out a two-way (paranoia group × gender) ANOVAwith AID as the dependent variable and age & years-in-education as covariates. Only the main effect of group was significant (table 6.2), while Simple Contrast analysis with the healthy-control group as reference revealed significant differences with the both the bad-me group and with the poor-me groups. With OID as the dependent variable, two way ANOVA (paranoia group × gender; age, years-in-education as covariates) again revealed significant differences for the main effect of group only. In this case Simple Contrast analysis with the healthy-control group as reference revealed a significant difference with the bad-me group (P < 0.005) and a trend-level difference with the poor-me group (table 6.2). In view of the similarity of the two self-discrepancy measures, we combined them into a simple, single Discrepancy-from-Ideal measure (DI) by taking their average for each participant. When DI was used as the dependent variable, the analogous two-way ANOVA indicated clearly significant differences between each of the two paranoid groups and the healthy control one (table 6.2). It is also of note that the clinical control group had similar discrepancy scores with the healthy control, despite the higher depression and anxiety scores found in the clinical control group. However the low numbers of the latter (N = 9) mean that a real difference may have been missed.

Undeserved paranoia does not help reduce Actual-Ideal discrepancy

As the key hypotheses of this study concerned defensive avoidance in poor-me paranoia, which is characterised by low deservedness, it is useful to weigh the PADS-P score with deservedness, to obtain an estimate of the extend of undeserved paranoia that each participant feels under: PU = PADS-P × (1 - PADS-D). Here the PADS-D score was expressed as a fraction of complete deservedness that the maximum score of the PADS-D 1 subscale would represent . Table 6.1 shows that the PU measure characterised the experimental groups in the expected manner. The following question could then be examined: does undeserved paranoia improve how a person assesses themselves to be (AID) when they feel themselves perceived in a certain way by others? This would be another way of detecting a defensive role of undeserved-paranoia, even in the presence of a negative average evaluation of the self.

However calculating the partial correlation between PU and AID while controlling for OID reveals no signiﬁcant correlation (r = −0.16,P = 0.17). Similar results are obtained if PADS-P rather than PU is used in this analysis (r = −0.14,P = 0.28).

1This is not the only way to derive an undeserved-paranoia measure out of the PADS-P and PADS-D scales; for example, each item of the PADS-P could be weighed by its own deservedness. However this would run into problems with the PADS-P items marked 1 out of 4, for which no deservedness estimate is collected but which contribute to the total PADS-P score.

136 6.6. RESULTS

Table 6.2: Paranoia grouping vs. Self-Discrepancies. ANOVAs with age and years-in- education as covariates are shown, with post-hoc Simple Contrast analyses when signiﬁ- cant differences are detected.

ANOVA with dependent variable: Other-Actual Discrepancy Effect P F df (factors) df (error) Paranoia group (main) 0.18 1.68 3 61 Gender (main) 0.78 0.08 1 61 Paran. gp. × Gender 0.46 0.87 3 61 ANOVA with dependent variable: Actual-Ideal Discrepancy Effect P F df (factors) df (error) Paranoia group (main) < 0.05 3.73 3 61 Gender (main) 0.58 0.45 1 61 Paran. gp. × Gender 0.58 0.65 3 61 Significant Simple Contrasts, reference group = Healthy Control Index group P Ref. mean Index mean Bad-Me paranoid <0.01 -0.25 -0.43 Poor-Me paranoid <0.005 -0.25 -0.42 ANOVA with dependent variable: Other-Ideal Discrepancy Effect P F df (factors) df (error) Paranoia group (main) < 0.05 3.26 3 61 Gender (main) 0.56 0.34 1 61 Paran. gp. × Gender 0.58 0.65 3 61 Significant & trend Simple Contrasts, reference group = Healthy Control Index group P Ref. mean Index mean Bad-Me paranoid <0.005 -0.26 -0.46 Poor-Me paranoid 0.066 -0.26 -0.34 ANOVA with dependent variable: Combined Discrepancy-from-Ideal Effect P F df (factors) df (error) Paranoia group (main) < 0.05 3.88 3 61 Gender (main) 0.46 0.55 1 61 Paran. gp. × Gender 0.33 1.17 3 61 Significant Simple Contrasts, reference group = Healthy Control Index group P Ref. mean Index mean Bad-Me paranoid <0.005 -0.25 -0.45 Poor-Me paranoid <0.01 -0.25 -0.40

137 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE

Figure 6.3: Illustrations of the relationship between self-discrepancies and paranoia. a. Scatter plot illustrating that paranoia increases with worsening Other-Ideal discrepancy; a few individuals have small discrepancies but high paranoia. b. The paranoid groups have worse self-discrepancies than the healthy controls; Actual-Ideal discrepancies are shown here.

138 6.6. RESULTS

Anxiety and depression are strongly related to large self-discrepancies

At the same time, it was observed that self-discrepancy measures were highly correlated with depression and anxiety, as measured by the HADS-Depression and HADS- Anxiety scores. For example the partial correlation between AID and HADS-D was 0.695 (corrected for age, gender, socioeconomic achievement and years-in-education), while the one between AID and HADS-A was 0.66. Affective variables and paranoia were also correlated; for HADS-D vs. PADS-P, r = 0.48,P = 0.00. In this study all three quantities, viz. self-discrepancies, paranoia and affective variables were measured simultaneously and the directionality between any two of them is uncertain. Therefore any mediation arguments have to be made with extreme care. This said, we can observe that the correlation between AID and HADS-D was very strong (r = 0.70, P < 0.001; figure 6.4) and controlling for PADS-P has little effect, reducing it to r = 0.60, P < 0.001. On the other hand controlling for AID reduces the magnitude of the correlation between HADS-D and PADS-P from 0.48 to 0.21 and, more importantly, renders it statistically nonsignificant. These results are consistent with AID being closely tied to depression and mediating between paranoia and depression (or possibly between the causal processes giving rise to each). As expected from the above result, controlling for depression also rendered the relationship between PADS-P and AID non-significant (uncorrected for HADS-D: r = −0.42, P < 0.001; corrected for HADS-D: r = −0.14,P = 0.23). It is interesting that despite the close relationship between AID and OID, the latter did not conform to the same pattern of relationships. Controlling for HADS-D also reduces the correlation between OID and PADS-P, from -0.46 to -0.281 but statistical significance was maintained (P < 0.05). It is therefore likely that OID is more closely related to paranoia than depression, or, in other words, that the correlation between OID and paranoia is not adequately accounted for by affect.

Low ‘acceptance and action’, but not high social desirability, characterises the paranoid groups

The second key hypothesis was that Poor-me persecuted patients would score lower than controls on the Avoidance and Action Questionnaire (AAQ-2) and higher on the Marlowe-Crowne social desirability scale. Two-way ANOVA with ‘acceptance and action’ (AAQ-2) as the dependent variable HC was highly signiﬁcant (table 6.3). The Healthy group had the highest score (AAQµ = 0.81), which was signiﬁcantly different from all the other groups. It can be seen that the two paranoid groups had equal mean AAQ scores (table 6.1). The mean of the non- threatened clinical control group had an intermediate value. There is therefore evidence that paranoia is associated with lower AAQ scores, but there is no evidence that it dif-

1We note that ‘high self-discrepancies’ are more negative numbers, while high psychopathology corresponds to more positive scale scores, hence the signs of the coefﬁcients reported here.

139 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE

Figure 6.4: Illustration of the relationship between self-discrepancies and HADS- Depression; they are highly correlated. ferentiates between bad-me and poor-me paranoia. ANOVA with social desirability (MC score) as the dependent variable only showed trend-level significance for the main effect of paranoia group. However, this trend was due to the means being arranged in an order inconsistent with the study hypothesis, the PM group having the lowest mean and the CC the highest (table 6.1). The results for the AAQ-2 and MC scores are compared in figure 6.5. An additional analysis was carried out where the total sample was grouped not according to paranoia × deservedness, but paranoia × depression. This was done to further illustrate the role of affect, but also for easier comparison with the literature (e.g. Kinderman, Prince, Waller, & Peters, 2003). As shown in table 6.4, depressed paranoid participants have significantly lesser AAQ scores than nondepressed paranoid ones, who in turn score significantly lower than the ‘healthy’ (nondepressed nonparanoid) ones. In this analysis both AI and OI self-discrepancy scores discriminate between ‘healthy’ and depressed paranoid, but not between ‘healthy’ and nondepressed paranoid groups.

Interactive measures of defensive avoidance

The third key hypothesis held that poor-me persecuted patients would show higher levels of avoidance than controls in the Computerised Engagement with Negative At- tributes task. We expected healthy control participants to show less pronounced avoidance and BM participants even less (or reversed) avoidance. This hypothesis was operationalized in the following manner.

140 6.6. RESULTS

Table 6.3: Paranoia grouping vs. AAQ-2 and social desirability, with post-hoc ‘simple contrast’ analyses when signiﬁcant differences were detected.

ANOVA with dep. variable: Acceptance and Action Questionnaire score Effect P F df (factors) df (error) Paranoia group (main) <0.001 14.76 3 61 Gender (main) 0.97 0.002 1 61 Paran. gp. × Gender 0.36 1.08 3 61 Signiﬁcant Simple Contrasts, reference group = Healthy Control Index group P Ref. mean Index mean Non-threatened Clin. 0.01 0.81 0.58 Bad-Me paranoid <0.001 0.81 0.40 Poor-Me paranoid <0.001 0.81 0.40 ANOVA with dependent variable: Marlowe-Crowne score Effect P F df (factors) df (error) Paranoia group (main) 0.067 2.52 3 59 Gender (main) 0.091 2.96 1 59 Paran. gp. × Gender 0.729 0.73 3 59

Table 6.4: Here the sample has been divided not with respect to deservedness, but with respect to depression, using a threshold of HADS-D = 0.33, the upper limit of the healthy group scores. According to this criterion there were only ﬁve non-threatened depressed participants, too few to be included in this analysis. A two-way ANOVA, group × gender with age and years-in-education as covariates, was performed. a. Key descriptive statistics. Simple contrast analysis with the non-depressed paranoid group as reference resulted in the signiﬁcant results shown in bold. b. ANOVA table for the AAQ analysis.

a. Key descriptive statistics Alternative Non-paranoid Paranoid Paranoid Grouping non-depressed Depressed non-depressed N = 26 N = 23 N = 17 M (SD) M (SD) M (SD) AAQ 0.79 (0.16) 0.28 (0.14) 0.55 (0.16) P < 0.005 P < 0.001 (reference) AI Discr. -23.4 (13.4) -54.22 (19.8) -25.9 (20.0) n.s. P < 0.005 (reference) OI Discr. -25.3 (13.6) -50.1 (22.8) -28.7 (15.1) n.s. P < 0.01 (reference) AO Discr. 1.84 (8.62) -4.12 (23.2) 2.80 (13.1) n.s. n.s. (reference) MC (Social 0.59 (0.14) 0.52 (0.16) 0.51 (0.12) Desirability) n.s. n.s. (reference) b. ANOVA with dep. variable: Acceptance and Action Questionnaire score Effect P F df (factors) df (error) Alt. Paranoia (main) <0.001 40.9 2 58 Gender (main) 0.63 0.24 1 58 Alt. Paran. × Gender 0.47 0.75 2 58

141 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE

Figure 6.5: a. Marlowe-Crowne scores did not separate the paranoid groups from each other or from the healthy controls. b. AAQ-2 scores strongly discriminated between healthy, clinical control and paranoid groups in the expected direction; however they did not distinguish between poor-me and bad-me threatened groups.

142 6.6. RESULTS

1. Healthy people should spend less time talking about what it would be like to have highly self-discrepant, negative characteristics compared to talking about low-discrepant and positive characteristics. On the basis of theoretical considerations the self-discrepancy that would be of most relevance to healthy people to be the Ideal-Actual one.

2. Poor-me paranoid participants should show a more pronounced gradient in the same direction as the healthy controls, i.e. a greater difference in time talking about high- discrepancy vs. low-discrepancy self-characteristics (HDSC vs. LDSC) . A secondary prediction here was that bad-me people were expected to show a less pronounced or even reversed gradient.

3. Within-subject correlations between aversiveness and time spent talking about a self-

characteristic (tSC ) should follow the same pattern.

4. Poor-me participants should be more likely to avoid highly discrepant self-characteristics altogether.

5. Engagement, rather than avoidance, with talking about negative self-attributes should be lower for PM participants compared both to healthy and clinical controls, when engagement is rated by investigators familiar with the concept of defensive avoidance and experienced in recognising it clinically. This involved establishing the simple scale described on page 128.

Time spent talking about self-characteristics increases with aversiveness irrespective of psychopathology

For each participant the median time spent talking about positive and LDSC was computed, t+med. The analogous quantity for HDSC, t−med, was also computed. As people may well differ with respect to the time they spend on each item for reasons other than F aversiveness, the fractional change ∆ (tmed) = (t+med − t−med)/t+med was considered. If aversive self-characteristics are avoided in a way similar to physical aversive stimuli, ∆F should mostly be a positive quantity, at least for non-depressed, non-bad-me participants. This prediction was not supported by the data. Figure 6.6 shows the distribution of ∆F for healthy and clinical participants when the Ideal-Actual discrepancy was used to gauge stimulus aversiveness. They are very similar, and the Kolmogorov-Smirnov test for the null hypothesis of these deriving from the same distribution gives P = 0.98. Table 6.6 shows the descriptive statistics derived from merging all clinical and healthy participant data. We note that no matter which measure of aversiveness was used, the mean t−med was longer than t+med by 15-25%, contrary to the hypothesis that people would spend shorter time on more aversive items. The standard error of the mean was small, but σ∆F was large, reﬂecting that some participants varied widely in this measure (cf. long ‘tail’ of distributions in ﬁg. 6.6). Descriptive statistics for this measure, as well as for experimenter-rated engagement (see below) are presented in table 6.5.

143 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE

h M

f f

o o

. .

o o

n n

a. Fractional time difference b. Fractional time difference

Figure 6.6: Distribution of change in response time with Ideal-Actual discrepancy a. Healthy control participants (median=-0.056). b. Mental health service users (median=- 0.056). The distributions are similar, with a ‘tail’ of people taking more time on more discrepant items (negative values of the fractional difference).

As the data were not normally distributed, they were log-transformed1. ANOVA showed that there were no significant differences between the groups with respect to F ∆ (tmedian); there is therefore no evidence that poor-me group was more avoidant on this measure (table 6.7). We also tested whether there was a detectable correlation between aversiveness and time spent on an item within each individual. Simple correlation coefficients were calculated between AID and response time within each participant, and the correlation coefficients were entered in a similar ANOVA as above. Again, no significant differences between the groups were detected (table 6.7). Participants vary rarely performed ‘avoidance responses’ - in fact there was no clear avoidance responses emitted by any participant. Only one behavioural avoidance response was recorded in the whole dataset, and therefore no statistical analyses were meaningful for this outcome variable.

Experimenter-rated engagement does not differ between groups

Using the four-point, observer-rated engagement scale (ORES) the experimenter rated all 2251 voice-recorded responses with respect to avoidance or engagement with the self- attribute stimulus. Responses were subdivided into low-engagement (scale score = 0 or 1)

1The Lilliefors test was used to test for deviations from the Gaussian distribution. After the transforma- tion ∆F → 1 − log(∆F ) this test failed to reject at the 0.05 level the null hypothesis that any of the three recruitment groups were drawn from Gaussian distributions.

144 6.6. RESULTS

Table 6.5: Means (& Standard Deviations) for the interactive measures pertaining to the four groups of participants. The fractional change in response time pertains to time spend answering for low AI discrepancy minus high AI discrepancy stimuli.

Healthy Clin. Control Bad-Me Poor-Me M (SD) M (SD) M (SD) M (SD) Fractional change in response time F ∆ (tmed) -0.049 (0.37) -0.12 (0.32) -0.23 (0.38) -0.26 (0.47) Experimenter-rated fraction of high engagement responses High Eng. fraction 0.76 (0.17) 0.70 (0.22) 0.73 (0.23) 0.71 (0.19)

Table 6.6: Fractional change in median response times averaged over all participants.

Discrepancy N Mean Standard Error Standard Deviation F type µ∆F of Mean ∆ σ∆F Actual-Other 75 -0.24 0.071 0.62 Ideal-Actual 75 -0.18 0.056 0.48 Ideal-Other 75 -0.14 0.058 0.50

Table 6.7: ANOVA of fractional change in response time with aversiveness, and of the correlation coefﬁcient between response time and aversiveness. Aversiveness was measured by Ideal-Actual discrepancy throughout this table.

ANOVA with dep. var.: transformed fractional change in responding time Effect P F df (factors) df (error) Paranoia group (main) 0.56 0.69 3 61 Gender (main) 0.90 0.016 1 61 Paran. gp. × Gender 0.92 0.17 3 61 ANOVA with dep. var.: within-participant time-IAD correlations Effect P F df (factors) df (error) Paranoia group (main) 0.28 1.30 3 59 Gender (main) 0.57 0.33 1 59 Paran. gp. × Gender 0.14 1.88 3 59

145 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE

Table 6.8: ANOVA of the fraction of high-engagement response during aversive trials only. There was a trend level difference for gender (women have slightly higher mean engagement)

ANOVA with dep. var.: Fraction of high-engagement responses Effect P F df (factors) df (error) Paranoia group (main) 0.67 0.52 3 60 Gender (main) 0.064 3.55 1 60 Paran. gp. × Gender 0.066 0.53 3 60 vs. high-engagement (score 2 or 3). A subset of 103 randomly selected responses was also blindly rated by Prof. R. P. Bentall, who supervised this study. The inter-rater agreement based on the scoring system above was 82%. Two-way ANOVA (Gender × paranoia group) with the proportion of high-engagement responses in aversive trials per subject as the dependent variable, and age and years-in-education as covariates, demonstrated no signiﬁcant differences (table 6.8). There was a trend for women to show more high- engagement responses.

6.6.4 Regression analyses

Analysis strategy & the effect of possible confounders

In the analyses presented here PADS-P was the primary dependent variable. Most regression analyses were multi-step, with inclusion of independent variables guided by the study hypotheses and the findings of the ANOVAs. A large number of putative explanatory variables were measured (or derived) for each participant. Some, like grandiosity, conceptual disorganisation and age were assessed so as to look for possible confounding effects if the need arose. There were, however, only a few variables which previous research indicated would predict – in the statistical sense – levels of paranoia. Of these, the level of intellectual ability, here assessed by ‘years in full-time education’ was not directly relevant to the hypotheses of the study. This variable was therefore always entered as the first step in the regression analysis, and retained1. The analyses of variance did not indicate that gender was predictive of paranoia, and gender was therefore not included in the regression analyses. In addition, as the hypotheses of the study mostly pertained to undeserved paranoia, analyses were replicated with undeserved paranoia (rather than PADS-P) as the dependent variable. The second step of the analyses concerned the role of the key measured or derived variables which we argued could reflect defensive avoidance. Here we aimed to include: • the Marlowe-Crowne score (MC), where we expected higher scores to be associated

with higher PU and PADS-P, i.e. a positive β, 1Apart from certain exploratory analyses - please see below.

146 6.6. RESULTS

• the ‘fraction of high-engagement responses to questions about aspects of the self’, where we expected a negative β,

• the fractional difference in median response times between low and high aversiveness questions.

• the ‘acceptance and action’ questionnaire (AAQ-2).

A third step aimed to examine variables, mostly symptom-based, which might explain paranoia scores better than, or instead of, the ‘psychological’ variables of the second step. If signiﬁcant relationships of the second step survived inclusion of these variables, that would strengthen the case as to their aetiological (rather than epiphenomenal) role. Here we aimed to include:

• HADS Anxiety score

• HADS Depression score

• BPRS Hallucinations

• BPRS Grandiosity

• BPRS Conceptual Disorganization

• Age

• Antipsychotic dose: total daily dose expressed as Chlorpromazine equivalent (CPZ eq.) .

Unfortunately if all the above variables of interest are entered in a linear regression, it becomes apparent that the sample cannot resolve the contributions of all of them. This is not surprising, as an oft-quoted rule of thumb is that a total sample of 75 cases should only support a regression analysis of about 7-8 variables. In our case, entering all 12 measures of interest results in a multicollinearity problem, with the Condition Index associated with the smallest eigenvalue CImin = 43.9 (Lattin et al., 2003). A second diagnostic linear regression was therefore performed, entering only the variables of the first and third steps. This allowed identification of variables that did not contribute to predicting PADS scores and thus would be unlikely to explain any effects of the (psychological) variables of interest. The result of this diagnostic regression analysis is shown in table 6.9; we can see that Age, Conceptual Disorganization, Hallucination score and antipsychotic dose (daily dose Chlorpromazine equivalent) are unlikely to explain paranoia scores. HADS-Depression was also non-significant in this analysis. Calculation of Variance Inflation Factors (VIF) reveals that VIFHADS−A = 3.35 and VIFHADS−D = 3.31, while all other variables have VIF < 2. This suggests that the two subscales of the HADS were substantially collinear, and hence that the calculation of β and significance for these variables might be unreliable. We therefore subsequently used only the total HADS score as the primary focus of

147 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE this study was not in the ﬁrst instance the resolution of the effects of anxiety vs. depression. The resulting four-variable model showed no diagnostic anomalies (such as outliers or marked heteroskedasticity) and was taken as the frame for the subsequent analyses.

Low ‘acceptance and action’ strongly predicts general and undeserved paranoia scores

Linear regression analysis with PADS-P as the dependent variable revealed that years- in-education were a strong predictor of paranoia, but the strongest one was the AAQ-2 score (table 6.10). Addition of the symptom variables brought up grandiosity as another important predictor of paranoia. These three had β"s in the expected direction – paranoia correlated with low education, low AAQ2 score and high grandiosity. The interactive & behavioural measures of defensive avoidance showed no significant correlations with paranoia; there was only a trend-level association with Marlwoe-Crowne score, but in the opposite direction to that predicted (P < 0.1; higher paranoia associated with presenting a less socially desirable image). We note that the HADS score did not emerge as a significant predictor, contrary to what might be expected from the literature. However we also note the high VIF coefficients shared by the HADS and AAQ-2. In fact the simple Pearson correlation coefficient was very high r = −0.85, P < 0.001. The same analysis was repeated with undeserved paranoia as the dependent variable (table 6.11). We note the same overall patter of results, with the interactive & behavioural measures of avoidance not explaining undeserved paranoia. The inclusion of both HADS and AAQ-2 still caused substantial variance inflation, and of course they were still highly correlated with each other. In this analysis the MC score attained significance at the 5% level, again in the direction opposite to the one we hypothesized. Years-in-education was not as strong a predictor of lower undeserved paranoia scores, as it was of lower PADS-P scores. The collinearity between AAQ-2 and HADS made it difficult to see the separate contributions of the processes that these two instruments measure to paranoia. However, and with the caveats expressed earlier, we note that controlling for AAQ greatly reduces, and renders non-significant, the simple correlation between PU and HADS: from r = 0.47, P < 0.001 to r = 0.055,P = 0.67. On the other hand, controlling for

HADS score reduces the correlation between PU and AAQ from r = −0.59, P < 0.001 to r = −0.40, P < 0.005. In this sample, therefore, the ‘acceptance & action’ process assessed via the AAQ appears to cover the inﬂuence of affect on paranoia, but to go further in accounting for paranoia.

148 6.6. RESULTS

Table 6.9: Regression analysis for auxiliary variables.

Linear Regression with dependent variable: PADS-Paranoia Model variable B Std. Err. B β Sig. level VIF (Constant) 0.29 0.20 0.16 Concept. Dis. -0.014 0.04 -0.035 0.75 1.84 Age 0.00 0.002 -0.035 0.69 1.23 Hallucinations 0.015 0.02 0.08 0.46 1.91 CPZ eq. dose 0.00 0.00 0.12 0.24 1.62 HADS-Depression 0.135 0.17 0.11 0.43 3.31 HADS-Anxiety 0.54 0.16 0.48 0.002 3.35 Grandiosity 0.08 0.03 0.255 0.008 1.40 Educ. years -0.02 0.01 -0.22 0.049 1.88 R2 = 0.6; Adjusted R2 = 0.61; Sig. level of F Change < 0.001

Table 6.10: Regression analysis - PADS-P.

Linear Regression with dependent variable: PADS-Paranoia Variables B Std. Err. B β Sig. level VIF Step 1 (Constant) 0.99 0.124 0.000 Educ. years -0.043 0.01 -0.507 < 0.001 1.000 R2 = 0.26; Adjusted R2 = 0.25; Sig. level of F Change < 0.001 Step 2 (Constant) 1.16 0.13 0.000 Educ. years -0.021 0.007 -0.253 < 0.005 1.19 MC score -0.239 0.132 -0.137 0.076 1.065 AAQ-2 -.0668 0.089 -0.623 < 0.001 1.289 F transf. ∆ (tmedian) 0.062 0.058 0.08 0.29 1.038 Hi Eng. Frac. 0.055 0.11 0.037 0.62 1.015 R2 = 0.66; Adjusted R2 = 0.63; Sig. level of F Change < 0.001 Step 3 (Constant) 0.87 0.198 0.000 Educ. years -0.018 0.007 -0.216 < 0.01 1.23 MC score -0.21 0.126 -0.122 0.097 1.07 AAQ-2 -0.59 0.158 -0.551 < 0.001 4.45 F transf. ∆ (tmedian) 0.047 0.055 0.06 0.402 1.05 Hi Eng. Frac. 0.14 0.109 0.092 0.22 1.11 HADS score 0.12 0.173 0.096 0.50 4.07 Grandiosity 0.073 0.025 0.22 < 0.01 1.23 R2 = 0.70; Adjusted R2 = 0.67; Sig. level of F Change < 0.05

149 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE

Table 6.11: Regression analysis - undeserved paranoia.

Linear Regression with dependent variable: Undeserved Paranoia Variables B Std. Err. B β Sig. level VIF Step 1 (Constant) 0.62 0.12 < 0.001 Educ. years -0.022 0.009 -0.31 < 0.05 1.0 R2 = 0.10; Adjusted R2 = 0.08; Sig. level of F Change < 0.05 Step 2 (Constant) 0.811 0.148 < 0.001 Educ. years -0.014 0.008 -0.20 0.087 1.125 MC score -0.299 0.134 -0.24 < 0.05 1.061 AAQ-2 -0.351 0.095 -0.44 < 0.001 1.222 F transf. ∆ (tmedian) 0.016 0.071 0.026 0.822 1.123 Hi Eng. Frac. 0.061 0.117 0.056 0.607 1.05 R2 = 0.39; Adjusted R2 = 0.33; Sig. level of F Change < 0.001 Step 3 (Constant) 0.66 0.22 < 0.05 Educ. years -0.013 0.008 -0.18 0.13 1.217 MC score -0.29 0.13 -0.24 < 0.05 1.072 AAQ-2 -0.29 0.17 -0.36 0.10 4.114 F transf. ∆ (tmedian) 0.012 0.072 0.019 0.87 1.149 Hi Eng. Frac. 0.1 0.12 0.093 0.42 1.146 HADS score 0.088 0.19 0.096 0.64 3.701 Grandiosity 0.032 0.026 0.146 0.23 1.249 R2 = 0.39; Adjusted R2 = 0.33; Sig. level of F Change = 0.47

150 6.6. RESULTS

Exploratory analyses

In addition to the above regression analyses, which were based on the study hypotheses and on the results of the analyses of variance, variants of linear regression analysis were performed to explore two additional issues. The first was to make a preliminary exploration as to whether variables over and above the ones we hypothesised about explain paranoia. The second was to further understand the nature of the AAQ-2. As these were exploratory analyses stepwise variable entry was used, with probability of variable entry 0.05 and variable removal 0.10. The first analysis used as candidates for entry all the psychological variables of interest, as well as the key affective variables (HADS-A, HADS-D and grandiosity). The former included not only the measures of avoidance, but also the Actual-Ideal and Actual- Other discrepancies. The stepwise regression chose four variables as significantly predictive of PADS-Paranoia scores. These were the ones above (AAQ-2, grandiosity, years- in-education) but also Other-Ideal discrepancy. However when undeserved paranoia was used as the dependent variable, neither years-in-education nor OID were significant predictors but Marlowe-Crowne score was (albeit in the ‘wrong’ direction, as above; table 6.12). It is of note that the adjusted proportion of variance explained as far as PADS-P is concerned was about twice as great as the proportion of undeserved paranoia similarly explained (0.67 vs. 0.33).

AAQ-2 scores are highly predicted by paranoia and affective symptoms

The AAQ-2 score was set as the dependent variable, with candidate independent variables: PADS-Paranoia, HADS total, grandiosity, AI discrepancy, OI discrepancy, Marlowe-Crowne score, high-engagement response fraction, fractional difference in response time and years-in-education. The entry and exit criteria were as above. The object of this exploration was to examine whether psychological and symptom variables might independently contribute to AAQ-2 scores. The analysis showed that only two variables were sufﬁcient to explain a very large proportion of the variance in AAQ-2 scores, namely HADS total and PADS-P (table 6.13).

151 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE

Table 6.12: Exploratory regression analysis - PADS-P. Only the ﬁnal model is shown, after the stepwise algorithm has converged.

Stepwise Linear Regression with dependent variable: PADS-Paranoia Variables B Std. Err. B β Sig. level VIF (Constant) 0.84 0.108 < 0.001 AAQ-2 -0.61 0.09 -0.57 < 0.001 1.61 Grandiosity 0.063 0.023 0.19 < 0.01 1.04 Educ. years -0.019 0.001 -0.22 < 0.01 1.24 Other-Ideal Disc. -0.002 0.001 -0.17 < 0.05 1.38 R2 = 0.69; Adjusted R2 = 0.67 Stepwise Linear Regression with dependent variable: Undeserved Paranoia (Constant) 0.69 0.08 < 0.001 AAQ-2 -0.40 0.088 -0.50 < 0.001 1.05 MC score -0.28 0.13 -0.23 < 0.05 1.05 R2 = 0.35; Adjusted R2 = 0.33

Table 6.13: Exploratory regression analysis - AAQ-2. Only the ﬁnal model is shown, after the stepwise algorithm has converged.

Stepwise Lin. Regr. with dependent variable: AAQ-2 Variables B Std. Err. B β Sig. level VIF (Constant) 0.95 0.027 < 0.001 HADS total -0.70 0.08 -0.61 < 0.001 1.68 PADS-P -0.35 0.065 -0.37 < 0.001 1.68 R2 = 0.81; Adjusted R2 = 0.80

152 6.7. DISCUSSION

6.7 Discussion

6.7.1 Poor-me paranoia is not due to increased avoidance of self-discrepant thoughts

This study used questionnaire measures as well as an experimental procedure to assess defensive avoidance in paranoia. It utilised one healthy-control & three clinical groups of participants and was reasonably well tolerated. ANOVA demonstrated that Actual- Ideal and Other-Ideal discrepancy scores differentiated between the paranoid and healthy groups, but Actual-Other discrepancy did not. Self discrepancies were largely correlated with affective (anxiety & depression) measures. BM paranoia was characterized by the greatest OID scores. Paranoia scores were weighed by deservedness to yield an index of undeserved paranoia (PU ). High levels of PU did not confer protection from higher AID for a given level of OID. Exploratory stepwise regression indicated that greater OID also predicted overall paranoia (β = 0.17). ‘Acceptance and action questionnaire’ scores were highly predictive of paranoia levels but did not differentiate between poor-me and bad-me types. AAQ-2 scores were also the best predictor of paranoia, overall or undeserved, in linear regression analyses. Marlowe-Crowne social desirability scores did not differentiate between groups in

ANOVA, but higher MC scores were a signiﬁcant predictor of lower PU in linear regression. The adjusted variance of PU explained by the best linear model was approximately half that explained for PADS-Paranoia. Regression analyses also showed that fewer years- in-education, here used as a measure of lesser intellectual ability, were an important predictor of paranoia. Higher levels of grandiosity also predicted overall paranoia. AAQ-2 scores were themselves highly predicted by a combination of HADS and PADS scores. In the experimental intervention, higher paranoia scores were neither associated with more avoidant, shorter responses, nor with more altogether-avoided aversive questions, nor with lower observer-rated engagement with aversive thoughts about the self. On average participants spent longer answering questions that dealt with more aversive potential aspects of themselves. However participants varied considerably in this measure, irrespective of psychopathology.

6.7.2 Interpretation of the results

The finding that a negative view of self closely accompanies paranoia replicates a number of existing experimental findings (e.g. Bentall et al., 2009). The present study extended these findings in that self-discrepant evaluations were assessed by means of a computerised version of the ‘self-lines’ questionnaire. The finding that AID was also strongly coupled with affective variables, especially HADS-Depression, also replicated existing findings (Kinderman & Bentall, 1996).

153 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE

The finding that levels of affective symptoms, both in terms of anxiety/depression and of grandiosity, were associated with paranoia is also consistent with the literature. Despite the statistical association with grandiosity, levels of grandiosity were in general very low as care was taken to exclude participants that might be experiencing a hypomanic or manic episode. This was because it has been suggested that psychological defensiveness found in paranoia may be due to grandiosity which coexists with the paranoia (Jolley et al., 2006). The present study of discrepancies between how one is seen by others versus one’s ideals, or versus how one actually evaluates oneself (OID and OAD) was based on the findings of Kinderman and Bentall (1996) but only achieved limited replication. The present study also found similar levels of AID between non-depressed paranoid and ‘healthy’ participants (table 6.4); It also found some evidence that OID is predictive of paranoia (table 6.12), especially of the bad-me kind (table 6.2). Here, however, OAD did not discriminate between any groups of interest; in contrast, the above authors found that high Self-actual:Parent-actual (and Self-actual:Parent-ideal) discrepancies characterised their paranoid group. If we take both studies at face value, Self-actual:Parent-actual/ideal conflicts do not apply to paranoid patients’ representation of the self and others in general. The aetiological significance of the self – parent discrepancies thus awaits further evaluation. The fact that low ‘acceptance and action’ scores were the best predictor of paranoia is also consistent with recent research (Udachina et al., in submission). As the AAQ-2 was meant to give a global measure of experiential avoidance, this provides evidence that participants who experienced paranoia engaged in much experiential avoidance. The finding that this was not modulated by deservedness was in the first instance surprising, as PM participants were expected to avoid unpleasant private experiences, while BM patients not to do so. However closer inspection of the AAQ-2 suggests that it uses perceived psychological impairment (worrisome or feared aversive private experiences believed to cause impairment) as a proxy for avoidance. This approximation may well not hold in the case of BMP. Here conscious negative private experiences are distressing but psychological disability may be due to non-accepting preoccupation, not due to efforts to avoid. The AAQ-2 scores showed substantial collinearity with HADS scores (table 6.10) and were highly predicted by a combination of PADS and HADS scores. The most parsimonious explanation is that AAQ-2 scores substantially reflect symptom severity per se. The finding that high undeserved paranoia does not offer relative protection against a negative self-evaluation runs against the study hypothesis. We expected poor-me patients to see themselves more in line with social standards and demands, at least when explicitly asked, as they would generally have less negative views of themselves. We expected also that defensive avoidance would largely operate outside conscious awareness, as advocated both by psychoanalytic and learning-theory accounts. We therefore expected Marlowe- Crowne scores to be high (self-deceptive positivity). This was not the case: low MC

154 6.7. DISCUSSION scores were one of the only two psychological predictors of undeserved paranoia. There is no single interpretation of this result. Poor-me patients may perceive relatively low standards for social behaviour, so they give themselves relatively low MC scores, while considering others to be unreasonably harsh in areas where they have got into conflict. Alternatively, low MC scores could be expressive of distress associated with undeserved paranoia. A lower number of years-in-education was associated with overall paranoia. Intellec- tually more able people may more flexible in their attributions, keeping in mind several potential causes for the conflicts they experience. Alternatively they may live, in general, in a more powerful, autonomous social stratum and may have encountered less expression of other people’s bad intentions. They may thus have weaker priors for threat-beliefs. In addition, mental disorders often limit academic performance (e.g. causing one to drop out of education) and are correlated with paranoid ideation. The results of the experimental part of the study also ran contrary to the study hypotheses. We first observed a lengthening, rather than shortening, of answers given to a request to think about more negative self-statements. There was no indication at all of learning to avoid exposure to negative self-statements as if they were physical noxious stimuli (cf. electric shocks). The fact that modulation of the response duration with aversiveness was observed militates against the possibility that our stimuli were all more or less as aversive as each other. On the contrary, it argues for processes other than straightforward avoidance to have predominated. First, at a cognitive level, thinking about more negative potential aspects of the self may be cognitively more taxing, more effortful. Sec- ond, the negative affect involved may have recruited the Behavioural Inhibition System (Gray, 1987), making both thought and speech more difficult. The observer ratings of engagement versus avoidance did not differentiate between groups, nor did they predict levels of paranoia. Again this stood against the study hypothesis, which was that paranoid participants would engage in a variety of verbal strategies, such as talking vaguely or about peripheral matters, denying that the negative self- characteristicts in question might possibly apply to them etc. more often than controls. We had also expected bad-me and depressed subjects to engage strongly with negative self-characteristics but to do so in an overly self-blaming manner, possibly turning the hypothetical questions (“What would it mean for you if you appeared X”) into concrete ones. Numerous instances of such strategies were observed, but their distribution did not significantly differ according to psychopathology. We note that ratings were made blind to the levels of paranoia and deservedness, although the recruitment group of the participants were known to the principal rater (Dr Moutoussis). Overall, the evidence from this study stands against the hypothesis that greater avoidance of negative thoughts about the self motivates paranoia, or that poor-me paranoid patients engage in more defensive avoidance than bad-me patients. In addition, the evidence from this study points towards an important differentiation whereby efforts at con-

155 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE scious avoidance and control of distress are reported by patients (consciously, through the AAQ-2) but there is no evidence for increased defensive avoidance (MC, Discrepancies, response-times, observer-ratings). This holds from either a learning-theory or a psychoanalytic perspective. There was, for example, no hint of increased OAD in the PMP group, a strong expectation on the basis of the projection theory of paranoia.

The possibility of false-negative ﬁndings

A number of factors may explain why different findings in this study did not support its hypotheses, even if defensive avoidance does indeed cause a heightened sense of feeling threatened (paranoia). First, it could be that the procedures used in this study did not reliably assess defensive avoidance. As far as the AAQ-2 is concerned, “experiential avoidance’ is a term that describes a whole class of strategies to deal with unpleasant private experiences. It refers to efforts to control negative thoughts, unpleasant memories, images, emotions – in fact, any kind of personal perception – by avoiding it or controlling it. This stands in contradistinc- tion to ‘allowing oneself’ to experience such private experiences. The AAQ-2 enquires about experiences to avoid or control such experiences, without asking about their extent, or indeed about the extent of relevant experiences that are accepted and do not stand in the way of adaptive action. As an example, one AAQ item is “My painful memories prevent me from having a fulfilling life”. This carries a completely different meaning depending on how fresh, how prominent and how traumatic the memories in question might be. These issues are not that important if the AAQ-2 is used to judge how one copes with perceived adversity for the purposes of therapy, where intra-individual change is required and the perception of adversity can be taken as a starting point, but they are important for scientific enquiry where very different social contexts and experiences need to be taken into account. More importantly, the AAQ-2 may have failed to show differences between the poor- me and bad-me group because of a type II error. Indeed, the PM group contained 27 participants while the BM only 13. The fact, however, that the AAQ does distinguish between the N = 23 depressed-paranoid and the N = 17 nondepressed paranoid participants (table 6.4) with the depressed paranoid group having the most avoidant scores renders this less relevant. The Marlowe-Crowne scale is not specifically designed to assess negative thoughts about the self, but social desirability as a whole. The argument that self-deceptive positivity can be assessed with this scale in clinical populations is plausible but untested. It could be argued that the timing measures depend on judgements made by both the participants and the experimenter (who pressed the timing button), and that uncertainty in these may obscure the results. A theoretical possibility exists that participants were so defended and avoidant that they ‘saw through’ the study and avoided bringing forward to the self-lines part items that

156 6.7. DISCUSSION were truly discrepant. However the high correlation of the discrepant items offered with the affective measures in the study strongly argues against the possibility that trivial or inoffensive items were offered. It might be that the relatively high inter-rater reliability in the measure used (the proportion of high-engagement vs. low-engagement answers) is falsely reassuring, as the two raters have collaborated for a number of years. Their possible misunderstandings as to what constitutes defensive avoidance may thus be correlated. It could be that personal factors and biases have contaminated the results. Within a psychoanalytic framework, the unconscious of the investigator may determine what they ﬁnd. Investigators might harbour unconscious aggression against the object that the hypothesis represents and thus sabotage it. It might also be argued that the containing psychological structures of the study process may have contributed towards the negative results. The reassuring approach, the safeguards given, the care with which the stimuli were worded, the neutral, interested and therapeutic in quality contact with the experimenter, the moral and monetary reward given, the conﬁdential, boundaried framework, may all have produced a setting which lowered – and hence hid – psychological defensiveness.

Study strengths

Despite these possibilities, on the grounds of consistency, parsimony and overall plau- sibility the results of this study should not be dismissed. First, results obtained by means of the AAQ-2 and the mood scales are consistent with the literature. Second, several of the self-discrepancy ﬁndings are also consistent with the literature, although, as described above, some differ in important ways. As far as personal and possibly unconscious factors are concerned, the study was discussed both at the design and during the execution stage in an on-going manner with a variety of senior clinicians. Several of these were psychotherapists of different ori- entations, including psychoanalysts, who found the design appropriate for investigating defensive avoidance. In addition, both Dr Moutoussis and Prof. Bentall are qualiﬁed specialist clinicians with many years of experience in observing and working with defensive behaviours in mentally ill patients clinically. Part of Dr Moutoussis’ training has been many years of insight-oriented psychotherapy. An important strength of this study was the idiographic approach to eliciting self- discrepant thoughts. This, together with exploratory nature of the stimuli exposing the participants to the aversive self-thoughts, the free speech format of the responses and the assessment of response content by experienced clinicians lends considerable ecological validity to the study. The most important reason, however, to take the negative results of this study seriously is that they fall into a clear and consistent pattern – with good evidence for the recruitment of avoidance and control strategies at a conscious level and a variety of measures

157 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE furnishing no evidence for increased nonconscious defensive avoidance.

6.7.3 Limitations of the present empirical study

Important limitations of the current study include:

1. Relatively small numbers of bad-me paranoid participants. In this study this was likely to be due to the effort to match the levels of clinical impairment, and may thus have been important in creating a false negative result, especially with respect to the AAQ-2 scores, in the bad-me group.

2. Small number of clinical control participants with levels of threat similar to the healthy controls.

3. About half the patients invited to participate declined to do so, introducing (self-) selection bias.

4. Several participants complained that the negative-self-thoughts task was repetitious. In fact most of the time there were differences between the question-stems presented, but despite reading them aloud many participants appeared to focus very much on the negative characteristic presented, rather than consider the question- stem that set the context. Observer-ratings of avoidance took this into account, but any future versions on the task should improve on this.

5. Despite the idiographic character of the self-thoughts, they did not make immediate contact with the morbid threat-beliefs, especially paranoid delusions, which the clinical participants had. Defensive avoidance may not relate to multiple important aspects of the self, as elicited in this study, but be specific to the central paranoid beliefs. If we use the conditioned avoidance analogy, the ‘subjects’ may learn not only that the CS predicts the US, and hence avoid it, but that the CS within a specific context, within a spcific ‘shuttle box’, predicts the US.

6. There was substantial collinearity between AAQ-2 and HADS scores. This impaired the assessment of how acceptance/EA on the one hand and depression/anxiety on the other simultaneously relate to paranoia (see table 6.11 for an example).

6.7.4 Future directions

We would advocate the routine introduction of all mental health service users to the possibility of participating in research to reduce selection bias. On a technical level future research should clarify what the best dimensional measures of deserved and underserved paranoia are, possibly building on the measure introduced here. This is important as the current study showed that the explanatory factors we used, including the AAQ-2, only explain a modest amount of the undeserved paranoia score.

158 6.8. EMPIRICAL INVESTIGATION OF DEFENSIVE AVOIDANCE: CONCLUSIONS

This suggests that the core processes underlying undeserved paranoia have not been well understood, and that its measurement is important. Each person, after all, may feel threatened both in relation to their own perceived characteristics (e.g. if they fear they may be naive or slow) and because of how they see others are (e.g. as deceitful), in a dimensional manner. There is an urgent need to examine the psychological mechanisms involved in paranoia as close to their native context as possible. Building on the idiographic approach of this study, future studies should look at psychological functioning in paranoia in the context of idiographically important threat-related preoccupations. Such a context should include not only references to the content of these beliefs, but also eliciting of the relevant threatened affect. It would be more helpful if a questionnaire measuring experiential avoidance enquired separately about the psychosocial context and the strategies used to deal with it. The heterogeneity of the AAQ-2 (Shallcross, Troy, Boland, & Mauss, 2010) and the multicollinearity problem may be alleviated by this. Udachina, Varese, Oorschot, Myin-Germeys, and Bentall (personal communication) recently observed a pattern of low ‘acceptance and action’ in paranoia, combined with lack of evidence that it conferred relief from negative self-evaluations, analogous to the present ﬁndings. They asked why people should engage in such strategies, if they offer no substantial beneﬁt to the self-esteem. Future research should examine if the employment of such strategies is the result of an interaction between how one sees oneself and how one judges others’ intentions within the context of the person’s current social standing. Most importantly, future research should ask what the optimal, normative way of perceiving possible threat is and what the optimal way of evaluating oneself is in that context.

6.8 Empirical investigation of defensive avoidance: Conclusions

In conclusion, it first appears that experiential avoidance is an important feature of paranoia, as paranoid patients were found to have the lowest ‘acceptance and action’ (AAQ-2) scores. This lends some support to the idea that increased psychological defensiveness operates in paranoia. However the attribution-theoretical prediction that poor-me patients would show higher EA was not upheld, which casts some doubt on the attribution- theoretical account of the differences between the two types of paranoia. Several study findings, however, detract from the credibility of the paranoia-as-defence hypothesis in both its psychodynamic (unconscious projection) and cognitive (attributional) forms. On the basis of the paranoia-as-defence hypothesis, Other-Actual discrepancy was expected to be greater in the poor-me group, but the data did not support this. Poor-me paranoid patients were also expected to have relatively preserved social desirability scores. However a trend-level significant correlation in the opposite direction was

159 CHAPTER 6. EMPIRICAL STUDY OF DEFENSIVE AVOIDANCE found, the poor-me group having the lowest mean social desirability. Similarly, the proportion of clinically avoidant (observer-rated) answers did not predict paranoia of any type. Against the study hypotheses, participants gave longer answers when considering more aversive self-characteristics, irrespective of their persecutory ideas. Conditioned avoidance-like mechanisms are thus unlikely to underpin psychologically defensive paranoid thinking. The longer response times to more aversive stimuli may be explained by an a greater ‘affective/cognititive load’ or by behavioural inhibition brought on by such stimuli. These conclusions have to be considered in the light of the limitations of this study, especially with respect to (i) participant selection bias, (ii) the overlap between experiential- avoidance and affective measures and (iii) the lack of idiographic analysis of the participants speciﬁc threat-beliefs. However the consistency of the overall results suggests that defensive avoidance is unlikely to make a major aetiological contribution to paranoia, though it may still be an important impediment to recovery.

160 CHAPTER SEVEN

DEFENSIVE AVOIDANCE IN PARANOIA: SYNTHESIS AND GENERAL DISCUSSION

7.1 Summary

This thesis investigated the role of avoidance in paranoid delusions. The clinical psychological, animal-behavioural and neuropharmacological evidence prima facie suggested that paranoia is associated with a generally heightened perception of threat. It also suggested that both overt behavioural avoidance and avoidance of negative thoughts may contribute to the stabilisation of paranoid ideas. The evidence suggested that the conditioned avoidance paradigm is more informative for the understanding of the psychology of paranoia than had hitherto been thought. However the ability of dopamine antagonists to quell paranoia appeared puzzling, as the involvement of dopamine in processing aversive information has been much disputed (Chapter2; Moutoussis et al., 2007). Overall, a heightened perception of threat appeared so prominent in paranoia that it might contribute to cognitive biases found in such patients, especially the ‘Jumping to Conclusions’ bias. As major contemporary accounts of threat-related learning appeared inconsistent with the neuropharmacological evidence, detailed modelling of the conditioned avoidance paradigm, with and without dopaminergic modulation, was undertaken. This revealed ﬁrst, that a temporal-difference, advantage-learning implementation of the two-factor theory was a minimal account consistent with the data. Second, it revealed an important asymmetry in the roles dopamine plays. Dopamine must have different roles in learning expectations about the environment itself, versus learning what to expect from a chosen action. Thirdly, the persistence of avoidance in the absence of fear could be well explained by a lack of cumulative experience of the different outcomes associated with different actions. Overall the model explained many aspects of the data but offered no suggestions as to why paranoid delusions appear resistant to experiencing dis-conﬁrming evidence (Chapter4; Moutoussis, Bentall, Williams, & Dayan, 2008). Could it be that seeing high potential costs where none exist explains the propensity of patients with paranoid delusions to give precipitous answers in cognitive tasks (Chap- ter5)? Detailed modelling of their answers in the ‘Jumping-to-Conclusions’ task did not

161 CHAPTER 7. DEFENSIVE AVOIDANCE IN PARANOIA: SYNTHESIS AND GENERAL DISCUSSION support this intriguing hypothesis. An ideal-observer Bayesian model was ﬁtted to the data obtained by Corcoran et al. (2008). This was compared with a cognitively less demanding, threshold-based decision making process, one that the literature on the JTC bias often refers to. As expected, the decision making of paranoid participants deviated more from the Bayesian ideal than that of never-paranoid participants. However, and contrary to the ‘high cost hypothesis’, a subjective perception of exaggerated costs in the paranoid participants did not account well for their precipitous decisions. Instead, an increased ‘cognitive noise’ in these participants explained their hastiness (Chapter5; Moutoussis, Bentall, El-Deredy, & Dayan, 2011). A detailed empirical study was then undertaken to discover whether avoidance of negative perceptions of the self fuelled paranoia, as might be expected from an attribution- theory, a psychodynamic or an ACT 1 standpoint (Chapter6). We hypothesised that poor- me paranoid patients would show the highest levels of avoidance, whereas bad-me patients might tend to engage in ruminations about important negative aspects of the self. Poor-me patients were also expected to show a relatively preserved overt view of the self, both in terms of discrepancies from their ideals and in terms of social desirability. It was found that negative thoughts about the self are not avoided by paranoid patients, neither in a way analogous to avoidance in the CAR nor according to clinical-psychotherapeutic criteria. In addition, neither paranoid patients in general nor the poor-me subgroup display any relative preservation of self-image. Only non-depressed paranoid participants show relatively normal self-discrepancies. However, both poor-me and bad-me patients showed very low self-reported tolerance of negative mental contents, consistent with high levels of experiential avoidance.

7.2 An integrative approach to paranoid delusions

This set of studies followed an integrative approach, rather than a purist or an eclec- tic one. It brought the broadest practicable spectrum of evidence to bear on whether exaggerated use of avoidance mechanisms accounts for unwarranted threat beliefs (paranoid delusions). Mathematical modelling was a key methodology used to integrate the evidence2. At ﬁrst glance, the models used are of peripheral concern: neither the conditioned avoidance nor the beads-in-a-jar paradigm have much to do with beliefs such as “Darth Vader is after me”. However the central task here is to establish if avoidance is a key general principle underlying such beliefs. A powerful way to put such a principle to the test is to - 1. Consider limiting cases where key mathematical approximations can be used. The choice of these limiting cases is of crucial importance, as they should retain the essence of the principle studied yet make large simpliﬁcations. 1Acceptance and Commitment Therapy (Hayes et al., 1999). 2This paragraph is adapted from the author’s contribution to the published article: Huys, Q., Moutous- sis, M., & Williams, J. (in press). Are computational models of any use to psychiatry? Neural Networks.

162 7.2. AN INTEGRATIVE APPROACH TO PARANOID DELUSIONS

2. Focus experiments at these computationally tractable cases; the quantitative results of the experiment will then provide evidence not just about a qualitative hypothesis but about a rigorous relationship between key variables.

3. Rigorously relate the mathematical description of limiting, tractable cases to the putative general principles.

In the physical sciences such a methodology led to the Galilean revolution and the establishment of powerful general principles, e.g. that free-fall motion is the same for all objects. In mental health, our best guess is that the key principles involve information processing and probabilistic inference. Information processing is a computational concept. Hence computational models, more than any other approach, allow us to relate findings to information-processing principles. Crucially, this approach also links psychopathology to normal psychology and neuroscience (Huys, Moutoussis, & Williams, in press). When it comes to defensiveness and delusions, much of the debate has been formulated around whether persecutory beliefs are (a) defensive of self-representation or (b) a direct reflection of emotion (e.g. Bentall et al., 2008; Smith, Freeman, & Kuipers, 2005). The present integrative, modelling-informed approach not only helps to link up the affective, neuroscientific and cognitive aspects of decision-making relevant to paranoia; it also points towards a rigorous, normative reformulation of the issue of self-representation itself: What kind of self-representation would optimally assist to detect and deal with social threats? (cf. Haselton & Nettle, 2006). Such a change in persepective may help transcend the ambiguities of the ‘defensive vs. expressive role’ debate.

7.2.1 Avoidance and self-discrepancies in paranoia

Some crucial components of the ‘defensive avoidance’ hypothesis passed the test of our temporal-difference analysis, some components remained unexplained and some con- tradictions worthy of further study emerged. Most straightforwardly, modelling supported the hypothesis that dopaminergic antipsychotics may work by dampening incentive motivation to actively avoid aversive outcomes – not ‘inappropriate salience’ (Kapur, 2003) in general. It is most interesting in this respect that the research group that proposed the inappropriate salience hypothesis more recently tested people with schizophrenia and healthy controls using a classical-conditioning aversive paradigm (Jensen et al., 2008). They found that people with schizophrenia showed increased responses to neutral stimuli, both in terms of BOLD response in the ventral striatum and in terms of GSR, leading them to subjective difﬁculty in distinguishing the predictor of the aversive stimulus from the predictor of the safe state (CS-). It is remarkable – and also an important confound – that the patients involved received stable therapeutic doses of antipsychotic medication and displayed moderate levels of symptoms. In terms of our TD model the increased response to CS- may involve the dopamine-independent component reporting aversive predictions, here being inappropriately overactive. Unfortunately this is not the only in-

163 CHAPTER 7. DEFENSIVE AVOIDANCE IN PARANOIA: SYNTHESIS AND GENERAL DISCUSSION terpretation, as additional controls and larger participant numbers would be required to firm up such a conclusion. Many intriguing research questions and implications arise (cf. section 4.5.2), but here we applied the lessons of the CAR to the verbal and interpersonal realm to specifically study paranoia. Leaving the study of overt avoidance aside, could internal states, especially thoughts, act as warning stimuli? Could they lead to internal, covert avoidant actions that bolster paranoia? The attributional model (Bentall et al., 1994; Bentall et al., 2008) and the psychodynamic one (Segal, 1994) would affirm so. The CAR itself is, of course, too limited to say anything on this. We therefore performed a multifaceted empirical study whose design was informed by the CAR to examine this possibility. It had several further strengths:

1. It combined a cross-sectional and an interventional/experimental approach to examine whether avoidance of negative thoughts about the self contributes to paranoid ideation. That is, it didn’t just examine whether measures of avoidance and measures of negative self-thoughts show increased correlation in people with paranoia, but whether exposure to negative thoughts is responded to by increased avoidance in paranoia.

2. The probes used had high clinical validity and relevance. Aversive thoughts about the self which were important to each individual participant were elicited. Open- ended questions about them were asked that directly related to widely adopted clinical approaches. In addition, several senior psychiatrists and psychoanalytic practitioners were consulted to ensure that the design and probes reﬂected these clinical approaches reasonably.

3. The outcome variables assessed proximal measures of defensive avoidance (response times, content of answers) as well as proximal measures of self-image (self- discrepancies), as opposed to more distal potential effects, such as self-esteem, whose measures may be confounded by many other inﬂuences.

4. A wide variety of possible confounding variables were assessed, so as to examine whether any positive ﬁndings may be explained by their presence. Admit- tedly changes in paranoia were not measured subsequent to avoidance. If exposure to negative thoughts were found to cause avoidance in paranoid participants, this might still be interpreted as an effect, rather than a cause, of paranoia.

In view of the above considerations, the consistent absence of any indication that increased covert or unconscious avoidance was associated with paranoia was significant. Though one or some of the criteria used (increased actual-other discrepancies; more frequent behavioural avoidance; shorter response times to more aversive thoughts; or observer-rated avoidant answers) might be flawed, the overall pattern is clear. The absence of behavioural avoidance is easy to interpret: it appears more fitting to resort to

164 7.2. AN INTEGRATIVE APPROACH TO PARANOID DELUSIONS verbal/cognitive processes in response to negative thoughts. In addition, once commit- ting themselves people are remarkably obliging towards experimenters. The other results, however, cannot be similarly explained. We can conclude that threat-beliefs are unlikely to be adopted mainly because of their promise to help avoid negative self-attributes. These findings are also in line with those of the Jumping-to-Conclusions (JTC) study, which also failed to detect any motivational bias (avoidance of high implicit, misperceived costs) in the deluded patients. Paranoid patients are not ‘paranoid’ about gathering more information. In accordance with the CAR study, and contrary to clinical intuition, the JTC task reveals no hints as to the provenance of ‘delusional fixity’ in paranoia (Fine et al., 2007), whether motivationally based or not. Although our analysis of the JTC bias is novel, and our results stand against a motivational explanation for this bias, some recent studies support our analysis and suggestions for further research. In the structural equation analysis of Bentall et al. (2009) themselves, for example, the JTC bias was found to contribute to the cognitive rather than the affective component of paranoia. Lincoln et al. (2010) also observed a correlation of JTC with cognitive abilities and started to include explicit costs in the task, as the present study also naturally suggests. In a separate study Lincoln, Lange, Burau, Exner, and Moritz (2010) demonstrated that experimentally induced state anxiety worsens JTC as well as paranoia in susceptible individuals. This result is consistent with a threatening affect increasing cognitive noise, as discussed in section 5.5.2. However, using mediation analysis these authors showed that the association between induced anxiety and increased paranoia was statistically mediated by increased JTC. They interpreted this to mean that ‘threat related thoughts, triggered by state anxiety, will be more rapidly accepted in the presence of JTC’. It would be most interesting, in the light of the present study, to investigate whether increased state anxiety simply increases noise or in addition distorts cost perception – in which case fixity as well as precipitousness of paranoid explanations might be observed. As far as defensive avoidance is concerned, modelling the JTC highlighted the crucial role of quantifying internal variables that cannot be observed directly. It clarified that statements about motivation need to be qualified relative to an appropriate yardstick. Postulating that an ‘internal action’, such as drawing a conclusion, is taken because the alternative would be ‘too costly’ is meaningless unless the costs involved can be independently estimated, especially with respect to their motivational impact. In contrast to the absence of evidence for implicit, covert or unconscious avoidance, the present study found clear evidence for the role of self-reported, more overt avoidance in the low ‘acceptance and action’ scores of both paranoia groups. The fact that this held regardless of poor-me / bad-me status argued against avoidant-defensive processes pushing people into the poor-me state. We note here that the bad-me group only contained 14 participants, raising the possibility of a type 2 error; however, the mean AAQ-2 scores were identical between the two groups. To the extent that AAQ-2 scores reflect defensive avoidance, this specific finding does not contradict a psychoanalytic account of paranoia:

165 CHAPTER 7. DEFENSIVE AVOIDANCE IN PARANOIA: SYNTHESIS AND GENERAL DISCUSSION from a psychoanalytic point of view the defensive style is different in the two groups, but the avoidance of suffering associated with the ‘depressive position’ is equally in place (see page 32).

With respect to experiential avoidance this study replicated the recent results of Udachina et al. (2009) in a nonclinical population, which was then extended to a clinical paranoid sample (Udachina et al., personal communication, in submission). They used an Experi- ence Sampling methodology, repeatedly administering a slightly modified subset of items from the AAQ-2 to measure EA. The items were: “Since the last [time I was asked] my emotions have got in the way of things which I wanted to do”; “Since the last [time I was asked] I’ve tried to block negative thoughts out of my mind”; and “Since the last [time I was asked] I’ve tried to avoid painful memories”. They found that EA was a powerful predictor of paranoia, not only statistically but also temporally (with some non-trivial caveats). However, contrary to their expectations they did not find evidence of any func- tionally useful avoidance, i.e. EA associated with successful protection of self-esteem. Parenthetically, this is also consistent with the idea that this short measure of EA, as well as the full AAQ-2 used here, may conflate having strong aversive experiences (negative emotions and thoughts, painful memories) and using control strategies to deal with them. In summary, having a low tolerance of negative mental contents (as measured by the AAQ) is not the same as having an effective strategy to avoid them.

The present results say very little about the possibility that poor-me (or maybe non- depressive) paranoia may have a protective effect on self-discrepancies, or on self-esteem. Still, some important studies in this area need to be taken into account in interpreting the present results. Kinderman et al. (2003), for example, observed an improvement in self-actual:self-ideal discrepancies in paranoid patients with relatively low BDI scores after exposure to threat words. Udachina et al. (in submission) observed that in poor-me patients paranoia scores predicted an increase in self-esteem between consecutive mea- surements, while in bad-me patients there was a corresponding decrease. One reconciling explanation may be that non-depressed and/or poor-me patients may respond with more active, ‘ﬁght – like’ thoughts in response to threat, whereas depressed and/or bad-me patients respond with more ‘learned helplessness – like’ thoughts (cf. Fornells-Ambrojo & Garety, 2009). The former would be associated with images of the self as empowered, whereas the latter with images of uselessness, thus having opposite effects on self-esteem. Such effects do not need to be based in defensive avoidance but may arise independently. This last claim is also compatible with the qualitative analysis of Campbell and Morrison (2007). In that study most people with paranoia saw themselves more negatively, e.g. as vulnerable or strange, because of their paranoia. Some of the clinical participants only, however, talked about paranoia having a positive effect on their self-esteem and some thought that it offered a unique perspective on life. Clinical participants also described more intense anger. The above ﬁndings and those of the present study would be compatible if the effect of paranoia on self-esteem rather than be determined by defensive

166 7.2. AN INTEGRATIVE APPROACH TO PARANOID DELUSIONS avoidance depended on an individualistic appraisal of what being under threat meant for each person.

7.2.2 The role of self-esteem in paranoia

In this thesis proximate and behavioural correlates of defensive avoidance were examined, rather than higher-level perceptions like self-esteem. This is because the ultimate effects of defensiveness on self-representation may be subject to complications that could spoil the ultimate success of the defensive manoeuvre. As an example, concluding that a relative has hostile intent might imply that one is less at fault, but also less lovable. Nevertheless several careful studies in this area offer challenges to the findings presented here, whereas others appear consistent with them. Many studies have assessed the explicit, consciously reported self-representation of people and have related it to paranoia. In different studies the self-representation assessed has included questions about the over-pessimistic (and over-optimistic) view of the self found in depression (or grandiosity), questions about a singular, overall concept of self- esteem, about separate positive and negative aspects of self-esteem, about self-esteem as derived from self-discrepancies, or about a self- (and other- ) representation in terms of core schemas. A broad-brush summary would be that in both healthy (e.g. Green et al., 2011) and clinical (e.g. Bentall et al., 2009; Thewissen et al., 2010) samples paranoia is associated with a more negative view of the self. The clinical extreme of this spectrum is bad-me paranoia, which from a cognitive-behavioural point of view represents either an un(der)defended state or one where the defences against low self-esteem have failed. This picture is consistent with the findings of the present study. Here we note that if defensive avoidance were indeed the motive behind paranoid ideation in healthy people, an explanation would be required as to the apparent failure of defensive function in this group (and also why people would choose it as a defence). An alternative, normative explanation which does not involve self-esteem directly is that in healthy people paranoid ideas are a ‘sinister attribution error’ (Kramer, 1994) that serves to manage false-negative errors in the detection of social difficulties. A second set of studies have looked at defensiveness from the perspective of attributions, the idea being that paranoia would help defend a possibly vulnerable view of the self if it were the result of attributing the origin of negative events to other people instead of the self. Indeed, studies have largely shown that clinically paranoid participants tend to attribute a greater proportion of negative events to the actions of others (Kinderman & Bentall, 1997; Lincoln, Mehl, Exner, Lindenmeyer, & Rief, 2010). One would log- ically expect that they would correspondingly avoid attributing such negative events to themselves; this however has not been confirmed. Fornells-Ambrojo and Garety (2009), for example, found that poor-me first-episode patients displayed an excess of external- personal attributions, but showed no increased externalising bias1. The latter, but not the

1‘Externalising bias’, or ‘Self-serving bias’, is an increased difference between the number of internal

167 CHAPTER 7. DEFENSIVE AVOIDANCE IN PARANOIA: SYNTHESIS AND GENERAL DISCUSSION former, correlated with self-esteem (see also Humphreys & Barrowclough, 2006). One interpretation of these attribution studies consistent with the present one is that paranoid patients are not so much generally averse towards internal-personal attributions as they are attracted towards external-personal ones. Evidence for a defensive role of paranoia has been furnished by studies demonstrat- ing differences between overt and covert self-esteem, with some paranoid patient groups showing higher (healthy-like) overt self-esteem but low (depressed-like) covert self-esteem. In an important paper, McKay et al. (2007) reviewed the relevant literature and replicated this ﬁnding, of overt – covert self-esteem discrepancy in paranoia, using the Implicit Asso- ciation Test (IAT) . This was further replicated and augmented by Mehl et al. (2010) who used a novel task to assess implicit attributional style. However, Cicero and Kerns (2010) failed to replicate these results in a substantial (N = 186) analogue sample using the IAT, while Vazquez, Diez-Alegria, Hernandez-Lloreda, and Moreno (2008) found implicit- explicit differences inconsistent with the defence hypothesis using the Self-Referent In- cidental Recall task. It may be that the acute psychotic state deteriorates performance in reaction-time tasks and that this distortion smothers out the difference in task reaction times on which the IAT depends. This might be somewhat similar to the ‘cognitive noise’ effect described in the present analysis of the JTC task. It would explain why actively deluded participants show high error rates in the IAT, and why the remitted-paranoid state is associated with a normal IAT effect (Moritz, Werner, & von Collani, 2006; McKay et al., 2007). Such explanations aside, however, several covert-self-esteem studies are difﬁcult to reconcile with those of the present study. Overall, although it is likely that paranoia which is felt to be undeserved may not de- press self-esteem, and may buffer against the consequences for the self-image of thinking that one is at fault for stressful situations (Udachina et al., in submission), this may best be thought of as a protective effect, a silver lining to the cloud, rather than an increased motivation to psychologically defend oneself compared to mentally healthy people. Ref- erence to a normative account of self-representation in socially threatening situations may provide a good framework to explore these issues.

7.3 Limitations

The studies reported here have a number of limitations. With respect to the Temporal- Difference model of avoidance (Chapter4), we note that -

1. Evidence has been reported for a direct role of dopamine in reporting aversive states (Brischoux et al., 2009), a possibility that our model did not examine. Un- fortunately space does not permit detailed discussion of this finding, which refines attributions for positive events minus the number of internal attributions for negative events. An excess of external-personal attributions is often quantified by the ratio of external-personal attributions for negative events over the total number of personal attributions for negative events and is termed the ‘Other person bias’.

168 7.3. LIMITATIONS

rather than overturns the conclusions reported here (Boureau & Dayan, 2011).

2. The vigour with which alternative actions (or thoughts) are performed and explored by the organism in question is also recognised to play an important role. We only performed a preliminary analysis of the role of vigour.

3. The models only concern generic, strong threats. Speciﬁc, social threats – as in paranoia – are likely to recruit specialised psycho-biological information processing.

4. There is a great scarcity of CAR-like data in patients with paranoid ideas, in suitably medicated healthy controls and in unmedicated patients.

5. Neither the animal model nor our simulations provide an adequate analogue for delusional ﬁxity.

With respect to the analysis of the Jumping-to-Conclusions we note that -

6. The Bayesian Ideal-Observer is a reference point; it is uncertain how the normal mind might approximate it, either in terms of psychological or biological mechanisms.

7. We had too few trials per participant to measure perceived costs for each individual participant.

8. We had no data on the actual decisions made, as opposed to the number of draws taken to decide.

9. There were many early decisions made, so ceiling effects may have prevented dis- cerning subtle effects of the ‘words’ as opposed to the ‘beads’ version of the task.

10. There were no external costs imposed, something which would give a better idea of participants’ own motivation.

11. Most importantly, there was no way to compare a neutral version of the task with one that involved ‘hot’ probabilistic reasoning, directly relevant to the patient’s own threatening objects.

12. Finally, the technical side of model-ﬁtting could be made more efﬁcient, as the following example demonstrates. In this study the denominator of equation 3.5.8, i.e. likelihood of a piece of data under a particular model, was calculated using a full Monte-Carlo numerical integration. A good approximation to this integral, called the Laplace approximation (Daw, in press), would be faster to use instead.

We now turn to the limitations of the empirical investigation of defensiveness in paranoia. We note that -

169 CHAPTER 7. DEFENSIVE AVOIDANCE IN PARANOIA: SYNTHESIS AND GENERAL DISCUSSION

13. In retrospect, an important control group would be a clinical group that is more likely than the paranoid one to show defensive avoidance of aversive self-thoughts, as well as a control procedure able to detect such defensiveness. A very interesting suggestion, albeit in a nonclinical sample, is that of Cicero and Kerns (2010), who utilised narcissism as the deﬁning feature of high-defensiveness controls. Other- wise the possibility remains that whatever emotional load caused the prolongation of response times or the high-avoidance response content detected might be unrelated to defence of self-esteem.

14. The aversive self-thoughts utilised were derived from the participants’ conscious accounts. The possibility thus exists that ‘really hot’ topics were not mentioned at all. Measures were taken to prevent this as much as possible through the way that self-attributes were elicited and by informally checking whether key aspects of the clinical picture appeared deliberately left out. However these are quick-and- dirty measures compared to a clinical psychotherapy assessment. Ideally discrepant self-attributes should also be derived from such a detailed psychotherapeutic formulation.

15. There are difﬁculties as to what it is that the AAQ-2 measures, as discussed above.

16. Too little relevant anxiety may have been present in the context in which participants were exposed to the self-discrepant stimuli. Ethical problems apart, however, a procedure that would raise relevant anxiety might introduce considerable psychological reactance (Kingdon & Turkington, 2005), difﬁcult to distinguish from defensiveness.

Section 6.7.3 also gives detail on some of these points.

7.4 Implications for future research & therapy

7.4.1 Clinical implications

Mental health professionals are divided into a cornucopia of schools when it comes to explaining exaggerated threat beliefs, and the treatments offered have very little to do with addressing the cause of such beliefs in an evidence-based manner. When it comes to paranoia, British psychiatry offers a biopsychosocial approach composed of parts adopted from all different theoretical approaches (dealing with social ‘life events’; biological ‘antipsychotic agents’; psychodynamic thinking about ‘projective mechanisms’ and ‘contain- ment’; cognitive-behavioural ‘Socratic questioning’ of beliefs; even legal ‘sectioning’, i.e. detainment). This Macedoine is made of whatever appears to be useful, but integrating the relevant evidence and clearly differentiating evidence-based science versus plausible but unjustiﬁed extrapolation in all these theories is long overdue.

170 7.4. IMPLICATIONS FOR FUTURE RESEARCH & THERAPY

A key component of the clinical encounter is to share a good understanding of the problems with the patient. The clinician who would be aware of the studies presented here would first of all exercise the utmost caution in conveying to patients troubled by exaggerated persecutory beliefs that they should consider defensive avoidance as an important cause. The clinician should be aware that our culture greatly values attributes such as honesty, courage and self-efficacy. A formulation that implies that the patient is ‘running away’ from their negative attributes or responsibilities can be extremely hurt- ful to patients already vulnerable in self-esteem. It is unethical for health practitioners to hurt the patient unless (a) there is the strongest evidence from well-controlled studies in support of such an intervention and (b) informed consent, or an equivalent legal safe- guard, is in place. The first condition is not fulfilled at the moment when it comes to the aetiological role of defensiveness in paranoia, therefore primum non nocere. A discussion of the importance of the interaction of paranoia and self-esteem might then be useful on a normalising (and even normative) basis. It could be said that the most common effect of feeling threatened on self-esteem is a negative one, but some effects may be positive. This would open up the possibility that either ‘positive’ (e.g. angry empowerment) or ‘negative’ (e.g. defensive withdrawal) reactions might, or might not, help the individual to be safer in the long term. Clinicians may also pay more attention to the patient’s feeling of threat and associated troublesome thoughts which are accompanied by avoidance of private experiences. The present studies do not indicate whether measures to reduce avoidance of these private experiences would ameliorate feeling threatened but they certainly do not cast doubt on this possibility. We can draw an exaggerated medical analogy: meningococcal sepsis in young people is nothing to do with a deficiency of penicillin, but penicillin may still be a lifesaving treatment for this condition. Finally, the likely psychological effects of antipsychotic drugs need to be kept in both the clinician’s and the patient’s minds. These agents are often prescribed clinically, and also often avoided, because of presumed emotional & motivational effects. However the present evidence about these effects in psychosis is too limited to provide authoritative guidance. The present studies (chapters2&4) replicate the claim that antipsychotics reduce motivation (and vigour); but also note that they may help speed up the abolition of avoidance of states which no longer need to be aversive. In the absence of solid evidence both patients and doctors could approach this in a cautious, heuristic manner, to explore the possibility that these drugs may help, or indeed hinder, the learning of more desirable patterns of avoidance.

7.4.2 Research implications

Implications for future research stem, on the one hand, from the uncertainties of clinical practise. On the other hand they stem from the opportunities that the current studies open up to address these uncertainties.

171 CHAPTER 7. DEFENSIVE AVOIDANCE IN PARANOIA: SYNTHESIS AND GENERAL DISCUSSION

Several issues of scientiﬁc and possibly clinical signiﬁcance are worth considering with respect to fundamental threat-related learning mechanisms :

1. Whether there is an inappropriately generalised (yet context-dependent, according to Jensen et al., 2008) over-reporting of worse-than-expected prediction errors in the striatum during paranoid psychosis; and whether a relevant, subjectively felt correlate can be detected psychologically.

2. In terms of basic pharmacology, whether the dopamine-independent (possibly serotonergic) error-reporting component is responsible for exaggerated perception or persistence of threat.

3. In terms of clinical pharmacology, whether the effects of antipsychotic drugs on threat-related psychological mechanisms, especially the dampening of avoidance, aid speciﬁc psychological learning (e.g. via exposure to previously avoided helpful experiences) and hence mediate speciﬁc therapeutic effects.

4. Whether the processing of social threat stimuli is very different from that of physical threats. Speciﬁcally, whether the role of the cached / habit system (pages 50 and 92), in which we investigated the dopamine-dependence of threat processing, is relevant for social threats. Alternatively, the dopamine-dependent and independent mechanisms of goal-directed / tree-search models (see page 89), currently inconsistent with the evidence, need to be updated and given priority.

5. What changes there are, both in terms of activation and in terms of plastic connectivity, corresponding to the waning of persecutory ideation with treatment. The prediction based on our studies would be that behavioural tendencies (overt or in thought, and including the relevant subjectively felt motivations) would wane ﬁrst. This would involve dopamine-dependent circuits and the motivational prominence of beliefs. However other implicit and explicit attitudinal elements of beliefs about the world, more closely associated to dopamine - independent aversive learning, would decay more slowly, if at all, in a manner dependent on new learning.

Some future research directions peripheral to psychopathology are discussed on page 92. We reiterate here that our TD model is highly applicable to manic, as opposed to paranoid, states. All the above suggestions would beneﬁt from a combination of modelling and imaging methodologies. If we now consider our analysis of the JTC phenomenon, the key overall question is how the emotional and cognitive components (Bentall et al., 2009) of paranoia interact.

6. The key methodological advance that would beneﬁt future research is to consider ‘prematurity of decision making’ in terms of appropriate normative, ideal-observer models rather than simply in an agnostic comparison to ‘normal’ controls.

172 7.4. IMPLICATIONS FOR FUTURE RESEARCH & THERAPY

7. It appears crucial – almost mandatory – for future studies to include an explicit measure of error rates, or cognitive noise, in accounting for participant choices.

In addition it would be proﬁtable to investigate -

8. Whether a heightened perception of threat induces or exacerbates a cognitive impairment, and whether this is detectable in affect modulated versions of the ‘beads task’. A straightforward way to do this is to apply the current modelling methodology to data such as those of Lincoln et al. (2010).

9. Whether paranoia in the more cognitively impaired, ‘high noise’ patients is different than in cognitively intact, ‘low noise’ individuals. The task would be much more useful if more trials per participant, with shorter sequences and including explicit costs, were delivered.

In the light of the present empirical study of defensive avoidance, further research could fruitfully investigate -

10. Whether, ﬁrst of all, the present results can be reliably replicated while addressing the methodological limitations identiﬁed above.

11. Whether, if the present results are replicated, mechanisms other than avoidance of negative aspects of the self link self-esteem with paranoia. What evidence might be gathered for speciﬁc self- and other- representations lending themselves to a paranoid role-lock?1.

12. Recruitment biases and difﬁculties need to be addressed. The institutional (NHS) difﬁculties encountered in the present study have already informed vigorous debate (Moutoussis, 2010).

13. Further development of the observer-rated engagement with negative thoughts rating scale would be desirable.

14. Whether the AAQ-2 can be further developed so as to give separate ratings of the intensity and frequency of troublesome experiences on the one hand and the control (vs. acceptance) strategies that were carried out to deal with them on the other. Fol- lowing the philosophy of the PADS, items might be split up - for example: “Painful memories have entered my mind in the last week” followed by “If you have an- swered 2 (sometimes) or more, please tell us how often you have dealt with this by turning your mind away from these memories”.

15. Whether deserved and undeserved paranoia can be examined separately in the same participants. The present study suggested calculating undeserved paranoia as the

1Different therapy schools, each with a limited evidence base, describe similar constructs: systems- centered therapists refer to ‘role locks’, cognitive-analytic therapists to ‘reciprocal role relationships’ while in mentalisation-based therapy paranoia represents a catastrophic failure of mentalising within relationships.

173 CHAPTER 7. DEFENSIVE AVOIDANCE IN PARANOIA: SYNTHESIS AND GENERAL DISCUSSION

overall paranoia score weighed by a factor of (100% − per cent Deservedness). Other decompositions are possible, but any would avoid the rigid categorising into bad-me and good-me people. This would also allow investigation of how well each of these variables is explained, as the current study suggests that the undeserved- paranoia fraction is not well explained by the variables considered here.

16. People with paranoia may think differently about themselves if placed in a context that activates their speciﬁc fears. Comparison should be with control participants that also have threat concerns but are not severely mentally ill – for example people undergoing threatening legal proceedings.

17. Whether a normative rationale can be developed for perceiving interpersonal threat, possibly in a controlled situation. In what way could a person’s self-representation adaptively help interactions? When and how, for example, should one optimally adopt a poor-me threatened attitude?

One particularly interesting paradigm which may be able to bring many of these strands together is that of game-theoretical tasks. Such tasks have recently started to be used in psychopathology research (King-Casas et al., 2008). They provide in vivo, interactive measures of such concepts as mistrust-of-others which are likely to be central to paranoia. There is a large literature of such tasks in healthy people, and profound theoretical thinking on the ideal-normative ways of carrying out such tasks has been developed (Yoshida, Seymour, Friston, & Dolan, 2010). It remains, of course, to be seen whether similar tasks that tap into paranoia can be developed.

174 CONTRIBUTION

This set of three studies has made several novel contributions to elucidating the role of defensive avoidance in paranoia. It was found that the Conditioned Avoidance Response paradigm reﬂects important aspects of threat perception, relevant to paranoid psychosis. An advantage-learning temporal- difference model was the simplest one to account for key data on how dopamine antagonists modulate the Conditioned Avoidance Response. Within this, a largely dopamine- independent process is likely to signal that a particular state has led to a worse-than- expected outcome. On the contrary, both worse-than-expected and better-than-expected signals involve dopamine when it comes to learning about actions. None of the dopaminergic manipulations considered resulted in effects analogous to delusional ﬁxity. The psychological mode of action of dopamine blockade may involve dampening both the vigour of the avoidance response and the prediction-errors that drive the learning of avoidant actions. The second study involved modelling of the Jumping-to-Conclusions bias. We hypothesised that avoidance of inappropriately perceived high costs of delaying a decision resulted in hasty decision-making in paranoid patients. The experimental data did not support our hypothesis. We considered a normative ‘ideal Bayesian observer’ performing this task; the decisions of control participants were closer to this Bayesian ideal than those of the paranoid participants. In the latter a higher ‘cognitive noise’, relative to the motivation to avoid errors, explained the hasty decisions. Finally we hypothesised that avoidance of negative thoughts about the self may contribute to paranoid beliefs in patients whose paranoia is felt to be undeserved. We conducted a detailed study where we elicited important self-attributes in paranoid patients and controls. We used several measures to assess avoidance while we exposed the participants to aversive thoughts about these important self-attributes. None of these measures showed any evidence of increased avoidance in the paranoid groups in this task. How- ever both ‘bad-me’ and ‘poor-me’ patients scored very low on the Acceptance and Action Questionnaire. We concluded that experiential avoidance is prominent in both bad-me and poor-me paranoid patients, but that avoidance of negative thoughts about the self is unlikely to play a major aetiological role in paranoia.

175 176 REFERENCES

Aguilar, M. A., Minarro, J., & Simon, V. M. (2004). Morphine potentiates the impairing effects of neuroleptics on two-way active conditioned avoidance response in male mice. Prog Neuropsychopharmacol Biol Psychiatry, 28 (2), 225–37. American Psychiatric Association Task Force on DSM-IV. (2000). Diagnostic and Statis- tical Manual of Mental Disorders: DSM-IV-TR (4th ed.). Washington DC: Ameri- can Psychiatric Association. Anisman, H. (1978). Psychopharmacology of aversively motivated behavior. In H. Anis- man & G. Bignami (Eds.). (Chap. Aversively motivated behavior as a tool in psy- chopharmacologic analysis, pp. 1–62). New York: Plenum Press. Bach, P., & Hayes, S. C. (2002). The use of acceptance and commitment therapy to prevent the rehospitalization of psychotic patients: a randomized controlled trial. J Consult Clin Psychol, 70 (5), 1129–39. Balleine, B. W., Liljeholm, M., & Ostlund, S. B. (2009). The integrative function of the basal ganglia in instrumental conditioning. Behav Brain Res, 199 (1), 43–52. Bardin, L., Auclair, A., Kleven, M. S., Prinssen, E. P., Koek, W., Newman-Tancredi, A., & Depoortere, R. (2007). Pharmacological profiles in rats of novel antipsychotics with combined dopamine D2/serotonin 5-HT1A activity: comparison with typical and atypical conventional antipsychotics. Behav Pharmacol, 18 (2), 103–18. Batchelder, W. H., & Riefer, D. M. (2007). Advances in Clinical Cognitive Science. In R. Neufeld (Ed.). (Chap. Using Multinomial Processing Tree Models to Measure Cognitive Deficits in Clinical Populations, pp. 19–50). Washington, D.C.: APA. Bayer, H. M., & Glimcher, P. W. (2005). Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47 (1), 129–41. Belin, D., & Everitt, B. J. (2008). Cocaine seeking habits depend upon dopamine- dependent serial connectivity linking the ventral with the dorsal striatum. Neuron, 57 (3), 432–41. Beninger, R., Mason, S., Phillips, A., & Fibiger, H. (1980a). The use of conditioned suppression to evaluate the nature of neuroleptic-induced avoidance deficits. J Pharma- col Exp Ther, 213 (3), 623–7.

177 REFERENCES

Beninger, R., Mason, S., Phillips, A., & Fibiger, H. (1980b). The use of extinction to investigate the nature of neuroleptic-induced avoidance deﬁcits. Psychopharmacol- ogy (Berl), 69 (1), 11–8. Bentall, R. P. (1994). The neuropsychology of schizophrenia. In A. David & J. Cutting (Eds.). (Chap. Cognitive biases and abnormal beliefs: Towards a model of persecutory delusions, pp. 337–360). London: Lawrence Erlbaum. Bentall, R. P. (2003). Madness explained: psychosis and human nature. London: Penguin. Bentall, R. P., Corcoran, R., Howard, R., Blackwood, N., & Kinderman, P. (2001). Per- secutory delusions: a review and theoretical integration. Clin Psychol Rev, 21 (8), 1143–92. Bentall, R. P., & Kaney, S. (2005). Attributional lability in depression and paranoia. Br J Clin Psychol, 44 (Pt 4), 475–88. Bentall, R. P., Kaney, S., & Bowen-Jones, K. (1995). Persecutory delusions and recall of threat-related, depression-related and neutral words. Cognitive Ther Res, 19, 331– 343. Bentall, R. P., Kinderman, P., & Kaney, S. (1994). The self, attributional processes and abnormal beliefs: towards a model of persecutory delusions. Behav Res Ther, 32 (3), 331–41. Bentall, R. P., Rowse, G., Kinderman, P., Blackwood, N., Howard, R., Moore, R., Cum- mins, S., & Corcoran, R. (2008). Paranoid delusions in schizophrenia spectrum disorders and depression: the transdiagnostic role of expectations of negative events and negative self-esteem. J Nerv Ment Dis, 196 (5), 375–83. Bentall, R. P., Rowse, G., Shryane, N., Kinderman, P., Howard, R., Blackwood, N., Moore, R., & Corcoran, R. (2009). The cognitive and affective structure of paranoid delusions: a transdiagnostic investigation of patients with schizophrenia spectrum disorders and depression. Arch Gen Psychiatry, 66 (3), 236–47. Bentall, R. P., & Swarbrick, R. (2003). The best laid schemas of paranoid patients: auton- omy, sociotropy and need for closure. Psychol Psychother, 76 (Pt 2), 163–71. Bentall, R., Kinderman, P., & Moutoussis, M. (2008). Persecutory delusions: assessment, theory and treatment. In D. Freeman, R. Bentall & P. Garety (Eds.). (Chap. The role of self-esteem in paranoid delusions: the psychology, neurophysiology and development of persecutory beliefs, pp. 143–173). Oxford: OUP. Berrios, G. E. (1991). Delusions as "wrong beliefs": a conceptual history. Br J Psychiatry Suppl (14), 6–13. Bertsekas, D. (1995). Dynamic programming and optimal control. Belmont, Mass.: Athena Scientiﬁc. Blum, H. (1980). Paranoia and beating fantasy: an inquiry into the psychoanalytic theory of paranoia. J Am Psychoanal Assn, 28, 331–361.

178 REFERENCES

Blum, H. (1994). Paranoia: new psychoanalytic perspectives. In J. Oldham & S. Bone (Eds.). (Chap. Paranoid Betrayal and Jealousy: The Loss and Restitution of Object Constancy). Maddison, Connecticut: International Universities Press. Blundell, P., Hall, G., & Killcross, S. (2003). Preserved sensitivity to outcome value after lesions of the basolateral amygdala. J Neurosci, 23 (20), 7702–9. Boureau, Y. L., & Dayan, P. (2011). Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharmacol, 36 (1), 74–97. Brischoux, F., Chakraborty, S., Brierley, D. I., & Ungless, M. A. (2009). Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc Natl Acad Sci U S A, 106 (12), 4894–9. Busemeyer, J. R., & Stout, J. C. (2002). A contribution of cognitive decision models to clinical assessment: decomposing performance on the Bechara gambling task. Psychol Assess, 14 (3), 253–62. Cagniard, B., Beeler, J. A., Britt, J. P., McGehee, D. S., Marinelli, M., & Zhuang, X. (2006). Dopamine scales performance in the absence of new learning. Neuron, 51 (5), 541–7. Campbell, M., & Morrison, A. (2007). The subjective experience of paranoia: Compar- ing the experiences of patients with psychosis and individuals with no psychiatric history. Clin. Psychol. Psychother. 14, 63–77. Campbell, W. K., & Sedikides, C. (1999). Self-threat magniﬁes the self-serving bias: a meta-analytic integration. Rev Gen Psychol, 3, 23–43. Carveth, D. (1994). Dark epiphany: the encounter with ﬁnitude or the discovery of the object in the body. Psychoanal Contemp Thought, 17, 215–250. Chadwick, P., Trower, P., Juusti-Butler, T.-M., & Maguire, N. (2005). Phenomenological evidence for two types of paranoia. Psychopathology, 38, 327–333. Chadwick, P., Birchwood, M., & Trower, P. (1996). Cognitive therapy for delusions, voices and paranoia. Chichester, UK: John Wiley and Sons. Chater, N., Tenenbaum, J. B., & Yuille, A. (2006). Probabilistic models of cognition: conceptual foundations. Trends Cogn Sci, 10 (7), 287–91. Christy, D., & Reid, L. (1975). Methods of deconditioning persisting avoidance: Am- phetamine and amobarbital as adjuncts to response prevention. B Psychonomic Soc, 5, 175–177. Cicero, D., & Kerns, J. (2010). Is Paranoia a Defence Against or an Expression of Low Self-esteem? Eur. J. Pers. DOI: 10.1002/per.794. Cooper, S., Coon, K., Mejta, C., & Reid, L. (1974). Methods of deconditioning persisting avoidance: Amphetamine, chlorpromazine, and chlordiazepoxide as adjuncts to response prevention. PhysioL Psychol, 2, 519–522. Corcoran, R., Cummins, S., Rowse, G., Moore, R., Blackwood, N., Howard, R., Kinder- man, P., & Bentall, R. (2006). Reasoning under uncertainty: heuristic judgments in patients with persecutory delusions or depression. Psychol Med, 36 (8), 1109–18.

179 REFERENCES

Corcoran, R., Rowse, G., Moore, R., Blackwood, N., Kinderman, P., Howard, R., Cum- mins, S., & Bentall, R. P. (2008). A transdiagnostic investigation of ’theory of mind’ and ’jumping to conclusions’ in patients with persecutory delusions. Psychol Med, 38 (11), 1577–83. Coutureau, E., & Killcross, S. (2003). Inactivation of the infralimbic prefrontal cortex reinstates goal-directed responding in overtrained rats. Behav Brain Res, 146 (1-2), 167–74. Crowne, D. P., & Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. J Consult Psychol, 24, 349–54. David, A. S. (1999). On the impossibility of deﬁning delusions. Philos Psychiatr Psychol, 6, 17–20. Daw, N. D., & Doya, K. (2006). The computational neurobiology of learning and reward. Curr Opin Neurobiol, 16 (2), 199–204. Daw, N. (in press). Attention and Performance XXIII. In E. Phelps, T. Robbins & M. Del- grado (Eds.). Attention and Performance (Chap. Trial-by-trial data analysis using computational models). Oxford: Oxford University Press. Daw, N., Kakade, S., & Dayan, P. (2002). Opponent interactions between serotonin and dopamine. Neural Networks, 15, 603–616. Daw, N., Niv, Y., & Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci, 8 (12), 1704– 11. Dayan, P., & Abbot, L. (2001). Theoretical neuroscience: computational and mathematical modeling of neural systems. Cambridge, MA: MIT Press. Dayan, P., & Balleine, B. W. (2002). Reward, motivation, and reinforcement learning. Neuron, 36 (2), 285–98. Dayan, P., & Daw, N. D. (2008). Decision theory, reinforcement learning, and the brain. Cogn Affect Behav Neurosci, 8 (4), 429–53. Dayan, P., & Huys, Q. J. (2008). Serotonin, inhibition, and negative mood. PLoS Comput Biol, 4 (2), e4. Dayan, P., Niv, Y., Seymour, B., & Daw, N. D. (2006). The misbehavior of value and the discipline of the will. Neural Networks, 19 (8), 1153–60. de Oliveira, A. R., Reimer, A. E., & Brandao, M. L. (2006). Dopamine D2 receptor mechanisms in the expression of conditioned fear. Pharmacol Biochem Behav, 84 (1), 102–11. Deakin, J., & Graeff, F. (1991). 5-HT and mechanisms of defence. J Psychopharmacol, 5, 305–315. Dempster, A., Laird, N., & Rubin, D. (1977). Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc, Series B, 39 (1), 1–38.

180 REFERENCES

Dickinson, A., & Balleine, B. (2002). Steven’s Hanbook of Experimental Psychology. In C. Gallistel (Ed.). (Chap. The role of learning in motivation, Vol. 3: Learning, Motivation and Emotion). New York: Wiley. Doya, K. (2002). Metalearning and neuromodulation. Neural Netw, 15 (4-6), 495–506. Drake, R. J., Pickles, A., Bentall, R. P., Kinderman, P., Haddock, G., & Tarrier, N. (2004). The evolution of insight, paranoia and depression during early schizophrenia. Psy- chol Med, 34, 285–292. Dudley, R. E., John, C. H., Young, A. W., & Over, D. E. (1997a). Normal and abnormal reasoning in people with delusions. Br J Clin Psychol, 36 ( Pt 2), 243–58. Dudley, R. E., John, C. H., Young, A. W., & Over, D. E. (1997b). The effect of self- referent material on the reasoning of people with delusions. Br J Clin Psychol, 36 ( Pt 4), 575–84. Dudley, R., & Over, D. (2003). People with Delusions Jump to Conclusions: A Theoretical Account of Research Findings on the Reasoning of People with Delusions. Clin Psychol Psychot, 10, 263–274. Efron, B., & Tibshirani, R. (1993). An Introduction to the Bootstrap. Boca Raton, Florida: CRC Press. Fear, C. F., & Healy, D. (1997). Probabilistic reasoning in obsessive-compulsive and delusional disorders. Psychol Med, 27 (1), 199–208. Fear, C. F., Sharp, H., & Healy, D. (1996). Cognitive processes in delusional disorder. Brit J Psychiat, 168, 61–67. Fine, C., Gardner, M., Craigie, J., & Gold, I. (2007). Hopping, skipping or jumping to conclusions? Clarifying the role of the JTC bias in delusions. Cogn Neuropsychiatry, 12 (1), 46–77. Fischhoff, B., & Beyth-Marom, R. (1983). Hypothesis evaluation from a Bayesian Per- spective. Psychol Rev, 90, 239–260. Fletcher, P. C., & Frith, C. D. (2009). Perceiving is believing: a Bayesian approach to explaining the positive symptoms of schizophrenia. Nat Rev Neurosci, 10 (1), 48– 58. Fornells-Ambrojo, M., & Garety, P. (2005). Bad me paranoia in early psychosis: a relatively rare phenomenon. Br J Clin Psychology, 44, 521–528. Fornells-Ambrojo, M., & Garety, P. A. (2009). Understanding attributional biases, emotions and self-esteem in ’poor me’ paranoia: ﬁndings from an early psychosis sample. Br J Clin Psychol, 48 (Pt 2), 141–62. Forsyth, J., Eifert, G., & Barrios, V. (2006). Fear and Learning: from basic processes to clinical implications. In M. Craske, D. Hermans & D Vansteenwegen (Eds.). (Chap. Fear Conditioning in an Emotion Regulation Context: A Fresh Perspective on the Origins of Anxiety Disorders, pp. 133–153). Washington, DC: American Psychological Association.

181 REFERENCES

Francis, J., Boldero, J., & Sambell, N. (2006). Self-Lines: A New, Psychometrically Sound, ’User-Friendly’ Idiographic Technique for Assessing Self-Discrepancies. Cognitive Ther Res, 30 (1), 69–84. Frank, M. J. (2005). Dynamic dopamine modulation in the basal ganglia: a neurocompu- tational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J Cogn Neurosci, 17 (1), 51–72. Freeman, D., Garety, P., Fowler, D., Kuipers, E., Dunn, G., Bebbington, P., & Hadley, C. (1998). The London-East Anglia randomized controlled trial of cognitive-behaviour therapy for psychosis. IV: Self-esteem and persecutory delusions. Br J Clin Psychol, 37 ( Pt 4), 415–30. Freeman, D., Garety, P., Kuipers, E., Colbert, S., Jolley, S., Fowler, D., Dunn, G., & Beb- bington, P. (2006). Delusions and decision-making style: use of the Need for Clo- sure Scale. Behav Res Ther, 44 (8), 1147–58. Freeman, D., & Garety, P. A. (2004a). Paranoia: The Psychology of Persecutory Delusions (1st ed.). The Maudsley Monographs. Hove, England: Psychology Press. Freeman, D., & Garety, P. A. (2004b). Paranoia: The Psychology of Persecutory Delu- sions. (1st ed., Chap. Delusions and disconfirmatory evidence, pp. 61–75). The Maudsley Monographs. Hove, England: Psychology Press. Freeman, D., & Garety, P. A. (2005). Connecting neurosis to psychosis: the direct influence of emotion on delusions and hallucinations. Behav Res Ther, 41, 923–947. Freeman, D., Garety, P. A., Bebbington, P. E., Smith, B., Rollinson, R., Fowler, D., & et al. (2005). Psychological investigation of the structure of paranoia in a non-clinical population. Brit J Psychiat, 186, 427–435. Freeman, D., Garety, P. A., & Kuipers, E. (2001). Persecutory delusions: developing the understanding of belief maintenance and emotional distress. Psychol Med, 31 (7), 1293–306. Freeman, D., Garety, P. A., Kuipers, E., Fowler, D., & Bebbington, P. E. (2002). A cognitive model of persecutory delusions. Brit J Clin Psychol, 41, 331–347. Freeman, D., Gittins, M., Pugh, K., Antley, A., Slater, M., & Dunn, G. (2008). What makes one person paranoid and another person anxious? The differential prediction of social anxiety and persecutory ideation in an experimental situation. Psychol Med, 38 (8), 1121–32. Freeman, D., McManus, S., Brugha, T., Meltzer, H., Jenkins, R., & Bebbington, P. (2010). Concomitants of paranoia in the general population. Psychol Med, 1–14. Freud, S. (1911). Collected papers (Vol. III, pp. 387-466). In S. Freud (Ed.). (Chap. Psy- choanalytic notes upon an autobiographical account of a case of paranoia (Dementia Paranoides)). London: Hogarth Press. Fuchs, T. (1992). Life events in late paraphrenia and depression. Psychopathology, 32, 60–69.

182 REFERENCES

Garety, P., & Freeman, D. (1999). Cognitive approaches to delusions: a critical review of theories and evidence. Brit J Clin Psychol. Garety, P. A., Fowler, D. G., Freeman, D., Bebbington, P., Dunn, G., & Kuipers, E. (2008). Cognitive–behavioural therapy and family intervention for relapse prevention and symptom reduction in psychosis: randomised controlled trial. Br J Psychiat, 192 (6), 412–23. Garety, P. A., Freeman, D., Jolley, S., Dunn, G., Bebbington, P. E., Fowler, D. G., Kuipers, E., & Dudley, R. (2005). Reasoning, emotions, and delusional conviction in psychosis. J Abnorm Psychol, 114 (3), 373–84. Garety, P. A., & Hemsley, D. R. (1987). Characteristics of delusional experience. Eur Arch Psychiatry Neurol Sci, 236 (5), 294–8. Garety, P. A., Hemsley, D. R., & Wessely, S. (1991). Reasoning in deluded schizophrenic and paranoid patients. Biases in performance on a probabilistic inference task. J Nerv Ment Dis, 179 (4), 194–201. Gelman, A. (2002). Encyclopedia of Environmetrics. In A. El-Shaarawi & W. Piegorsch (Eds.). (Chap. Posterior distribution, Vol. 3). Chichester: Wiley. Glockner, A., & Moritz, S. (2008). A Fine-grained Analysis of the Jumping to Conclu- sions Bias in Schizophrenia: Data-Gathering, Response Conﬁdence and Informa- tion Integration. Bonn. Gold, J. I., & Shadlen, M. N. (2001). Neural computations that underlie decisions about sensory stimuli. Trends Cogn Sci, 5 (1), 10–16. Goto, Y., Otani, S., & Grace, A. A. (2007). The Yin and Yang of dopamine release: a new perspective. Neuropharmacology, 53, 583–587. Grace, A. A. (2000). The tonic/phasic model of dopamine system regulation and its implications for understanding alcohol and psychostimulant craving. Addiction, 95 Suppl 2, S119–28. Gray, J. A., & McNaughton, N. (1996). The neuropsychology of anxiety: reprise. Nebr Symp Motiv, 43, 61–134. Gray, J. (1987). The psychology of fear and stress. New York, NY: McGraw-Hill. Green, C. E., Freeman, D., Kuipers, E., Bebbington, P., Fowler, D., Dunn, G., & Garety, P. A. (2011). Paranoid explanations of experience: a novel experimental study. Behav Cogn Psychother, 39 (1), 21–34. Green, D., & Swets, J. (1966). Signal detection theory and psychophysics. New York: John Wiley and Sons. Green, D., & Swets, J. (2008). Signal detection theory and psychophysics. Los Altos: Peninsula Publishing. Grossberg, S. (1972). A neural theory of punishment and avoidance. Math Biosci, 15, 39–68. Hartmann, H. (1958). Ego psychology and the problem of adaptation. New York: Inter- national Universities Press.

183 REFERENCES

Haselton, M. G., & Nettle, D. (2006). The paranoid optimist: an integrative evolutionary model of cognitive biases. Pers Soc Psychol Rev, 10 (1), 47–66. Hayes, S. C., Strosahl, K. D., & Wilson, K. G. (1999). Acceptance and commitment therapy: An experiential approach to behavior change. New York: Guildford. Hayes, S. C., Wilson, K. G., Gifford, E. V., Follette, V. M., & Strosahl, K. (1996). Exper- imental avoidance and behavioral disorders: a functional dimensional approach to diagnosis and treatment. J Consult Clin Psychol, 64 (6), 1152–68. Hayes, S., Strosahl, K., Wilson, K., Bissett, R., Pistorello, J., Toarmino, D., Polusny, M., Dykstra, T., Batten, S., Bergan, J., Stewart, S., Zvolensky, M., Eifert, G., Bond, F., Forsyth, J., Karekla, M., & McCurry, S. (2004). Measuring Experiential Avoidance: A Preliminary Test of a Working Model. Psychol Rec, 54, 553–578. Hemsley, D. R., & Garety, P. A. (1986). The formation of maintenance of delusions: a Bayesian analysis. Br J Psychiat, 149, 51–6. Herrnstein, R. J. (1961). Relative and absolute strength of response as a function of frequency of reinforcement. J Exp Anal Behav, 4, 267–72. Hoenig, J. (1982). Kurt Schneider and anglophone psychiatry. Compr Psychiat, 23, 391– 400. Horvitz, J. C. (2000). Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience, 96 (4), 651–6. Humphreys, L., & Barrowclough, C. (2006). Attributional style, defensive functioning and persecutory delusions: symptom-speciﬁc or general coping strategy? Br J Clin Psychol, 45 (Pt 2), 231–46. Huq, S. F., Garety, P. A., & Hemsley, D. R. (1988). Probabilistic judgements in deluded and non-deluded subjects. Q J Exp Psychol A, 40 (4), 801–12. Huys, Q., Moutoussis, M., & Williams, J. (in press). Are computational models of any use to psychiatry? Neural Networks. Iordanova, M. D., Westbrook, R. F., & Killcross, A. S. (2006). Dopamine activity in the nucleus accumbens modulates blocking in fear conditioning. Eur J Neurosci, 24 (11), 3265–70. Janssen, I., Hanssen, M., Bak, M., Bijl, R. V., de Graaf, R., Vollebergh, W., McKenzie, K., & van Os, J. (2003). Discrimination and delusional ideation. Br J Psychiat, 182, 71–6. Janssen, I., Versmissen, D., Campo, J. A., Myin-Germeys, I., van OS, J., & Krabben- dam, L. (2006). Attributional style and psychosis: evidence for externalizing bias in patients but not individuals at high risk. Psychol Med, 27, 1–8. Jaspers, K. (1913/1959). General Psychopathology. Manchester: Manchester University Press. Jensen, J., McIntosh, A. R., Crawley, A. P., Mikulis, D. J., Remington, G., & Kapur, S. (2003). Direct activation of the ventral striatum in anticipation of aversive stimuli. Neuron, 40 (6), 1251–7.

184 REFERENCES

Jensen, J., Willeit, M., Zipursky, R. B., Savina, I., Smith, A. J., Menon, M., Crawley, A. P., & Kapur, S. (2008). The formation of abnormal associations in schizophrenia: neural and behavioral evidence. Neuropsychopharmacol, 33 (3), 473–9. Johnson, J., Li, W., Li, J., & Klopf, A. (2002). A computational model of learned avoidance behavior in a one-way avoidance experiment. Adaptive Behavior, 9, 91–104. Johnson, S., Sandrow, D., Meyer, B., Winters, R., Miller, I., Solomon, D., & Keitner, G. (2000). Increases in manic symptoms after life events involving goal attainment. J Abnorm Psychol, 109 (4), 721–7. Jolley, S., Garety, P., Bebbington, P., Dunn, G., Freeman, D., Kuipers, E., Fowler, D., & Hemsley, D. (2006). Attributional style in psychosis–the role of affect and belief type. Behav Res Ther, 44 (11), 1597–607. Jorgensen, P., & Jensen, J. (1994). Delusional beliefs in ﬁrst admitters. Psychopathology, 27, 100–112. Kahneman, D., Slovic, P., & Tversky, A. (1982). Judgement under uncertainty: heuristics and biases. Cambridge: Cambridge University Press. Kaney, S., & Bentall, R. P. (1989). Persecutory delusions and attributional style. Brit J Med Psychol, 62, 191–198. Kaney, S., Bowen-Jones, K., Dewey, M. E., & Bentall, R. P. (1997). Frequency and con- sensus judgements of paranoid, paranoid-depressed and depressed psychiatric patients: subjective estimates for positive, negative and neutral events. Brit J Clin Psy- chol, 36, 349–364. Kaney, S., Wolfenden, M., Dewey, M. E., & Bentall, R. P. (1992). Persecutory delusions and the recall of threatening and non-threatening propositions. Brit J Clin Psychol, 32, 85–87. Kapur, S. (2003). Psychosis as a state of aberrant salience: a framework linking biology, phenomenology, and pharmacology in schizophrenia. Am J Psychiatry, 160 (1), 13–23. Retrieved from http://eutils.ncbi.nlm.nih.gov/entrez/ eutils/elink.fcgi?cmd=prlinks&dbfrom=pubmed&retmode= ref&id=12505794 Kapur, S., Mizrahi, R., & Li, M. (2005). From dopamine to salience to psychosis–linking biology, pharmacology and phenomenology of psychosis. Schizophr Res, 79 (1), 59–68. Kaslow, N., Reviere, S., Chance, S., Rogers, J., Hatcher, C., Wasserman, F., Smith, L., Jessee, S., James, M., & Seelig, B. (1998). An empirical study of the psychodynamics of suicide. J Am Psychoanal Assn, 46, 777–796. Kay, S. R., & Opler, L. A. (1987). The positive-negative dimension in schizophrenia: its validity and signiﬁcance. Psychiatr Dev, 5 (2), 79–103. Kendler, K. S., Glazer, W., & Morgenstern, H. (1983). Dimensions of delusional experience. Am J Psychiat, 140, 466–469.

185 REFERENCES

Killcross, S., & Coutureau, E. (2003). Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb Cortex, 13 (4), 400–8. Killcross, S., Robbins, T. W., & Everitt, B. J. (1997). Different types of fear-conditioned behaviour mediated by separate nuclei within amygdala. Nature, 388 (6640), 377– 80. Kim, H., Shimojo, S., & O’Doherty, J. P. (2006). Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain. PLoS Biol, 4 (8), e233. Kinderman, P., & Bentall, R. P. (1996). Self-discrepancies and persecutory delusions: evidence for a model of paranoid ideation. J Abnorm Psychol, 105 (1), 106–13. Kinderman, P., & Bentall, R. P. (1997). Causal attributions in paranoia and depression: internal, personal, and situational attributions for negative events. J Abnorm Psychol, 106 (2), 341–5. Kinderman, P., & Bentall, R. P. (2000). Self-discrepancies and causal attributions: Studies of hypothesized relationships. Br J Clin Psychology, 39, 255–273. Kinderman, P., Prince, S., Waller, G., & Peters, E. (2003). Self-discrepancies, attentional bias and persecutory delusions. Br J Clin Psychol, 42 (Pt 1), 1–12. King-Casas, B., Sharp, C., Lomax-Bream, L., Lohrenz, T., Fonagy, P., & Montague, P. R. (2008). The rupture and repair of cooperation in borderline personality disorder. Science, 321 (5890), 806–10. Kingdon, D., & Turkington, D. (2005). Cognitive therapy of schizophrenia. (Chap. Case formulation and intervening with delusions, pp. 96–118). NY: Guilford Press. Klein, M. (1975/1936). The writings of melanie klein:, volume I: love, guilt and reparation and other works 1921-1945. (Chap. Weaning). London: Hogarth Press. Kording, K. (2007). Decision theory: what "should" the nervous system do? Science, 318 (5850), 606–10. Kramer, R. (1994). The Sinister Attribution Error: Paranoid cognition and collective dis- trust in organizations. Motiv Emotion, 18 (2), 199–230. Kruglanski, A. W., Webster, D. M., & Klem, A. (1993). Motivated resistance and openness to persuasion in the presence or absence of prior information. J Pers Soc Psychol, 65 (5), 861–76. Kumar, P., Waiter, G., Ahearn, T., Milders, M., Reid, I., & Steele, J. D. (2008). Abnormal temporal difference reward-learning signals in major depression. Brain, 131 (Pt 8), 2084–93. Laming, D. (1968). Information theory of choice reaction time. New York: Wiley. Langdon, R., McKay, R., & Coltheart, M. (2008). Persecutory Delusions: Assessment, Theory and Treatment. In D. Freeman, R. Bentall & P. Garety (Eds.). (Chap. The cognitive neuropsychological understanding of persecutory delusions, pp. 221– 236). Oxford: Oxford University Press.

186 REFERENCES

Laruelle, M. (2008). Persecutory Delusions: Assessment, Theory and Treatment. In D. Freeman, R. Bentall & P. Garety (Eds.). (Chap. Dopamine and Persecutory Delu- sions, pp. 239–266). Oxford: Oxford University Press. Laruelle, M., Abi-Dargham, A., Gil, R., Kegeles, L., & Innis, R. (1999). Increased dopamine transmission in schizophrenia: Relationship to illness phases. Biol Psy- chiat. Lattin, J., Carroll, J., & Green, P. (2003). Analyzing multivariate data. Belmont, CA: Brooks Cole. Lincoln, T. M., Lange, J., Burau, J., Exner, C., & Moritz, S. (2010). The effect of state anxiety on paranoid ideation and jumping to conclusions. An experimental investigation. Schizophr Bull, 36 (6), 1140–8. Lincoln, T. M., Peter, N., Schafer, M., & Moritz, S. (2010). From stress to paranoia: an experimental investigation of the moderating and mediating role of reasoning biases. Psychol Med, 40 (1), 169–71. Lincoln, T. M., Ziegler, M., Mehl, S., & Rief, W. (2010). The jumping to conclusions bias in delusions: speciﬁcity and changeability. J Abnorm Psychol, 119 (1), 40–9. Lincoln, T., Mehl, S., Exner, C., Lindenmeyer, J., & Rief, W. (2010). Attributional Style and Persecutory Delusions. Evidence for an Event Independent and State Speciﬁc External-Personal Attribution Bias for Social Situations. Cogn Ther Res, 34, 297– 302. Link, S. (1992). The Wave Theory of Difference and Similarity. Hillsdale, NJ: Lawrence Erlbaum Associates. Lovibond, P. (2006). Fear and Learning: from basic processes to clinical implications. In M. Craske, D. Hermans & D. Vansteenwegen (Eds.). (Chap. Fear and Avoid- ance: an integrated expectancy model, pp. 117–132). Washington, DC: American Psychological Association. Lyon, H. M., Kaney, S., & Bentall, R. P. (1994). The defensive function of persecutory delusions. Evidence from attribution tasks. Br J Psychiat, 164 (5), 637–46. Ma, T., & Yu, Q. (1993). Effect of 20(S)-ginsenoside-Rg2 and cyproheptadine on two- way active avoidance learning and memory in rats. Arzneimittelforschung, 43 (10), 1049–52. Maher, B., & Ross, J. (1984). Comprehensive Handbook of Psychopathology. In H. Adams & P. Suther (Eds.). (Chap. Delusions). New York: Plenum. Marczewski, P., Van der Linden, M., & Laroi, F. (2001). Further investigation of the Su- pervisory Attentional System in schizophrenia: Planning, inhibition, and rule ab- straction. Cogn Neuropsychiatry, 6 (3), 175–192. Marr, D. (1982). Vision. New York: Freeman. McAllister, D., & McAllister, W. (1991). Fear, avoidance, and phobias: A fundamental analysis. In M. Denny (Ed.). (Chap. Fear theory and aversively motivated behavior: Some controversial issues, pp. 135–163). Hillsdale, NJ: Erlbaum.

187 REFERENCES

McAllister, D., McAllister, W., Hampton, S., & Scoles, T. (1980). Escape-from-fear performance as affected by handling method and an additional CS-shock treatment. Anim Learn Behav, 8, 417–423. McAllister, W. R., McAllister, D. E., Scoles, M. T., & Hampton, S. R. (1986). Persis- tence of fear-reducing behavior: relevance for the conditioning theory of neurosis. J Abnorm Psychol, 95 (4), 365–72. McKay, R., Langdon, R., & Coltheart, M. (2005). Paranoia, persecutory delusions and attributional biases. Psychiat Res, 136, 233–245. McKay, R., Langdon, R., & Coltheart, M. (2006). Need for closure, jumping to conclusions, and decisiveness in delusion-prone individuals. J Nerv Ment Dis, 194, 422– 426. McKay, R., Langdon, R., & Coltheart, M. (2007). The defensive function of persecutory delusions: an investigation using the Implicit Association Test. Cognit Neuropsy- chiatry, 12 (1), 1–24. Mehl, S., Rief, W., Lullmann, E., Ziegler, M., Muller, M. J., & Lincoln, T. M. (2010). Implicit attributional style revisited: evidence for a state-speciﬁc "self-decreasing" implicit attributional style in patients with persecutory delusions. Cogn Neuropsy- chiatry, 15 (5), 451–76. Melo, S., Taylor, J., & Bentall, R. P. (2006). "Poor me" versus "bad me" paranoia and the instability of persecutory ideation. Psychol Psychother Theor Res Pract, 79, 271– 287. Menon, M., Jensen, J., Vitcu, I., Graff-Guerrero, A., Crawley, A., Smith, M. A., & Ka- pur, S. (2007). Temporal Difference Modeling of the Blood-Oxygen Level Depen- dent Response During Aversive Conditioning in Humans: Effects of Dopaminergic Modulation. Biol Psychiat, 62, 765–772. Menon, M., Mizrahi, R., & Kapur, S. (2008). ’Jumping to conclusions’ and delusions in psychosis: relationship and response to treatment. Schizophr Res, 98 (1-3), 225–31. Menon, M., Pomarol-Clotet, E., McKenna, P. J., & McCarthy, R. A. (2006). Probabilistic reasoning in schizophrenia: a comparison of the performance of deluded and nonde- luded schizophrenic patients and exploration of possible cognitive underpinnings. Cogn Neuropsychiatry, 11 (6), 521–36. Mezulis, A. H., Abramson, L. Y., Hyde, J. S., & Hankin, B. L. (2004). Is there a universal positivity bias in attributions? a meta-analytic review of individual, developmental and cultural differences in the self-serving attributional bias. Psychol Bull, 130, 711–747. Mineka, S. (1979). The role of fear in theories of avoidance learning, ﬂooding and extinction. Psychological Bulletin, 86, 985–1010. Mirowsky, J., & Ross, C. E. (1983). Paranoia and the structure of powerlessness. Am Socio Rev, 48, 228–239.

188 REFERENCES

Miyamoto, J., Tsuji, M., Takeda, H., Ohzeki, M., Nawa, H., & Matsumiya, T. (2004). Characterization of the anxiolytic-like effects of ﬂuvoxamine, milnacipran and risperidone in mice using the conditioned fear stress paradigm. Eur J Pharmacol, 504 (1-2), 97–103. Mobbs, D., Petrovic, P., Marchant, J. L., Hassabis, D., Weiskopf, N., Seymour, B., Dolan, R. J., & Frith, C. D. (2007). When fear is near: threat imminence elicits prefrontal- periaqueductal gray shifts in humans. Science, 317 (5841), 1079–83. Montague, R., Dayan, P., & Sejnowski, T. (1996). A framework for mesencephalic dopamine systems based on predictive hebbian learning. J Neurosci, 76, 1936– 1947. Moritz, S., Werner, R., & von Collani, G. (2006). The inferiority complex in paranoia readdressed: a study with the Implicit Association Test. Cogn Neuropsychiatry, 11 (4), 402–15. Moritz, S., & Woodward, T. S. (2007). Metacognitive training in schizophrenia: from basic research to knowledge translation and intervention. Curr Opin Psychiatry, 20 (6), 619–25. Moritz, S., Woodward, T. S., & Hausmann, D. (2006). Incautious reasoning as a pathogenetic factor for the development of psychotic symptoms in schizophrenia. Schizophr Bull, 32 (2), 327–31. Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006). Midbrain dopamine neurons encode decisions for future action. Nat Neurosci, 9 (8), 1057–1063. Morrison, A. P., French, P., Lewis, S. W., Roberts, M., Raja, S., Neil, S. T., Parker, S., Green, J., Kilcommons, A., Walford, L., & Bentall, R. P. (2006). Psychological factors in people at ultra-high risk of psychosis: comparisons with non-patients and associations with symptoms. Psychol Med, 36 (10), 1395–404. Morrison, A. P., Renton, J., Dunn, H., Williams, S., & Bentall, R. P. (2003). Cognitive therapy for psychosis: a formulation-based approach. Hove: Brunner-Routledge. Morrison, A. (1998). Treating complex cases: the cognitive behavioural therapy approach. In N. Tarrier, A. Wells & G. Haddock (Eds.). (Chap. Cognitive behaviour therapy for psychotic symptoms in schizophrenia, pp. 195–216). Chichester: Wiley. Moutoussis, M. (2010). Falling research in the NHS: Tackle deprofessionalisation and fragmentation. BMJ Letter; 340:c3456. Moutoussis, M., Bentall, R. P., El-Deredy, W., & Dayan, P. (2011). Bayesian modeling of Jumping-to-Conclusions Bias in delusional patients. Cognit Neuropsychiatry. Moutoussis, M., Bentall, R. P., Williams, J., & Dayan, P. (2008). A temporal difference account of avoidance learning. Network, 19 (2), 137–60. Moutoussis, M., Williams, J., Dayan, P., & Bentall, R. P. (2007). Persecutory delusions and the conditioned avoidance paradigm: towards an integration of the psychology and biology of paranoia. Cognit Neuropsychiatry, 12 (6), 495–510.

189 REFERENCES

Mowrer, O. (1947). On the dual nature of learning: a reinterpretation of conditioning and problem solving. Harv Educ Rev, 17, 102–148. Myin-Germeys, I., Marcelis, M., Krabbendam, L., Delespaul, P., & van Os, J. (2005). Sub- tle fluctuations in psychotic phenomena as functional states of abnormal dopamine reactivity in individuals at risk. Biol Psychiat, 58 (2), 105–10. National Collaborating Centre for Mental Health. (2010). Schizophrenia: the NICE guide- line on core interventions in the treatment and management of schizophrenia in adults in primary and secondary care. Updated Edition. British Psychological Soci- ety and Royal College of Psychiatrists. Leicester and London. Ndetei, D. M., & Vadher, A. (1984). Frequency and clinical significance of delusions across cultures. Acta Psychiatr Scand, 70, 73–76. Niv, Y. (2007). Cost, benefit, tonic, phasic: what do response rates tell us about dopamine and motivation? Ann N Y Acad Sci, 1104, 357–76. Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007). Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl), 191 (3), 507–20. Oaksford, M., & Chater, N. (1995). Information gain explains relevance which explains the selection task. Cognition, 57 (1), 97–108. Oaksford, M., & Chater, N. (2001). The probabilistic approach to human reasoning. Trends Cogn Sci, 5 (8), 349–357. O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. (2004). Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning. Science, 304, 452–454. Ogden, T. (1983). The concept of internal object relations. Int J Psychoanal, 64, 227–241. Pankey, J., & Hayes, S. (2003). Acceptance and Commitment Therapy for psychosis. Int J Psychol Psychol Ther, 3 (2), 311–328. Penny, W., Holmes, A., & Friston, K. (2003). Random effects analysis. In R. Frackowiak, K. Friston, C. Frith, R. Dolan, K. Friston, C. Price, S. Zeki, J. Ashburner & W. Penny (Eds.), Human Brain Function (2nd). Academic Press. Peters, E., Joseph, S., & Garety, P. (1999). Measurment of delusional ideation in the normal population: Introducing the PDI. Schizophr Bull, 25, 553–576. Pleskac, T. J., Dougherty, M. P., Rivadeneira, A. W., & Wallsten, T. S. (2009). Random error in judgment: The contribution of encoding and retrieval processes. J Mem Lang, 60, 165–179. Poremba, A., & Gabriel, M. (1995). The amygdala is necessary for the initial acquisition but not for maintenance of discriminative avoidance behavior in rabbits. Soc Neurosci Abstr, 21, 1930. Poremba, A., & Gabriel, M. (1999). Amygdala neurons mediate acquisition but not maintenance of instrumental avoidance behavior in rabbits. J Neurosci, 19 (21), 9635– 41.

190 REFERENCES

Pretzer, J. (2007). Cognitive therapy of personality disorders. In A. Beck, A. Freeman & D. Davis (Eds.). (2nd ed., Chap. Paranoid personality disorder, pp. 115–137). NY: Guilford Press. Puterman, M. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Pro- gramming. New York: John Wiley and Sons. Raftery, A. (1995). Bayesian model selection in social research. Sociol Methodol, 25, 111–163. Rassin, E., Merckelbach, H., & Muris, P. (2000). Paradoxical and less paradoxical effects of thought suppression: a critical review. Clin Psychology Review, 20, 973–995. Ratcliff, R., & Smith, P. L. (2004). A comparison of sequential sampling models for two- choice reaction time. Psychol Rev, 111 (2), 333–67. Raune, D., Bebbington, P., Dunn, G., & Kuipers, E. (2006). Event attributes and the content of psychotic experiences in ﬁrst-episode psychosis. Psychol Med, 36, 221–230. Reis, F. L., Masson, S., de Oliveira, A. R., & Brandao, M. L. (2004). Dopaminergic mechanisms in the conditioned and unconditioned fear as assessed by the two-way avoidance and light switch-off tests. Pharmacol Biochem Behav, 79 (2), 359–65. Rosenkranz, J. A., & Grace, A. A. (2001). Dopamine attenuates prefrontal cortical suppression of sensory inputs to the basolateral amygdala of rats. J Neurosci, 21 (11), 4090–103. Rosenkranz, J. A., & Grace, A. A. (2002). Cellular mechanisms of infralimbic and pre- limbic prefrontal cortical inhibition and dopaminergic modulation of basolateral amygdala neurons in vivo. J Neurosci, 22 (1), 324–37. Rushworth, M. F., & Behrens, T. E. (2008). Choice, uncertainty and value in prefrontal and cingulate cortex. Nat Neurosci, 11 (4), 389–97. Rycroft, R. (1995). A critical dictionary of psychoanalysis (2nd ed.). London: Penguin. Samaha, A. N., Seeman, P., Stewart, J., Rajabi, H., & Kapur, S. (2007). "Breakthrough" dopamine supersensitivity during ongoing antipsychotic treatment leads to treatment failure over time. J Neurosci, 27 (11), 2979–86. Satel, S., Southwick, S., & Gawin, F. (1991). Clinical features of cocaine-induced paranoia. Am J Psychiat, 148, 495–498. Schmajuk, N., & Zanutto, B. (1997). Escape, avoidance and imitation: a neural network approach. Adapt Behav, 6, 63–129. Schneider, K. (1949/1974). Themes and variations in european psychiatry: An anthology. In S. Hirsch & M. Shepherd (Eds.). (Chap. The concept of delusion, pp. 33–39). Translated by H. Marshall. Bristol, UK: John Wright and Sons. Schultz, W., Apicella, P., & Ljungberg, T. (1993). Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci, 13 (3), 900–13. Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275 (5306), 1593–9.

191 REFERENCES

Schwarz, G. (1978). Estimating the dimension of a model. Ann Stat, 6 (2), 461–464. Segal, H. (1994). Paranoia: new psychoanalytic perspectives. In J. Oldham & S. Bone (Eds.). (Chap. Paranoid Anxiety and Paranoia). Madison, Connecticut: Interna- tional Universities Press Inc. Seligman, M., & Johnston, J. (1973). Contemporary approaches to conditioning and learning. In F. McGuigan & D. Lumsden (Eds.). (Chap. A cognitive theory of avoidance learning, pp. 69–110). Washington, DC: Winston. Selten, J.-P., & Cantor-Graae, E. (2005). Social defeat: risk factor for schizophrenia ? Brit J Psychiat, 187, 101–102. Seymour, B., Daw, N., Dayan, P., Singer, T., & Dolan, R. (2007). Differential encoding of losses and gains in the human striatum. J Neurosci, 27 (18), 4826–31. Seymour, B., O’Doherty, J. P., Dayan, P., Koltzenburg, M., Jones, A. K., Dolan, R. J., Friston, K. J., & Frackowiak, R. S. (2004). Temporal difference models describe higher-order learning in humans. Nature, 429 (6992), 664–7. Shallcross, A. J., Troy, A. S., Boland, M., & Mauss, I. B. (2010). Let it be: Accepting negative emotional experiences predicts decreased negative affect and depressive symptoms. Behav Res Ther, 48 (9), 921–9. Shannon, H., Hart, J., Bymaster, F., Calligaro, D., DeLapp, N., Mitch, C., Ward, J., Fink- Jensen, A., Sauerberg, P., Jeppesen, L., Sheardown, M., & Swedberg, M. (1999). Muscarinic receptor agonists, like dopamine receptor antagonist antipsychotics, in- hibit conditioned avoidance response in rats. J Pharmacol Exp Ther, 290 (2), 901– 7. Sims, A., Mundt, C., Berner, P., & Barocka, A. (2000). New oxford textbook of psychiatry. In M. Gelder, L.-I. J. Jnr & A. N.C. (Eds.). (Chap. Descriptive Phenomenology). Oxford: Oxford University Press. Siuciak, J. A., Chapin, D. S., McCarthy, S. A., Guanowsky, V., Brown, J., Chiang, P., Marala, R., Patterson, T., Seymour, P. A., Swick, A., & Iredale, P. A. (2007). CP- 809,101, a selective 5-HT2C agonist, shows activity in animal models of antipsychotic activity. Neuropharmacology, 52 (2), 279–90. Smith, A., Becker, S., & Kapur, S. (2005). A computational model for the functional role of ventral-striatal d2 receptor in the expression of previously acquired behaviors. Neural Comput, 17, 361–395. Smith, A., Li, M., Becker, S., & Kapur, S. (2006). Dopamine, prediction error and associative learning: a model-based account. Netw Comput Neural Syst, 17, 61–84. Smith, A., Li, M., Becker, S., & Kapur, S. (2007). Linking animal models of psychosis to computational models of dopamine function. Neuropsychopharmacology, 32 (1), 54–66. Smith, N., Freeman, D., & Kuipers, E. (2005). Grandiose delusions: an experimental investigation of the delusion as defense. J Nerv Ment Dis, 193 (7), 480–7.

192 REFERENCES

Smith, P. L., & Ratcliff, R. (2004). Psychology and neurobiology of simple decisions. Trends Neurosci, 27 (3), 161–8. Solomon, R. L., & Corbit, J. D. (1974). An opponent-process theory of motivation. I. Temporal dynamics of affect. Psychol Rev, 81 (2), 119–45. Solomon, R., Kamin, L., & Wynne, L. (1953). Traumatic avoidance learning: the outcomes of several extinction procedures in dogs. J Abnorm Soc Psychol, 48, 291– 302. Solomon, R., & Wynne, L. (1953). Traumatic avoidance learning: acquisition in normal dogs. Psychol Monogr, 67, 19. Stompe, T., Friedman, A., Ortwein, G., Strobl, R., Chaudhry, H. R., Najam, N., & Chaudhry, M. R. (1999). Comparison of delusions among schizophrenics in Austria and in Pakistan. Psychopathology, 32 (5), 225–34. Sutton, R., & Barto, A. (1998). Reinforcement learning: an introduction. Cambridge, Massachusetts: MIT Press. Swanson, J. W., Swartz, M. S., Van Dorn, R. A., Elbogen, E. B., Wagner, H. R., Rosen- heck, R. A., Stroup, T. S., McEvoy, J. P., & Lieberman, J. A. (2006). A national study of violent behavior in persons with schizophrenia. Arch Gen Psychiatry, 63 (5), 490–9. Swazey, J. P. (1979). Chlorpromazine’s entrance into psychiatry. An extremely interesting product. Minn Med, 62 (2), 81–5. Tarrier, N., Lewis, S., Haddock, G., Bentall, R., Drake, R., Kinderman, P., Kingdon, D., Siddle, R., Everitt, J., Leadley, K., Benn, A., Grazebrook, K., Haley, C., Akhtar, S., Davies, L., Palmer, S., & Dunn, G. (2004). Cognitive-behavioural therapy in ﬁrst- episode and early schizophrenia. 18-month follow-up of a randomised controlled trial. Br J Psychiat, 184, 231–9. Thewissen, V., Bentall, R. P., Oorschot, M., Campo, J. A., van Lierop, T., van Os, J., & Myin-Germeys, I. (2010). Emotions, self-esteem, and paranoid episodes: An experience sampling study. Br J Clin Psychol. Tidey, J., & Miczeck, K. (1996). Social defeat stress selectively alters mesocorticolimbic dopamine release: an in vivo microdialysis study. Brain Res, 721, 140–149. Titov, S. A., Shamakina IIu, & Ashmarin, I. P. (1983). [Effect of lysyl vasopressin and va- sotocin on a disorder of the conditioned avoidance reaction by a serotonin receptor blockader]. Biull Eksp Biol Med, 95 (2), 31–3. Tobler, P. N., Dickinson, A., & Schultz, W. (2003). Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J Neurosci, 23 (32), 10402–10. Todorov, E. (2006). Linearly-solvable Markov decision problems. In B. Scholkopf, J. Platt & T Hoffman (Eds.), Proceedings of the Twentieth Annual Conference on Neu- ral Information Processing Systems (Vol. 19, pp. 1369–1376). Vancouver: Canada: MIT Press.

193 REFERENCES

Trower, P., & Chadwick, P. (1995). Pathways to defense of the self: a theory of two types of paranoia. Clin Psychol Sci Pract, 2, 263–278. Udachina, A., Thewissen, V., Myin-Germeys, I., Fitzpatrick, S., O’kane, A., & Bentall, R. P. (2009). Understanding the relationships between self-esteem, experiential avoidance, and paranoia: structural equation modelling and experience sampling studies. J Nerv Ment Dis, 197 (9), 661–8. Udachina, A., Varese, F., Oorschot, M., Myin-Germeys, I., & Bentall, R. (in submission). Dynamics of self-esteem in Poor-Me and Bad-Me paranoia. Udachina, A., Varese, F., Oorschot, M., Myin-Germeys, I., & Bentall, R. (personal communication). The role of Experiential Avoidance in paranoid delusions. Unger, W., Evans, I., Rourke, P., & Levis, D. (2003). The s-s construct of expectancy versus the s-r construct of fear: which motivates the acquisition of avoidance behaviour ? J Gen Psychol, 130, 131–147. Ungless, M. (2004). Dopamine: the salient issue. Trends Neurosci, 27, 702–706. Ungless, M., Magill, P., & Bolam, J. (2004). Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science, 303, 2040–2042. Unterrainer, J. M., & Owen, A. M. (2006). Planning and problem solving: from neuropsychology to functional neuroimaging. J Physiol Paris, 99 (4-6), 308–17. Usher, M., & McClelland, J. L. (2001). The time course of perceptual choice: the leaky, competing accumulator model. Psychol Rev, 108 (3), 550–92. Vazquez, C., Diez-Alegria, C., Hernandez-Lloreda, M. J., & Moreno, M. N. (2008). Im- plicit and explicit self-schema in active deluded, remitted deluded, and depressed patients. J Behav Ther Exp Psychiatry, 39 (4), 587–99. Volans, P. J. (1976). Styles of decision-making and probability appraisal in selected ob- sessional and phobic patients. Br J Soc Clin Psychol, 15 (3), 305–17. Wadenberg, M. L. (2010). Conditioned avoidance response in the development of new antipsychotics. Curr Pharm Des, 16 (3), 358–70. Wadenberg, M., & Hicks, P. (1999). The conditioned avoidance response test re-evaluated: is it a sensitive test for the detection of potentially atypical antipsychotics ? Neurosci Biobehav Rev, 23, 851–862. Wadenberg, M., Soliman, A., VanderSpek, S., & Kapur, S. (2001). Dopamine D(2) receptor occupancy is a common mechanism underlying animal models of antipsychotics and their clinical effects. Neuropsychopharmacology, 25 (5), 633–41. Wald, A. (1945). Sequential Tests of Statistical Hypotheses. Ann Math Stat, 16, 117–186. Warman, D. M., Lysaker, P. H., Martin, J. M., Davis, L., & Haudenschield, S. L. (2007). Jumping to conclusions and the continuum of delusional beliefs. Behav Res Ther, 45 (6), 1255–69. Watkins, C. (1989). Learning from delayed rewards. (Doctoral dissertation, University of Cambridge).

194 REFERENCES

Wijkstra, J., Lijmer, J., Balk, F., Geddes, J., & Nolen, W. A. (2005). Pharmacological treatment for psychotic depression. Cochrane Database Syst Rev (4), CD004044. Williams, J., & Dayan, P. (2005). Dopamine, learning, and impulsivity: a biological account of attention-deficit/hyperactivity disorder. J Child Adolesc Psychopharmacol, 15 (2), 160–79; discussion 157–9. Wise, R., Spindler, J., deWit, H., & Gerberg, G. (1978). Neuroleptic-induced "anhedonia" in rats: pimozide blocks reward quality of food. Science, 201 (4352), 262–4. Wolpert, D., & Miall, R. (1996). Forward Models for Physiological Motor Control. Neural Netw, 9 (8), 1265–1279. World Health Organisation. (1992). The ICD-10 International Classification of Men- tal and Behavioural Disorders. clinical descriptions and diagnostic guidelines (10th ed.). Geneva: WHO. World Health Organization. (1997). Schedules for clinical assessment in neuropsychiatry (SCAN version 2). Geneva: World Health Organization. Xu, F., & Tenenbaum, J. B. (2007). Word learning as Bayesian inference. Psychol Rev, 114 (2), 245–72. Yacubian, J., Glascher, J., Schroeder, K., Sommer, T., Braus, D. F., & Buchel, C. (2006). Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain. J Neurosci, 26 (37), 9530–7. Yoshida, W., Seymour, B., Friston, K. J., & Dolan, R. J. (2010). Neural mechanisms of belief inference during cooperative games. J Neurosci, 30 (32), 10744–51. Young, A. M. (2004). Increased extracellular dopamine in nucleus accumbens in response to unconditioned and conditioned aversive stimuli: studies using 1 min microdialysis in rats. J Neurosci Meth, 138 (1-2), 57–63. Young, H. F., & Bentall, R. P. (1997). Probabilistic reasoning in deluded, depressed and normal subjects: effects of task difficulty and meaningful versus non-meaningful material. Psychol Med, 27 (2), 455–65. Yu, A. J., & Dayan, P. (2005). Uncertainty, neuromodulation, and attention. Neuron, 46 (4), 681–92. Zullow, H., Oettingen, G., Peterson, C., & Seligman, M. (1988). Pessimistic explanatory style in the historical record. CA Ving LBJ, Presidential Candidates, and East versus West Berlin. Am Psychol, 43, 673–682.

195