sign in sign up
<<
Home , Moral psychology

Florida State University Libraries

Electronic Theses, Treatises and Dissertations The Graduate School

2011 The Fragmentation of Moral : Reason, , and Moral Judgment in and Science Christopher Zarpentine

Follow this and additional works at the FSU Digital Library. For more information, please contact [email protected] THE FLORIDA STATE UNIVERSITY

COLLEGE OF ARTS AND SCIENCES

THE FRAGMENTATION OF MORAL PSYCHOLOGY: REASON, EMOTION,

MOTIVATION AND MORAL JUDGMENT IN ETHICS AND SCIENCE

BY

CHRISTOPHER ZARPENTINE

A Dissertation submitted to the Department of in partial fulfillment of the requirements for the degree of Doctor of Philosophy

Degree Awarded: Spring 2011 The members of the committee approve the dissertation of Christopher Zarpentine defended on March 29, 2011.

______Alfred R. Mele Professor Directing Dissertation

______John Kelsay University Representative

______David McNaughton Committee Member

______Michael Ruse Committee Member

Approved:

______J. Piers Rawling, Chair, Department of Philosophy

The Graduate School has verified and approved the above-named committee members.

ii

Writing this dissertation made me realize how much I have learned from my parents. I dedicate this work to them.

iii

ACKNOWLEDGEMENTS

It would be true to say that this project started my very first semester of graduate school. Back in 2004, Eddy Nahmias and Zach Ernst taught the fMRI papers by Josh Greene and colleagues. I’m grateful to them for setting me on this path. But a project like this doesn’t get completed without a lot of help along the way. I should first thank my committee members: John Kelsay and David McNaughton—both of whom were pinch-hitters, stepping in to replace Frank Marlowe and Josh Gert, whose own career paths led a different way—and my advisor and committee chair, Al Mele, who has been absolutely irreplaceable. My colleagues in the Department of Philosophy have been terrific and provided support in numberous ways. Thanks to Steve McFarlane, Josh Shepherd and Aron Vadican for helpful discussion. Steve and Aron both offered generous comments on parts of this dissertation and Steve has provided very helpful discussion on a number of issues. Ali Brown has taught me how beer tastes after a hard day of writing; I’m grateful for her support and friendship. I must also thank Clifford Sosis, whose friendship and conversation have been with me my entire philosophical career. Heather Cipolletti has been my friend and collegue throughout this process and I am grateful to her. Michael and Lizzie Ruse have provided invaluable moral and dietary support throughout the process, as has the rest of the Sunday crew. My parents never wavered in their support, even when they wondered what was taking so long. Finally, Samantha Muka deserves a medal for her love and support over the last 5 years. I could not have done it without her.

iv

TABLE OF CONTENTS

Abstract ...... viii

1. INTRODUCTION...... 1 1.1 The new synthesis in moral psychology...... 1 1.2 Some distinctions...... 3 1.3 Moral judgment ...... 4 1.4 Preview...... 6

2. TOWARD AN INTEGRATED THEORY OF AND MORAL JUDGMENT...... 8 2.1 What is morality?...... 10 2.1.1 Philosophical accounts ...... 11 2.1.2 Psychological accounts and the moral/conventional distinction...... 15 2.1.3 and functional accounts of morality...... 17 2.2 A three-level functionalist account of morality...... 21 2.2.1 The function of moral concepts ...... 22 2.2.2 The content and reference of moral concepts ...... 25 2.2.3 Two worries ...... 30 2.2.4 Conclusion ...... 32

3. AND BEYOND ...... 34 3.1 Humean and Kantian moral psychologies...... 35 3.2 Dual process theory ...... 36 3.3 A cockroach in a glass of apple juice ...... 41 3.3.1 The cognitive-development and social interactionist approaches.... 42 3.3.2 The social intuitionist model...... 44 3.3.3 Problems for the social intuitionist...... 51 3.4 Trolleys, Ecologists and Terrorists...... 53 3.4.1 Greene’s dual process model ...... 56

4. CHALLENGING THE BASIC DUAL PROCESS MODEL ...... 63 4.1 Complicating the model: moral rules?...... 63 4.1.1 Causal representations ...... 66 4.1.2 Beyond the basic dual process model...... 70 4.2 Linguisticism...... 71

5. ‘KNOWING THE WORDS BUT NOT THE MUSIC’: AND THE ROLE OF EMOTION IN MORAL JUDGMENT ...... 77 5.1 What is a psychopath? ...... 77 5.2 in psychopaths...... 82

v 5.3 Moral judgment in psychopathy...... 83 5.4 Emotional learning...... 85 5.4.1 Amygdala dysfunction...... 86 5.4.2 Response reversal and vmPFC dysfunction...... 88 5.4.3 Psychopathy and the developmental hypothesis...... 90 5.5 Compensation...... 95

6. GAGE WAS “NO LONGER GAGE:” VMPFC DAMAGE AND THE ROLE OF EMOTION IN MORAL JUDGMENT ...... 100 6.1 vmPFC damage: the clinical profile ...... 101 6.2 vmPFC function: representing ...... 104 6.2.1 OFC-amygdala connections...... 108 6.2.2 control: DLPFC modulation ...... 115 6.2.3 Dissociations ...... 118 6.3 Moral judgment in vmPFC patients...... 122 6.4 Early onset vmPFC damage ...... 126 6.5 Psychopathology and emotion in moral judgment: conclusions ...... 127

7. CAUSAL COGNITION...... 130 7.1 What is causal cognition?...... 132 7.1.1 Concepts and causal cognition...... 137 7.1.2 Sketching the two-systems account...... 140 7.1.3 Evidence from learning and memory research ...... 142 7.2 A two-system account of causal cognition...... 144 7.2.1 Causal cognition: a variety of views ...... 144 7.2.2 Animal learning and causal cognition...... 146 7.2.3 Cognitivist accounts of causal cognition...... 150 7.2.4 Cues to causal structure ...... 155 7.2.5 The two-system explanation ...... 157 7.2.6 Awareness...... 159 7.3 Conclusion...... 165

8. CAUSAL COGNITION IN MORAL JUDGMENT ...... 166 8.1 Against linguisticism ...... 166 8.1.1 A preliminary consideration against linguisticism...... 167 8.1.2 The nativist challenge...... 169 8.1.3 An explanation of the trolley/footbridge asymmetry ...... 173 8.1.4 Some loose ends...... 177 8.1.5 Causation, deviance and mental state...... 178 8.2 Conclusion...... 184

9. MORAL PSYCHOLOGY: PUTTING THE PIECES TOGETHER...... 185 9.1 Computationalism...... 185 9.2 Multiple kinds of representations in moral psychology...... 189 9.2.1 Core cognition...... 191 9.2.2 Affect...... 192

vi 9.2.3 The medial temporal lobe system ...... 201 9.2.4 The fundamental computational bias ...... 202 9.3 Patterns revisited...... 202 9.4 Taking stock: linguisticism ...... 207 9.5 Conclusion...... 211

10. UNTANGLING MORAL PSYCHOLOGY...... 212 10.1 The moral problem...... 212 10.2 A neuropsychological argument against motivation internalism ...... 213 10.2.1 Characterizing motivation internalism ...... 214 10.2.2 Telling stories: for and against internalism...... 215 10.2.3 A new argument against internalism...... 216 10.2.4 Qualifying internalism...... 221 10.2.5 Explaining the connection between moral judgment and motivation...... 224 10.2.6 Fetishism...... 225 10.3 A neuropsychological problem for Humean theories of motivation ...... 227 10.3.1 Characterizing Humean theories of motivation ...... 228 10.3.2 A problem for Humean theories...... 233 10.3.3 A related worry: affective disorders...... 235 10.4 Dissolving the moral problem ...... 236 10.4.1 Humean externalism, anti-Humean internalism...... 237 10.4.2 Whither noncognitivism?...... 239 10.4.3 Making sense of complexity ...... 241 10.5 and rationalism ...... 242 10.5.1 Moral judgment and moral agency...... 242 10.5.2 Morphological rationalism...... 249 10.6 Conclusion: science and ethics...... 255

APPENDIX A...... 257

APPENDIX B...... 263

REFERENCES ...... 264

BIOGRAPHICAL SKETCH...... 307

vii

ABSTRACT

Increasingly, and neuroscientists have become interested in moral psychology and moral judgment. Despite this, much of moral philosophy remains isolated from this empirical research. I seek to integrate these two literatures. Drawing on a wide range of research, I develop an empirically adequate account of moral judgment. I then turn to issues in philosophical moral psychology, arguing that empirical research sheds light on old debates and raises new questions for investigation. The neuropsychological mechanisms underlying moral judgment exhibit a large degree of complexity. Different processes contribute to moral judgment under different conditions, depending both upon the kind of case under consideration and on individual differences. Affective processes subserved by a broad base of brain regions including the orbitofrontal cortex, ventromedial prefrontal cortex, amygdala, and basal ganglia are crucial for normal moral judgment. These affective processes also provide an important link to motivation. More explicit cognition dependent upon areas of the medial temporal lobe and the dorsolateral prefrontal cortex also play a crucial role in some kinds of moral judgment though they exhibit less direct connections to motivation. The descriptive account of moral judgment I defend makes sense of debates in moral psychology over two influential views: motivation internalism, according to which moral judgment necessitates motivation to act accordingly and the Humean Theory of Motivation, according to which belief and desire are distinct and motivation requires both a desire and an appropriate means-end belief. Moral judgments that derive from affective processes exhibit a connection between motivation and moral judgment. However, not all moral judgments derive from such processes. More explicit representations are not closely connected to motivation, thus motivation can come apart from moral judgment. While explicit beliefs are distinct from desires, affective representations have both cognitive (albeit nonpropositional) content and direct connections to motivation. This challenges Humean theories of motivation.

viii This account helps resolve these traditional disputes. Anti-Humean, internalist theories offer an approximately accurate account of these affective mechanisms. Externalist, Humean theories offer an approximately accurate account of more explicit cognitive processes. Thus, several prominent philosophical theories offer a plausible account of some aspect of moral psychology. Because of the complexity of moral psychology, none of these accounts offers a complete account. This account also raises new questions for investigation. Some researchers have argued that the representation of a moral rule like the Doctrine of Double Effect helps explain the pattern of judgments in response to different kinds of Trolley cases. I argue that these judgments are better explained in terms of the details of the associative mechanisms underlying these judgments and not in terms of the representation of a moral rule. These findings raise a unique concern about the evidential value of our intuitions in these cases—a concern that could not arise from armchair reflection alone. The approach taken in this dissertation illustrates how integrating the results of empirical research contributes to philosophical work in ethical theory.

ix

CHAPTER 1

INTRODUCTION

1.1 The new synthesis in moral psychology? During the first thirty years of the twentieth century, the field of biology was divided among competing evolutionary theories. By the 1930s, two groups of researchers working independently had each solved major problems faced by evolutionary theory. Population geneticists had developed rigorous mathematical methods in order to understand the phenomena of gradual adaptive change in populations by . Unaware of the work of the population geneticists, a group of European naturalists had managed to solve the problem of how speciation occurs. Later in the decade and into the early 1940s the field of biology was unified by the synthesis of the work of these two groups into a new theory of evolution, in what became known as the ‘Modern Synthesis’. The modern synthesis vindicated the ideas of Charles Darwin, and established an enormously successful scientific theory, gaining wide acceptance, and generating testable hypotheses and accurate predictions (Mayr 2005). Around thirty-five years later, biologist E.O. Wilson called for an extension of this synthesis. In his monumental and controversial book Sociobiology: The New Synthesis, he argued for the integration of sociobiology into evolutionary theory, as “the systematic study of the biological bases of all social behavior” (1975, 4). He writes: “It may not be too much to say that sociology and the other social sciences, as well as the humanities, are the last branches of biology waiting to be included in the Modern Synthesis” (1975, 4). Wilson envisions all branches of human knowledge integrated in a grand consilience. This amplified the vision of Darwin himself, writing over a century earlier in the final chapter of his On the Origin of Species: “In the distant future I see open fields for far more important researches. Psychology will be based on a new foundation…Light will be thrown on the origin of man and his history” (1859, 458). In the final chapter of Wilson’s book, devoted to human sociobiology, he goes even further

1 suggesting that, “Scientists and humanists should consider together the possibility that the time has come for ethics to be removed temporarily from the hands of the philosophers and biologicized” (1975, 562). Nearly another thirty-five years later, (2007) sees Wilson’s predicted integration of ethics into the new synthesis of sociobiology already in progress. He argues that beginning in the 1990s the new synthesis has “transformed the study of morality today” (2007, 998). In Haidt’s view the key factor in bringing about the new synthesis in moral psychology was the so-called affective revolution in psychology of the 1980s, a time of increasing research on emotion following two decades of almost exclusive focus on cognition. Philosophers have also been interested in issues in moral psychology, yet up until recently these two groups of researchers have been nearly as isolated from each other as population geneticists and naturalists had been in the time leading up to the modern synthesis. In their 1992 review of nearly 100 years of metaethics, Stephen Darwall, Alan Gibbard, and Peter Railton wrote, “Too many moral philosophers and commentators on moral philosophy…have been content to invent their own psychology and anthropology from scratch” (1992, 188-9). In the past ten years, this has begun to change as the work of many philosophers has been increasingly informed by results from the empirical sciences (e.g. Doris 2002; Greene et al. 2001; Greene et al. 2004; Nichols 2004a; Prinz 2007). By 2006, the Stanford Encyclopedia of Philosophy had added an entry entitled “Moral Psychology: Empirical Approaches” (Doris and Stich 2006). The goal of this dissertation is to add to this growing body of work at the intersection of science and philosophy. While I do not follow Wilson and Haidt in arguing that ethics should be synthesized with evolutionary theory or as Wilson put it: “biologicized,” I do think that the increased interaction between scientific and philosophical approaches to moral psychology can benefit both sides. Scientific theories of moral psychology can be improved through increased philosophical scrutiny. At the same time, increased attention to our best scientific theories about human moral psychology can improve the quality and relevance of much philosophical work.

2 1.2 Some distinctions In the previous section, I roughly characterized two different approaches to the the study of moral psychology. In this section and the next, I introduce some distinctions in order to clarify the difference between these two approaches and to focus on the subject of this dissertation. In the last section, I provide a preview of the rest of the dissertation. I have been discussing the distinction between these different approaches to the study of moral psychology as if it were largely a disciplinary division. In some ways, this is true. However, it would be misleading to draw the distinction in this way. With increasing interaction between these two approaches, the disciplinary boundaries have begun to blur. To take two examples, even as far back as the 1970s, E.O. Wilson was making what appeared to be normative moral claims. Likewise, Joshua Greene, who holds a Ph.D. in philosophy but currently works in a psychology department, has devoted much effort to defending a descriptive theory of moral judgment (as well as discussing the implications for moral philosophy). Partly in response to the interest in morality by comparative and evolutionary psychologists, Bernard Gert (2008) distinguishes between two different definitions of the word ‘morality’. According to Gert, the word ‘morality’ may be used descriptively to refer to a code of conduct put forward by a society, group, or individual, or normatively to refer to “a code of conduct that, given specific conditions, would be put forward by all rational persons” (2008). This is certainly on the right track. However, I am not sure that ‘morality’ is ambiguous in this way. Even if it is, we can still draw a distinction between approaches to the study of morality with different aims. I draw the distinction in the following way: descriptive approaches to the study of morality aim at correctly describing actual moral practices or belief systems. Such approaches aim, for example, to give an account of why and how people make the moral judgments that they make. Normative approaches aim at justifying a particular moral theory or theories or at articulating the true moral theory. They aim to offer an account of the truth or justification of moral judgments. In short, the aims of descriptive approaches to morality are explanatory; the aims of normative approaches are justificatory. This allows us to see these two approaches as different approaches to the same phenomena. We should be able

3 to describe extant moral beliefs and practices in such a way that—even if they are normatively problematic—we can still recognize them as moral beliefs and practices. Philosophical approaches to the study of morality traditionally divide the issues in moral theory into first-order and second-order concerns (e.g. M. Smith 1994, 2). First- order concerns include questions about, for example, the permissibility of or whether we are morally obligated to give to famine relief. These first-order concerns fall under what I will call substantive ethics. Second-order concerns involve questions about first-order concerns. These issues include the semantics of moral language, the ontology and epistemology of moral properties and facts, and issues in moral psychology. These second-order concerns fall under metaethics. It should be noted that the substantive/metaethical distinction and the descriptive/normative distinction cut across each other. For example, we can aim at describing either the first-order moral beliefs or practices of a society or group or their second-order metaethical beliefs (however inchoate or indeterminate they may be). Similar points apply to normative approaches to both substantive ethics and metaethics. The subject of this dissertation is a topic in moral psychology: moral judgment.

1.3 Moral judgment Imagine you are living in a remote stretch of jungle with a group of ecologists that includes eight children. The whole group is taken hostage by a band of terrorists. One of your captors takes a liking to you and tells you that the leader of the terrorists is going to kill you and the rest of the hostages the next morning. He offers to help you and the children escape, but in order to ensure that you won’t go to the police later, he demands that you kill one of the other hostages and allow him to video-tape the killing. If you refuse, all of the hostages including you and the children, will be killed in the morning. Is it morally permissible for you to kill one of your fellow hostages in order to escape and save the lives of the eight children? Dilemmas like the above have been used in studies of moral judgment by a number of researchers (e.g. Greene et al. 2001). Although most of us will never face a moral dilemma as dramatic as this, we do face moral dilemmas in our daily lives; we engage in moral reasoning and we make moral judgments. Given the role such

4 judgments play in human life, it is not surprising that moral judgments have been the subject of investigation by researchers in philosophy and the human sciences. Whereas traditional philosophical approaches to moral theorizing have often started with the materials available from reflection and the consideration of hypothetical cases, I begin by developing and defending a descriptive account of moral judgment, drawing on empirical work from a wide range of disciplines. I then go on to investigate how this account bears on issues of traditional philosophical interest in moral psychology. There is a broad body of empirical research on moral judgment upon which to draw. Though there is some work on morality in anthropology, there is little work on moral judgment itself. During the middle of the twentieth century, a few books at the intersection of anthropology and philosophy offered a promising start. Alexander Macbeath’s Experiments in Living (1952) reviewed much of the ethnographic research on ethics then available. Richard Brandt’s study Hopi Ethics (1954) combined anthropological scholarship, philosophical theorizing, and his own fieldwork. May and Abraham Edel’s Anthropology and Ethics (1959) was the product of interdisciplinary collaboration between an athropologist and a philosopher on ethics across cultures. In , research by (1932), (1969), Elliot Turiel (1983) and others has investigated the stages in the development of moral judgment, the role of reasoning in generating such judgments, and the differentiation of moral and conventional transgressions in the judgments of children and . More recently, social Jonathan Haidt (2001) has developed the Social Intuitionist Model of moral judgment, stressing the role of intuitive, emotional processes in generating moral judgments and giving moral reasoning a more indirect role in moral judgment. Based on their neuroimaging studies, Greene and his colleagues (2001; 2004) have developed a dual-process model of moral judgment in which both cognitive reasoning processes and emotional processes play a role in the generation of moral judgment. I review much of this work in Chapter 3. Othere have emphasized the analogy (originally suggested by Rawls 1971) between moral judgments and grammatical judgments (e.g. Hauser 2006a; Mikhail 2007). This approach, which I dub linguisticism, draws upon work in Chomskian

5 Linguistics to help inform the investigation of moral competence. I discuss this approach further in Chapter 4 and offer criticisms of it in Chapter 9. In philosophy, the nature of moral judgment has been the subject of much debate—but little consensus. One central issue in philosophical moral psychology is whether moral judgment is a kind of cognitive judgment, expressing something which can be true or false, or whether moral judgment is the expression of an emotion or a prescription, for example, the expression of one’s disapproval of an action. Another central issue is whether moral judgments are inherently motivational or instead rely on external to moral judgment (such as the desire to do the morally right thing) to produce behavior. Related to this, philosophers have been concerned with the relation between moral judgment and human rational faculties.

1.4 Preview Having discussed some of the issues to be addressed in this dissertation, it will be helpful to offer a preview of the chapters to come. In Chapter 2, I offer a rough characterization of moral judgment that is neutral between the descriptive and normative approaches. The goal is not to give an exhaustive analysis, but rather to give a rough account of the nature of morality and moral judgment that demonstrates how the descriptive and normative approaches are different approaches to the study of the same phenomenon. Moreover, as work in comparative ethics emphasizes, moral thinking and judgment are activities; they are things that we do in a social and cultural context: morality is embedded in the fabric of human social life. An adequate theory of moral judgment must acknowledge this fact. In Chapter 2, I address these difficulties while clarifying the subject of investigation for the remainder of the dissertation. In Chapters 3 through 9 I develop a descriptive account of moral judgment. I draw on work from several fields seeking to situate moral judgment in the broader context of human psychology. As noted above, Chapter 3 reviews work from developmental and as well as more recent research utilizing neuroimaging. I sketch a basic dual process model of moral judgment. Chapter 4 raises some issues for this basic dual process model and introduces linguisticism, its main theoretical competitor.

6 Linguisticism denies that contribute to moral judgment, emphasizing the importance of moral . In the next four chapters I develop a more model of moral judgment, according to which several kinds of process contribute to moral judgment. In Chapters 5 and 6 I draw primarily from research on psychopathology and discuss the contribution of emotion to moral judgment. Chapters 7 and 8 I develop an account of causal cognition and its contribution to moral judgment. This account offers an alternative to linguisticism. In Chapter 9 I draw together the discussion of the previous four chapters and argue against linguisticism. Chapter 10 returns to the philosophical issues discussed at the end of the previous section. I draw on the account developed in the previous chapters to shed light on the debate in philosophical moral psychology. I claim that this descriptive account points toward the resolution of certain issues in moral psychology as well as offering an explanation for the debate.

7

CHAPTER 2

TOWARD AN INTEGRATED THEORY OF MORALITY AND MORAL JUDGMENT

In this chapter, I develop a schematic account of morality and moral judgment. Since a primary goal of this dissertation is to investigate how empirical research bears on philosophical issues in moral psychology, particularly about the nature of moral judgment, it must be clear why such empirical work is relevant. The account offered in this chapter directly addresses this issue. The recent explosion of interest in morality and moral judgment across the sciences only underscores the importance of this project. A preliminary issue for such a project is that theoretical debate about the nature of morality is at different points in different fields. In their early foray into comparative ethics, May and Abraham Edel describe the state of things as follows:

Anthropology has no established definitions of the moral…Philosophers, on the other hand, have plenty of definitions. The trouble is, they have too many, so that controversy permeates the very starting-points. (1959, 7-8)

Although there has been some work since then, their comments still ring true. Anthropologist James Laidlaw has recently reiterated the former claim: “despite the interest shown in the matter by some of the very greatest anthropologists our discipline has not developed a body of theoretical reflection on the nature of ethics” (2002, 311). With regard to the latter claim, as is usually the case, philosophy has almost as many accounts of morality as it does philosophers writing on morality. I discuss some of these accounts later in the chapter (§2.1.1). The goal of this chapter is to establish an account which can serve as a starting point for a more integrated approach to morality.

8 I proceed by discussing and criticizing several accounts of morality and moral judgment in science and philosophy (§2.1). Drawing on some of these, I put forward my own schematic account and discuss some of the ways it needs to be filled in (§2.2). The aim of investigating the relation between scientific work and philosophical work on morality and moral judgment places an important constraint on this account. The account should remain neutral on as many substantive issues as possible. I call this the neutrality constraint. To clarify the neutrality constrain, consider an issue briefly discussed in the previous chapter (§1.2). Bernard Gert argues that the word ‘morality’ may be used descriptively to refer to a code of conduct put forward by a society, group, or individual, or normatively to refer to “a code of conduct that, given specific conditions, would be put forward by all rational persons” (2008). I expressed skepticism that ‘morality’ is ambiguous in this way and suggested that distinguishing between different approaches to the study of morality can remove the need to posit ambiguity. The account I offer here aims to be neutral with respect to descriptive and normative approaches to the study of morality, making sense of how they can be different approaches to the same object of study. Often the same phenomenon may be studied by researchers with different goals. For example, evolutionary biologists may be interested in the phylogenetic origin of a trait, while developmental biologists may be interested in the ontogenetic origin of that same trait. I have characterized descriptive approaches as those whose aim is to correctly describe actual moral practices or belief systems and normative approaches as those whose aim is the justification of moral theories (or beliefs). This helps to distinguish between the following two questions: (1) which judgments are moral judgments? and (2) which moral judgments are true?1 Gert’s normative sense of ‘moral’ can be seen as a particular answer to (2): the true moral judgments are the judgments that would be put forward by rational, impartial people. However, we can ask what makes a judgment a moral judgment, without

1 Catherine Wilson, for example, points out that we can distinguish the task of determining whether a rule is a moral rule from the question of whether anyone should endorse such a rule (2004, 8). Paul Taylor (1978) makes a similar point with regard to moral principles.

9 worrying about whether or not it is true, or whether or not it would be accepted by all rational, impartial persons. On Gert’s view, judgments that would not be agreed upon by all rational, impartial persons do not count as moral judgments (in his ‘normative’ sense). On the account I am putting forward, such judgments may be moral judgments, and if they are, then they may or may not be true (or justified).2 These two questions are distinct and require distinct kinds of answers. The first question is a conceptual question which must be addressed by both descriptive and normative approaches to the study of morality. Answering the second question is of central importance to normative approaches, though it may not be of much interest to those engaged in descriptive approaches. Here, I will largely focus on the first question. To do so, I will begin by discussing some accounts of morality and moral judgment in science and philosophy.

2.1 What is morality? A number of attempts have been made by researchers interested in morality to distinguish between moral rules, principles, or norms and nonmoral rules, principles, or norms, or to distinguish moral judgments from nonmoral judgments. In this section I discuss some philosophical accounts, drawing on work by Bernard Gert (§2.1.1). I then turn to work in psychology. In particular, I discuss accounts based on the moral/conventional distinction and recent criticisms of that distinction (§2.1.2). The discussion then turns to cultural psychology and functional accounts of morality (§2.1.3). I then offer a brief summary of this discussion (§2.1.4) and then proceed to my own account (§2.2). Before turning to these accounts, it will be useful to introduce some terminology. I will use the term ‘moral system’ to refer to the collection of moral beliefs and practices (including moral judgment), moral rules (or principles or norms) and so on that are (largely) shared by a group of people. I take ‘morality’ to refer to all such moral systems and a theory of morality to be applicable to all such moral systems. All moral systems

2 This dispute may be largely terminological. A philosopher with Gert’s view may still be interested in what he may label ‘putative moral judgment’, even if—on this view—some such judgments are not considered genuinely moral.

10 involve the use of moral concepts in their constituent beliefs and practices. The account of morality I go on to develop relies on this fact and our ability to identify moral concepts.3 In a discussion of moral judgment, (1965) distinguishes between two kinds: judgments of obligation and judgments of (moral) value. Each of these kinds of judgments may be either general, as in “You ought to keep your promises,” or particular, as in “Hitler was morally reprehensible.” A crucial point is that all moral judgments make use of moral concepts in their expression. With these preliminary remarks out of the way, I now turn to some accounts of morality and moral judgment on offer.

2.1.1 Philosophical accounts I begin by discussing the work of Bernard Gert. I do this for two reasons. First, Gert raises a serious problem for three traditional theories of moral judgment. The second reason is that his own account faces two difficulties that my account attempts to overcome. Gert criticizes a number of what he calls “linguistic theories of moral judgments,” on the grounds that they fail to adequately distinguish moral from nonmoral judgments . Gert discusses three such theories that have been popular in the philosophical literature: the imperative theory, the commending theory, and the emotive theory. The imperative theory construes moral judgments as commands: they function to instruct people what to do. According to the commending theory, moral judgments function as evaluations analogous to judgments like “That is a good painting.” The emotive theory regards moral judgments as expressions of emotion. As Gert points out, these theories are each plausible because moral judgments sometimes function in precisely the way each theory specifies. The problem is that moral judgments do not always function in any of these ways, and furthermore, nonmoral judgments may sometimes function in these ways as well. Thus, these theories fail to adequately characterize moral judgment. Gert offers the following account of morality:

3 I use ‘concept’ to denote mental representations and not abstracta (for some discussion, see Sutton 2004).

11 Morality is an informal public system applying to all rational persons, governing the behavior that affects others, and includes what are commonly known as the moral rules, ideals, and and has the lessening of or harm as its good. (2005, 27)

The problem with this account is that it does not allow us to distinguish moral judgments from nonmoral judgments in the absence of a complete account of morality. Gert himself says: “only after providing a clear account of the moral system, identifying the moral rules, moral ideals, and moral virtues and distinguishing them from other kinds of rules, ideals, and virtues, is it possible to distinguish moral judgments from nonmoral judgments” (309). One advantage of distinguishing the conceptual question “what makes a judgment a moral judgment?” from the normative question, “what makes a moral judgment true?” is that it allows us to gain a clearer descriptive view of moral judgment before we address the normative aspects of moral judgment. Gert’s account ties the making of a moral judgment too closely to the making of a true moral judgment and thus violates the neutrality constraint. Gert’s account may be plausible if it is considered as an answer to the normative question “what makes a moral judgment true?” However, on the account I offer later (§2.2), we are able to identify moral systems and moral judgments before we address their normative status. Publicity, morality, and religion. Another concern about Gert’s account lies in its separation of religion and morality on the basis that “every feature of morality must be known to, and could be chosen by all rational persons,” but, “No religion is known to all rational persons” (2005, 6). I do not want to deny that there is a difference between morality and religion. However, in many societies there is quite a bit of overlap. Using publicity to distinguish between the two does not help because religious views may very well be common knowledge in a particular society. A society may have a moral system very closely linked to their religious views, these religious views may be learned by all members beginning at a very young age and they may provide the basis for their views of personhood or . Such a society may even be sufficiently isolated that they are unaware that other human societies with distinct moral systems exist. They may well

12 consider their moral system to be public and universal. This moral system might ultimately be normatively problematic, but I see no reason to deny that it is a moral system. 4 An additional problem is raised by Gert’s claim that “One of the characteristics that a person must have to be subject to moral judgment is knowledge of a very general sort,” for example, that “one person can inflict pain or disable another person, and persons do not normally want to be inflicted with pain or disabled” (2005, 24). Individuals in many societies may share such beliefs, but disagree with regard to when pain, for example, is justifiably inflicted. And they may do so for metaphysical or religious reasons which, though universal in their society, are not held by the members of other societies. It may not be easy to distinguish such religious beliefs from basic metaphysical beliefs about the nature of persons or freedom. For example, a communitarian minded society may have a very different conception of personhood from a Western, liberal conception.5 This may have effects on their views about the appropriate relations between people. Nevertheless, such a society may still have a system recognizable as a moral system. In Alexander Macbeath’s early treatment of ethics in the ethnographic literature, he examines the relation between primitive morality and religion (1952). His survey reveals quite a bit of diversity. He finds that there are some societies whose moral codes have no supernatural sanctions, there are other societies where most moral rules have supernatural sanctions, and there are—most commonly, he suggests—societies where some moral rules have such sanction but most do not. All societies seem to have purely religious duties which have only supernatural sanction (1952, 336-50). The relation between morality and religion across societies is not entirely surprising. However, Macbeath also found that there were no systematic cross-cultural differences between the

4 While Gert prefers to think of such differences as variations in one system of a universal morality, he does admit that if one wants to think about these as different moralities sharing the same framework, nothing rests on this point (2005, 3-4). As noted above, I will speak in terms of different moral systems. Whether such systems will ultimately converge on a single universal system or whether irresolvable disagreement will remain are empirical questions, which—on some views (e.g. M. Smith 1994)—have normative importance. 5 See the discussion of cultural psychology below (§2.1.3).

13 rules that had supernatural sanctions and those that did not. Where overlap occurs, it seems to depend on the particular features of individual societies. This indicates that simple and general criteria of demarcation between morality and religion may be impossible. Macbeath goes on to discuss several ways that religious beliefs and practices can influence morality. Religious sanctions can create additional incentives to follow moral rules. Belief in supernatural or magic forces—even if mistaken—can have real psychological and social effects. They can “give primitive man the hope and confidence which enable him to face the difficulties and overcome the dangers with which his life is beset” (1952, 346). The rituals of religion can serve functions for the individual and the group: they “refresh and reinvigorate the individual; they strengthen the bonds which bind the members of the group together; they promote good will and mutual ” (1952, 347). Religious duties may function to enhance the self-discipline of individuals and their loyalty to the group. Religious sanctions may provide an beyond that of the individual or group, which can increase the likelihood of the observance of such rules. Attachment of religious sanctions to moral rules can ground the latter in the picture of moral perfection offered by religious views. The flip side of the stabilizing effects that religious sanctions may provide for moral rules is that such rules may become resistant to change—even in the face of changing circumstances. Despite the interconnections between morality and religion across many societies, the basis of Macbeath’s discussion is that they are nevertheless different. It is unlikely that there is a general account that easily sorts the beliefs or judgments of a society into moral and religious categories. Sorting out the differences between morality and religion is something that may have to be done on a case-by-case basis. This does not mean that there cannot be guidelines for such an undertaking. However, this is a problem for any account of morality or moral judgment. The criticism here is that Gert’s account fails to adequately appreciate the complexity of the problem. This discussion of Gert’s work brings out three points: (1) that we must be wary of accounts which identify moral judgment with a single function, (2) accounts of moral judgment which are too closely tied to normative issues may run afoul of the neutrality constraint, and (3) some sophistication is required for an account to successfully

14 distinguish morality from religion. In the next section, I discuss work in developmental psychology on the moral/conventional distinction.

2.1.2 Psychological accounts and the moral/conventional distinction In developmental psychology, a popular approach to morality attempts to characterize moral rules in opposition to conventional rules. Over the past 30 years, a large body of research (e.g. Turiel 1983; Turiel, Killen, and Helwig 1987; Smetana 1993; Nucci 2001), has found a distinction between moral and conventional rules in the judgments of individuals across cultures and ages, ranging from adults to children as young as three. On the basis of this evidence, it is argued that this distinction is both psychologically real and significant. This approach has received some attention in philosophy as well. Shaun Nichols (2004a) offers an account of ‘core moral judgment’ which focuses on judgments about harm-based violations.6 His account draws on this work in developmental psychology. Recently, however the conclusions this research in developmental psychology has been criticized in a series of articles by Dan Kelly and co- authors (D. Kelly and Stich 2007; D. Kelly et al. 2007; Nado, D. Kelly, and Stich 2009). I briefly discuss this research and Kelly et al.’s criticism of it. The core of the experimental paradigm in this literature is the moral/conventional task. Researchers give participants scenarios representing typically moral or conventional violations and ask them to make judgments about the following: (a) whether the action is wrong, and if so, how serious it is; (b) whether the action’s wrongness depends on the pronouncement of an authority figure; (c) whether the scope of the rule violated is general and applies to everyone, or only to a restricted group of people in certain circumstances; and (d) whether the participants justify the rule with reference to harm, , or . Kelly and Stich (2007) identify three core results:

6 The criticisms discussed later do not refute Nichols’ view because he is specifically concerned with harm violations, not moral violations in general. The account I propose later attempts to capture this wider notion of morality. For similar reasons, I will not here discuss philosophers like Thomas Scanlon who state explicitly that they are not concerned with morality “in the broad sense” but only “our duties to other people” (1998, 6).

15 (1) Responses in the moral/conventional task generally tend to cluster around two response patterns. In the moral response pattern, rules are judged to be authority-independent, general in scope, violations are wrong and judged more serious, and are justified by appeal to harm, justice, or rights. In the conventional response pattern, rules are judged to be authority-dependent, not general in scope, violations are less serious wrongs, and are not justified by appeal to harm, justice, or rights. (2) The moral response pattern is generally triggered by violations involving harm, rights, or justice, while the conventional response pattern is given in response to violations that do not involve harm, rights, or justice. (3) The results discussed in (1) and (2) are pancultural and emerge early in development.

While there are many studies consistent with these conclusions, there are also several studies which appear to undermine each of these results. Stephen Stich (2007) has gone so far as to suggest that these results are largely an artifact of the narrow range of scenarios used in this research. In a study of the judgments of children in traditional Arab villages in Israel, Mordecai Nisan (1987) found that all tested violations elicited the signature moral pattern, despite the fact that some of these did not appear to involve harm, rights or justice—for example, co-ed bathing or addressing a teacher by his first name. Similarly, Jonathan Haidt and his colleagues, in a study of adults and children in Philadelphia and

Recife and Porto Alegre, Brazil found that low socioeconomic status participants in both places judged certain actions, such as washing a toilet with the national flag or having sex with a chicken carcass before cooking and eating it, to be seriously and generally (not locally) wrong, and that this wrong was authority-independent (Haidt, Koller, and M.G. Dias 1993).7 Even participants who agreed that these violations did not involve harm made such judgments. These studies demonstrate that harmless violations do not always elicit the conventional response pattern in some groups, thus challenging conclusions (2) and (3).

7 For the full text of some of these scenarios, see Appendix A.

16 A study by Shaun Nichols (2002) found that disgusting actions (e.g., spitting into one’s water glass before drinking out of it at a dinner party) were judged by children to be serious, authority-independent, and general violations. College students in America judged these violations to be serious, authority-independent, but not general in scope. Studies like this demonstrate that the clustering of response patterns can break down, challenging (1). Since they involve transgressions not involving harm, rights or justice, they also represent another challenge to (2). Another study challenges the conclusion that harm violations elicit the moral response pattern (D. Kelly et al. 2007). Participants were asked whether it was OK to whip a drunken sailor on cargo ship 300 years ago, given that it was common practice and not illegal. Many participants judged that this rule was not authority-independent or general—despite the fact that it involved harm. A full 52% of participants said that it was OK to whip a drunken sailor 300 years ago, but only 6% said that it is OK today (D. Kelly and Stich 2007, 364). There are further studies which provide support for this kind of criticism of the moral/conventional distinction.8 However, these suffice to indicate how the moral/conventional distinction can break down in different groups. Furthermore, while morality in Western cultures tends to focus on violations involving harm, rights, or justice, the domain of morality across cultures appears to be much broader. A satisfactory account of morality must accommodate this variability. In the next section, I discuss work in cultural psychology which attempts to do just that.

2.1.3 Cultural psychology and functional accounts of morality Richard Shweder and his colleagues have developed the idea that there are three distinct codes of ethics, each of which may be emphasized by different cultures to different degrees. They are the ethics of , the ethics of community, and the ethics of divinity (Shweder et al. 1997). The ethics of autonomy focuses on individuals

8 Heather Cipolletti and I extended these results seeking to identify some of the factors that affect the pattern of judgments made. Preliminary results indicate that systematically varying the level of , the degree to which a response is inappropriate, and the severity of the consequences of the same action produce patterns of judgments along a spectrum between the conventional response pattern and the moral response pattern. This provides further evidence that people’s judgments do not always fall into either the signature moral or conventional response patterns.

17 and the role of morality is to increase individual autonomy. It aligns well with the conception of morality from developmental psychology discussed above, which emphasizes harm, rights, and justice. The ethics of community focuses on social hierarchy. The focus is on duties of respect and obedience based on an individual’s position in the social hierarchy: their age, sex, caste, etc. In the ethics of divinity, the focus is on purity. It prohibits disgusting acts that may ‘pollute’ the individual, even if they harm no one. While morality in Western nations is largely exhausted by the ethics of autonomy, work in cultural psychology indicates that in some non-Western cultures the scope of morality is significantly wider (Shweder et al. 1997; Haidt, Koller, and M.G. Dias 1993).9 Subsequent work has sought to link each of these codes of ethics to distinct . and his co-authors have argued that the three main other-regarding moral emotions—contempt, , and disgust—map onto the three codes of ethics: community, autonomy, and divinity, respectively (P. Rozin et al. 1999). (This is called the CAD hypothesis, because of the correspondence of first letters between the codes of ethics and the moral emotions associated with them). Building on this work, Jonathan Haidt and Craig Joseph set out to discover the “psychological primitives” of morality by examining morality across cultures and the evolutionary precursors to morality in nonhuman primates (2004; 2007b). They claim that there are five innate, psychological foundations for morality, each corresponding to a particular adaptive challenge faced by our ancestors and each associated with one or more moral emotions. These five foundations are: (1) harm/care, focused on protecting kin and associated with ; (2) fairness/reciprocity, focused on cooperation between individuals and associated with , anger, and gratitude; (3) in-group/loyalty, focused on group cooperation and associated with pride in one’s group and rage at traitors; (4) authority/respect, focused on successful navigation of human social hierarchy and associated with fear and respect; and (5) purity/sanctity, focused on avoiding microbes and parasites and associated with disgust.

9 It is interesting to note that there is some evidence that political conservatives—as opposed to liberals—in America make use of a broader conception of morality as well (Haidt and Joseph 2007a).

18 On the basis of this work, Haidt develops his account of morality. He claims: “Moral systems are interlocking sets of values, practices, institutions, and evolved psychological mechanisms that work together to suppress or regulate selfishness and make social life possible” (2008, 70). A similar account has been put forward by Catherine Wilson. She offers the following ‘semi-essentialist’ account of moral rules: “Moral rules are restrictive and prohibitory rules whose social function is to counteract short- or long-term advantage possessed by a naturally or situationally favored subject” (2004, 9). Both of these accounts are functional accounts of morality: they identify morality in part by reference to some function, e.g., regulating selfishness, or reducing advantage. These accounts certainly handle the issue of diversity much better then previous accounts, however they face another problem. This problem is related to Gert’s criticism of the imperative, commending, and emotive theories of moral judgment. His concern was that, while moral judgments may sometimes function in the ways specified by each theory, they do not always function in any of these ways, and additionally, nonmoral judgments may function in these ways as well. Similarly, while morality may well sometimes or even usually function in these or other ways, we should not identify morality with the performance of a single function, whether evolutionary, social, or otherwise. Such accounts face a serious dilemma. If the function proposed is too specific, it risks making the account of morality too idiosyncratic. Such an account would fail to appreciate the diversity of morality across cultures. If the proposed function is too broad, it may fail to distinguish morality from religion or other domains which can also contribute to such social or evolutionary functions. Such broad functional accounts run the risk of making it a matter of definition that every society has a moral system.10

10 Some have claimed that Colin Turnbull’s description of the Ik of Uganda portrays a population approaching a state of society without a moral system (1972). Turnbull’s account is, however, controversial (see Heine 1985; for some discussion, see Prinz 2007, 275). Others, for example Robert Parkin, have criticized Emile Durkheim on the grounds that he “so conflated the moral with the social that ethnographers could not isolate for analysis those moments of moral reflexivity that…so typify human activity and predicaments” (1985, 4-5). Though it may in fact be true that all societies have moral systems, this should not be so as a matter of definition.

19 This dilemma represents a long-standing concern in attempts to give an account of morality. Edel and Edel, for example, distinguished between ethics wide and ethics narrow (1959, 8-12; see also Brandt 1954, 55). Ethics wide conceives of morality as dealing with all values and aims of humans: anything that is characterized as good. Ethics narrow, on the other hand, focuses on moral phenomena related to the ideas of duty and obligation.11 The strategy adopted by Edel and Edel was to cast a wide net and risk including elements in the survey that would later be excluded. I think this is a promising strategy. We need not settle these issues a priori. Rather, we can deal with them by considering morality as system which functions in a family of ways. We can rely on an intuitive understanding of morality as a starting point from which to begin investigation. Richard Joyce follows a similar strategy in offering an account of human moral judgment, not in terms of necessary and sufficient conditions, but rather as an accounting of “important characteristics” of moral judgment: moral judgments often express conative attitudes, but they also express beliefs; they are impartial and do not depend on the interests of the individuals involved; they purport to be “inescapable” and to “transcend human conventions;” they apply to interpersonal relations; they “imply notions of desert and justice;” and the emotion of guilt functions as “an important mechanism for regulating one’s moral conduct” (Joyce 2006, 70). I adopt a similar approach here, offering a three-level functionalist account of morality, which relies on identify moral concepts as those that function in a family of related ways. I then go on to offer an account of moral judgment. It should be noted that I am not denying that there might be an essence to morality that could be discovered a posteriori. If there is, discovering it will require much more research. At this point I am agnostic. Nevertheless, I think the framework I offer is a

11 This distinction is similar to ’ distinction between ‘ethics’ and ‘morality’. He writes: “I am going to suggest that morality should be understood as a particular development of the ethical, one that has a special significance in modern Western culture. It peculiarly emphasizes certain ethical notions rather than others, developing in particular a special notion of obligation…” (2006, 6). He uses ‘ethical’ as a broad term to refer to the subject matter of moral philosophy. While this is a helpful distinction, I do not adopt Williams’ usage and will continue to use the terms ‘moral’ and ‘ethical’ largely interchangeably, and thus to use ‘moral’ broadly unless otherwise indicated. The account offered below depends on the identification of moral concepts, not on the peculiarity of some subset of such concepts.

20 good starting point. At the end of this chapter I offer an account of moral judgment. This should not be taken as a definition but as a schematic for a full account.

2.2 A three-level functionalist account of morality The first part of this chapter identified two important components of a theory of morality: moral emotions and moral concepts. It also raised a number of difficulties which must be addressed. Accommodating the appearance of moral diversity across cultures and delineating the boundaries of morality within a culture are two of the biggest difficulties. In what follows I attempt to address these and other difficulties. In offering this account, I will sometimes flag issues which I take up in later chapters. For example, I take up discussion emotions again in Chapter 3 and subsequent chapters. For now, the focus is on moral concepts. This account has three levels, but the second and third level fit together closely. The first level offers a way of distinguishing moral concepts from nonmoral concepts. The second level offers an account of the content and reference of particular moral concepts. My account of these latter two levels is related to a family of views that have come to be known as the ‘Canberra Plan’ (Braddon-Mitchell and Nola 2009). Views of this kind have been developed most thoroughly by David Lewis (1970; 1972) and Frank Jackson (1998). I discuss the first level of this account in §2.2.1 and the second and third levels in §2.2.2. In §2.2.3 I consider two worries about the view offered and conclude in §2.2.4 by briefly discussing how this account deals with the issues raised in the first half of this chapter. The starting point for my account is the idea that moral systems essentially make use of moral concepts. As a component of moral practices, moral judgments necessarily make use of moral concepts in their expression. How then, are we to identify moral concepts? As suggested earlier, we must begin with our intuitive understanding. We may think immediately of such concepts as DUTY, OBLIGATION, and the moral OUGHT— the concepts expressed by some of the moral terms of English. This is a start; however, we cannot simply list the moral terms of English. This is so for at least two reasons.

21 First, it is not always clear when such terms are being used to express moral concepts. Words like ‘ought’, which seem to have a clear moral import, may be used nonmorally. For example, the use of ‘ought’ in “You ought to believe q if you believe p and p implies q,” does not express a moral concept. Words like ‘must’ or ‘should’ which do not have explicit moral import are often used to express moral concepts, as in “You must not steal from the elderly!” Second, moral concepts may be expressed by different terms in different languages. This may suggest that what we really want is an accounting, not of moral terms, but of moral concepts. Consider the following method. Presumably, we can easily recognize the moral terms of English. From there we could generate the moral concepts expressed by moral terms in English, and then we could get a handle on moral terms in other languages by seeing if terms in other languages express these concepts. There are several problems with this suggestion. First, there is always the possibility of introducing new moral concepts, so we could never be that sure our accounting was complete. Second, it does not account for the possibility of alien moral concepts. Other moral systems may have concepts which simply have no counterpart in English. Just as beings with different visual systems may see alien colors, beings which belong to another culture or species may have concepts that are distinct from our moral concepts, but which are nevertheless, moral concepts. Richard Joyce considers the example of European explorers encountering the Polynesian concept TAPU (Joyce 2001, 1; see also MacIntyre 2007, 111-113). At the time, the Europeans did not know what to make of this concept, yet it seems to be recognizable as a moral concept. Third, a brute accounting of moral concepts does not explain why these are the moral concepts. In the next section I offer a proposal which attempts to address these problems.

2.2.1 The function of moral concepts I suggest that we identify moral concepts by the way they function. Moral language is very useful, having many but also very particular uses. Concepts like DUTY for example, have a special function. Any community that lacks such a concept has a moral system that is significantly different from ours. However, rather than identify morality with any particular function, I propose to identify moral concepts by reference to

22 the family of uses that moral language has. With some grasp on how moral concepts function, we can utilize the method of working from the moral terms of English outward from our moral concepts to the moral concepts of other language groups. This method allows us to utilize our intuitive understanding of the function of moral concepts as a starting point, while not falling victim to the problems of a simple accounting of the moral concepts expressed by terms in English. It gives us a way of recognizing new and alien moral concepts as moral concepts, on the basis of the way they function, and it explains a concept’s status as moral concept in terms of a family of related functions. We have already seen many of the functions of moral concepts in the context of moral judgments: commands, commendations, and the expression of emotions. Across different contexts, moral judgments can function in a variety of ways. An understanding of the functions of moral judgments and language is an important part of being a member of a moral community. Terms like ‘duty’ and ‘ought’ express concepts that have distinctive purposes for humans. They differ from purely descriptive terms like ‘red’ and ‘tall’ and even from other normative terms like ‘should’ as used in the following sentence: “If you want your house to be stable, you should use building materials of this strength.” In addition to deontic terms like ‘duty’, there are evaluative terms like ‘good’, and so-called thick terms like ‘courage’. These also serve particular kinds of functions when used morally. Some of the ways moral concepts are distinctive are captured by Joyce’s characterization of moral judgments discussed above (§2.1.3). Similarly, Paul Taylor identifies six characteristics of moral norms. Moral norms are (1) general in form, (2) universal in scope, (3) intended to be overriding, (4) intended to apply independently of the preferences of any particular individual, (5) intended to be publicly adopted and recognized, and (6) must give equal consideration to every person (Taylor 1978, 37). Again, I do not think we should construe this list as giving necessary conditions, but rather as a rough account of the ways moral concepts can function in the expression of norms. This suggestion for identifying terms that express a moral concept can be illustrated by reflecting on the way we resolve ambiguous judgments. For example, how

23 would we determine what a speaker means by saying ‘You ought not eat meat’? She might mean that eating meat is morally wrong, or that it offends her aesthetic sensibilities, or even that for health reasons, the rational person should not eat meat. We can ask her further questions in order to establish what was meant: whether ‘ought’ was meant to express a moral concept or a rational one, or some other nonmoral normative type of concept. We can ask what kinds of considerations might undermine the judgment (if any). For example, if someone is anemic, or doesn’t care about their health, or is not repulsed by animal flesh, then the judgment might be withdrawn. If none of these considerations undermine the judgment, we might start to suspect that the judgment is a moral judgment. We can use a similar method when trying to identify the moral concepts of another language, though it may be more difficult. Of course, there may not be always be a determinate answer. This is to be expected—as Gert is right to point out: morality is an informal system (2005, 27). We can always try to resolve ambiguity by appeal to the relevant counterfactuals, e.g., whether a particular consideration would lead to withdrawal of the judgment—but some indeterminacy might remain. Similarly, there are problems when we attempt to identify moral concepts across cultures. The example of TAPU highlights the fact that the task of translation is not always straightforward. Accordingly, it may sometimes be difficult to determine whether such concepts ought to count as genuinely moral or not. This is further complicated by the fact that some languages do not explicitly have moral terms. Anthropologists Anita Jacobson-Widding’s account is representative:

When I first began to do fieldwork among the Shona-speaking Manyika of Zimbabwe…I tried to find a word that would correspond to the English concept ‘morality’. I explained what I meant by asking my informants to describe the norms for good behavior toward other people. The answer was unanimous. The word for this was tsika But when I asked my bilingual informants to translate tsika into English, they said that it was ‘good manners’. And whenever I asked someone to define tsika, they would say: ‘Tsika is the proper way to greet people’, or ‘Tsika is to show respect’. (Jacobson-Widding 1997, 48)

24

Obviously this is not a straightforward matter. Above (§2.1.1), I claimed that distinguishing moral systems from religions systems must be done on a case-by-case basis. The same point holds for these difficulties. We must address them on a case-by- case basis by focusing on the family of functions which moral concepts serve. Some indeterminacy may always remain: there is a complex relationship between morality and religion in most, if not all, societies. We can, however, get some handle on this by thinking of the ways morality can function and seeing how some of these functions may overlap with religion. I hold that these difficulties are unavoidable; they will be faced by any account of morality. Though they are not simple to address, I think it is an advantage of the account on offer here that it squarely faces up to them and provides a method for addressing them. Thus, the first level of this account distinguishes moral concepts from other concepts by reference to a family of functions. The next section gives an account of the content and reference of particular moral concepts.

2.2.2 The content and reference of moral concepts The most influential development of the Canberra Plan in ethics has been in the work of Frank Jackson under the name ‘moral functionalism’ (Jackson and Pettit 1995; Jackson 1998). Moral functionalism gets its name from the well-known view in the , where, according to Jackson, “we have a network of interconnected and interdefinable concepts that get their identity though their place in the network” (1998, 130). Rather than , in ethics, we have “folk morality: the network of moral opinions, intuitions, principles and concepts whose mastery is part and parcel of having a sense of what is right and wrong, and of being able to engage in meaningful debate about what ought to be done” (1998, 130). Thus, the content of moral concepts is given by their place in this network. The details of folk morality are still a matter of debate. Jackson suggests that these details will be settled through a process of reflective equilibrium characterized by (1971), in which we “attempt to balance compelling general principles against considered judgments about how various options should be characterized” (1998,

25 133). He calls the product of such a process mature folk morality. Ultimately, the mature folk morality will give us the content of particular moral concepts. To do this, we articulate the mature moral theory replacing terms expressing moral concepts with property names. We then generate a Ramsey sentence by replacing all mention of moral properties (e.g. ‘rightness’, ‘fairness’) with variables bound by existential quantifiers. We can identify the moral properties, rightness, say, as whatever property fills the rightness role according to the theory of mature folk morality (Jackson 1998, 140-3). The determination of whether and what properties fill such roles is accomplished through empirical inquiry.12 Such is an exceptionally brief description of Jackson’s view. I want to contrast it with my view by focusing on the details of generating a folk theory of morality: how exactly do we go about doing this and what materials do we utilize?13 Jackson’s account involves reflective equilibrium on the basis of mastery of folk morality and where conceptual inquiry proceeds from the armchair, in isolation and (in some sense) prior to empirical inquiry. The account I favor embraces a base of materials for theory construction wider than a mastery of folk morality and sees conceptual inquiry as properly done in concert with empirical inquiry. I begin by discussing the role of reflective equilibrium. Reflective equilibrium comes in narrow and wide forms (Daniels 1979). In its narrow form, reflective equilibrium works by mutual adjustment of our intuitions about particular cases and general principles, aiming to produce a theory that best unifies these. Wide reflective equilibrium takes into account a set of considered judgments, principles, and a set of

12 Moral functionalism as defended by Jackson is explicitly cognitivist. It might be argued that this begs the question against noncognitivists and thus violates the neutrality constraint. I think that this account could be suitably modified by noncognitivists. Most expressivists make room for some form of truth-conditions or application-conditions. So ‘extension’ may be replaced by a suitably expressivist-friendly notion. The foregoing comments notwithstanding, plausible contemporary accounts provide roles for both cognitive and conative elements. Pure seems to face immense difficulties (e.g. the Frege-Geach Problem, see Geach 1965). A number of recent accounts are explicitly hybrid, for example, David Copp’s (2001) realist-expressivism, Michael Ridge’s (2006) ecumenical expressivism, and Richmond Campbell’s (2006; 2007) belief- desire theory. I discuss noncognitivism further below (§10.4.2). 13 Others have questioned this part of Jackson’s account as well (see, e.g. Denis Robinson 2009).

26 relevant background theories. Jackson’s characterization of reflective equilibrium sounds like its narrow form: we balance “general principles” against “considered judgments” on the basis of our mastery of folk morality. There are well-known criticisms of reflective equilibrium (for example Brandt 1979; Stich 1990). Brandt, for example, worries that the method of reflective equilibrium might amount to nothing more than a “reshuffling of moral prejudices” (1979, 21). The diversity of morality across cultures discussed above, underscores this worry. Similarly, above I discussed several kinds of indeterminacy which appear in the attempt to distinguish moral concepts from nonmoral concepts. Similar kinds of indeterminacy will reappear at the level of particular moral concepts. Consider the dispute between deontological and consequentialist accounts of what is right. Do committed deontologists and committed consequentialists (merely) have different concepts of rightness, or is their dispute more substantive? It is not easy to see how to go about resolving such issues or how mastery of folk morality alone (even in the context of sustained critical reflection) could provide the resources to do so. The way my account deals with such concerns is to expand the base of materials relevant for theory construction in ethics. Rather than rely only on what is available from the armchair, conceptual work needs to engage with the relevant empirical work. According to the approach I will follow, conceptual inquiry and empirical inquiry are tackled together, rather than separately. In other fields of inquiry, we do not demand a complete conceptual analysis before we embark on empirical work. Rather, we begin with a rough and ready conceptual understanding and proceed, realizing that we will likely make modifications to our conceptual account as inquiry proceeds. I propose to take the same approach to moral theorizing. Before I continue, it will be useful to consider one commonly raised issue. This is a concern about whether my account risks changing the subject. Taking an example from the philosophy of mind, Jackson puts the worry as follows: “if we give up too many of the properties common sense associates with belief as represented by the folk theory of belief, we do indeed change the subject, we are no longer talking about belief” (1998, 38). Thus, if the moral theory we end up with rejects many of the beliefs we started with,

27 then we may be accused of changing the subject—we are no longer talking about morality. I am not persuaded by this worry. If one accepts a causal-historical theory of reference, we may continue to be talking about the same thing, provided the right causal grounding is there, even if many of our beliefs about that thing turn out to be false (for further discussion, see Lycan 2009). Still, I’m not sure why we should expect to find that all of our beliefs remain unchanged. In other fields, our common-sensical understanding of a phenomenon often provides only a starting point. Physicists simply do not worry that their theories do not align with folk physics.14 In ethics, it is not clear how much our theorizing is necessarily constrained by our pre-theoretic beliefs. Constraining moral theorizing too much by such constraints simply raises Brandt’s concern that such a theory represents no more than a reshuffling of prejudices. We must be open to the fact that our pre-theoretical beliefs may need to be revised. On the account I am proposing the second and third levels of this account are closely related. Revision of our concepts may be motivated either by theoretical considerations or failure of reference. There is no guarantee that all of our concepts successfully refer. Some of our current moral concepts may simply find no place in the final theory. In fact, moral functionalism does not guarantee that any of our moral concepts refer to instantiated moral properties. This is the view of error theorists like John Mackie (1977). This raises an extremely difficult issue. How much are we allowed to revise our moral concepts, before we are simply talking about something else? In other

14 Moreover, in the case of physics, we also have an explanation for the initial plausibility of folk physics. First, it is approximately true. Second, psychology has uncovered psychological mechanisms that explain the pervasiveness of certain aspects of folk physics (see, e.g. B. Smith and Casati 1994). Of course, folk physics does not remain static either. Many aspects of theoretical physics are widely known and so change the content of folk physics. In the case of folk morality, I will argue both that aspects of folk morality are approximately true and also that psychology can offer an explanation for the initial plausibility of certain aspects of moral psychology. What these considerations seem to illustrate is that folk physics and folk morality play their most crucial role at the outset of inquiry. After the early stages of theory building, their evidential value will be limited, especially after there is an explanation of their initial plausibility. See further discussion in Chapters 10 and 11.

28 words, what is the difference between eliminativist error theories and revisionist realist theories? David Lewis, for example, speaking about values, writes:

Strictly speaking, Mackie is right: genuine values would have to meet an impossible condition, so it is an error to think there are any. Loosely speaking, the name may go to a claimant that deserves it imperfectly…What to make of the situation is mainly a matter of temperament. (D.K. Lewis 1989, 93)

On one hand, philosophers like Richard Joyce argue that any account which fails to capture features like the “inescapable practical authority” of morality cannot be taken to be theories of morality at all (even “loosely speaking”) (Joyce 2006, 208-9; see 199-209 for discussion). On the other hand, Simon Blackburn (1985) criticizes Mackie (1977), on the grounds that, after arguing for his error theory, he goes on to do what appears to be straight-forward moralizing. To Blackburn, this undermines the original claim that such a practice was in error. He writes:

It would be much more natural to say that Hume and Mackie moralize, just as ordinary people do, but with a developed and different theory about what it is that they are doing. The error theory then shrinks to the claim that most ordinary moralists have a bad theory, or at least no very good theory, about what it is to moralize, and in particular that they falsely imagine a kind of objectivity for values, obligations, and so on. (Blackburn 1985, 150)

To the extent that this issue is substantive and not terminological, it appears to rest on what ordinary people think they are doing when they engage in moral activity. My account explicitly acknowledges the relevance of such empirical evidence, and thus, while it does not immediately offer an answer, it does have the resources to address it.15 To summarize the three level account: at the first level, moral concepts are distinguished from nonmoral concepts by reference to a family of functions played by

15 For some relevant work, see e.g. Nichols (2004b). This research is far from conclusive, and much work remains to be done.

29 moral concepts. At the second level, the content of particular moral concepts is picked out by their place in the network of moral concepts. The content of moral concepts is used to specify the extensions of such concepts at the third level, i.e. to say, for example, which (kinds of) acts fall under a particular moral concept. In the next section I discuss how this account deals with two worries.

2.2.3 Two worries One issue raised by Gert is that there are constraints on the appropriate targets of moral judgment (2005, 21). Generally, they apply to actions, intentions, people, and character traits. How does my account accommodate this constraint? On this view, moral concepts are picked out by the ways they function. Given those functions, applying moral concepts in certain ways does not make sense. It does not, for example, make sense to make moral judgments about inanimate objects like rocks. Actually, this conclusion is too quick. The constraint on what moral judgments may appropriately apply to must be taken to refer to a proper subset of moral judgments. One reason for this is the existence of examples like “Rocks cannot do anything morally wrong,” and “All actions by rocks are permissible.” These seem to be moral judgments (and some of them may even be true). Walter Sinnott-Armstrong discusses a similar problem faced by moral nihilists who wish to deny the truth of moral facts (2006, 32-37). This problem is raised by assertions like “Abortion is not morally wrong,” “If abortion is wrong, so is contraception,” and “Either abortion is wrong or euthanasia is not” (2006, 33). Presumably, nihilists will agree with such assertions (provided assertions like the former are not taken to entail that interfering with an abortion is morally wrong), but then nihilists seem committed to the truth of certain moral facts—which is what they wish to deny. Sinnott-Armstrong discusses a number of strategies for rendering nihilism coherent. One such strategy is for nihilists to refine their thesis to the denial of “positive moral facts” (2006, 34). One proposal for defining positive moral facts relies on the entailments of supposed moral facts:

Error theorists could define a supposed moral fact as positive when and only when its description entails any claim about what it is morally wrong to do or not

30 to do, what anyone morally ought or ought not to do, what is morally good or bad, and so on. (Sinnott-Armstrong 2006, 35)

We can adopt a similar strategy in dealing with the problem raised above about the appropriate objects of moral judgment. The constraint on the appropriate objects of moral judgments should be taken to apply to what can be called positive moral judgments. More tentatively, such judgments can be defined by reference to the same entailments specified in the right hand side of the biconditional in Sinnott-Armstrong’s proposal.16 A second worry is raised by ’s assertion that “Morality is necessarily connected with such things as justice and the common good, and it is a conceptual matter that this is so” (2002, 92). Similarly, Gert suggests that to speak of “Nazi morality” is paradoxical, because it is to speak of an “immoral morality” (2005, 10). My account relies on morality serving a family of functions and so makes no necessary conceptual connection between morality and the common good. Despite the evil perpetrated, there is no obstacle to thinking of Nazi Germany as having a moral system. This worry can be resolved by appreciating the three levels of this account. The first level allows us to pick out the moral concepts by appreciating the functions of such concepts. Whether or not Nazi morality was really a moral system depends on whether or not their society included beliefs and practices which utilized concepts which functioned in a family of related ways. Historian Claudia Koonz offers a compelling case that Nazi Germany did have a moral system: a peculiar kind of ethnic relativism (2003). The proper diagnosis of the error of Nazi morality is not that they lacked moral beliefs, but that they had false moral beliefs (in addition to false nonmoral beliefs), i.e. that their beliefs were normatively problematic. But this does not challenge the framework proposed here. It is no surprise that adherence to a false morality should give rise to radically immoral behavior—but contra Gert, there is nothing paradoxical about this. Rather than think of justice as conceptually connected with morality, as Foot does, what constrains the content of moral principles are facts about human biology and

16 In what follows, unless otherwise noted, discussion of moral judgments should be taken to apply to the proper subset of positive moral judgments.

31 social life. These are not conceptual constraints: they are constraints based in the contingencies of human life. As anthropologist Clyde Kluckhohn put it, “All cultures constitute so many somewhat distinct answers to essentially the same questions posed by human biology and by the generalities of the human situation” (1953, 520). Other species could have radically different biologies or social structures from humans; they might nevertheless have a kind of moral system. Facts about human biology and social life will constrain the possible moralities that can arise, but the extent of variation is a matter for empirical investigation. Foot mistakes an empirical generalization based in these contingent constraints on moral concepts and their extensions for a necessary conceptual truth about morality. What is necessary is that moral concepts fit into a family of functions. As explained above, this does place constraints on, for example, the things to which moral judgments can apply, and these may be conceptual constraints.

2.2.4 Conclusion The three-level account of morality helps clarify the two questions asked at the beginning of this chapter: (1) what makes a judgment a moral judgment? and (2) what makes a moral judgment true? A judgment is a moral judgment if it makes use of a moral concept, picked out at the first level. Finding out which moral judgments are true will depend on determining the extension of these moral concepts. This three-level account also addresses the issues raised in the first half of this chapter. It accommodates the broad and varying range of morality across cultures by identifying morality with a family of functions. It also provides a method for delineating the realm of morality within a particular society. It does so without begging the question on normative issues, thus satisfying the neutrality constraint. Additionally, this account remains neutral with respect to relativism, pluralism, and objectivism about morality. It leaves open the possibility of (hypothetical) agreement (given enough time and suitably motivated inquiry) on the content of particular moral concepts. On some accounts (e.g. M. Smith 1994), such convergence is necessary for their being any moral truths at all. It also leaves open the possibility of pluralism if more than one distinct moral system could be true, or relativism if irresolvable disagreements persist.

32 I can now offer a schematic account of moral judgment. A moral judgment is whatever can be expressed by the utterance of a sentence which (a) makes use of a term which expresses a moral concept (as specified by their functions), (b) is either a statement of a principle17 or the application thereof to a particular case, and (c) has an appropriate object.18 In short: a judgment is a moral judgment if it functions like a moral judgment, given a noncircular specification of such functioning. This is purposefully schematic. However, it provides enough guidance to develop a descriptive account of moral judgment. I turn to this in the next several chapters. We have looked at some of the ways moral judgments and moral language function. However, moral judgments also have a certain place in the psychological economy of individuals: indeed, that is how the expressions of moral judgments are able to perform the functions that they do. So how do they fit? Within individuals, moral judgments seem to have a dual nature. On the one hand, they seem to have cognitive content: they seem to be about something, they appear to ascribe properties to appropriate objects, and at least some of them seem to be true or false. On the other hand, moral judgments also seem to have a deep connection to conative elements. In fact, examination of the empirical literature will reveal that moral judgment is subserved by a variety of complex psychological mechanisms. Let us now turn to this research.

17 I take principles to include general statements of obligation and (moral) value. See §2.1 above. 18 As noted above (§2.2.3), this should be taken to be an account of positive moral judgments.

33

CHAPTER 3

DUAL PROCESS THEORY AND BEYOND

In the previous chapter, I laid out a framework for an integrated theory of morality. Here, the focus narrows to one central feature of moral practice: moral judgment. In recent years, there has been extensive empirical research on moral judgment, especially among Western participants. On the framework discussed in the previous chapter, we locate moral beliefs and practices across cultures, by working outwards from our own moral concepts. This is not an instance of ethnocentrism, but a recognition of the fact that we must rely on our own conception of morality (and its functions) as a starting point. While this cross-cultural diversity raised problems for some of the accounts in Chapter 2, it may still be that these beliefs and practices are subserved by the same kind of underlying psychological mechanisms. Understanding the development and function of these mechanisms in Western participants could help us understand their function and development across cultures. In this chapter and the next, I build the case for the view that moral judgment depends on a wide variety of different kinds of cognitive and affective processes. I begin by setting the stage with some brief historical remarks regarding the differences between Humean and Kantian moral psychologies. Though I lay no particular claim to the views of either Hume or Kant, my project is firmly in line with Hume’s overall project of situating an account of morality in human nature. However, the best way of pursuing this project is not to rely on 18th century accounts of human nature, but rather to draw on our best 21st century psychology and neuroscience. I introduce a recent version of dual process theory that I use as a broad framework for understanding the research discussed in the rest of this chapter. I introduce two recent models of moral judgment: Jonathan Haidt’s social intuitionist model and Joshua Greene’s dual process model. I review a wide range of research bearing on these models. I argue that the social intuitionist model gets part of the puzzle right. Greene’s dual process theory adds another important piece. Recent evidence, however, suggests that Greene’s model also needs to be supplemented. In the following chapter, I review some work critical of accounts like Greene’s

34 and in subsequent chapters I develop a more complex account that accommodates these concerns.

3.1 Humean and Kantian moral psychologies On many issues in philosophical moral psychology, and in ethical theory more broadly, the views attributed to and stand in stark opposition to one another. This is especially so for the issues which are of interest here: the role of reason and emotion in moral judgment and their importance for morality, and the nature of practical reason especially as it pertains to the role of moral belief in moral motivation. Some brief comments about these differences sets the stage for further discussion.1 The full title of Hume’s first book is A Treatise of Human Nature: Being an attempt to introduce the experimental method of reasoning to moral subjects. As he articulates in the introduction, he seeks to apply the experimental methods, which had already been so successful in the natural sciences, to what he calls “moral subjects:” human reasoning, morality, criticism, and politics (Hume 1739, 4). His view places morality in the context of an account of human nature. For Kant, morality (as opposed to what he calls “practical anthropology”) is based not on human nature, but on the a priori conclusions of reason (Kant 1785, 4). Thus for Kant, reason is the foundation of morality. Hume does not deny reason a role in morality, but for him morality emerges not from reason but from sentiment. Reason plays at most a secondary role. In practical reason, Humean theories assert that belief and desire are distinct existences and so moral beliefs alone cannot motivate action. Thus, in the well-known phrase: “Reason is, and ought only to be slave of the passions” (Hume 1739, 195). Kantian theories of practical reason deny this, claiming that reasoned moral belief alone has the power to produce motivation (e.g. T. Nagel 1970). The proper motivation for moral action is not sentiment (e.g. benevolence or generosity) as it is for Hume, but duty. The possibility of moral action does not depend on motivations external to morality. It is interesting to look back at Hume because his ideas have had a wide sphere of influence. He was obviously important among the British moral sentimentalists of the 18th century, a movement which had some effect on Kant. Hume’s work also helped pave the way for the development of by in the 19th century (Slote 2005). In

1 For more detailed discussion, see Denis (2008).

35 addition, the influence of Hume is also often claimed by proponents of a variety of noncognitivist and antirealist views including emotivist, expressivist, quasi-realist, and error theoretic accounts, as well as versions of response-dependent realism and virtue theory. It should be clear that the approach adopted here is much more in line with Hume’s project. But beyond that, I do not want to lay specific claim to any particular aspect of Hume’s thought. As just mentioned, a significant number of contemporary theories trace their lineage back to Hume. This is possible, in part, because there is significant controversy over what Hume’s view actually was. It is likely that resolution of this controversy will require understanding how Hume’s remarks on morality fit with his overall (Garrett 1997, chap. 9). It is my goal to pursue a broadly Humean project by drawing on contemporary empirical research. Thus, in the next section I briefly introduce dual process theory, which provides a convenient contemporary idiom in which to discuss the issues in moral psychology that Hume and Kant grappled with.

3.2 Dual process theory Over the last 15 years, dual process theories have been increasingly appealed to in social and cognitive psychology (e.g. Sloman 1996; Chaiken and Trope 1999; Stanovich 1999; Evans and Frankish 2009). Keith Stanovich lists over 20 dual process theories (Stanovich 2004, 35). The roots of this idea can be traced back to ’ distinction between association and reasoning (James 1890, 329-330). More recently, work in the 1970s (Evans and Wason 1976) and especially work by and in the heuristics and biases program (Kahneman and Tversky 1982) has led to the development of dual process theory. Kahneman writes “Tversky and I always thought of the heuristics and biases approach as a two- process theory” (Kahneman 2000, 682). There is also some overlap with Jerry Fodor’s distinction between modules and central processing (Fodor 1983). It is commonly suggested (e.g. Kahneman and Sunstein 2005) that such distinctions capture traditional distinctions between ‘intuition’ and ‘reason’. It will be useful to introduce a recent account of dual process theory here, in order to help unify discussion of recent empirical research on moral judgment. The label ‘dual process theory’ is in some ways misleading. First, the terminology adopted by Keith Stanovich (1999) distinguishing between System 1 and System 2 suggests that the theory posits two distinct systems. A better characterization of this distinction is between two

36 kinds of processes. It is believed that there are many systems involved in so-called System 1 processing, leading Stanovich (2004) to label them The Autonomous Set of Systems (TASS). For these reasons, I will adopt the terminology of Evans (2009) and distinguish between Type 1 processes and Type 2 processes. In characterizing dual-process theory, I rely heavily on Stanovich’s recent model (2009a; 2009b). As the label given by Stanovich (2004) suggests, an important characteristic of Type 1 processes is their autonomy. These processes are triggered automatically by certain stimuli, they occur quickly without conscious attention or effort, they do not rely on input from higher-level processing, and they can occur in parallel with other Type 1 processes. Type 1 processes include emotional processes, the processing done by the modules postulated by evolutionary psychologists, , and responses that have been automatized through repetition. Type 2 processing, sometimes referred to as analytic processing, contrasts with all of the attributes of Type 1 processes. They are generally consciously initiated and maintained and so require conscious effort and computational resources; they are slow and involve serial processing, so generally only one Type 2 process can occur at one time. It is the kind of process which occurs during what is called “conscious problem solving” (Stanovich 2009b, 22). Since many of these features can be a matter of degree, rather than think in terms of only two kinds of processes, it is better to see Type 1 processes and Type 2 processes as two ends of a spectrum with a significant amount of variety in between. Additionally, Stanovich’s recent dual- process model distinguishes between two different levels within Type 2 processing (2009a). In fact, this model bears some similarity to ’s tri-partite theory of the soul offered in the Republic. Type 1 processes loosely correspond to the appetitive part of the soul. The rational part of the soul can be identified with the Type 2 processes identified by Stanovich (2009a) as the reflective mind. The reflective mind houses “higher-level goal states and epistemic thinking dispositions” (Stanovich 2009, 157). What Stanovich calls the algorithmic mind can be identified with the spirited part of the soul as it is largely subservient to the goals and attitudes of the reflective mind. I discuss this distinction between levels of Type 2 processing further below. One of the primary functions of Type 2 processes is to override the results of Type 1 processes. This helps explain why dual process theory has been thought so useful for understanding the heuristics and biases literature. For Type 2 processing to override Type 1 processing requires two things. First, Type 1 processing must be interrupted and the Type 1

37 response suppressed. Second, Type 2 processing must substitute a better response in place of the suppressed Type 1 process response. This usually involves hypothetical reasoning. Accounts of such reasoning suggest that hypothetical reasoning involves creating mental models of the world and simulating possible actions and their effects. When doing this, it is important to maintain the distinction between real representations of the world and simulated representations. This is sometimes called decoupling. Type 2 processes involve maintaining the decoupled representation while the process of simulation takes place. It is thought that such processes incur heavy computational costs that are thought to explain the inability of Type 2 processes to be executed in parallel (Stanovich 2009b). To illustrate the relation between Type 1 and Type 2 processes, consider the following question:

A bat and a ball cost $1.10 in total. The bat costs $1 more than the ball. How much does the ball cost?2

Many people immediately answer “10 cents.” That is the quick response generated by Type 1 processing. Of course, further reflection reveals that that cannot be correct; it would require the total to be $1.20. Succeeding on this task requires overriding the Type 1 response—this requires recognizing that one’s initial response was in error, and then quickly figuring out the right answer utilizing Type 2 processing. Because Type 1 processes occur without conscious monitoring, we do not have access to the causes of responses generated by Type 1 processes. Thus, when we must interpret such responses, we often confabulate. Type 2 processes can generate plausible reasons—but they don’t necessarily have any connection with the Type 1 processing which generated the response in the first place—rather, they are based on a priori theories. In many cases, we don’t notice such confabulation. However, such processes can be dramatically illustrated in split-brain patients (Gazzaniga 1998). Richard Nisbett and Timothy Wilson review evidence for such processing in normal individuals (Nisbett and T.D. Wilson 1977; see also T.D. Wilson 2002). Sometimes, however Type 2 processes don’t involve full-blown decoupled representation. For example, some Type 2 processes involve what Stanovich calls “serial

2 From Frederick (2005).

38 associative cognition with a focal bias” (2009a, 68). The idea is that such processing is not quick like Type 1 processes, but does remains relatively inflexible, utilizing a single representation that is either given by the task or involves the most easily constructed model. For example, consider the following problem:

Jack is looking at Anne but Anne is looking at George. Jack is married but George is not. Is a married person looking at an unmarried person?

(A) Yes (B) No (C) Cannot be determined.3

Most people answer (C): the answer cannot be determined. But, the answer is actually (A). Most people do engage in some Type 2 processing to reach the answer (C). They engage in enough serial associative cognition to determine that, since the problem does not say whether Anne is married, that it is not known whether Jack is looking at an unmarried person, or whether a married person is looking at George. Some people infer from this that the answer cannot be determined. However, whether or not Anne is married, the answer is yes. If she is married then she is looking at George, who is unmarried. If she is not married then Jack is looking at her. Stanovich distinguishes between the algorithmic mind and the reflective mind. The algorithmic mind handles the implementation of the goals based on the beliefs and desires at the level of the reflective mind. The algorithmic mind can be evaluated for its efficiency, but because it is largely directed by the reflective mind, errors due to poor reasoning strategy lie at the level of the reflective mind. In cognitive psychology the differences among the reflective mind are known as thinking dispositions or cognitive styles and include things like need for cognition, open-minded thinking, need for closure, and others (Stanovich 2009b, 31). This distinction is also supported by evidence of dissociation in testing and psychopathology. The failure to engage in fully decoupled simulation can be a result of a pervasive feature of human cognition, what Stanovich calls the “fundamental computational bias” (Stanovich 1999, 190): the fact that humans are “cognitive misers” (Stanovich 2009b, 63). When humans

3 This is adapted by Stanovich et al. (2008) from Levesque (1986).

39 engage in Type 2 processing, such as simulating a state of affairs, they will often depend on the most easily constructed model. Consider, for example, the Grandfather Paradox. Some have argued that time travel is impossible on the grounds that if it were possible, someone could go back in time and kill her grandfather, which would then mean that she could never have existed producing a contradiction (e.g. Dowden 1993, 202). However, as David Lewis (1976) has pointed out, this does not follow at all. What does follow is that if time travel is possible and one goes back in time intending to kill her grandfather, she knows in advance that she will fail: her existence is proof that she failed to kill her grandfather. Such a world would be strange, but not impossible. Sometimes Type 2 processing does not “involve simulating alternative worlds and exploring them exhaustively” (Stanovich 2009b, 175). When we think about the Grandfather paradox, we may be easily misled by the stipulation that someone kills her grandfather. This involves a contradiction and so we assume that time travel is impossible. We do not consider the possibility that one could successfully go back in time and fail to kill her grandfather. There is nothing contradictory about time travel, the plausibility of the argument depended on not engaging in fully decoupled simulation. There is some evolutionary rationale for the cognitive. Brains are expensive to produce and maintain so, as Richerson and Boyd put it, “animals are under stringent selection pressure to be as stupid as they can get away with” (Richerson and Boyd 2005, 135). Similarly, animals are often under time pressure to make decisions. Engaging in fully decoupled simulation to determine whether the tiger tracks by the watering hole were really caused by a tiger will likely leave one with no ancestors with the inclination to engage in such decoupled simulation. The contention to be defended here is that processes of all these varieties can contribute to moral judgment. For example, we know that manipulating emotions can affect moral judgment. We also have evidence that some nonaffective Type 1 processes play a role in moral judgment. For example, in some cases, people’s judgments accord with the Doctrine of Double Effect (2004, 9), even though very few people can produce this rule as a justification for these judgments. Other rules are consciously accessible suggesting that they might be a result of Type 2 processes. For example, people appear to use the rule that harms due to acts are worse than those due to omissions (e.g. Cushman, Young, and Hauser 2006). Additionally there are

40 processes that weigh costs and benefits of actions (Nichols and Mallon 2006). I return to this work in the next chapter. In the remainder of this chapter, I discuss two recently proposed accounts of moral judgment in order to illustrate some of the evidence which supports this contention. I begin by discussing Haidt’s social intuitionist model and evidence which demonstrates the role of affect and emotion in moral judgment (§3.3). This represents a shift from earlier work which emphasized the role of moral reasoning. In §3.4, I discuss Joshua Greene’s dual process model according to which some moral dilemmas engage both a negative emotional response and a competing ‘cognitive’ response to the beneficial consequences of a harmful action.

3.3 A cockroach in a glass of apple juice Imagine I offer you a glass of your favorite juice. There’s one catch, before I give it to you I’m going to dip a sterilized dead cockroach in it. Still interested? How about a piece of chocolate fudge shaped like dog feces?4 Why not? There’s no danger. The cockroach has been sterilized and then removed before you drink it. And after all, fudge is fudge, who cares what it is shaped like, right? Of course, there is a difference. Cockroaches and dog feces are disgusting. Their presence or appearance can affect our judgments even when there is no real reason to be concerned. While these cases are not moral, there is a similarity between our reluctance to drink ‘roached’ juice and certain moral judgments. Consider the Chicken case:

A man goes to the supermarket once a week and buys a dead chicken. But before cooking the chicken, he has sexual intercourse with it. Then he cooks and eats it. (Haidt, Koller, and M.G. Dias 1993, 617)

When using such cases in moral judgment interviews, Jonathan Haidt and his colleagues (Haidt, Koller, and M.G. Dias 1993; Haidt, Bjorkland, and S. Murphy 2000; Haidt and Hersh 2001) discovered that participants will often quickly judge that the action is wrong, and then try to justify their judgment with reasons. Participants would sometimes pause, laugh, or stutter. Even when unable to give a plausible reason, participants stand by their judgment, sometimes just

4 These examples come from a study by Rozin et al. (1986)

41 asserting the judgment: “It’s just wrong to have sex with a chicken” (Haidt, Koller, and M.G. Dias 1993). Haidt and Hersh (2001) called this phenomena moral dumbfounding.

Recall that Haidt et al.’s participants included participants from Brazil (Recife and Porto Alegre) and the United States (Philadelphia) of varying socio-economic status. They found that some participants, primarily “college students at elite universities,” did not moralize such harmless but offensive actions—despite having similar affective responses to such stories (as measured by participants’ response to a probe about whether or not they would be bothered by the act under judgment) (Taylor 1978, 37). In the rest of their participants, they found that the best predictor of participants’ judgments in such cases was their affective responses.5 This research has had an important impact on moral psychology. It challenged the hegemony of rationalist approaches and led to an appreciation of the importance of affect and culture in moral psychology. More concretely, it led to the development of Haidt’s social intuitionist model of moral judgment. I discuss these rationalist approaches in the next section (§3.3.1). In the following section I discuss Haidt’s social intuitionist model and the evidence of the role of affect and emotion in moral judgment (§3.3.2). I then discuss some criticisms of the social intuitionist model (§3.3.3). This leads into the discussion of Greene’s dual process model in the following section (§3.4).

3.3.1 The cognitive-development and social interactionist approaches The finding of moral dumbfounding offers a particularly clear challenge to traditional rationalist views of moral judgment. In psychology this tradition grew out of the work of Jean Piaget (1932) and was most thoroughly developed in the cognitive-development approach of

5 In fact, 37% of Haidt et al.’s participants said that the man in the Chicken case was harming himself. This is ironically reminiscent of Kant’s view: That such an use contrary to nature (therefore abuse) of one’s property of sex is a violation of the duty to one’s self jarring in the highest degree with morality, directly strikes every body at once with the thought of it, and excites such aversion to this thought, that the very distinguishing of such a vice by it own name is holden immoral…But the proof of reason of the inadmissibleness of the unnatural use, and even of that use, which is merely disconformable-to-end, of ones properties of sex as a violation (and, as to the former, in the highest degree) of the duty to one’s self, is not so easily made….he, who gives himself up entirely to animal inclination, makes of man a thing to be enjoyed, but herein at the same time a thing contrary to nature, id est, a disgusting object, and thus deprives himself of all reverence for himself. (1799, 2:5-6)

42 Lawrence Kohlberg (1969). Kohlberg developed a method for measuring as a kind of cognitive development. Kohlberg’s approach involved presenting moral dilemmas6 to participants as part of a moral judgment interview. The most famous of these dilemmas involves a man named Heinz who steals a drug in order to try to save his dying wife, when the druggist refuses to sell it cheaper or to let Heinz pay later. Kohlberg would present the dilemma and then ask whether Heniz should have stolen the drug and why.7 By coding the responses made by participants in theses moral judgment interviews, Kohlberg developed a framework describing six stages of progressive moral development: from preconventional (judgments of right or wrong are driven by the direct consequences of the action, e.g. avoiding punishment), to conventional (judgments made on the basis of societal expectations), and finally to postconventional (judgments are made with reference to abstract or universal principles). Kohlberg’s approach dominated moral psychology in the 1970s and 1980s. The biggest subsequent development during this time came from the social interactionist perspective, developed in large part by Larry Nucci and Elliot Turiel (1978). They proposed that “social convention and morality constitute two distinct conceptual domains and that their development stems from different aspects of the child’s social interactions” (1978, 401). Nucci and Turiel demonstrated that children as young as preschool age distinguish the social conventional domain from the moral domain, thus undermining Kohlberg’s view that “moral development entails a process of progressively differentiating morality from convention” (1978, 406). The research tradition that developed out of this work attempted to give an account of this distinction and emphasized the psychological reality and importance of this distinction (e.g. Nucci and Turiel 1978; Turiel 1983; Turiel, Killen, and Helwig 1987; Smetana 1993; Nucci 2001). This has been an incredibly influential research paradigm. In Chapter 2 (§2.1.2), we saw some reasons to doubt the account of morality emerging from this perspective. First, research in cultural psychology shows that the realm of morality extends beyond harm violations. Second, work by Dan Kelly and others (e.g. D. Kelly et al. 2007) suggests that the moral/conventional distinction is an artifact based on using too narrow a

6 It is important to note that I use the word ‘dilemma’ as a case with considerations on both sides and which calls for a judgment. I take no stand on whether there are moral dilemmas in the stronger sense involving a case in which taking either option would be wrong. 7 For the full case, see Appendix A.

43 range of scenarios. Using a wider range of scenarios undermines the moral/conventional distinction, even among Western participants. The phenomena of moral dumbfounding shows that people are sometimes unable to provide reasons for their moral judgments, but nevertheless do not retract those judgments. The finding that for some scenarios and some participants, affective reaction predicts moral judgment better than harm (many participants agreed that the acts in such scenarios did not cause harm) suggests that the rationalist accounts just surveyed, if not completely inadequate, are at least missing an important component of moral judgment.

3.3.2 The social intuitionist model Building on his work challenging the cognitive-development and social interactionist models of moral judgment, Haidt proposed the social intuitionist model of moral judgment and marshaled a significant amount of evidence in support of it (Haidt 2001). Though I will argue shortly that Haidt’s account needs supplementation, it is noteworthy as one of the first dual process models of moral judgment in recent moral psychology. According to this model, moral judgment is the result of an immediate moral intuition which can be followed by a process of post-judgment moral reasoning. The account depends first, on distinguishing moral intuition from moral reasoning. In line with dual process theories, Haidt argues that moral intuition is the result of quick, effortless, automatic processing. These processes are not generally consciously accessible, though their products are. Moral reasoning is, on the other hand, the result of slower, more effortful, consciously executed cognition. One important aspect of this model is that reasoning is almost always an ex post facto affair. The goal of reasoning is not to contribute to the making of a judgment, but rather to provide reasons for the judgment that has already been made and expressed. In his explanation of moral reasoning, Haidt draws on the work of Nisbett and Wilson (1977), according to which people do not have privileged introspective access to the causes of their decisions and judgments, but rather use ‘a priori’ theories to come up with plausible rationalizations after the fact. Participants do not recognize their affective reactions as the cause of their judgments and they make up plausible rationalizations of their judgments after the fact. However, in Haidt’s work, the cases like the Chicken case are constructed so that there are no plausible reasons. For participants who do not override their affective response, the result is moral dumbfounding.

44 The social aspects of Haidt’s model involve the effects of one person’s moral judgments or reasoning on the intuitions of other people. While the expression of your moral reasoning may not be an accurate account of the etiology of your judgment, it may help to persuade others to agree with you. Similarly, the mere expression of your judgment may exert pressure on others to conform. Haidt does not deny that moral reasoning could play a role in moral judgment, but maintains that this happens only rarely (e.g. among professional philosophers). Individuals may sometimes reach a moral judgment through a process of reasoning or come to revise their original judgment in the course of reasoning about it. In his original article, Haidt drew on a wide range of evidence for his view, including some of the work on moral dumbfounding and moral judgment across cultures previously discussed (§3.3, §2.1.2, and §2.1.3). Since then, he and his collaborators have produced several more studies in support of it. For example, Haidt and Thalia Wheatley found that hypnotically induced disgust affected participants’ moral judgments (2005). In this study, the experimenters planted a post-hypnotic suggestion that the participants experience disgust upon hearing a specific word, e.g. ‘often’. They were then given moral-violation vignettes containing the word and asked to judge the moral wrongness and disgustingness of certain moral transgressions, e.g. a man eating his dead pet dog, ambulance chasing lawyers, or second cousins who have sex with each other. They found the judgments of participants with the post-hypnotic suggestion indicated a higher degree of wrongness and a higher degree of disgust for the same moral violations than the judgments of those who were not hypnotized.8 Thus, the severity of moral judgment can be manipulated by inducing greater feelings of disgust. What is most interesting about this study are the results of a control story which contains no obvious moral violation but did contain the disgust-inducing word. One such story involves Dan, a Student Council representative who is in charge of scheduling academic discussions and “often picks topics that appeal to both professors and students” (2005, 782). Some participants reported that Dan’s actions were wrong, even though they could not exactly say why. One participant wrote, “I don’t know, it just seems like he’s up to something” (2005, 783). Thus, the

8 Out of 100, the mean disgust rating for those with the post-hypnotic suggestion was 68.04, compared with 43.11 in the control group. For moral wrongness, the mean for the post-hypnotic group was 73.94, while the control mean was 64.67. Both differences were significant (p < .001 and p < .05, respectively) (T. Wheatley and Haidt 2005).

45 disgust manipulation led to a negative moral judgment and then confabulation in the justification of the judgment. In an extension of these studies, Schnall et al. (2008) tested the effects of other methods of disgust induction on moral judgments. In one study, the experimenters used a novelty ‘fart spray’ on a nearby trash can. In another experiment, they had some participants complete a survey while seated in a disgusting environment including a sticky desk, a chair with a torn and dirty cushion, a chewed-up pen, a dried-up smoothie, and a trash can overflowing with greasy pizza boxes and dirty tissues. Experimenters also used a disgust-inducing clip from the film Trainspotting and a writing task in which participants were asked to write about an event in their life when they felt physically disgusted. The results replicated the earlier findings that disgust causes judgments of wrongness to be more severe. This study extends the previous work in interesting ways. First, Schnall et al. were able to identify an interesting individual difference mediating this effect. They hypothesized that participants more sensitive to their own bodily states (as measured by Miller et al.’s Private Body Consciousness (PBC) measure (1981)) would exhibit the effect of disgust on moral judgments. In fact, they found that induced disgust affected the moral judgments only of participants who rated high on the PBC item and were more sensitive to their own bodily states. These experiments also investigated the effects of disgust induction on a range of judgments. They asked for two kinds of moral judgments: those with moral violations involving disgust (e.g. a man eating his deceased dog or cannibalism among the survivors of a plane crash) or no disgust (e.g. keeping the cash from a found wallet or falsifying a resume). They also asked participants to make two kinds of nonmoral judgments: public policy judgments both involving contamination (e.g. about border patrol or waste management) and not (e.g. about nondenominational school prayer or increasing funding for social scientific research). The results indicate that disgust affected moral judgments regardless of whether the violation under judgment involved disgust and that the effect did not extend to any of the nonmoral public policy judgments. The experiment involving film clips tested whether other ‘negative’ emotions like sadness would have the same effect on moral judgment. The results from sadness-induction trended (but were not statistically significant) in the opposite direction, indicating that the effects

46 were specifically due to the induction of disgust. In addition, the PBC finding shows that the effect is not mediated by semantic priming, but rather by the induction of affective processes. Another study investigated the link between physical cleansing and moral behavior (Zhong and Liljenquist 2006). In one experiment, participants were asked to recall in detail either an ethical or an unethical deed, describing any emotions they experienced. They were then given a free gift: they were offered a choice between a pencil and an antiseptic wipe. Participants who had recalled an unethical deed were twice as likely (67%) to take the antiseptic wipe as compared with the pencil (33%). Such results provide evidence for what the authors call the Lady Macbeth effect: “exposure to one’s own and even to others’ moral indiscretions poses a moral threat and stimulates a need for physical cleansing” (Zhong and Liljenquist 2006, 1452). In a follow-up experiment, Zhong and Liljenquist were able to demonstrate this effect in moral behavior and correlate it with experienced emotions. Previous research shows that contemplating actions at odds with one’s personal values can lead to intentions to perform actions that further those values (Tetlock et al. 2000). Baumeister et al. found a similar result with respect to feelings of guilt (1994). Thus, participants who recalled unethical deeds should be more likely to pursue moral options, if given the choice. However, if physical cleansing offers a way of “washing away sins” then participants who use an antiseptic wipe will be less likely to pursue moral behaviors in order to compensate for recently recalled past misdeeds. These predictions were confirmed. Participants were again asked to recall either an ethical deed or an unethical deed. Some participants cleansed their hands with an antiseptic wipe while others did not, and then they filled out a survey about their current emotional state. They were then asked if they would volunteer (without further compensation) to participate in an additional experiment to help a graduate student in need. Only 41% of participants who used the antiseptic wipe offered to help, compared with 74% of those who didn’t clean their hands. In addition, participants who used the wipe reported less moral emotion (disgust, regret, guilt, , embarrassment and anger) than those who did not. There was no difference in nonmoral emotions across conditions. This suggests that the effects of recalling immoral behavior discussed above are mediated by moral emotions and that physical cleansing reduces the tendency toward prosocial behavior brought about by contemplating immoral deeds by reducing moral emotions like guilt and disgust. In corroboration, Schnall, Benton and Harvey (2008) found that cleansing mitigates the

47 effect of induced-disgust on moral judgment. Participants that washed their hands after watching the disgust-inducing clip from Trainspotting made less severe moral judgments than those who had not washed their hands. A more recent study found that washing can attenuate cognitive dissonance following a decision, indicating that this effect extends beyond the moral realm (S.W.S. Lee and Schwarz 2010). Another recent study found that priming “self cleanliness” (mediated by an “inflated moral self”) can lead to more severe moral judgments about contentious social issues like pornography, drug use, and adultery (Zhong, Strejcek, and Sivanathan 2010). The work reviewed here has focused primarily on disgust, which is only one of the so- called “other-critical” moral emotions (Haidt 2003). Above (§2.1.3), I mentioned the CAD hypothesis, which links three moral emotions: contempt, anger (sometimes called ‘moral outrage’ to distinguish it from nonmoral anger), and disgust, with the three moral domains identified by Shweder et al. (1997): the ethics of community, the ethics of autonomy, and the ethics of divinity (P. Rozin et al. 1999). The effect of disgust in Schnall et al.’s (Schnall et al. 2008) study was not selective for moral judgments involving the ethics of divinity. As noted, they appeared to affect all moral judgments, but not nonmoral judgments. It is possible that the other moral emotions could have similar effects on moral judgments. In addition to the other-critical moral emotions, there are also three main “self-directed” moral emotions: shame, guilt, embarrassment, as well as positive moral emotions like gratitude, admiration and (Haidt 2003). The work by Zhong and Liljenquist sheds light on how self-directed moral emotions affect moral behavior. Work by Haidt has also suggested that ‘elevation’, characterized as “a response to acts of moral beauty” associated with a feeling of “opening in the chest, combined with the feeling that one has been uplifted,” can motivate prosocial behavior (Algoe and Haidt 2009, 106). A study of moral elevation in nursing women suggests that its effects may be mediated by oxytocin, a hormone and neurotransmitter associated with bonding and attachment (Silvers and Haidt 2008). Thus, Haidt’s social intuitionist model of moral judgment and subsequent research by Haidt and others have demonstrated the importance of affect in moral judgment. In addition, there is evidence supporting the ex post fact nature of moral justification. Cushman et al. (2006) and Hauser et al. (2007) found that individuals made moral judgments in accordance with certain principles (e.g. the Doctrine of Double Effect), but had difficulty articulating some of these

48 principles when asked to justify their judgments. I discuss this research more in the next chapter (§4.1). It will be useful to integrate this work with the dual process framework introduced above. One way of seeing this is to look at a failed attempt to manipulate moral judgment through affect. Schnall et al. (2008) report an attempt to induce disgust using a bucket full of a gooey substance created by mixing creamed corn, collard greens, and chocolate pudding. Participants were asked to immerse their hand in the bucket (or a bucket filled with water in the control condition) and then asked to make a series of moral judgments. Contrary to their expectations, there was no difference in judgments and the authors hypothesized that in this case the disgusting experience was too obvious. Participants’ disgust was rightly attributed to their experience with the gooey substance rather than experienced as a response to the cases under judgment. This suggests that people who rate high on the BPC use information about their occurrent emotions (in this case, disgust) to inform their moral judgments, unless they are cued to the fact that the cause of those emotions is some extraneous factor, e.g. dipping your hand in a gooey substance. This fits nicely with the dual process theory framework introduced above. Recall that one of the functions of Type 2 processes is to interrupt Type 1 processes. This involves recognizing when the result of a Type 1 is likely to be in error. Kahneman reviews some of the conditions under which this is known to occur (2003, 711). For example, time pressure, cognitive load, and being in a good mood all tend to preclude Type 2 processes from correcting Type 1 processes. On the other hand, intelligence, need for cognition, and education in statistics all tend to facilitate greater Type 2 correction. In experimental studies, making the manipulation salient can cause the participants to (over)compensate for Type 1 processing and not exhibit the predicted effect. This seems to be exactly what happened in the failed disgust induction reported by Schnall et al. (2008). For this reason, within-participant designs have been discouraged in the heuristics and biases tradition since they tend to “induce the effect they are intended to test” (Kahneman and Tversky 1982, 500). Under certain conditions, such cues can sometimes lead to overcorrection. For example, Schwartz and Clore (1983) found that both laboratory induced- and naturally weather-induced mood could affect participants’ judgments of their satisfaction with their lives and at the moment. However, when participants were given cues by which they could attribute their

49 negative mood to extraneous factors (a sound-proof room and rainy weather, respectively), they did so. Their judgments did not differ from those of participants who had no mood induced or had positive moods induced. In fact, the sound-proof room was a red herring. The experimenters had no reason to think that it had any effect on the mood of participants. Raising the salience of the setting, led participants to ‘correct’ for something that had actually had no effect. It looks like people use their mood as a source of information to make judgments, unless they are alerted to the fact that their mood might be due to some extraneous factor (or are induced to attribute their mood to some other factor). Similar effects have been found with the availability heuristic (Oppenheimer 2004). Participants tend to overestimate the number of words that contain the letters in one’s initials. Daniel Oppenheimer found an effect in the opposite direction when participants were directed to first write down their initials. In a classic study, and Jerome E. Singer (1962) found that participants are less likely to misattribute arousal due to an injection of adrenaline to salient situational features (e.g. an offensive questionnaire or a hula hoop twirling confederate), if they have been told to expect certain side effects from the injection. As and Kathleen Vohs put it, “Apparently, the unnoticed emotion is more influential than the noticed” (2003, 195). I return to this research in my discussion of affect in Chapter 9 (§9.2.2). For the time being, the important point is that this work provides support for the basic dual process model. The disgust response is a Type 1 process. In normal cases, this Type 1 process can influence certain kinds of judgments. However, when the disgust is salient and attributable to some extraneous , participants override the Type 1 response which they see as unreliable precisely because it can be attributed to the extraneous stimulus. They must then generate an alternative response and usually rely on some kind of Type 2 process to do so. How things go in abnormal cases, sheds light on how things normally go. The manipulation works precisely because it is subtle. The system functions because (a) usually the source of our disgust is not the product of subtle manipulation, but is a response to features of the case under judgment, and (b) if our disgust is not due to the case under judgment, the source of our disgust is noticeable as an extraneous stimulus, and we correct for it. Sometimes this system breaks down. It can be tricked by intentional manipulation or by coincidental circumstances. For example, jurors’ judgments could be made more severe due to deliberation being conducted in a dirty jury room.

50 Some individuals are more likely to reevaluate their Type 1 responses than others. Frederick (2005) has developed the cognitive reflection task (CRT) a measure of the extent to which people are disposed to resist providing the first answer that comes to mind. In other words, it indicates the extent to which people will engage in subsequent cognition to check the outcome of a Type 1 process. The ball and bat question discussed above (§3.2) is one item of this measure. I found that CRT scores predicted permissible judgments in a case of consensual, brother-sister incest. Those disposed to reflect on their initial affective response, were more likely to judge that the action was permissible (Fisher’s exact test, two-tailed p < .05). In addition, this research is consistent with a broader body of work on affect primacy. (1980; 1984) reviewed a variety of research indicating that affective responses are relatively automatic and independent of cognition. One of the earliest sources of evidence for this involved the mere exposure effect: simply being presented with a stimulus previously can affect people’s preferences even in the absence of awareness. For example, people prefer melodies and shapes that they have seen before (sometimes for very brief periods of time), even when their judgments about whether they had seen the shape or heard the melody were at chance: “subjects were able to distinguish between the old and new stimuli if they used liking as their response, but they were not able to distinguish between them if they had to identify them as ‘old’ or ‘new’” (Zajonc 1980, 163).

3.3.3 Problems for the social intuitionist Obviously affect and emotion have a significant influence on moral judgment. I’ve discussed some of the evidence for this. I discuss much more work on this in Chapters 5 and 6, particularly with respect to psychopathology. While the social intuitionist model has helped to overturn the rationalist views in moral psychology that held sway for most of the 20th century, there is a danger that the scales may tip too far in the opposite direction. In fact, there are several reasons to think that affect can’t be the whole story when it comes to moral judgment. Joshua Greene and colleagues have developed a dual process model of moral judgment which makes room for both emotion and other ‘cognitive’ processes to play a part in moral judgment. I present this model in the next section (§3.4). This section paves the way for that discussion by reviewing some of the problems for the social intuitionist model—problems that Greene’s model helps address.

51 First, as just discussed, sometimes individuals come to see their initial affective response as due to factors irrelevant to the object of the moral judgment (e.g. having one’s hand in a gooey substance). In such cases, individuals must substitute an alternative response. However, the social intuitionist model has no good account of where these alternative responses come from. Of course, Haidt claims that reasoning can sometimes lead to moral judgment, but he’s not very clear on how this happens, and it is not clear that explicit reasoning is the only way to generate an alternative judgment. Greene’s model provides an account of how this might happen in response to certain kinds of moral dilemmas. Second, Haidt’s original model seems to link moral intuitions too closely with moral judgments. For example, Haidt writes that “moral intuitions (including moral emotions)…directly cause moral judgments” (Haidt 2001, 814). However, the research seems to suggest a more complicated relation between affective Type 1 processes and moral judgment. For example, as discussed above, some participants in Haidt et al.’s study have affective responses that do not lead to analogous moral judgments. High SES participants in Philadelphia tended to say that they would be bothered by harmless offenses (65%), but less likely to say that the offender should be stopped or punished (19%) or that the action was wrong (40%). Obviously some of these participants overrode a negative affective response and judged that the action was not wrong and that the individual should not be punished. Again, it is not clear how Haidt’s model can explain such cases. Similarly, as I discuss in the next section, certain moral dilemmas can trigger multiple but conflicting processes. These conflicts must then be resolved before a moral judgment can be made. Thus, the relation between intuitions and moral judgments is more complicated than the social intuitionist model proposes. Third, Haidt et al. call for moral psychologists to “place more emphasis on emotions” (1993, 626). However, they don’t really say what they mean by emotions or affect. This is complicated by the fact that on Haidt’s model, moral intuitions are not exclusively affective, but most of the research done by Haidt and his coauthors have concentrated on affective intuitions, especially disgust. There is a real need to explicate the nature of affective processes, how they differ from more cognitive kinds of processes, and the role they play in moral judgment. This is highlighted by the work discussed in the next section that demonstrates multiple and sometimes conflicting Type 1 processes which can play a role in moral judgment. A more general account

52 of affect, its role in moral judgment and its relation to cognition and motivation is discussed in the Chapter 9.9 Thus, Haidt and his colleagues’ work serves to highlight the role of affect in moral judgment and the ex post facto nature of some moral reasoning and justification. However, there is also reason to doubt whether Haidt’s model captures the whole story. In the next section, I discuss Greene’s dual process model, which promises to add another piece to the puzzle.

3.4 Trolleys, Ecologists and Terrorists Recall the cockroach in the apple juice. I discussed how the offer of ‘roached’ juice is likely to trigger an affective disgust response. However, it is possible to overcome this response. By focusing on the fact that the roach was sterilized so there is no danger of any contamination, we can bring ourselves around to accept the offer (especially if we are offered a reward for doing so). Our conscious overriding of the initial affective response is not likely to make the disgust go away immediately, suggesting that our effort to override is an additional process and not simply a matter of interrupting the initial affective response. Recent dual process theories of moral judgment suggest that something similar occurs when judging certain moral dilemmas. Part of the impetus for Joshua Greene’s research was a well known puzzle in moral philosophy. Consider the Trolley Case.10 A runaway trolley is barreling towards five people stuck on the tracks. You can flip a switch and divert the trolley to a side track where only one person lies in the path. Is it permissible to flip the switch and save the lives of five people, at the cost of the life of the person on the side track? Most people say that it is permissible. A puzzle arises when we consider a similar case, the Footbridge case: Again, a runaway trolley threatens five people trapped on the tracks. In this case, however, you are standing on a

9 Another reason to return to the nature of affect is as follows. On the account of moral judgment offered in the previous chapter, moral judgments necessarily make use of moral concepts in their expression. If this is right, then a moral judgment must itself have conceptual content. However, on some accounts of the emotions (e.g. Prinz 2004), including the account I sketch in Chapter 9 (§9.2), emotions do not necessarily have conceptual content. A complete account must explain the link between emotions and moral judgment. This provides another reason to more clearly understand the nature of affect and emotion. 10 The full text of the cases discussed in this chapter (Ecologists, Infanticide, etc.) are reproduced in Appendix A. Though there is some variation in the presentation of these cases across researchers (e.g. in Hauser et al. 2007), I include those versions used by Greene et al. (Greene et al. 2004).

53 footbridge over the tracks. There is a large man that you can push onto the tracks and stop the trolley. Is it permissible to save the lives of five people, at the cost of the life of the large man? Most people say that it is not permissible to push the large man onto the tracks. A large web-based survey with over 5,000 participants confirmed these judgments, finding that 85% of participants agreed that it was permissible to flip the switch in the Trolley case, while only 12% said it was permissible in the Footbridge case (Hauser et al. 2007). This pattern was found in a wide range of demographic subpopulations for which there were enough participants to perform analyses including national groups, ethnic groups, religious groups, groups of different levels of education, age groups, and sex groups (2007, 8-10).11 Our judgments about these cases are puzzling because both involve sacrificing one life in order to save five. What could explain the marked divergence in judgments about these cases? Greene et al. (2001) developed a descriptive explanation for this divergence, though they do not claim that this explanation offers any normative justification for the pattern of judgments. Their explanation depends on drawing a distinction between what they call personal and impersonal moral dilemmas. They hypothesized that personal moral dilemmas would engage people’s emotional responses, while impersonal moral dilemmas would not. The footbridge case is a more personal moral dilemma involving directly pushing a man to his death. The trolley case is less so, it merely involves flipping a switch. (I discuss Greene et al.’s operationalization of the personal/impersonal distinction shortly.) If their hypothesis is correct, the emotions triggered by the personal nature of the footbridge case might explain why people judge it impermissible to push the large man onto the tracks. In the absence of any emotional response in the trolley case, people will base their judgment on the consideration that five deaths are worse than one. For participants who do judge the action permissible in the footbridge case, an additional process may be required to override the initial emotional response.

11 There were two exceptions. A greater percentage of American Indians rated the Trolley case as permissible (40%) than the footbridge case (7%), but a majority rated both scenarios as impermissible. Interestingly, a majority of those who specified their current religion as Muslim rated the action in both the trolley and the footbridge case as permissible, and actually exhibited the opposite effect, with a higher percentage of participants rating the footbridge case as permissible (90%) than the trolley case (78%) (Hauser et al. 2007, 9-10).

54 One argument that personal moral dilemmas engage emotional processing more than impersonal moral dilemmas is evolutionary (Greene 2008). For example, it has been shown that humans and nonhuman primates can acquire a fear of snakes from as little as a single of an ’s expression of fear in response to a snake (Mineka et al. 1984). There are no similarly biased acquisition mechanisms for fear of guns and knives—things that are generally more dangerous to modern-day humans. The reason for this is that guns were not a part of human societies in evolutionary history until recently, thus there has been no time to evolve biased learning mechanisms for the acquisition of such fears. Snakes have been a dangerous part of the environment since before our ancestors were human; evolution has given us adaptive mechanisms for quickly acquiring fear of snakes. Similarly, Greene hypothesizes that our ancestors were rarely in a position to cause harm to another individual by flipping a switch. For our ancestors, the only way to harm another was “up close and personal” (Greene 2008, 43). The social environment of our ancestors would have favored individuals who were disinclined to harm members of their own community, so our emotions—relatively old, phylogenetically speaking—are set-up to respond negatively to (the possibility of) up close and personal harms, but not to less direct, impersonal harms. Thus, considering personal moral dilemmas is more likely to trigger emotional responses than impersonal moral dilemmas. Though this evolutionary story is overly simplistic, it is helpful in generating hypotheses for testing. Greene et al. proposed the following “first cut” distinction between personal and impersonal moral dilemmas (2001, 2107). Personal moral dilemmas involve (a) direct harm, (b) to a particular person(s), (c) that is the result of action and not a deflection of an existing danger to another individual. Impersonal moral dilemmas do not involve all three factors. They tested their hypothesis that personal moral dilemmas would activate greater emotional processing, as compared with impersonal moral dilemmas. They asked participants to make judgments about a series of dilemmas: nonmoral, moral personal, and moral impersonal, while collecting fMRI neuroimaging and behavioral data. Brain areas associated with emotion were significantly more active in participants considering personal moral dilemmas as opposed to impersonal moral dilemmas or nonmoral dilemmas. Areas associated with working memory are known to be less active when emotional processing is activated and were less active in the moral personal cases than the other two cases.

55 This lends support to their hypothesis that personal moral dilemmas selectively trigger emotional response. The personal/impersonal distinction was initially supported by reaction time (RT) data. Greene et al. reported that, in response to personal moral dilemmas, participants who judged the action ‘appropriate’ took longer than those who judged the response inappropriate (2001, 2107). They suggested that this was due the role of ‘cognitive’, controlled processes being engaged in overriding the emotional response. However, these RT data have been shown to be artifacts based on the inclusion of dilemmas in which there is no realistic competing utilitarian justification for a harmful action, and which tended to elicit very quick judgments of ‘inappropriate’ (McGuire et al. 2009; see also Greene et al. 2008, 1146 n.5; Greene 2009). Though the personal/impersonal distinction is not supported by RT data, it remains supported by the differential patterns of neural activation. However, as we will see in the next section (§3.4.1), an alternative way of sorting personal moral dilemmas into “high-conflict” and “low-conflict” dilemmas (Koenigs et al. 2007) generates RT data in support of a distinction in the vicinity of the personal/impersonal distinction suggested by Greene et al. (2001). On the basis of selective activation of emotion in response to certain moral dilemmas, they suggested a middle ground between traditional rationalist views and “emotivist” views like Haidt’s (Greene et al. 2001, 2107). This led to the development of a dual process model of moral judgment. In the next section, I describe the model developed by Greene and colleagues and review some of the evidence that has subsequently been uncovered in favor of it (§3.4.1). In the next chapter, I discuss a variety of research that challenges the basic model, but which, I argue, can be accommodated by significant modifications of the basic model.

3.4.1 Greene’s dual process model Building on the suggestive results of their first study (Greene et al. 2001), Greene and his colleagues performed an ingenious follow-up experiment (Greene et al. 2004). They began by replicating their original findings. Again they found that areas of the brain associated with emotion and were selectively activated for personal moral dilemmas: the medial prefrontal cortex (BA 9/10), posterior cinculate/precuneus (BA 31/7), and bilateral superior temporal sulcus/inferior parietal lobe (BA 39). In response to impersonal moral dilemmas, they found greater activation of areas associated with working memory, abstract reasoning, and other

56 “cognitive” processes: the right dorsolateral prefrontal cortex (DLPFC) (BA 46), and bilateral inferior parietal lobe (BA 40). In addition, the larger sample of the follow-up study revealed an increase in bilateral amygdala activation selective for personal moral dilemmas. This was in accordance with their predictions. A personal moral dilemma involves a direct harm likely to engage a fast, automatic, emotional response. Some personal moral dilemmas feature contrasting beneficial consequences of the harmful action, for example, saving five lives versus one in the Footbridge case. Thus, these cases should engage processes of abstract reasoning in response to these countervailing benefits. These two responses are obviously in conflict, and so additional processes associated with cognitive conflict and cognitive control will likely also be activated by personal moral dilemmas. They interpreted the longer RT in response to personal dilemmas in their initial study (Greene et al. 2001) as evidence of cognitive conflict between abstract reasoning and emotion, and the recruitment of areas associated with cognitive control to resolve the conflict.12 This led to their follow-up study. They distinguished between difficult personal moral dilemmas and easy personal moral dilemmas. This was done on the basis of individual reaction time data. Thus, some dilemmas were difficult for some and easy for others. But as one might expect, there was quite a bit of overlap across participants. For most participants the Footbridge case or the Ecologists case were difficult personal moral dilemmas:

You are part of a group of ecologists who live in a remote stretch of jungle. The entire group, which includes eight children, has been taken hostage by a group of paramilitary

12 As discussed above, the RT data supporting the personal/impersonal distinction was debunked by McGuire et al. (2009). However, that critique indicated that when “poorly endorsed” dilemmas, i.e. dilemmas for which less than 5% of responses were ‘appropriate’, were removed, the only significant effect was between personal and impersonal dilemmas. This suggests that personal dilemmas which are ‘true’ dilemmas (which can be operationalized by the degree to which they admit of divergent responses among participants) really do take participants longer to answer. In fact, this is quite similar to the distinction drawn by Koenigs et al. (2007) between what they call “High conflict” and “low conflict” personal moral dilemmas. They found a RT effect that supported this distinction. As stated in the text, this work shows that there is an empirically supported distinction in the vicinity of the distinction between personal and impersonal moral dilemmas suggested by Greene et al. (2001). This is discussed further in Chapter 6 (§6.3).

57 terrorists. One of the terrorists takes a liking to you. He informs you that his leader intends to kill you and the rest of the hostages the following morning. He is willing to help you and the children escape, but as an act of good faith he wants you to kill one of your fellow hostages whom he does not like. If you refuse his offer all the hostages including the children and yourself will die. If you accept his offer then the others will die in the morning but you and the eight children will escape. Is it appropriate for you to kill one of your fellow hostages in order to escape from the terrorists and save the lives of the eight children?

Easy personal moral dilemmas, for example, tended to include cases like the Trolley case or the Infanticide case: You are a teenage woman who must decide whether or not to kill your unwanted infant in a situation where there are no overriding benefits beside your own convenience.13 Greene et al. compared levels of activation within personal moral dilemmas, contrasting difficult with easy personal moral dilemmas. Their hypothesis was that difficult personal moral dilemmas would selectively activate areas associated with abstract reasoning, in response to the beneficial consequences of these cases and also areas associated with cognitive conflict and cognitive control engaged to resolve this conflict. This was confirmed. They found increased activation bilaterally in the anterior DLPFC (BA 10/46) and the inferior parietal lobes (BA 40/39), areas associated with abstract reasoning and cognitive control. They also found increased activation in the anterior cingulate cortex (ACC), an area found by some studies to be associated with cognitive conflict and control (e.g. Carter and van Veen 2007; Botvinick et al. 2001). In addition, they found increased activation in the posterior cingulate cortex (PCC) (BA 23/31). Of particular interest is the fact that the DLPFC was less active in response to personal moral dilemmas than impersonal moral dilemmas in the original study (Greene et al. 2001), whereas it is more active in response to difficult personal moral dilemmas when compared with easy ones. The PCC and anterior insula also exhibited this effect. This result is even more surprising, given that these areas are standardly associated with emotion. The authors suggest the following explanation for the selective activation of the insula, an area specifically associated with disgust, for difficult personal moral dilemmas: “In response to difficult personal moral

13 See Appendix A for full text.

58 dilemmas, people find themselves entertaining, and in some cases, endorsing actions that would otherwise be considered morally repugnant” (Greene et al. 2004, 395). I think there is something to this. I return to this in Chapter 6 (§6.2.3). In an additional layer of analysis, the authors contrasted activation patterns in response to difficult personal moral dilemmas for participants who judged the action permissible compared with those who judged the action impermissible. Since the obvious considerations in favor of a permissible judgment are the beneficial consequences, they called this kind of judgment ‘utilitarian’, in contrast to ‘nonutilitarian’ judgments. Even for participants who ultimately make nonutilitarian judgments, we would expect to see activation in areas associated with abstract reasoning (representing the beneficial consequences of the action) and cognitive conflict. Thus, they hypothesized that utilitarian judgments would be predicted by greater activation in areas associated with cognitive control (in order to overcome the opposing emotional response). They found increased activation in the DLPFC (bilateral BA 46) and the inferior parietal lobes (BA 40). In addition, they found increased activation of the posterior cingulate region (BA 23/31), a region they also found activated in response to impersonal—in contrast to personal—moral dilemmas. Since Greene et al. introduced their model, a substantial body of evidence can be marshaled in support of it. In addition to the neuroimaging data just reviewed showing differential activation in response to different kinds of moral dilemma, there are other neuroimaging studies showing similar patterns of activation (reviewed in Young and Koenigs 2007). Neuroimaging data, however, only correlate brain activation with certain functions. Causal claims must be supported by behavioral or manipulation studies. Some studies of this sort were reviewed above (§3.3.2) when I discussed work by Haidt and others in support of his social intuitionist model. Much of this work illustrated affective manipulation of moral judgment (e.g. Schnall et al. 2008). Another study of this sort is by Piercarlo Valdesolo and David DeSteno (2006). They found that inducing positive affect by having participants watch a funny clip from “Saturday Night Live,” increased the likelihood that they would make the utilitarian judgment in the Footbridge case as compared with a control group which watched a clip from a documentary. Induced positive affect did not have an effect on participants’ judgments in response to the

59 Trolley case. If nonutilitarian judgments in the Footbridge case are driven by negative emotional reactions to the personal moral violation, then positive affect should mitigate this response, while positive affect should not affect judgments in the Trolley case because according to the dual process theory, judgments in this case are not driven by negative emotional reactions. This is exactly what was found, providing further direct evidence for the dual process theory. In a related study, Nina Stohminger and colleagues investigated the effect of two positive emotions on moral judgment: mirth (humor) and elevation (Strohminger, R.L. Lewis, and Meyer 2009). They induced these with audio clips either of a stand-up comedian or a story about volunteering and found that these different positive emotions had different effects on moral judgment. In particular, mirth was correlated with decreased for the victim in cases like the Footbridge case, corroborating the results of Valdesolo and DeSteno (2006) Rather than use affective manipulation, Greene et al. (2008) devised a cognitive load study, in order to provide another test of the dual process theory. On the basis of behavioral evidence from normal controls and clinical participants with damage to the ventromedial prefrontal cortex (vmPFC), Koenigs et al. (2007) distinguished between “high-conflict” and “low-conflict” personal moral dilemmas (this distinction overlaps significantly with the difficult vs. easy personal moral dilemma distinction discussed above).14 All of these dilemmas involved the possibility of inflicting direct personal harm on another individual, however, the moral dilemmas eliciting 100% or near 100% agreement in both participant populations were labeled “low conflict.” “High-conflict” dilemmas did not elicit such agreement. This distinction is supported by RT data: normal participants took significantly longer to respond to “high-conflict” dilemmas than “low-conflict” dilemmas, no matter what their response was. Greene et al. (2008) instructed participants to monitor a string of digits and hit a button whenever they saw the number “5.” This cognitive load should selectively interfere with controlled cognitive processes. (This "cognitive load" task is adapted from Gilbert, Tafarodi, and Malone 1993.) While this manipulation did not affect the frequency of utilitarian judgment, it did significantly increase reaction time selectively for utilitarian judgments but not for non-utilitarian judgments. Since the cognitive load task selectively interferes with controlled cognitive processes, this provides evidence that such processes are implicated in utilitarian judgments. Since the manipulation did not affect the frequency of utilitarian judgments, Greene et al. liken

14 For more on vmPFC damage, see §5.4 and Chapter 6.

60 the effect to motorists stuck in traffic who are delayed but not stopped from reaching their destination. Other kinds of manipulations may help shed more light on the details. For example, E.J. Masicampo and Roy Baumeister (2008) found that a glucose beverage could restore the performance of Type 2 processing, after it had been diminished by a ‘depletion’ task. The idea is that Type 2 processes rely on certain physiological resources, viz. glucose. Certain kinds of ‘depletion’ tasks can temporarily exhaust these resources, but they can be restored through consumption of a beverage high in glucose. Gailliot et al. found similar results with respect to the effect of depletion on stereotypes (2009). The dual process theory predicts that depletion will affect moral judgments in response to difficult (or “high conflict”) personal moral dilemmas in ways similar to the cognitive load manipulation. In particular, response time may be selectively increased for utilitarian judgments but not for non-utilitarian judgments, if under conditions of depletion there are still resources available but they are more scarce and require more effort to bring them to bear on overriding emotional Type 1 processes. Alternatively, participants given a depletion task first may actually make fewer utilitarian judgments because they lack the resources to execute cognitive control processes necessary to override Type 1 processing and issue a utilitarian judgment. Furthermore, giving participants a glucose drink after a depletion task and before soliciting their judgments on various moral dilemmas should mitigate the effects of depletion on moral judgment. There may also be interesting individual differences recoverable from such experiments. Greene et al. (2008) separated participants into “high-utilitarian” and “low-utilitarian” on the basis of the percentage of utilitarian responses (high-utilitarian participants averaged 80% utilitarian judgments) and in the no-load condition found increased RT only for low-utilitarian participants. This suggests that high-utilitarian participants have automatized their utilitarian responses and so their utilitarian judgments depend on Type 1 processes and thus require less cognitive resources (and are generated more quickly). If so, we would predict no (or less) effect of depletion in high-utilitarian participants. Research like this will help fill in the details of dual process theory. In another paper, Greene reviews a wide range of evidence that traditionally deontological judgments (i.e. judgments which favor rights or duties over favorable consequences) are driven by emotional processes and traditionally utilitarian judgments (i.e.

61 those which favor the best consequences, sometimes at the cost of violating individual rights or duties) are driven by more ‘cognitive’ processes (Greene 2008). Of course, there is some trouble in grasping exactly what the difference between emotional and cognitive processes is, other than that they take place in different parts of the brain. Some definitions of ‘cognition’ take it to involve some kind of information processing. However, as Greene et al. point out, all the emotional processes at issue have information processing aspects (Greene 2004; Greene 2008). Additionally, both involve some kind of representational content. Greene et al. suggest that the essential feature of affective representations is their connection to motivation. I adopt Greene et al.’s convention of using single quotes to denote the traditional notion of cognition as involving higher-level reasoning. This should be distinguished from a more general notion of cognition as information processing as well as a more philosophical notion that indicates a state or process as having representational content. I discuss these different notions further in Chapter 9 (§9.2), where I discuss affect and its relation to motivation and cognition. Similarly, the discussion of cognitive load and depletion manipulations raises interesting questions about individual differences in moral judgment. These issues have to be addressed. Nevertheless, I think the basic idea behind the dual process model of moral judgment is solid. In the next chapter, I turn to research that challenges the basic dual process model. I think these criticisms can be met by modifying and supplementing this account. While the model I will develop in response to these criticism will look much different, the basic model serves as a firm base on which to build.

62

CHAPTER 4

CHALLENGING THE BASIC DUAL PROCESS MODEL

In the previous chapter I discussed evidence that both emotional processes and more ‘cognitive’ processes play a role in moral judgment. I call this the basic dual process model, and I take the research reviewed in the previous chapter to make a prima facie case for this model. In this chapter, I discuss a few recent challenges to the basic dual process model. These challenges can be met, though responding to some require developing the basic dual process model. I begin by discussing a friendly challenge to the basic dual process model (§4.1). In response, I argue that the basic dual process model needs to be supplemented by causal representations (§4.1.1). I discuss causal representations more in Chapters 7 and 8. Next, I discuss what I call Linguisticism (§4.2). As I use the term, linguisticism refers to a set of claims about methodology and psychological structure loosely related to each other by a commitment to an analogy between moral psychology and linguistics. Linguisticism represents the most serious challenge to the basic dual process model. I discuss some of the central claims of linguisticism and focus especially on linguisticism’s claim that emotion plays no role in moral judgment. This runs counter to the evidence discussed in the previous chapter. I think this claim is mistaken and in the next two chapters I argue that emotion contributes to moral judgment in at least two ways.

4.1 Complicating the model: moral rules? In the previous chapter, I discussed how Greene’s dual process model addresses the puzzling asymmetry between the Trolley case and the Footbridge case. The Footbridge case selectively activates emotional responses, leading subjects to judge that pushing the large man is impermissible. Shaun Nichols and Ron Mallon raise a difficulty for this kind of account. In particular, they argue that emotional responses are not sufficient to cause judgments of impermissibility and that emotions do not explain the asymmetry in judgments between the

63 trolley and footbridge cases, though they allow that the emotions do play a role in moral judgment. They suggest circumcision and self-defense as examples of actions that might generate emotional responses (and involve direct personal harm), but are not judged as wrong (Nichols and Mallon 2006, 532). Similarly, they test a pair of cases that generate a similar asymmetry to that found between the Trolley and Footbridge cases, but are each equally “impersonal,” i.e. in cases with “minimized emotional content” (Nichols and Mallon 2006, 534). They argue that, in addition to emotions and reasoning about the consequences of action, representations of moral rules also play a role in moral judgment. In fact, I’m not sure that the dual process model is committed to the sufficiency claim. What this view claims is that in the absence of other considerations, an emotional response may be sufficient to cause a moral judgment of ‘impermissible’. Thus, one response is that, other things being equal, people should judge circumcision to be impermissible—but things are not equal. Circumcision has a long history of traditional practice and many people believe that it has compensating benefits. So it is unsurprising that people do not judge it to be impermissible. But Nichols and Mallon do not put much weight on these examples. Their claim that an adequate theory of moral judgment must include the representation of moral rules is defended primarily by their interpretation of the results of a set of experiments they report. They constructed cases analogous to the Trolley and Footbridge cases, but to minimize the emotional impact, the cases involve tea cups instead of people. I call these Tea Cups Trolley and Tea Cups Footbridge. Tea Cups Footbridge is as follows:

When Susie’s mother leaves the house one day, she says “you are forbidden from breaking any of the teacups that are on the counter.” While Susie is playing in her bedroom, her 18 month old brother Fred has taken down several of the teacups and he has also turned on a mechanical toy truck, which is about to crush 5 of the cups. As Fred leaves the room, Susie walks in and sees that the truck is about to wreck the cups. She is standing next to the counter with the remaining teacups and she realizes that the only way to stop the truck in time is by throwing one of the teacups at the truck, which will break the cup she throws. Susie is in fact an excellent thrower and knows that if she throws the

64 teacup at the truck she will save the five cups. Susie proceeds to throw the teacup, which breaks that cup, but it stops the truck and saves the five other teacups.

Tea Cups Trolley features similar modifications to the original Trolley case. It involves a switch track but substitutes a toy trolley for a real trolley and tea cups for people.1 They asked participants whether the individual in the story had broken a rule and whether all things considered, the action was ok. They found a similar asymmetry in these tea cups cases. Participants were more likely to say that the individual had broken a moral rule in Tea Cups Footbridge (96%) than in Tea Cups Trolley (44%). They take this to provide evidence that moral rules explain the asymmetry: the footbridge cases involve the violation of a rule that is not violated in the trolley cases. Additionally, in the tea cups cases they found that participants make a distinction in their judgments between whether a rule was broken (“weak impermissibility”) and their judgments about whether the action was ok, all things considered (“all-in permissibility”). While most participants (70%) in their version of the original Footbridge case said both that a rule was broken and that it was wrong all things considered, in Tea Cups Footbridge, they found that 85% of participants judged that a rule was broken and that it was ok, all things considered. To test whether they could get this effect in cases with more emotional impact, they devised a variant of the footbridge case I call Catastrophic Footbridge, in which a train with a deadly virus is heading towards a bomb. If the bomb goes off, the virus will spread and cause billions of deaths. The only way to stop it is to push a large man onto the tracks to his death. As might be expected, a majority of participants said that the actor broke a rule (68%), but only 24% judged that it was wrong, all things considered (2006, 538). Thus, when the consequences are bad enough, people are willing to endorse what they consider a (pro tanto) moral violation. Nichols and Mallon argue that these results show that at least three factors are at work in people’s judgments. People represent moral rules and this is the basis for the asymmetry in judgments between the Trolley and Footbridge cases. In addition, people reason about minimizing bad consequences and in some cases these considerations trump the violation of a rule, e.g. in cases like Catastrophic Footbridge. Nichols and Mallon also make room for emotions. They argue that in cases where there are countervailing benefits to an action that

1 Full text is in Appendix A.

65 would violate a moral rule, the emotional responses of participants ‘compete’ with the beneficial consequences to determine whether individuals will judge it to be all-in impermissible.

4.1.1 Causal representations I think that the experiments by Nichols and Mallon are very suggestive. One view consistent with their results has been developed by Marc Hauser, Fiery Cushman and others (e.g. Hauser 2006a; Cushman, Young, and Hauser 2006; Hauser et al. 2007). They argue that the relevant moral rule in these cases is the Doctrine of Double Effect (DDE). According to (one formulation of) the DDE, harm intended as a means or an end is worse than unintended harm which results from a foreseen side-effect. Thus, in the Footbridge case you intentionally push a man to his death, using him as a means to save five people stuck on the tracks. But in the Trolley case, you only intentionally divert the trolley, though you know it will result in one death as a side-effect. If people do represent this rule in some way, it would explain why people are more likely to judge that pushing the man in the footbridge case is wrong than that flipping the switch in the Trolley case is. The asymmetry in the tea cup cases can be similarly explained. Michael Waldmann and Jörn Dieterich tested this idea against an alternative view (2007). Waldmann and Dieterich interpret the asymmetry in terms of differences in mental representation of these cases. These mental representations, on their view are causal models, they represent the causal structure of these scenarios differently. They argue that the difference between the Trolley case and the Footbridge case is that the former involves causal intervention on the agent, while the latter involves causal intervention on the patient. That is, in the Trolley case, the intervention under consideration targets the agent of the harm; it is a case of agent intervention. The Footbridge case involves patient intervention because it involves putting the large man in harm’s way, making him the patient of the harm caused by the trolley. Waldmann and Dieterich suggest that the asymmetry in judgments can be explained by what they call intervention myopia. The agent intervention cases focus attention on the trolley (i.e. the agent). Without intervention, five people will die; with intervention, one person will die. Comparing these, most people judge that the intervention is permissible. On the other hand, in patient intervention cases the focus is on the large man. In the absence of intervention, he will survive. With intervention, he will die.

66 They argue that these differences in causal structure lead people to represent them in different ways. This leads to differences in judgment. In agent intervention cases, they weigh the consequences of each outcome. In the patient intervention cases, however, the five people who will be killed in the absence of intervention are in the background, the focus is on the large man: the prospective patient of the harm in the event of intervention. Waldmann and Dieterich call this intervention myopia. Thus, they argue that the key difference between the Trolley case and the Footbridge case is not that the large man’s death is intended as a means to saving the other five in the Footbridge case but the deaths in the Trolley case are only a foreseen side effect of diverting the trolley. Rather, they hypothesize that the important difference is that the Trolley case involves agent intervention, focusing attention on the beneficial consequences of intervention, whereas the Footbridge case involves patient intervention, which focuses attention on the harm to the individual man in the event of intervention. In their study, they first replicated the Trolley/Footbridge asymmetry in a variety of contexts, using cases involving bombs, torpedoes, viruses, and trains. Their train cases are structurally similar to the Trolley and Footbridge cases except that they involve a train heading toward a bus with 10 people on it. In Agent Intervention Train, the train can be directed onto a side track upon which a bus with 2 people is stuck. In Patient Intervention Train, a truck can push several cars in front of it thus pushing a bus with 2 people onto the tracks in front of the bus with 10 people. (They used this rather indirect method of patient intervention in order to control for spatial proximity.) In their crucial follow-up experiment, they contrasted Patient Intervention Train and Agent Intervention Train with a ‘loop’ case. The Loop case resembles Agent Intervention Train in which a train is heading for a bus with 10 people and the train can be diverted onto a side track upon which a bus with 2 people is stuck. However, in Loop, the side track rejoins the main track so if the train is diverted the bus with 2 people is used as a means to stop the trolley and save the 10 people from harm. If the DDE explains the asymmetry, then participants should judge this case on a par with Patient Intervention Train. However, the hypothesis of intervention myopia predicts that people will judge Loop on a par with the Agent Intervention Train. Participants’ judgments about Loop were statistically indistinguishable from their judgments about Agent Intervention Train. However, there was a significant difference between

67 participants’ judgments about Loop about Patient Intervention Train, with participants judging the intervention more permissible in Loop than in Patient Intervention Train. This provides some evidence for Waldmann and Dieterich’s intervention myopia hypothesis. In fact, their hypothesis is consistent with the basic dual process theory sketched above if the causal representation explains why certain kinds of cases generate stronger emotional reactions. Though it adds the causal representation as an additional element to the basic model. In the terminology of dual process theory, intervention is a kind of focal bias (see §3.2). A crucial distinction here is between judgments which are in accordance with a rule and those in which the representation of a rule figures in the etiology of those judgments. The study by Nichols and Mallon only shows that people’s judgments are in accordance with a rule; they do not provide evidence that implicates the representation of moral rules in the etiology of these judgments. On the other hand, Waldmann and Dieterich offer a hypothesis about the psychological mechanism that generates the asymmetry in judgments, but which does not rely on the explicit representation of moral rules. This may also help explain another set of data. As mentioned in the previous chapter (§3.2), Cushman et al. (2006) and Hauser et al. (2007) investigated the relation between moral judgment and justification. Cushman et al. tested the role of three principles in moral judgment: (1) the act/omission principle, according to which harms caused by an act are worse than harms caused by an omission; (2) the doctrine of double effect (or what they call the intention principle), according to which harm inflicted as a means is worse than harms that results as a side-effect, and (3) the contact principle, according to which harm involving physical contact is worse than harm produced by less direct means (2006). Using a web survey with a large sample, they tested each of these principles using minimal pairs of scenarios. They asked participants to rate actions on a 7-point scale, where 1 was ‘forbidden’, 4 ‘permissible’, and 7 ‘obligatory’. Participants were then presented with pairs of scenarios that they had judged in accordance with one of the principles and asked to justify their judgments. Participants’ justifications were coded into five categories: sufficient: participant mentioned a factual difference in the two cases; failure: participant mentioned a principle that could not explain the judgments made; uncertainty: participant indicated that she could not justify her responses or was uncertain about how to do so; denial: participant indicated that there was no moral difference between cases; and alternative explanation: participant’s explanation

68 used facts not present in the scenarios or claimed to have made a mistake in one of their judgments. They found significant differences in means indicating that individuals “make use of” these principles in their moral judgments (2006, 1083). They found that these principles were accessible to varying degrees during justification. A majority of justifications were sufficient for the action/omission principle (81%) and the contact principle (60%), while only 32% of justifications were sufficient for the DDE. Justifications of the pairs involving the contact principle and DDE were more likely to involve denial (13% and 17%, respectively) or alternative explanation (32% and 29%, respectively) than for the action/omission principle, whose justifications involved denial only 2% and an alternative explanation only 10%. For the DDE, justifications were more likely to involve failure (16%) or uncertainty (22%) than for either the action/omission principle (6% and 5%, respectively) or the contact principle (10% and 4% respectively). On the basis of this, they conclude that participants use the action/omission principle in conscious reasoning, while the contact principle and the DDE are more likely used via unconscious intuitive processes. The high rate of denial and alternative explanation justifications for the contact principle suggested that it was driven by an intuitive process followed by confabulation. This suggests that while some participants came up with the contact principle, they rejected it as an adequate justification. On the other hand, there was a much higher rate of failure and uncertainty in the justifications of judgments in accordance with the DDE suggesting that participants were genuinely puzzled when asked to explain their judgments. There are a few problems with the interpretation of these results. The first problem is that the authors assume that “a hallmark of conscious reasoning is that the principles used in judgment are articulated in justifications” (2006, 1083) and thus, that judgments are guided by the same principles that are accessible to participants in justification. However, they do not rule out the possibility that the psychological mechanisms engaged in judgment are distinct from those engaged in justification. In light of evidence that moral justification can involve confabulation (see §3.3.2), this must be considered a real possibility. Secondly, the guidelines for coding a justification as sufficient were quite liberal. For a justification to be coded as sufficient, “It was not necessary for the subject to identify the target principle, so long as the principle generated by the subject could adequately account for his or

69 her pattern of responses on the scenario pair in question” (2006, 1084). So, the percentage of justifications coded as sufficient does not necessarily indicate the rate at which the relevant principle was invoked. Hauser et al. confirmed that people make judgments consistent with the DDE, but generally cannot access it in justification (2007). Suggestively, they also found that individuals who had been exposed to moral philosophy were more likely to give sufficient justifications (41%) than those who had not (27%). However, since Hauser et al. used loose coding criteria similar to that of Cushman et al., this result might only suggest that exposure to moral philosophy makes you better at confabulating. (Of course, it is important to point out that we are here concerned with a descriptive account of moral judgment, and not the normative justifications of those judgments. These results do not speak to the issue of how exposure to moral philosophy might affect the latter. See §10.5.2 for further discussion of confabulation in this context.) Thirdly, while Cushman et al.’s results show a statistically significant difference in means indicating that participants made judgments in accordance with these principles, the differences in means were relatively small (generally <1) and the authors do not report the means (only the difference between means). So it is unclear how these differences related to the scale anchors (forbidden, permissible, obligatory). These results may provide some insight into the kinds of principles that are accessible for post-hoc justification, but they fail to shed much light on the psychological mechanisms of judgment. Again it is important to remember the distinction between judgments that are in accordance with a rule (or principle) and judgments in which the principle figures in the etiology of that judgment. While the authors claim that, at least for the act/omission principle, it is likely that conscious reasoning making use of the principle is part of the etiology of those judgments, they fail to substantiate this claim. Their results show that participants’ moral judgments in accordance with these principles could easily be explained as a result of the way these scenarios are represented by causal models along the lines suggested above.

4.1.2 Beyond the basic dual process model Earlier in this section (§4.1), I discussed what I called a “friendly” criticism of the basic dual process model. I called it friendly because, while it suggests ways of supplementing the

70 basic model, it is consistent with the commitment of the basic model that both emotional and ‘cognitive’ processes play a role in moral judgment. The subsequent discussion indicated a number of important points. It suggested that causal representations play an important role in moral judgment. It also raised the issue of the role of representations of moral rules in moral judgment. I think that moral rules do play a role in moral judgment, though in a way different from that suggested by Nichols and Mallon. Similarly, I think that Nichols and Mallon are right to qualify the role of emotion in moral judgment. These later two issues will be clearer after the discussion of psychopathology in the next two chapters. In the next section, I deal with a less friendly criticism. This criticism has been bruited primarily by those associated with what I above labeled linguisticism. These theorists have argued that emotion plays no role in moral judgment.

4.2 Linguisticism Marc Hauser, Susan Dwyer, John Mikhail and others have strongly pushed an approach to moral psychology that I call linguisticism. Linguisticism involves a number of claims, some substantive others more methodological, some general others more specific. What unites linguisticism is the assumption that linguistic theory and moral theory are relevantly similar. Linguisticism represents the most influential alternative to dual process theory in moral psychology. I will briefly review some of the central claims of linguisticism as I see it. In the following chapters I focus on some of the particular claims. One element of linguisticism is the linguistic analogy (LA). Following up on a suggestion of John Rawls (1971, 46-53), a number of people have suggested an analogy between linguistics and moral theory (Harman 1999; Dwyer 1999; Mikhail, Sorrentino, and Spelke 1998). The LA is often introduced in terms of 5 questions, which are intended to be analogous to the questions asked by the Chomskyan or generative grammar program in linguistics (e.g. Hauser 2006a; Hauser 2006b; Mikhail 2007). Hauser lists the questions:

(i) what are the operative principles that capture the mature state of competence? (ii) how are the operative principles acquired? (iii) how are the principles deployed in performance? (iv) are the principles derived from domain-specific or general capacities?

71 (v) how did the operative principles evolve? (Hauser 2006b)

The key elements involve the competence/performance distinction (i and iii), ontogeny (ii), phylogeny (v) and psychological structure (iv). All theories of moral psychology will presumably want to answer questions about development, evolution and psychological structure. What seems to be unique about those employing the LA is an insistence on the competence/performance distinction and a borrowing of concepts from linguistics. There are additional claims which go beyond the basics of the LA. John Mikhail (2007), endorses what he calls Universal Moral Grammar (UMG). He writes: “UMG seeks to describe the nature and origin of moral knowledge using concepts and models similar to those used in Chomsky’s program in linguistics” (Mikhail 2007, 143). In addition, Mikhail adds that “UMG proceeds from assumptions that are mentalist, modular and nativist” and that UMG seeks a computational theory according to which “an adequate scientific theory of moral cognition will often depend more on the computational problems that have to be solved than on the neurophysiological mechanisms in which those solutions are implemented” (Mikhail 2007, 143). The upshot of such an approach is that there is a universal moral grammar, which is innate, embodies knowledge which is universal, and can account for the similarities across human cultures. Mikhail claims that there is a moral grammar and that a poverty of the stimulus argument2 similar to those offered in linguistics also works in the case of moral grammar. Thus, in addition to the claims of the LA, UMG endorses specific claims about ontogeny (nativism) and psychological structure (modularity). Mikhail also sees UMG as primarily a computational theory (as opposed to a neuropsychological). Some have attempted to spell out the idea of a moral grammar according to a Principles and Parameters (P&P) approach (e.g. Harman 1999; Dwyer 1999; Hauser 2006b). Hauser explains: We are endowed with a moral faculty that operates over the causal-intentional properties of actions and events as they connect to particular consequences…We posit a theory of universal moral grammar which consists of the principles and parameters that are part and parcel of this biological . Our universal moral grammar provides a toolkit for building possible moral systems. Which particular moral system emerges reflects

2 I discuss poverty of the stimulus arguments further in Chapter 8 (§8.1.2).

72 details of the local environment or culture, and a process of environmental pruning whereby particular parameters are selected and set early in development. Once the parameters are set for a particular moral system, acquiring a second one later in life— becoming functionally bimoral—is as difficult and different as the acquisition of Chinese is for a native English speaker. (Hauser 2006b, 215). In this development of linguisticism, Hauser endorses a strong analogy between linguistics and moral psychology, positing a specialized faculty whose particular parameters are set by the local environment. In addition to the positive claims of linguisticism, many of which are at odds with the approach being pursued here, some proponents have argued more specifically against the central claim of the basic dual process theory, that both emotional and ‘cognitive’ processes contribute to moral judgment. Bryce Huebner et al. (2009) have denied that emotions contribute to moral judgment. Rather, they claim: “our moral judgments are mediated by a fast, unconscious process that operates over causal-intentional representations. The most important role that emotions might have is in motivating action” (2009, 5). Obviously not all proponents of linguisticism endorse every part of it, but it is a useful label under which to collect a number of related claims. While it is not intended to be exhaustive, I offer the following list of the 10 central claims of linguisticism: (1) Moral processing and judgment are analogous to linguistic processing and judgment (Harman 1999; Dwyer 1999; Hauser 2006a; Hauser 2006b; Mikhail 2007; Dwyer and Hauser 2008; Dwyer, Huebner, and Hauser 2010). Going along with this is the methodological claim that: (2) Moral psychology can adopt many of the methods and concepts of linguistics (Dwyer 1999; Hauser 2006b; Mikhail 2007; Dwyer, Huebner, and Hauser 2010). There are specific claims about psychological structure and architecture. These include: (3) There is a specialized, domain-specific moral faculty, mechanism or organ (Dwyer 1999; Hauser 2006a; Hauser 2006b; Hauser, Young, and Cushman 2008; Dwyer, Huebner, and Hauser 2010; Dwyer 2009); and (4) There is a moral grammar or universal moral grammar (Hauser 2006a; Hauser 2006b; Mikhail 2007; Dwyer, Huebner, and Hauser 2010). These psychological structures are claimed to have a certain ontogeny:

73 (5) Some forms of moral knowledge are innate (and can be so shown by a Poverty of Stimulus argument) (Dwyer 1999; Mikhail 2007; Mikhail 2008; Dwyer, Huebner, and Hauser 2010). In addition, there are even more specific claims about the nature of moral processing: (6) Moral judgment derives from causal-intentional psychology (Hauser 2006b; Mikhail 2007; Huebner, Dwyer, and Hauser 2009); and (7) The notion of moral competence plays a central role in moral psychology (Dwyer 1999; Hauser 2006b; Mikhail 2007; Dwyer 2009; Dwyer, Huebner, and Hauser 2010). (8) Moral cognition depends more on the computational structure than on the underlying neuropsychological mechanisms (Mikhail 2007; Dwyer, Huebner, and Hauser 2010). Lastly, there are claims about the role of emotion in moral judgment: (9) Emotion plays no (synchronic) role in moral judgment (Huebner, Dwyer, and Hauser 2009); and (10) The primary function of emotion in moral psychology is in connecting moral judgment to action and the (diachronic) role of emotion in moral development is in attention modulation (Huebner, Dwyer, and Hauser 2009).

I am skeptical of many of these claims. However, I defer full evaluation until I have reviewed more evidence. Indeed, claims (1) and (2) are so general as to be difficult to evaluate given our current state of knowledge; in part they depend on how the rest of these claims come out. In Chapter 8 (§8.1) I will have something to say about claims (3) and (4), in light of the research reviewed in Chapters 5-7. The nativist claim in (5) is especially tricky since the notion of innateness often tends to run together several distinct things (see for example, Griffiths 2002; see also Griffiths 1997, Chapter 3). I think the research on moral development is instructive and remain skeptical of nativism because, due to general confusion about innateness, these claims are often not very clearly stated. I briefly discuss nativism in Chapter 8 (§8.1.2). We have already seen some evidence bearing on claim (6). I discuss the role of causal and intentional representations in moral judgment more in Chapters 7 and 8. Claim (8) is also a pretty broad claim, which makes it difficult to evaluate, but I will discuss it in Chapter 9 (§9.1). This leaves claims (9) and (10) about the role of emotion in moral judgment. I focus on these two

74 claims specifically in the next two chapters. Together, the evidence reviewed in the next four chapters makes a strong case against linguisticism. I review this case in Chapter 9 (§9.4). In light of the evidence reviewed in the previous chapter, it would be wildly implausible to assert that emotion is unrelated to moral judgment. Huebner et al. (2009) argue that, however much emotional processing accompanies moral judgments, the source of moral judgments is our causal-intentional psychology and that the primary function of emotions is in motivating moral behavior. Thus, while emotions may accompany moral judgment, the evidence does not show that “emotional processes are constitutive of moral cognition” (2009, 5). They question the relevance of neurological and behavioral data to causal claims about moral judgment and point out the poor temporal resolution of fMRI. While they grant that emotion may play a role in socialization, they suggest that it is through attention modulation rather than contributing to moral judgment itself. Huebner et al. are right to point out the limits of fMRI technology. The poor temporal resolution does leave questions about causation open. If imaging evidence were the only evidence for the claim that emotion plays a role in moral judgment, then Huebner et al. would be right. But it is not the only evidence. I reviewed several manipulation studies in the previous chapter. The study by Wheatley and Haidt (2005) provided some evidence that in some cases emotion is actually sufficient for moral judgment. Huebner et al. frequently challenge the interpretation of these studies, offer alternative hypotheses consistent with their views and point to ambiguous data, but rarely do they provide any direct evidence to support their views. In fact, I think that emotion is important for some kinds of moral judgments synchronically and that emotional learning plays a substantial role in the development of moral judgment diachronically. So I think that (9) is mistaken. The best evidence for these claims comes from research on psychopathology. I draw on it to help clarify the role of emotion in moral judgment. Two important claims that emerge from this research that are relevant here. Firstly, the particular pattern of dysfunction in moral judgment that occurs in individuals with damage to the ventromedial prefrontal cortex (vmPFC) indicates the role emotion plays in moral judgment synchronically. The second important claim is what I call the developmental hypothesis. It emerges from research on psychopathy and gains further support from research on early onset vmPFC damage. According to the developmental hypothesis, emotional learning plays an important role in normal moral development. However, in some individuals with

75 deficits in emotional learning, other kinds of processes (primarily subserved by the dorsolateral prefrontal cortex (DLPFC)) can compensate. These claims are consistent with the neuroimaging data reviewed in the previous chapter. As far as (10), the exclusive role of emotion is not in mediating the connection between moral judgment and moral action, but I think that individuals with deficits in emotional learning exhibit an interesting dissociation between different kinds of representations. Individuals who suffer no emotional deficit may have representations that are more closely connected to motivation and thus are better placed to influence moral action than other, more ‘cognitive’ kinds of representations. I turn to this evidence in the next two chapters.

76

CHAPTER 5

‘KNOWING THE WORDS BUT NOT THE MUSIC’: PSYCHOPATHY AND THE ROLE OF EMOTION IN MORAL JUDGMENT

There are a variety of pathologies that affect moral behavior and judgment. In this chapter I discuss psychopathy. In the following chapter I discuss psychopathology resulting from damage to the frontal lobes, especially to the ventromedial prefrontal cortex (vmPFC). These conditions can be characterized by significant emotional dysfunction and disruption of social behavior. This makes them important test cases for claims about the role of emotion in moral judgment. In Chapter 3 (§3.4) I reviewed imaging data indicating emotional processing in response to personal moral dilemmas. While inferring causation from correlation can be problematic in neuroimaging, lesion studies can provide strong evidence for causal claims. If emotion really does play a causal role in moral judgment, we should except these pathologies to affect moral judgment. Indeed, investigation of these conditions reveals not only that emotion does contribute to moral judgment, but sheds light on exactly how it contributes. In this chapter and the next I argue that emotion is crucial for moral judgment in at least two ways. Firstly, emotion contributes to moral judgment synchronically in response to certain kinds of moral dilemmas. Secondly, emotion plays a diachronic role in moral development. I draw primarily on clinical reports and lesion studies. I also draw on neuroimaging and psychological studies to support the results of these lesion studies. The strategy pursued in these chapters is to provide an account of the pathologies themselves in order to understand how they affect moral judgment.

5.1 What is a psychopath? The idea of the psychopath has come along way since Philipe Pinel described it as “mania without delirium” at the turn of the nineteenth century (1806). In fact, it is an idea with a

77 long history. Some scholars have seen the contemporary notion of the psychopath in the work of Theophrastus, a student of , in his discussion of the ‘unscrupulous man’ (Millon 1995, 430). Cultural psychiatrist Jane Murphy reports that the Eskimos have a word, kunlangeta, “his mind knows what to do but he does not do it,” that is used to refer to individuals resembling psychopaths (J.M. Murphy 1976, 1026). In literature, Shakespeare’s Iago seems an almost textbook case (F. West 1978). A.C. Bradley, for example, writes: Iago…illustrates in the most perfect combination the two facts concerning evil which seem to have impressed Shakespeare the most…that perfectly sane people exist in whom fellow-feeling of any kind is so weak than an almost absolute becomes possible to them…[and] that such evil is compatible, and even appears to ally itself easily, with exceptional powers of will and intellect. (Bradley 1904, 232-3) Such a man certainly seems to fit the common conception of a psychopath. It is important, however, to keep such popular conceptions distinct from our theoretical understanding of psychopathy. For example, many popular portrayals of psychopaths cast them as having above average intelligence. While some psychopaths may fit this conception, Robert Hare’s review indicates that there is no difference in average intelligence between psychopaths and nonpsychopaths, in fact there is some evidence that psychopaths may be less intelligent on average (2003). Psychopathy is not recognized in the Diagnostic and Statistical Manual, 4th edition (DSM-IV) (1994). Its symptoms fall under the broader category of Anti-social Personality Disorder (ASPD). There are different forms of anti-social behavior and many causal paths leading to them. Often a distinction is made between reactive aggression, which involves responding violently often in a way out of proportion with the provocation, and instrumental aggression, which involves committing acts of as a means to gaining other ends (R.J. Blair, D.G.V. Mitchell, and K.S. Blair 2005; Berkowitz 1993). James Blair and co-authors (2005, chap. 7) review evidence bearing on the etiology of these forms of aggression. The ultimate causes of anti-social behavior can include endogenous factors, abuse, and brain trauma. Proximally, reactive aggression can be caused by above average baseline activity of the brain’s threat system. Reactive aggression can also result from failures to regulate this circuitry, as is often seen in individuals with brain damage. Instrumental aggression,

78 on the other hand, is more rare and less well understood, but it appears that it can be caused by a failure of moral socialization. Many researchers argue that psychopaths represent a distinct sub-set of individuals with ASPD (R.D. Hare, S.D. Hart, and T.J. Harpur 1991; R.J. Blair, D.G.V. Mitchell, and K.S. Blair 2005). Though there are individual differences within the category of psychopathy (Weber et al. 2008), there is a sufficiently robust pattern of dysfunction that it is useful to consider them as a group. Emerging neuropsychological accounts are supporting this view by shedding light on the underlying basis of this pathology and its variations (R.J. Blair, D.G.V. Mitchell, and K.S. Blair 2005; R.J. Blair 2007).1

1 Obviously, there remains much theoretical dispute about psychopathy. The most important issues discussed in the recent literature include (1) clarifying the relation between measures (e.g. the PCL-R and derivative measures, as well as other proposed measures) and psychopathy as a psychological construct, (e.g. Skeem and Cooke 2010), especially with regard to whether criminal behavior is constitutive of psychopathy or merely a common accompaniment of it; and (2) the degree to which psychopaths are a homogenous and distinct group of individuals (e.g. Weber et al. 2008), for example, whether there are important differences between “successful” and “unsuccessful” psychopaths as distinguished by whether or not they have been convicted of a crime (see, for example, Yang et al. 2005). These are important issues for future research. In the text, I review evidence for a certain view of the dysfunction involved in individuals identified as psychopaths by the PCL-R and derivative measures, which have been standard in the literature, and also evidence relevant to the moral behavior and judgments of individuals so identified. I anticipate that these measures will continue to be revised and improved in the future. As I note in the text, I think that psychopaths as identified by currently used measures present with a robust enough pattern of pathology that it makes sense to treat them as a group, though I do not doubt that future research will reveal important intergroup differences. One interesting hypothesis is that psychopathy may manifest in different ways according to the economic and social opportunities available to an individual. Psychopaths developing in low-SES environments with few opportunities may find criminal behavior more appealing that similar individuals in high-SES environments (see, for example, R.J. Blair 2007). Additionally, as I discuss later (§5.5), there are psychopathic traits in ‘normal’ populations that appear to share an underlying neuropsychological basis. Psychopathy may blend into nonpathological populations in a number of ways. There are also disputes about the nature of the pathology itself. In addition to the amygdala/vmPFC dysfunction account that is defended here and by Blair and colleagues (R.J. Blair, D.G.V. Mitchell, and K.S. Blair 2005; R.J. Blair 2007; R.J. Blair 2010) and the account below dubbed the motivation hypothesis offered by Cima et al. (2010), there is the paralimbic hypothesis defended by Kent Kiehl (2006), accounts proposing psychopathy as a deficit of “information processing” (e.g. J. P Newman 1998), and others. The reasons I endorse the amygdala/vmPFC dysfunction account are largely reviewed in the text. It should be noted that there is some convergence between this and the paralimbic hypothesis endorsed by Kiehl.

79 Thirty years ago, Robert Hare introduced the Psychopathy Checklist, a diagnostic tool for identifying psychopaths. This has been significantly developed and revised over the years and now exists as the Psychopath Checklist-Revised (PCL-R) (R.D. Hare 1991; R.D. Hare 2003). The PCL-R utilizes a process combining a personal interview with a review of the individual’s personal history and institutional records (e.g. criminal records). It consists of twenty items, e.g. glibness, grandiosity, superficial charm, pathological lying, parasitic lifestyle, shallow feelings, proneness to boredom, impulsiveness, manipulativeness, and lack of remorse, guilt, and . Each of these items is scored out of 2 points. The cutoff for psychopathy is generally 30 out of 40, but some studies include slightly lower scores in their psychopathy group. Different versions divide these items differently. They can be arranged according to a two-factor model (dividing items into Factor 1: Interpersonal/Affective and Factor 2: Social Deviance, leaving a few items, e.g. promiscuous sexual behavior, falling under neither factor), or a four-factor model (1: Interpersonal, 2: Affective, 3: Lifestyle, and 4: Antisocial).2 What most distinguishes psychopathy from other forms of ASPD is that it is characterized by a combination of reactive and instrumental aggression, together with an underlying emotional deficiency but a relatively intact intellect. While this leads some psychopaths to engage in criminal behavior, others do not fall into the criminal justice system but may nevertheless be extremely disruptive to the lives of the people with whom they interact (R.D. Hare 1993; Babiak and R.D. Hare 2006). This cluster of symptoms led early clinicians to write that psychopaths “know the words but not the music” (Johns and Quay 1962, 217). This metaphor, however, can be unpacked in a variety of ways. Recently, Maaike Cima and coauthors have criticized other interpretations of psychopathy, arguing that “Psychopaths know what is right or wrong, but simply don’t care” (Cima, Tonnaer, and Hauser 2010, 66). On their view, emotion does not play a role in moral judgment per se, but in the connection between moral knowledge and behavior. Thus, since their dysfunction seems to be primarily emotional, psychopaths have normal moral knowledge. They differ from nonpsychopaths in not caring about their knowledge of right and wrong and in “not engaging with the kinds of motivational systems that inspire morally appropriate behavior”

However, the account defended by Blair and colleagues is more developed than that offered by Kiehl. 2 See Appendix B for both two- and four-factor models.

80 (2010, 60). I call this the motivation hypothesis. If this view of psychopaths is correct, it would support the view of Huebner et al. (2009) and others that emotion does not play a role in moral judgment. According to the motivation hypothesis, we should expect psychopaths to make normal moral judgments. That is what Cima et al. found. They tested psychopaths on a set of personal and impersonal moral dilemmas previously used by Greene et al. (2004). They found no statistically significant differences between psychopaths and either control group (healthy individuals and nonpsychopathic criminals). Psychopaths appear to exhibit the same pattern of moral judgments as normal individuals. Specifically, they found that all three groups judged impersonal moral dilemmas more permissible than personal moral dilemmas and that there was no significant difference in percentage of utilitarian judgments in other-serving, personal moral dilemmas.3 In fact, I think the motivation hypothesis is seriously mistaken. I think Cima et al.’s account is rather superficial and that an alternative account of psychopathy explains the available evidence far better. A proper understanding of psychopathy actually sheds light on the role of emotion in moral judgment in normal individuals. On the view I will argue for, psychopathy primarily involves a deficit in specific forms of emotional learning. Importantly, psychopaths appear to be unable to associate antisocial behavior with the distress of victims. This disrupts moral socialization. Psychopaths compensate for this deficit by utilizing more cognitive ways of learning moral rules. This process takes longer developmentally and the representations they acquire are not well-placed to contribute to moral behavior. This helps explain psychopaths’ tendency toward antisocial behavior. While psychopaths’ moral judgments appear normal, their representation of moral knowledge is significantly different from that of normal individuals. In a very real sense, psychopaths know the words but not the music. On this view, psychopathy has an important developmental component. It is characterized by a deficit in emotional learning, accompanied by compensation for this deficit.

3 This is especially surprising given the fact (discussed further below in §6.3) that Koenigs et al. (2007) found that the judgments of individuals with vmPFC damage exhibit a greater percentage of utilitarian judgments in response to personal moral dilemmas than both a normal control group and a group of individuals with other kinds of brain damage.

81 For this reason, I refer to this as the developmental hypothesis. In the remainder of this chapter, I compare the developmental and motivation hypotheses. When evaluating these hypotheses, it is important to keep the following three things distinct: moral behavior, moral reasoning, and moral judgment. Earlier (§3.3.2 and §4.1.1), I discussed some reason to think that the connection between moral reasoning and moral judgment may not be as close as some rationalist views have held. Indeed much moral reasoning may be post-hoc rationalization that draws on mechanisms (partially or completely) distinct from those engaged in moral judgment. Similarly, the results of Cima et al.’s study provide a reason to think that moral judgment and moral behavior come apart in psychopaths. However, one must be careful not to infer deficits in moral reasoning or judgment simply from the prevalence of antisocial behavior among psychopaths.4 I argue that the pattern of moral behavior, reasoning and judgment in psychopathy supports the developmental hypothesis and that this explains both their apparently normal moral judgment and their antisocial behavior.

5.2 Moral reasoning in psychopaths The results of research on moral reasoning in psychopaths are mixed. Some studies show a deficit, others show that the moral reasoning in criminal psychopaths is on a par with nonpsychopathic criminals —somewhat below average. In line with other criticisms of Kohlberg’s paradigm (e.g. Shweder, Mahapatra, and J. Miller 1987) and consistent with the hypothesis that moral reasoning is (at least some of the time) post-hoc rationalization, Blair et al. suggest that the assessment of moral reasoning via moral interviews really measures participants’ general intelligence and cultural experience (2005, 57). If so, we would expect offenders to score somewhat worse than noncriminals since average intelligence of offenders in general is below average. Research on moral reasoning has also been done on juveniles with psychopathic tendencies. Though they are not referred to as psychopaths, these juveniles are usually identified with a version of the PCL-R designed for use with youths (PCL:YV) (Forth, Kosson, and R.D.

4 As discussed in the first footnote to this chapter, the relation between psychopathy and criminality is currently in dispute. This issue is complicated by empirical ignorance: while criminal populations are a rather captive audience for data collection, it is much more difficult to obtain statistics on psychopathy in the general population.

82 Hare 2003). The research on this is mixed, but there is evidence that supports the long-term validity of this measure (Gretton, R.D. Hare, and Catchpole 2004). There is also evidence for stability of psychopathy from adolescence to adulthood (Lynam et al. 2007; Lynam, Loeber, and Stouthamer-Loeber 2008; Lynam et al. 2009), which supports the developmental hypothesis. The research on moral reasoning in juveniles with psychopathic tendencies indicates some impairment. Blair (1997) found that, relative to controls, such juveniles had more difficulty with the attribution of moral emotions and were also more likely to judge that moral transgressions were permissible if there were no rules against them. The developmental hypothesis would predict that, despite early difficulties, adult psychopaths may be able to use alternative strategies to compensate. According to the motivation hypothesis, psychopaths have normal moral knowledge, so it would not predict deficits in moral reasoning. The relative difficulty of juveniles compared to the relatively normal reasoning of adult psychopaths provides limited support for the developmental hypothesis.

5.3 Moral judgment in psychopathy Psychopathic adults and juveniles with psychopathic tendencies show abnormal performance on the moral/conventional task: specifically they show problems on the authority independence dimension. Earlier (§2.1.2, §3.3), I argued that the account of morality emerging from this research was flawed and that some have suggested that the body of research documenting the moral/conventional (M/C) distinction is largely an artifact. Nevertheless, it might capture something important about harm norms in Western cultures. If so, the abnormal performance of psychopaths on this task might be indicative of a psychological difference within Western culture between normal subjects and psychopaths. By providing insight into how psychopathy manifests in Western culture, it could give insight into psychopathy in general. It might be thought that the adult psychopaths’ failure on the M/C task manifests itself in their tendency to assimilate moral violations to conventional violations. Perhaps psychopaths think that harm-based violations are wrong only because there are laws and social norms prohibiting such actions. In a discussion of rational amoralism, Philippa Foot, for example, asks: “Why cannot the indifferent amoral man say that for him [the moral “should”] gives no reason for acting, treating [the moral “should”] as most of us treat [the “should” of etiquette]?” (1995,

83 451).5 The research shows that, for children with psychopathic tendencies, this is indeed the case. Blair (1997) found that children with psychopathic tendencies were more likely to judge moral transgressions to be permissible if there were no rules against them, than the control group from the same school for children with emotional and behavioral problems but which did not exhibit psychopathic tendencies. Thus, such juveniles seem to assimilate moral transgressions to conventional transgressions, just as Foot’s comment suggests. However, research with adult psychopaths shows that they actually treat conventional transgressions like moral transgressions. Blair (1995) found that psychopathic criminals (as compared with nonpsychopathic criminals) judged conventional transgressions to be authority independent. However, their justifications failed to involve the welfare of others, whereas nonpsychopaths did make reference to the welfare of others in their justifications of their judgments of moral transgressions. Rather, psychopaths tend to offer justifications which refer (implicitly or explicitly) to rules—the same kinds of justification they give with reference to conventional transgressions (R.J. Blair 1995, 24). Thus, it looks like adult psychopaths consider both conventional and moral transgressions to be wrong, independently of an authority, but the kinds of reasons they offer suggest that they do not really understand why such transgressions are wrong. Later (§5.5) I review evidence that confirms that the processing underlying moral judgment in psychopaths is significantly different from that seen in normal individuals. Cima et al. mention some of these results, but offer no explanation. The developmental hypothesis, however, does have something to say. Normal individuals make the M/C distinction from the age of around 39 months (Smetana 1993). It is likely that psychopaths’ emotional dysfunction explains their failure to make this distinction in the normal way. However, the difference between juveniles with psychopathic tendencies and adult psychopaths suggests that psychopaths compensate for their emotional dysfunction by engaging other, more ‘cognitive’ forms of learning. By adulthood, psychopaths have learned which actions are right or wrong, and thus make normal moral judgments as Cima et al. found. Nevertheless, their performance on the M/C task reveals that the particular representations underlying these judgments does not give rise

5 This appears in a footnote added to a reprint of her classic article “Morality as a System of Hypothetical Imperatives.” The text in brackets replaces Foot’s subscripted “shouldm” and “shoulde.”

84 to the standard responses on the M/C task or the normal justifications of such judgments, indicating psychological differences between psychopaths and normal individuals. Another line of support for the developmental hypothesis involves the treatment of psychopathy. It has long been thought to be incurable and untreatable. A recent review supports this view (G.T. Harris and Rice 2006). However, newer research suggests that intervention in adolescence can significantly reduce recidivism (Caldwell et al. 2007). It appears that early and intensive treatment can help mitigate the developmental deficits that would otherwise lead to an adult condition that is no longer treatable. Only the developmental hypothesis offers a way of explaining the differences between youths with psychopathic tendencies and adult psychopaths.

5.4 Emotional learning Some of the most important evidence in favor of the developmental hypothesis involves the particular deficits in emotional learning exhibited by psychopaths. Though there is quite a bit of research documenting these deficits, Cima et al. almost entirely ignore it. Much of this research is reviewed by Blair et al. (2005; see also R.J. Blair 2007). This research indicates that psychopathy is primarily a result of dysfunction of the amygdala and regions of the ventromedial prefrontal cortex (vmPFC) or orbitofrontal cortex (OFC), though it is unclear if these are distinct defects or if they are a result of diminished input from the amygdala to these frontal regions. Before we get to this research, it is important to get clear on terminology. There is no standardized nomenclature for many of the brain regions I will be discussing. Sometimes ‘vmPFC’ is used broadly to refer to the area including the OFC, other times is used to refer to the areas just above the OFC. For example, Finger et al. identify the vmPFC with BA 10 (2008), while other accounts have it including other neighboring areas, (e.g. BA 25, lower 24, 32, medial 11, 12 & 10 (S.W. Anderson et al. 2006)). Here I use the term ‘OFC’ to refer to a sub-area of the vmPFC directly behind the orbits and I will use ‘vmPFC’ to refer to a larger region encompassing the OFC and extending above the OFC along the medial wall of the cortex.6

6 Even among researchers in agreement with this general usage there is some variation. Kringelbach, for example, excludes the lateral and central regions of the OFC from the ventromedial prefrontal cortex (2005). These variations in terminology should not affect the argument here. For further discussion of anatomical details, see reviews by Kringelbach and Rolls (2004) and Wallis (2007).

85 5.4.1 Amygdala dysfunction Amygdala dysfunction disrupts certain kinds of learning: classical conditioning and instrumental learning. Often these forms of learning go together, but Everitt et al. demonstrate that dissociation is possible (2003).7 As in the famous experiments by , classical conditioning involves associating a neutral stimulus (called a conditioned stimulus (CS)) with a stimulus that evokes an (involuntary) autonomic response (called an unconditioned stimulus (US)). Ultimately, the formerly neutral stimulus evokes the autonomic response (called an unconditioned response (UR)) in the absence of the US. Thus, Pavlov paired a bell (CS) with the appearance of food (US), which triggers salivation. After conditioning, the bell alone resulted in salivation (UR). Amygdala damage can also disrupt different kinds of instrumental learning. One kind involves associating a CS with an affective response (distinct from a particular autonomic response), e.g. a hedonic response or an “emotional ‘tone’ that is tagged to a stimulus” (Everitt et al. 2003, 234). Another kind of instrumental learning associates a CS with the “valenced sensory properties” of an US (R.J. Blair, D.G.V. Mitchell, and K.S. Blair 2005, 113). For example, the sight of chocolate cake (CS) is initially a neutral stimulus, but it becomes associated with the taste (US) and ultimately leads us to pursue cake when it is presented visually.8 Blair et al. (2005) argue that psychopaths’ deficits in emotional learning are due to amygdala dysfunction. Psychopaths have difficulty with classical conditioning, especially with respect to aversive stimuli (Lykken 1957; Flor et al. 2002). For example, normal people who are conditioned to expect an electrical shock (US) after hearing a tone (CS) will exhibit a skin conductance response (SCR) after hearing the tone by itself. This kind of conditioning is impaired in psychopaths, though they do show normal URs, for example, SCRs in response to some US (e.g. loud noise, shock, foul odor). Lesion studies reveal a failure to develop conditioned autonomic responses, for example pairing a tone (CS) and a loud noise (US), despite developing declarative knowledge of this

7 For example, Everitt et al. (2003) demonstrate double dissociations with respect to the different kinds of learning subserved by the Central Nucleus (classical conditioning) and basolateral (instrumental learning) regions of the amygdala. For reviews of amygdala function in associative learning, see Cardinal et al. (2002); see also Rolls (2005, chap. 4). 8 While the amygdala is necessary for these forms of association, it appears that the acquired associations are stored in the insula (R.J. Blair, D.G.V. Mitchell, and K.S. Blair 2005, 113).

86 association, in individuals who have suffered bilateral amygdala damage (Bechara et al. 1995) and unilateral temporal lobectomy (including the amygdala) (LaBar et al. 1995). Animal studies corroborate these results (Ledoux 1996). Additionally, Bechara et al. (1995) demonstrated a double dissociation, finding that an individual with selective, bilateral damage to the hippocampus (known to be necessary for declarative knowledge, see §6.2.3) acquired conditioned responses but had no explicit knowledge of the association and an individual with bilateral damage to the hippocampus and amygdala developed neither the conditioned response nor declarative knowledge. A recent neuroimaging study reveals that, while normal controls exhibit activation in the amygdala, OFC, insula and ACC during fear conditioning, psychopaths do not exhibit such activation, nor do they show SCRs—though, both groups exhibit declarative knowledge of the contingencies (Birbaumer et al. 2005). Psychopaths also show significant divergence in autonomic response from normal individuals, particularly with respect to social stimuli (R.J. Blair, D.G.V. Mitchell, and K.S. Blair 2005, 52-6). While they show normal SCRs to threatening visual images, they show reduced startle reflex to threat (R.J. Blair et al. 1997). They exhibit reduced autonomic response to imagining fearful events and to the distress of others. For example, they show reduced SCRs in response to watching a confederate believed to be receiving electrical shocks and in passive viewing of distressing images (Aniskiewicz 1979; House and Milligan 1976). They also have selective difficulty in facial and vocal affect recognition: they cannot recognize sad and fearful facial expressions or vocal affect (R.J. Blair 1995; R.J. Blair et al. 2002). These are all similar to impairments demonstrated by individuals with amygdala damage (R. Adolphs et al. 1995; R. Adolphs, Tranel, and A.R. Damasio 1998; Buchanan, Tranel, and R. Adolphs 2009; Ohman 2009, 128-9). While the amygdala handles both aversive and appetitive conditioning (Baxter and Murray 2002), Flor et al. found that psychopaths have difficulty specifically with acquiring conditioned responses to aversive stimuli (2002, 516). More interestingly, they also found that “attentional processing” (as indicated by EEG-measured event related potentials) was equal or greater in psychopaths, suggesting that “deficits in emotional responding were not attributable to attentional defects” (2002, 514-6). Psychopaths also exhibit difficulty with instrumental learning, for example with passive avoidance tasks (J.P. Newman and Schmitt 1998). Such tasks involve learning to avoid

87 previously punished responses. They also appear to have more difficulty with associative learning involving stimulus-punishment association, than with stimulus-reward association (R.J. Blair et al. 2004). However, psychopaths do not have difficulty associating a CS with a conditioned response (CR) , a form of learning not subserved by the amygdala (Baxter and Murray 2002). Their performance is normal on object discrimination tasks that require learning to pair a particular response with a particular stimuli (i.e. CS-CR association), e.g. to press button one in response to a red light and button 2 in response to a green light (D. G. V. Mitchell et al. 2002). Such learning does not involve associations between a stimulus and a reward or punishment, since the reinforcement depends on the participants action, e.g. the red light stimulus might give rise to no reward if followed by the press of button 2. Thus, psychopaths have deficits in two forms of emotional learning subserved by the amygdala. They are impaired in classical (i.e. Pavlovian) conditioning. While they do not exhibit the UR, surprisingly, they do acquire declarative knowledge of the pairings. They also have difficulty instrumental learning, especially acquiring associations between a CS and a punishment.

5.4.2 Response reversal and vmPFC dysfunction Not only do psychopaths have difficulty with certain forms of emotional learning subserved by the amygdala, they also have difficulty with extinction and response reversal tasks. Such tasks involve changing one’s response when a response that was previously rewarded is either no longer rewarded (extinction) or now leads to punishment (response reversal). Building on similar studies in animals (R. Dias, T. W. Robbins, and A. C. Roberts 1996), Fellows and Farah demonstrated that individuals with vmPFC damage have difficulty with response reversal, but exhibit normal associative learning (2003). Thus, extinction and response reversal are likely subserved by the vmPFC (discussed further in §6.1) and difficulty with these kinds of tasks indicates an additional dysfunction in psychopathy. Joseph Newman et al. (1987) found that psychopaths do poorly on an extinction task involving a one-pack card game. Each card provides a monetary reward or punishment and participants are allowed to draw as many cards as they want from a one hundred-card deck. They can quit whenever they like. At the beginning, the majority of cards are rewarding, but as more

88 cards are drawn the probability of punishment increases. Normal subjects learn to extinguish their initial response to continue drawing cards when it becomes disadvantageous. Psychopaths have difficulty with this and continue to draw, leading them to lose much more money. In fact, in Newman et al.’s study, 75% of psychopaths drew all 100 cards. Similarly, Mitchell et al. found that psychopaths do poorly on the Iowa Gambling Task (IGT) (2002), a task requiring response reversal originally developed for use with vmPFC patients (Bechara et al. 1994; Bechara et al. 1996; Bechara et al. 1997). In this task, there are four decks from which participants must choose. Two decks contain high rewards, but even higher losses—choosing from these bad decks leads to a net loss. The other two decks contain lower rewards but also lower losses—choosing from these good decks leads to a net gain. Normal participants will develop SCRs before choosing from a bad deck and begin to choose from the good decks before they are aware that those decks are advantageous (Bechara et al. 1997). Psychopaths do not appear to learn during this task and end up making more risky choices than the control group.9 Again, this mirrors impairments of individuals with amygdala damage who do poorly on the IGT (Bechara et al. 1999).10 Indeed, psychopaths are impaired on response reversal even on tasks involving instrumental learning not subserved by the amygdala. Budhani et al. found that psychopaths have difficulty with response reversal in object discrimination tasks—tasks in which learning is not impaired in psychopaths (Budhani, Richell, and R.J. Blair 2006). Mitchell et al. also tested psychopaths on the intradimensional/extradimensional shift task (2002). This task requires both

9 One hypothesis is that psychopaths are more influenced by the appetitive ‘rewards’ than the aversive ‘punishment’ on the IGT. They persist in taking the high reward, net loss packs. Psychopaths exhibit similar behavior on the Newman one-pack task. While normal individuals stop after some high punishments, psychopaths persist despite this, some participants even expressed some awareness of this after the task. One is quoted as saying “I should’ve quit when I had all those chips” (J.P. Newman, Patterson, and Kosson 1987, 147). 10 In fact, while both normal individuals and those with vmPFC damage generate SCRs after the reward or punishment properties of their choice have been revealed, individuals with amygdala damage fail to produce both anticipatory SCRs (which vmPFC patients also lack) and SCRs in response to delivery of reward and punishment (Bechara et al. 1999). This could indicate that amygdala damage disrupts input to the psychological mechanisms that facilitate learning in these kinds of tasks. While these results are suggestive, I am not aware of any studies in which psychopaths were given the IGT in combination with physiological recording of SCRs. Nevertheless, the amygdala dysfunction hypothesis would predict that psychopaths, like amygdala patients, would also fail to generate SCRs following the delivery of reward and punishment.

89 response reversal and attention shifting, which is known to be disrupted in individuals with damage to the dorsolateral prefrontal cortex. As expected, relative to controls, psychopaths’ performance was impaired with respect to response reversal but not attention shifting. This indicates that psychopaths exhibit a dysfunction of the vmPFC, but not the DLPFC. Interestingly, Blair et al. (2001) found that children with psychopathic tendencies had less difficulty with the response reversal task. This suggests that vmPFC dysfunction in psychopathy could be a secondary dysfunction resulting from abnormal input from the amygdala. Surprisingly, though, Finger et al. (2008) found abnormal vmPFC (BA 10) activity in children with psychopathic tendencies when errors in response reversal were punished, despite the fact that behavioral responses were comparable to the control group. This suggests that such children are unable to integrate feedback about changing reward contingencies.

5.4.3 Psychopathy and the developmental hypothesis While some puzzling results remain, there is significant evidence that psychopathy is characterized by a severe deficit in particular forms of emotional learning subserved by the amygdala. Psychopaths also have difficulty with response reversal tasks, subserved by the vmPFC. Research on patients with vmPFC damage demonstrates a correlation between difficulty in reversal learning tasks and poor everyday functioning (especially involving disinhibition and socially inappropriate behavior (Rolls et al. 1994; Fellows and Farah 2003)). This suggests that this dysfunction could contribute to behavioral abnormalities in psychopaths as well. Blair and others argue that psychopaths’ reactive aggression is explained by their difficulty with response reversal (R.J. Blair, D.G.V. Mitchell, and K.S. Blair 2005; R.J. Blair 2010). In this way, psychopaths have some similarity to individuals with damage to the vmPFC—leading some to refer to the later condition as ‘acquired sociopathy’ (A.R. Damasio, Tranel, and H. Damasio 1990). This pathology will be the focus of Chapter 6. However, psychopaths are importantly different from vmPFC patients—the latter show no tendency toward instrumental aggression. It is this tendency that makes psychopaths so socially destructive. Individuals with vmPFC damage generally end up hurting themselves more than they hurt others (A.R. Damasio, Tranel, and H. Damasio 1990, 92). Though, ironically, their lack of insight for their condition renders them relatively insensitive to this damage.

90 When it comes to psychopaths, this is unfortunately not the case. Hare (1996) reports a rate of psychopathy between 15 and 25% in North American prisons and Blair et al. (2005) estimate the incidence in the general population at .75% for males. Rates are generally lower for females (Salekin, Rogers, and Sewell 1997), though there has been far less research. Despite this, Hare reports that psychopaths are estimated to be responsible for over half of all “serious crimes” (1993, 87). These figures are admittedly tentative, but it is very plausible that psychopaths are responsible for a disproportionate amount of crime relative to the percentage of the general population that they make up. Blair et al. (2005) argue that psychopaths’ deficits in emotional learning disrupt moral socialization. This helps fill in some of the details of the developmental hypothesis’ account of psychopathy and how it contributes to their antisocial behavior. As they put it: “the emotional deficit present in individuals with psychopathy interferes with socialization such that individuals with the disorder do not find the prospect of goal-directed anti-social behavior aversive” (2005, 37). I now turn to evidence supporting this claim. Moral socialization, at base, involves the shaping of psychological mechanisms to promote moral or pro-social behavior. This is often taken to involve the ‘internalization’ of moral norms and values (Grusec and Goodnow 1994; Hoffman 1994; Kochanska 1995; Kochanska 1997). Parents are certainly an important part of this (Maccoby 1992), though other parties surely have an effect as well, e.g. teachers, peers, etc. Unfortunately this process is difficult to study directly. One reason is that this processes is extended over a long period of time. Grazyna Kochanska and colleagues have done some longitudinal studies on parent-child relations and moral development in toddlers (Kochanska 1995; Kochanska 1997; Kochanska 1998; Kochanska et al. 2005; Kochanska, Aksan, and Joy 2007). As might be expected from the complexity of moral psychological mechanisms discussed so far (as well as that to be demonstrated in the following 3 chapters), the evidence suggests that there is no single causal pathway to successful moral socialization. However, there are important patterns that support the developmental hypothesis. Research from a number of fields highlights the importance of emotional learning for moral socialization. Of course, ‘cognitive’ processes are also important, but these tend to follow affect-driven development (Hoffman 1990; Hoffman 1994). Research in developmental psychology shows that parental induction tends to be the most successful parenting technique

91 (Hoffman 2000). Induction involves emphasizing others’ perspectives, drawing attention to the distress of victims and making it clear that the child’s action caused the harm. Induction tends to be better than other techniques, for example, assertion of power or withdrawal of love. Martin Hoffman (1994) argues that these latter techniques may induce too much arousal in the child, while induction takes advantage of a child’s natural anxiety and utilizes the optimal level of anxiety, which seems to be enough to get the child’s attention and facilitate learning. Too little anxiety and no learning will take place, too much risks triggering anger and resentment instead of learning. This is supported by research in learning theory indicating that, during instrumental learning, the aversive response will become attached to whatever stimulus is most reliably contiguous with the punishment (Dickinson 1980). Thus, aversive conditioning requires a close connection between the transgression and the punishment. The distress of others can function as a good unconditioned stimulus for moral socialization. This stimulus is naturally aversive and generally accompanies transgressions. In fact, Marsh and Ambady (2007) found that perceiving facial expressions of fear was associated with a (self-reported) tendency toward pro-social behavior. This also explains why techniques involving power assertion (e.g. punishment) are less effective. Punishment executed by parents is generally too far removed from the transgression. Such punishment risks conditioning the child to be afraid of the parent. This may get children to behave, but not necessarily for the right reasons. Such parenting may lead to other issues, as it can lead to anger and resentment on the part of the child, which can disrupt learning (Hoffman 1994). Parental induction works to facilitate the instrumental learning connecting the child’s transgression with the naturally aversive stimuli of others’ distress. Thus, emotional learning plays an important role in moral socialization. However, as indicated above, things are not so simple. Individual differences in children’s temperaments are also important. Kochanska’s longitudinal studies uncovered important individual differences involving what she calls ‘fearfulness’. Children high on this measure tend to respond well to what she calls ‘gentle discipline’ (Kochanska 1995; Kochanska 1997; Kochanska, Aksan, and Joy 2007). Gentle discipline involves parental induction as just discussed. Relatively fearful children are naturally at a level of arousal conducive to learning. Higher levels of anxiety as induced, for example by

92 assertions of power, are not as effective at internalization. In fact, Kochanska et al. found that, for relatively fearful children, fathers’ power assertion actually predicted poor socialization outcomes (2005). However, children who are relatively low in measures of ‘fearfulness’ do not respond as well to gentle discipline. This is likely because they have a higher threshold for anxiety or arousal. Interestingly, however, they do not respond well to techniques of power assertion either. This tends to induce anger. Kochanska found that a strong mother-child attachment best predicted successful socialization (Kochanska 1995; Kochanska 1997; Kochanska, Aksan, and Joy 2007). Strong attachment and expressed displeasure by the mother may function to induce the appropriate level of anxiety for relatively fearless children. Thus, moral socialization involves emotional learning, which is understandably mediated by the proper level of arousal and interesting individual differences. As I argued above, psychopaths are dysfunctional with respect to emotional learning. They are systematically insensitive to distress cues, failing to respond autonomically to fearful faces and other important social stimuli and they are impaired in emotional learning involving the association of stimuli with punishment. Without such input, psychopaths develop no aversion to goal directed anti- social behavior and thus exhibit greater rates of instrumental aggression. Consider a simple example. Suzy steals a toy from Johnny. She may experience negative affect due to the responsive crying of the victim. This may be further facilitated by parental induction that draws attention to this distress: “Suzy, do you see how you’ve made Johnny feel? That wasn’t very nice, was it?” If these natural distress cues do not function to connect the act of stealing with the negative affect due to the distress of the victim, in combination with induction, children may have difficulty developing an aversion to stealing and in the future will be more likely to engage in such behavior. Adult psychopaths, who have developed no aversive reaction to the thought of performing such actions, will, if they have learned that such behaviors can be profitable, have a tendency to engage in aggressive behavior to secure resources, power, or other objects of desire. This view is further supported by research involving more pathological individual differences. Wootton et al. found that youths with callous-unemotional (CU) traits do not respond to punishment cues and develop conduct problems regardless of the quality of parenting (1997). Callous-unemotional traits overlap significantly with the first factor in the 2-factor model

93 of psychopath discussed above (§5.1) as the Interpersonal/Affective factor. Only for children without significant CU traits did “ineffective parenting” correlate with conduct problems. Quality of parenting was assessed by a questionnaire measuring five different elements: parental involvement, use of positive reinforcement, monitoring and supervision, consistency of discipline, and use of corporal punishment (Wootton et al. 1997, 303). These results were replicated by Oxford et al. (2003) using a more demographically diverse sample and different ways of measuring quality of parenting and conduct problems and are supplemented by neuroimaging results indicating reduced amygdala activation in CU youths in response to fearful expressions (Marsh et al. 2008). As indicated, such expressions are a distress cue that can function to stop aggressive behavior (R.J. Blair 2001) or to facilitate pro- social behavior (Marsh and Ambady 2007). Thus, youths with CU traits, which overlap with one dimension of psychopathy, develop conduct problems regardless of parenting. This suggests that such youths share psychopaths’ deficits in emotional learning. This hypothesis gains confirmation from neuroimaging indicating reduced amygdala activation similar to that found in adult psychopaths across a variety of tasks. So there is quite a bit of evidence supporting what I have called the developmental hypothesis: that psychopathy results from a deficit in emotional learning and that this deficit contributes to failures of moral socialization and anti-social behavior. Cima et al. have no clear account of how this evidence is related to their hypothesis that psychopathy is a disruption of the relation between moral knowledge and motivation. The developmental hypothesis also claims that in psychopathy, these deficits in emotional learning are accompanied by differences in more cognitive forms of processing that are engaged to compensate for these deficits. Psychopaths present with a pattern of normal moral judgment with significant differences in behavior. Rather than a simple disconnect between knowledge and motivation, I argue that this pattern is the result of underlying processing in psychopaths significantly different from that in normal individuals. Specifically, psychopaths’ deficit in emotional learning is compensated for by more ‘cognitive’ forms of processing. This alternative developmental path leads to declarative knowledge of moral norms, and thus, to normal moral judgment. However, they lack underlying emotional representations present in individuals without this emotional deficit.

94 5.5 Compensation There is significant evidence of abnormal processing in psychopaths, especially with respect to emotional processing. Much of this research is reviewed by Blair et al. (2005, 59-62) and Kiehl (2006, 113-116). Here, I mention some representative findings. Research utilizing emotion attribution and lexical decision tasks indicates abnormal processing in psychopaths. Kiehl has suggested that this abnormal semantic processing associated with affect is the result of reduced affective input to semantic processing and that other types of processes are engaged to compensate (Kiehl 2006). Psychopaths’ performance differs from that of normal participants on tasks involving the processing of abstract concepts (Kiehl et al. 1999) and more importantly on tasks involving emotional concepts. For example, Intrator et al. (1997) found abnormal processing of emotion words during a lexical decision task using SPECT11 imaging despite the fact that there were no behavioral differences. Similarly, psychopaths show no RT differences in lexical decision tasks involving emotional words as compared with neutral words, whereas controls (in this experiment, nonpsychopathic criminals) show faster RTs for emotional words (Kiehl et al. 1999). Another study by Kiehl et al. (2001) indicates increased frontal activity in psychopaths engaged in emotional processing. Thus, there is evidence that psychopaths recruit alternative processes to compensate for their deficits in emotional learning. In fact, in many of these studies psychopaths do not exhibit behavioral differences. That is, psychopaths make judgments comparable to controls (in emotion attribution or lexical decision tasks, for example), but exhibit abnormal processing as indicated by event related potentials (i.e. EEG data) or neuroimaging. This makes differences between psychopaths and normal individuals relatively difficult to detect. It also helps explain why a superficial reading of the research on psychopaths might lead one to posit the motivation hypothesis: that psychopaths exhibit no abnormality in moral knowledge, only in the connection between moral knowledge and motivation. However, closer examination shows that this hypothesis is far too simplistic. The developmental hypothesis makes sense of this abnormal processing in terms of compensation for deficits in emotional learning.

11 Single Photon Emission Computed Tomography is a low spatial resolution (relative to fMRI) imaging technique involving a radioactive tracer that gives off gamma rays that are recorded and used by a computer to produce three-dimensional images.

95 One prediction of this aspect of the developmental hypothesis is that these compensating processes in psychopaths are more akin to Type 2 processing, than to the more automatic Type 1 processing exhibited in normal individuals. This indicates that, for tasks on which psychopaths engage different processes than normal individuals, psychopaths will be more sensitive to cognitive load from normal individuals. While this has not been directly tested, there are some results that could be interpreted as suggesting that psychopaths function under high-load conditions much of the time (see R.J. Blair, D.G.V. Mitchell, and K.S. Blair 2005, 71). Interestingly, there is evidence that on some response reversal tasks, adult psychopaths are more impaired than children with psychopathic tendencies (R.J. Blair, D.G.V. Mitchell, and K.S. Blair 2005, 135-6). This could be interpreted as evidence for increased compensation by adults and thus increased impairment on tasks in which these compensating processes induce cognitive load. More directly relevant to this aspect of the developmental hypothesis, recent neuroimaging studies show abnormal processing in psychopaths during moral judgment. In agreement with the results of Cima et al., a recent fMRI study of psychopaths found no relationship between their score on the 4-factor PCL-R score and the percentage of utilitarian judgments in response to personal moral judgments, indicating a normal pattern of moral judgment (Glenn et al. 2009). However, this study did find decreased amygdala activation in response to personal moral dilemmas in those scoring high on the PCL-R, as well as a negative correlation between individuals’ Factor 1 scores (on the 4-factor model) and the activation of the medial prefrontal cortex, posterior cingulate and anterior gyrus (Glenn, Raine, and Schug 2009), regions overlapping with those activated in studies of Greene et al. (2001; 2004) discussed above (§3.4.1). More interestingly, in personal moral dilemmas they found a positive correlation between right DLPFC activity and PCL-R score (specifically with Factors 3 and 4, and a nonsignificant trend with Factor 2). This suggests an explanation for the normal patterns of judgment in response to personal moral dilemmas: amygdala dysfunction in psychopaths disrupts certain kinds of learning processes that facilitate normal judgments in personal moral dilemmas, to compensate, psychopaths engage abstract reasoning processes subserved by the DLPFC. Again, Cima et al. mention this research, but offer no explanation for differences in processing. They seem too quick to assume that the processing that leads psychopaths to make

96 normal moral judgments is of the same kind as in normal individuals. If that were correct, then the motivation hypothesis would be far more plausible: moral knowledge in psychopaths is intact—they simply don’t care and don’t adjust their behavior in light of this knowledge. However, the evidence reviewed in this section shows that psychopaths exhibit different processing, even when exhibiting no behavioral differences. This provides significant support for the developmental hypothesis. That psychopathy is primarily caused by a deficit in emotional learning and that the brain of the psychopath compensates by recruiting alternative processes explains the disconnect between psychopaths’ antisocial behavior and their normal pattern of moral judgment. Adult psychopaths have had time to learn the ‘correct’ moral judgments, but they utilize a learning process which is slower and less closely connected to motivation. So, they know the ‘right’ answer, but they lack the emotional base of this knowledge present in normal individuals. Actually, ‘normal’ here must be qualified. Gordon et al. found evidence for similar processing differences in male college students (2004). Those rating high on the Psychopathy Personality Inventory (PPI), a measure designed for identifying psychopathic tendencies in noncriminal populations, exhibited increased activation in the right DLPFC during an task. Participants rating low on the PPI showed higher levels of activation in the inferior frontal cortex, medial prefrontal cortex and amygdala. Similarly, a neuroimaging study found processing differences in adolescents when compared to adults during a decision making task involving reasoning about consequences. Participants were asked to say whether each of a series of scenarios (e.g. swimming with sharks, going for a walk) was a good idea or a bad idea. Though there were no differences in their responses, Baird et al. (2005) found significantly greater right DLPFC activation in adolescents when answering ‘bad idea’, as compared to greater activation in bilateral insula and fusiform face areas in adults. This indicates that, while adolescents use more effortful cognition to answer these questions, adults utilize more efficient automatic processing. Not only does this shed light on why adolescents are prone to bad decision making, it also indicates how development can lead to improved decision making in nonpathological populations. Thus, within a population of male college students and between adolescents and adults, we see a similar pattern of distinct provessing with no behavioral differences on tasks that involve emotional processing in ‘normal’ adults. It appears that the alternative processing present

97 in psychopaths is also present in some ‘normal’ individuals. Some individuals utilize more emotional processing while others utilize more ‘cognitive’ processing on the very same task. In addition, we may see diachronic instances of similar processing differences. Following a reasoned change in moral belief our affective responses do not immediately change. For example, suppose you are convinced by the arguments of (1975) and become a vegetarian. It is unlikely that you will instantaneously develop a negative affective response toward eating meat. If you were previously disposed to appetitive responses toward meat, these responses will probably continue for a while. In fact, Rozin et al. (1997) found that moral vegetarians, as compared with health vegetarians, report being more disgusted by meat. Fessler et al. (2003) found that moral vegetarians are not more prone to disgust than those who avoid meat for nonmoral reasons, indicating that increased disgust is a result, rather than a cause, of a change in beliefs regarding the moral status of eating meat. A similar dissociation may occur in committed utilitarians. Consider Bernard Williams’ well-known story of ‘Jim and the Indians’ (1973). A utilitarian who believes that the morally right think to do is to kill one of the ‘indians’, may nevertheless feel a negative affective response towards the thought of it. If they found themselves in such a situation, they might be unable to perform the action, despite believing it to be morally required.12 It looks like the kind of dissociation present in psychopaths may be surprisingly prevalent. It is important to note that psychopaths seem to exhibit a global variety of this dissociation, while the instances in nonpathological individuals tend to be more local. I return to these issues in Chapter 9 (§9.3). To sum up, the research on psychopaths does not demonstrate that emotion plays no role in moral judgment. While Cima et al. are right that psychopaths exhibit a disconnect between

12 Incidentally, Peter Singer himself seems to exhibit something similar. When asked by a journalist about the tension between his utilitarian views and his decision to spend tens of thousands of dollars a year on the professional care of his mother, stricken with Alzheimer’s disease, he responds that it is “probably not the best use you could make of my money” (Specter 1999). There need not be any doxastic (as opposed to practical) inconsistency for utilitarians here. Sometimes doing what you believe to be morally right is hard. Peter Singer notes: “I think this has made me see how the issues of someone with these kinds of problems are really very difficult…Perhaps it is more difficult than I thought before, because it is different when it’s your mother” (Specter 1999). Similar dissociations may occur in individuals who have strong appetitive responses toward having premarital relations with their romantic partner despite their sincere belief that it is immoral.

98 their representations of moral norms and motivational structures, this does not show that emotions do not play an important role in moral judgment in normal individuals. Rather, the developmental pattern exhibited by psychopaths indicates that when certain kinds of processes (e.g. emotional learning) are disrupted, other kinds of processes can compensate. There is evidence of similar processing differences across ‘normal’ individuals. This supports the contentions that moral judgment is subserved by a variety of kinds of processes and that emotion does play a role in moral judgment. However, research on psychopathy reveals that the psychology of moral judgment is far more complicated than might be suggested by the basic dual process model. An adequate model must go beyond this model in explaining the roles of emotion and ‘cognition’ in moral judgment. In the following chapter, I discuss the roles that frontal regions, and especially the vmPFC, play in moral judgment. Despite important differences, individuals with brain damage to the vmPFC are similar in some ways to psychopaths. I argue that research on vmPFC patients further supports my claims about the importance of emotion in moral development. Investigation of moral judgment in these individuals also reveals an important synchronic role for emotion and clarifies the role that representations of mental rules may play in moral judgment.

99

CHAPTER 6

GAGE WAS “NO LONGER GAGE:” VMPFC DAMAGE AND THE ROLE OF EMOTION IN MORAL JUDGMENT

Perhaps the most famous human to have damage to his vmPFC was the unfortunate railway man Phineas Gage. In 1848, an accidental explosion blew a tamping iron—nearly 4 feet in length and over 13 pounds—through Gage’s left front cheek, exiting at the top of his skull. Physical recovery took 2 months, but the damage resulted in profound behavioral changes. His doctor reported that Gage went from “the most efficient and capable foreman” to a man “fitful, irreverent…manifesting but little reverence for his fellows, impatient of restraint or advice when it conflicts with his desires, at times pertinaciously obstinate, yet capricious and vacillating, devising many plans of future operation, which are no sooner arranged than they are abandoned;” according to friends, he was “no longer Gage” (Harlow 1868). If Phineas Gage is the most famous, Antonio Damasio’s patient, identified as E.V.R. and featured in his Descartes’ Error, is probably one of the most studied individuals with vmPFC damage (A.R. Damasio 1994). E.V.R. had been a successful businessman, married happily, and a good father. Surgery to remove a meningioma in the orbitofrontal region left him with bilateral OFC lesions. Following the surgery, he was divorced, lost his job, and entered into several unsuccessful business enterprises, losing a large sum of money to a scam artist. His second marriage, to a prostitute he had known for a month, lasted only six months. Unable to hold a job due to his disorganization and interpersonal difficulties, he moved back in with his parents. Despite the fact that his intelligence and memory seemed untouched, he had severe deficits in decision making. Simple decisions like deciding where to eat dinner left him mulling over every possible consideration—to the point of driving to every

100 restaurant to see if they were busy. Most tragic of all, E.V.R. was emotionally detached, even lacking a full appreciation of his own misfortunes.

6.1 vmPFC damage: the clinical profile Damasio, his colleagues and others have done extensive work on patients with this kind of damage, trying to uncover the functions of this brain region. Focal lesions to the vmPFC produce a peculiar pattern of symptoms.1 Such patients present with profound emotional dysfunction, but with intact intellectual skills and social and moral knowledge (Saver and A.R. Damasio 1991). As we will see, this combination makes them an important test case for hypotheses about the role of emotion in moral judgment. As noted above, vmPFC patients bear some similarity to psychopaths in their tendency toward reactive aggression, but they do not demonstrate instrumental aggression. Their poor decision making often makes them the greatest victims of their condition—though they exhibit a noticeable lack of insight into their own condition (A.R. Damasio 1994; Beer et al. 2006). While they exhibit generally blunted affect and diminished autonomic responses, they also have a tendency to respond with irritation or anger to relatively minor provocation (A.R. Damasio, Tranel, and H. Damasio 1990; A.R. Damasio 1994; S.W. Anderson et al. 2006). One patient is described as berating her almost 90-year-old mother in a clinic waiting room, when her mother could not produce her appointment slip quickly enough (Koenigs and Tranel 2007, 954). These clinical symptoms suggest a problem with emotional regulation. Patients with vmPFC lesions (especially the OFC) have difficulty with response reversal tasks, though they exhibit normal associative learning (Rolls et al. 1994; Fellows and Farah 2003; Fellows and Farah 2005a). This is unsurprising given the results in the

1 Of course, there is significant diversity in the extent and location of these lesions across patients. This introduces a fair bit of variation in the particular pathologies of these patients. For example, J.M., the subject of Blair and Cipolotti (2006), has difficulty in response reversal only in social contexts, he does fine on the Iowa Gambling Task. Increasing understanding of the function of sub-regions of the vmPFC and OFC are shedding light on this variability (see e.g. Fellows and Farah 2007, 2672). Another difference that gives rise to systematic variation involves the age of injury. Early onset vmPFC damage manifests in importantly different ways than adult onset vmPFC damage (S.W. Anderson et al. 1999). I discuss early onset damage below (§6.4).

101 previous chapter that associative learning is subserved by the amygdala while the vmPFC is necessary for response reversal. Furthermore, their difficulty with response reversal is highly correlated with socially inappropriate behavior (Rolls et al. 1994; Fellows and Farah 2003). As stated above, vmPFC patients have difficulty with the IGT (Bechara et al. 1994; Bechara et al. 1996; Bechara et al. 1997). This is most likely because performance on the gambling task requires response reversal. Where normal individuals begin taking from the packs which offer high rewards but net losses and then switch to the advantageous decks, patients with vmPFC damage learn to take the high reward packs and then perseverate even after it becomes clear that the high reward, net loss pack is disadvantageous (A.R. Damasio 1994, 213). In a modified version of this task, in which there is no initial bias toward the high reward, net loss packs, vmPFC patients perform normally (Fellows and Farah 2005a). This poor performance is correlated with abnormal autonomic responses. Bechara et al. (1997) found that normal individuals generate SCRs following the selection of a card issuing in a reward or punishment and, as the task proceeds, they begin to generate anticipatory SCRs before risky choices, even prior to any conscious awareness that their choices are risky. While vmPFC patients do generate normal SCRs following reward or punishment, they failed to develop SCRs prior to choosing a risky card (Bechara et al. 1996; Bechara et al. 1997). This is in contrast to individuals with amygdala damage who also do poorly on the IGT, but fail to produce SCRs in response to the delivery of rewards and punishments (Bechara et al. 1999). This suggests that, because they fail to produce these emotional responses, amygdala patients cannot learn, while vmPFC patients respond to rewards and punishments, but are unable to transfer these emotional responses into anticipatory emotional responses that could guide decision making. This research led Damasio, Antoine Bechara and colleagues to put forward the somatic marker hypothesis (SMH) (A.R. Damasio 1994; Bechara and A.R. Damasio 2005; Reimann and Bechara 2010). Because of the correlation between physiological responses (e.g. SCRs) and performance on the IGT, it was posited that these physiological responses contribute to decision-making, by biasing choice. These physiological responses are called somatic markers and include changes in heart rate,

102 blood pressure, and other aspects of bodily function under the control of the autonomic nervous system. These somatic markers are facilitated by the vmPFC, so when this brain area is damaged individuals do not produce somatic markers prior to choice and so are unable to use them to guide decision-making. The SMH is an attempt to explain how emotion influences decision making. It shares much in common with the James-Lange theory of emotion, according to which, emotions are the of bodily changes (discussed further in §9.2.2). In contrast, according to the SMH, somatic markers can function unconsciously and they can also by- pass the body by using a cortical “as-if loop” (A.R. Damasio 1994, 155; Bechara and A.R. Damasio 2005) This theory has received significant critical attention, much of which is nicely reviewed by Dunn et al. (2006). Some of the main issues of contention include the timing and extent to which participants in the IGT are consciously aware of which decks are advantageous and the extent to which the bodily responses identified as somatic markers are necessary for normal decision-making. As noted, Bechara et al. had claimed that individuals were choosing advantageously before they were consciously aware of which decks were advantageous (1997). Using more probing questions, Maia and McClelland (2004) found evidence indicating that normal participants had more conscious access to the relative value of each deck than had previously been thought. However, a more recent study utilizing a post-decision wager method to gauge knowledge of the value of each deck compared IGT performance of a group in which Bechara et al.’s original method of questioning was used, with a no-question group and a group in which Maia and McClelland’s more probing questions were used (Persaud, McLeod, and Cowey 2007). They found that, in both the group given Bechara et al.’s original questions and the no-question group, wagers lagged behind advantageous decision by about 30 trials (where each trial is the choice of single card). The third group showed no such lag, suggesting that Maia and McClelland’s method of probing may have influenced participants’ awareness rather than measuring it. Of course, these results are not conclusive and there remains general controversy over the existence or so-called implicit learning and it’s underlying

103 substrates (for reviews of this debate and some skepticism, see Shanks 2005; Shanks 2010). I return to this issue in Chapter 7 (§7.2.6). One problem, given our purposes here, is the SMH’s lack of specificity. Indeed, Reimann and Bechara claim that the SMH is “the only neurological theory of decision- making” (2010, 773). They argue that, for example, a theory about dopamine’s role in error-prediction of reward value (to be discussed shortly) “is not an alternate view to the somatic marker hypothesis, but rather one specific link in a broader neural model of decision-making described under the somatic marker framework” (Reimann and Bechara 2010, 768). Recent research utilizing not just lesion studies but also work on substance abusers and results from neuroimaging and single-neuron recording studies is shedding much new light on the vmPFC. Whether this research is consistent with the SMH is an issue I leave to one side (though I discuss perceptual theories of emotion again in Chapter 9). Here I present evidence that the vmPFC is the central node of a network of interconnected systems crucial for decision making and the representation of the value. This will help shed light on the etiology of vmPFC patients’ abnormal pattern of moral judgment.

6.2 vmPFC function: representing value In the last 10 years, a wealth of research on valuation and choice has been done, especially in the new field of (for a review, see Glimcher et al. 2009). This research has utilized single-neuron recordings in primates as well as human neuroimaging studies. One result of this research is the emergence of a two-stage model of choice (Kable and Glimcher 2009; Niv and P.R. Montague 2009). I introduce the broad outlines of this account. The first stage involves the assessment of the value of different options. This value is variously referred to as reward value, subjective value, offer value or goal value.2 The second stage involves the use of this information to make a choice. Several

2 In fact, Kable and Glimcher (2009) discuss three different signals related to value, each coded by different populations of neurons in the OFC: offer value, chosen value, and taste. Each of these signals plays different roles in processes of learning and evaluation related to reward value. For ease of exposition, I will often refer to this simply as ‘value’ or ‘subjective value’.

104 components are thought to contribute to the valuation stage. The vmPFC (especially the OFC) and the dorsal striatum (part of the basal ganglia) represent the value of different options in a “common currency,” which allows disparate to be compared. The midbrain dopamine system, which projects to the nucleus accumbens (part of the dorsal striatum), the amygdala, and the medial PFC, is thought to facilitate the learning of subjective values in the vmPFC by representing a reward prediction error (Schultz, Dayan, and P.R. Montague 1997). This signal represents the difference between the expected value of a reward and the actual value. An unexpected reward, for example juice paired with a cue for the first time, leads to a spike of above-baseline phasic dopamine (DA) activity. As a reward is consistently paired with a cue, the DA signal shifts in timing, so that it follows the cue and precedes the reward. An expected reward is not followed by a DA spike. However, when an expected reward is not received, DA fires below baseline (Balleine, Daw, and O'Doherty 2009). In other words, DA signals whether a reward was as good, better, or worse than expected. As common sense would suggest and is confirmed by prominent computational theories of learning, this is a key parameter to be represented by any system capable of learning.3 The second stage of decision is implemented by more lateral regions of the PFC and the parietal cortex, which makes sense given their proximity and connections to motor regions, including the motor cortex, premotor cortex and supplementary motor areas. Focusing on the way the brain represents value reveals a central role for the OFC. The OFC receives information from all sensory modalities, as well as having reciprocal connections with subcortical structures like the amygdala and hypothalamus, and to medial regions of the PFC like the ACC (Kringelbach and Rolls 2004; Wallis 2007). This makes it an place to integrate information about value. Understanding how this works requires a bit of terminology, which has been adopted by researchers in this area from earlier uses in the animal learning literature. The OFC codes the value of reinforcers. Positive reinforcers, i.e. those that increase the

3 Specifically, a reward prediction error signal is necessary in temporal difference models, which have been influential in the animal learning literature. For more on the dopamine system and its role in learning and reward, see the chapters in Part IV of Glimcher et al. (2009). I discuss this further in §7.2.

105 probability of the associated response, are known as rewards, and negative reinforcers are known as punishers (Rolls 2005; Kringelbach 2005). Rewards and punishers affect the frequency of behaviors that lead to them. Reinforcers are primary if they do not depend on learning and secondary if they are learned. A meta-analysis of neuroimaging studies indicates that the value of rewards is handled by the medial OFC, while that of punishers is handled by the lateral OFC and that the posterior OFC handles simple reinforcers like pleasant tastes and pain, while the anterior OFC handles more abstract reinforcers, like monetary gains or losses or even musical experience (Kringelbach and Rolls 2004). There is even evidence that the value of social reinforcers, e.g. faces of (unfamiliar) infants, are represented in the medial OFC (Kringelbach et al. 2008). In a widely-cited study using rhesus monkeys, single-neuron recording found populations of neurons in the OFC that coded for the subjective value of different rewards, i.e. the value a reward has for a particular individual (Padoa-Schioppa and Assad 2006). One individual may prefer apple juice to grape juice, another may have the opposite preference. These are differences in subjective value. In choice, various kinds of information must be integrated. For beverages, this may involve the taste of different kinds of beverages and the quantities of these available. For example, monkeys may prefer equal volumes of apple juice to tea. But, they may prefer more tea to less apple juice. Making decisions like this requires integrating information from multiple senses. This is done in the OFC. Padoa-Shioppa and Assad studied monkeys choosing between different quantities of different kinds of juice. These choices were used to calculate subjective values for these juices. These values were found to correlate with activation patterns of certain neuronal populations in the OFC. A subsequent study found that these populations of neurons coded for absolute subjective value, i.e. neuronal response to juice rewards did not depend on the particular alternative reward it was paired with in any given choice (Padoa-Schioppa and Assad 2008). Correspondingly, monkeys’ preferences obeyed transitivity. Thus, it appears that the value of different rewards is represented in a “common currency” in the OFC.

106 Human neuroimaging studies provide confirmation of this. Kable and Glimcher (2007) used fMRI to investigate temporal discounting, i.e. the tendency of individuals to value rewards less, the more temporally distant they are. For example, many people may take $25 tomorrow over $20 today, but many fewer will take $25 in a year over $20 today. Of course, the extent to which people do this varies, ranging from the patient and ‘rational’, e.g. preferring $21 in a year over $20 dollars right now, to the impatient and desperate, e.g. preferring $20 today over $50 tomorrow. Kable and Glimcher presented participants with intertemporal choices of just this kind. They found that activity in the medial PFC (an area of vmPFC) and dorsal striatum corresponded to individuals’ subjective value of different offers as indicated by their choices between particular pairs. In addition, across choices, the function according to which different individuals discounted delayed rewards (i.e. the relation between increasing delay and decreasing subjective value) mirrored the function according to which neural activity in response to offers decreased with increasing delay. This provides strong evidence that these regions represent the subjective value of both immediate and delayed rewards in a common currency. These results are supported by lesion studies. Fellows and Farah (2007) investigated decision-making in patients with vmPFC damage. Participants were asked to make preference judgments for three different categories: food, colors and famous people. For example, while being shown pictures of Britney Spears and Shaquille O’Neal, they might be asked “which do you like better?” Despite making normal perceptual judgments (e.g. about which of 2 color swatches appears more blue), patients with vmPFC damage were found to make preference judgments that were significantly more inconsistent, i.e. violated transitivity more, than either a normal control group or a control group with damage to other frontal regions. Similarly, in an fMRI study of normal individuals, the vmPFC was more active during judgments of how much participants would enjoy events in the present, than when making judgments about events in the past (J.P. Mitchell et al. 2010). More interestingly, the greater the difference in activation level during judgments of present and future pleasures, the more likely the individual was to discount future rewards at a later date (between 2 and 110 days later).

107 In fact, it appears that the common currency representation of value in the vmPFC enables humans to literally compare apples and oranges. FitzGerald et al. found that the medial OFC reflects the subjective value in choices of “incommensurable goods” such as a mug or a box of chocolates and a sum of money (2009, 8392). Collectively, these results suggest that the OFC integrates a variety of kinds of information, e.g. about the quantity, quality, and delay of a reinforcer, into an abstract representation of the value of that reinforcer. OFC’s input from every sensory modality makes it well placed to perform this integrating function. Other elements more closely related to choice than to value are represented in other regions. For example, the cost of expending effort to bring about an outcome does not appear to be integrated into the representation of value in the OFC. This makes some sense, as appreciating the cost needed to bring about an outcome involves planning how to bring it about. This is handled by lateral regions of the PFC more closely connected to motor cortex (Kable and Glimcher 2009). Thus, integrating the cost of effort into decision making is handled by the ACC (most closely connected to both motor and reward regions—the OFC has relatively less motor connections) (Wallis 2007, 46). Many decisions also involve risk, requiring decision-makers to integrate information about the probability of success into the decision-making process. This appears to be done by the ACC, parietal cortex and by midbrain dopamine neurons (Wallis 2007, 45). While the vmPFC (especially the OFC) is crucial for the representation of value, it is important to note two other important connections: the OFC’s relation to the amygdala and to the DLPFC.

6.2.1 OFC-amygdala connections The OFC is thought to modulate amygdala activation. As we saw earlier, the amygdala is involved in emotional learning. Both amygdala damage and OFC damage can impair performance on the IGT (Bechara et al. 1999). On the IGT, normal individuals are first drawn to the high-reward, net-loss packs, and must reverse this response going for the low-reward, net-gain packs. Since the OFC is necessary for response reversal, individuals with damage to OFC cannot do this and so continue with the previously learned response. Individuals with bilateral amygdala damage are impaired in processes

108 of emotional learning, fail to generate SCRs—either in response to or anticipation of rewards and punishments—and exhibit impaired performance on the IGT. Thus, both the amygdala and OFC are necessary for normal IGT performance. Individuals with substance abuse problems (e.g. alcohol, methamphetamines, and cocaine dependence) also exhibit abnormal IGT performance (Grant, Contoreggi, and London 2000; Bechara et al. 2001). Indeed, in the DSM-IV, substance abuse is defined in part by behavioral problems involved in work, family, interpersonal, or legal situations or physically dangerous behavior (American Psychiatric Association 1994). Decision- making problems in substance abusers have been linked to OFC dysfunction (Bechara 2005; Schoenbaum, Roesch, and T.A. Stalnaker 2006). Some have suggested that OFC dysfunction is a predisposing cause of substance abuse (Bechara 2005). Animal models also indicate that drug use can lead to changes in the OFC, suggesting that substance abuse can lead to greater decision-making difficulties (Schoenbaum, Roesch, and T.A. Stalnaker 2006). It is plausible that OFC dysfunction is both a cause and an effect of drug abuse. Research investigating the role of the OFC in addiction, as well as research on vmPFC patient indicates that some kinds of decision-making crucially depend on the interaction of the OFC and the amygdala. As we saw earlier, the amygdala is necessary for emotional learning, for example, learning simple associations between an auditory tone and an electric shock or a green light and a food reward. On it’s own, however, this kind of associative learning has a few drawbacks: it is slow, largely dependent on trial- and-error, and it is relatively short-sighted, most sensitive to immediate stimuli and outcomes (Wallis 2007, 46). When the emotional learning capacities subserved by the amygdala are coupled with the OFC, the system has much more flexibility and power. As we’ve seen, the OFC integrates a variety of information into the representation of value. This is particularly useful when comparing two outcomes (e.g. whether to choose a small quantity of apple juice or a larger quantity of water, or whether to major in engineering or literature) and in thinking about future outcomes rather than just immediate ones (as just discussed, the OFC represents both immediate and delayed rewards).

109 And so, the amygdala without the OFC is a short-sighted, slow-learner. The OFC without the amygdala has no emotional input. As Bechara et al. note, vmPFC patients often engage in behaviors that lead to poor financial outcomes or damaged relationships, but never in behaviors that cause physical harm to themselves or others; whereas patients with bilateral amygdala damage exhibit overly trusting behaviors and do engage in behaviors that put themselves or others in physical danger (1999).4 Thus, the dynamics of amygdala-OFC interaction suggest that the amygdala drives affective responses to immediate rewards and punishments, while the OFC’s representations of value incorporate future outcomes and allow the direct comparison of alternatives. Schoenbaum et al. discuss research from animal studies supporting this (2006; e.g. Saddoris, Gallagher, and Schoenbaum 2005; see also Phelps 2006 for a review of amygdala modulation by regions of the PFC). Functional connectivity analysis in a human fMRI study reveals reciprocal activation of the OFC and amygdala when looking at angry faces, in contrast to no such activation in participants with intermittent explosive disorder, a condition characterized by a tendency toward reactive aggression (Coccaro et al. 2007). Thus, the OFC appears crucial to modulate immediate affective responses. In cases of OFC damage, the amygdala activation is not modulated by the OFC and we see the well-known pattern of impulsivity and poor decision-making, which Bechara et al. aptly called “‘myopia’ for the future” (1994, 14). On the IGT, individuals must reverse previously learned responses. This requires integrating information about rewards and punishments over time and using this to change responses. This is exactly the function enabled by the OFC and its modulation of the amygdala. Consistent with this, Fellows and Farah (2005b) found foreshortened future time perspective in vmPFC

4 Indeed, they cannot recognize untrustworthy or unapproachable individuals by sight, though they have no problem when given verbal descriptions of such individuals (R. Adolphs, Tranel, and A.R. Damasio 1998). Additionally, individuals with bilateral amygdala damage lack any sense of social distance in interpersonal interactions, yet they are aware of the relevant norms (D.P. Kennedy et al. 2009).

110 patients, indicating that when vmPFC patients imagine the future, they do not look as far ahead as controls.5 Patients with vmPFC damage also exhibit abnormal behavior in the Ultimatum Game (Koenigs and Tranel 2007). This is an economic decision making game requiring two players to split a sum of money. One player is the proposer and makes an offer to the second player, the responder, to split the money. For example, the proposer may offer to split $10 by giving $4 to the responder and keeping $6 for himself. The responder can either accept the offer, in which case the sum is split in accordance with the offer, or reject the offer, in which case neither player receives any money. Individuals with vmPFC damage rejected more unfair offers (e.g. $7/$3, $8/$2, and $9/$1) than control groups. Although systematic evidence was not collected, anecdotally, vmPFC patients exhibited stronger emotional reactions in response to these unfair offers. This is unsurprising given the results of clinical work (discussed §6.1) revealing a tendency toward reactive aggression. A congenial hypothesis about this evidence is that accepting an unfair offer (the ‘rational’ thing to do: after all, $3 is better than none, right?) requires modulating a negative emotional reaction to the unfair offer. This modulating function is likely subserved by the OFC. Indeed, this tendency toward such outbursts, especially in response to relatively small provocation, would be puzzling in a population of patients otherwise characterized by flattened or blunted affect. The hypothesis that the OFC is responsible for modulating emotional responses in other areas, e.g. the amygdala, helps explain this otherwise paradoxical result.6

5 Fellows and Farah (2005b) also found normal temporal discounting in these individuals. This represents a lingering puzzle. One suggestion is that there is functional differentiation within this region. For example, Fellows and Farah (2007) found that difficulty with response reversal was correlated with posteromedial OFC damage while inconsistent preferences were correlated with damage to the ventral wall of the medial PFC. 6 Of course, things are likely quite complicated. An fMRI study of the ultimatum game found that responses to unfair offers from a human, as compared with both fair offers from a human and unfair offers from a computer, involved significantly higher activation in bilateral anterior insula (Sanfey et al. 2003). In addition, the level of activation correlated with the degree of unfairness of the offer and unfair offers that were later rejected involved significantly greater insula activation than those that were accepted.

111 Some emotions can be triggered by learning about alternative outcomes. For example, finding out that a movie was bad may make one feel relief at having decided not to go. Alternatively, finding out that the movie was terrific may lead to regret. Similarly, finding out alternative outcomes can sometimes facilitate learning. For example, by seeing the next card turned over, naïve Black Jack players may come to hit on 12 rather than stay. We have seen that the OFC facilitates the comparison of alternatives. If so, we should expect individuals with damage to the OFC to be impaired

Thus, behavioral responses to unfair offers seem to be driven by activity in the anterior insula. The anterior insula has been implicated in, among other things, both the experience of disgust and observing facial expressions of disgust (Wicker et al. 2003). Sanfey et al. do not report selective activation of the amygdala in response to unfair offers as might be expected by a simple version of the hypothesis that the OFC modulates amygdala activation in normal participants in response to unfair offers and that, lacking this modulation, vmPFC patients exhibit increased rejection of unfair offers. This suggests that things are more complicated than this simple hypothesis holds. We do know that there are significant reciprocal connections between the anterior insula and the amygdala and this circuit is crucial to normal emotional processing, especially emotional experiences (J.S. Morris 2002). Indeed, Bud Craig has recently suggested that the insula functions as a center for the integration of emotionally relevant processes, including input from the amygdala, OFC, and ventromedial and dorsolateral prefrontal regions (Craig 2009). Additionally, that amygdala activation is not reported by Sanfey et al. does not necessarily undermine the view sketched in the text that the OFC plays a role in modulating amygdala activation in the ultimatum game. As mentioned above, the amygdala responds to both positive and negative reinforcers. Analysis of fMRI data generally requires comparisons between conditions. Absolute activation level is often not useful. Thus, the hypotheses under test in various studies will determine the comparisons made. This will sometimes have the effect of canceling out activations seen across conditions. In situations like the ultimatum game, there may be amygdala activation across both control and experimental conditions. This activation would not show up in analyses contrasting activation between experimental and control conditions. Sanfey et al. did find that the DLPFC was also selectively activated in response to unfair offers, though the pattern of activation in DLPFC does not predict acceptance or rejection behavior. The authors suggest that this activation may indicate the “representation and active maintenance of the cognitive demands of the task, namely the goal of accumulating as much money as possible” (2003, 1757). This is consistent with the role of the DLPFC in self-control as discussed below (§6.2.2). This suggests that vmPFC patients’ abnormally high rejection of unfair offers (and possibly their difficulties with emotional regulation) in the ultimatum game stems from a failure to integrate both emotional and cognitive processes in arriving at a ‘rational’ evaluation of the situation.

112 in learning from information about alternatives as well as to exhibit diminished regret in response to such information. To test this, a lesion study utilizing a gambling task manipulated whether participants saw information about unchosen alternatives (Camille et al. 2004). Participants choose between the gambles present on two ‘spinners’. For example, on one trial participants choose between two gambles: (1) a 50% chance of either $50 or -$50 and (2) a 20% chance of $200 and an 80% chance of -$50. (These gambles corresponded to French francs in the actual experiment.) In one condition, participants learned the outcome of the chosen gamble and the outcome of the unchosen gamble. Enabling them to see what they would have one had they chosen the other gamble. In another condition, they learned only the chosen outcome. Both vmPFC patients and normal controls responded emotionally (as measured by self-reports and SCRs) to gains and losses. However, only control participants exhibited responses affected by information about unchosen gambles. Controls experienced regret when the outcome of unchosen gambles were better than chosen ones and only controls were able to utilize this feedback about unchosen alternatives to learn. And (because of the outcome structure of the experiment) by modifying their strategy controls were able to finish the experiment in the black, while OFC patients averaged net losses. Another study utilizing economic decision making games indicates that vmPFC patients exhibit diminished guilt—an emotion that depends not just on what has happened to the subject of that emotion, but also to others (Krajbich et al. 2009). The authors suggest that the vmPFC “contributes to planning for the future for the same reason that it contributes to negotiating interactions with other people: it mediates the elicitation of emotions based on outcomes that are not directly experienced, but imagined” (2009, 2191-2). Indeed, this idea gains support from a recent study finding similarity in participants’ real (versus hypothetical) decisions affecting themselves in the future and others, and that these contrasted with their decisions affecting themselves in the present (Pronin, Olivola, and K.A. Kennedy 2008). The poor decision-making of some substance abusers may be given a parallel explanation. One subset of addicts demonstrates behavioral and physiological

113 performance on the IGT similar to that of vmPFC patients.7 This suggests that OFC dysfunction and its failure to regulate the amygdala is behind these decision making problems and contributes to compulsive drug use and an undervaluing of the future consequences of this continued use. Additionally, there is a subset of normal individuals with behavioral and physiological similarities to vmPFC patients. This suggests that decision-making deficits in individuals without brain damage may also be traceable to OFC dysfunction and that these individuals may also be at greater risk for addiction. So the amygdala-OFC connection represents a key part of normal decision making. The OFC is crucial both when reward contingencies change and a learned response must be reversed (as in response reversal tasks), as well as in cases in which individuals must compare alternatives (for example: should I continue to use drugs or not; should I enter this business relationship?). The poor decision-making of vmPFC patients, which is notable in part because it is so obviously poor, is a result of a failure of the OFC to integrate information into representations of the value of outcomes and to use these representations to modulate the amygdala and to guide behavior. We have also seen how this hypothesis helps explain a variety of results involving vmPFC patients, for example, their diminished future perspective (Fellows and Farah 2005b), their tendency toward angry outbursts, and their impairment in emotions trigged by representations of outcomes and their alternatives (e.g. regret and guilt). However, as might be expected, situations can be set up in which the defects present in vmPFC patients that are normally disadvantageous are actually beneficial. Baba Shiv et al. tested vmPFC patients, individuals with substance dependence and normal controls on a simple investment game (Shiv, Loewenstein, and Bechara 2005). Participants were given $20 and for each of 20 rounds, they could invest $1 for a 50% chance of gaining $2.50 and a 50% chance of nothing—forfeiting the dollar. Since the expected value of investing is $1.25, the perfectly ‘rational’ individual should invest in every round. In fact, vmPFC patients and substance dependence individuals invested on

7 This subset of decision makers is in contrast to two other groups. On the one hand is a group of addicts which exhibits neither behavioral or physiological abnormalities. Bechara describes these as “‘functional’ addicts” (2005, 1462). In contrast, there is another group of addicts which exhibit SCRs, but are hypersensitive to reward and so overvalue the continued use of the drug.

114 significantly more trials, on average (83.3% and 80.9% of trials, respectively), than the control group (57.6%). Another study found similar results comparing vmPFC patients to a normal control group and a group of patients with brain damage to areas unrelated to emotion (Shiv et al. 2005). This suggests that emotional responses to risk, especially after losing rounds, pushed normal participants to avoid risk by not investing. The vmPFC patients and substance dependent participants lacked these emotional responses and so were more likely to take risks on each round and, in this game, ended up with (statistically significantly) greater profits, on average ($27.20 and $24.60 for the substance dependence and vmPFC groups, respectively; in comparison to $22.80 in the control group).

6.2.2 Self control: DLPFC modulation Another crucial part of normal decision-making involves self-control or what is sometimes called “top-down” or executive control. A classic case of this involves dieting. People not on diets (or not otherwise watching what they eat) will make decisions based on how rewarding an option is thought to be (e.g. taste, texture, quantity, etc.) and other homeostatic factors (e.g. hunger level, etc.). For (successful) dieters, these decisions will also take into consideration information about the healthiness (say) of particular options. Often this can require the exercise of some self-control, as when individuals take an apple instead of a far tastier bit of chocolate cake. A recent neuroimaging study sheds light on just this kind of decision making. Todd Hare and colleagues (2009) used fMRI to examine the neural bases of decisions about food consumption. Participants included a group of a dieters and a group of individuals on no particular diet. They were asked to make a series of ratings about the taste and health of different foods and then asked to make decisions about which food to eat (at the end of the study, one of these decisions was randomly selected and the participant consumed the chosen item). On the basis of these decisions, participants were separated into self-controllers and non-self-controllers.8

8 To be included in the self-control group participants were required to exercise self- control in at least 50% of the decisions requiring it, i.e. declining liked, but unhealthy foods or choosing disliked but healthy foods. In addition, their pattern of health and taste

115 As expected, activity in the vmPFC represented the subjective value of the food items. When successfully exercising self-control, both groups had greater activation in the left DLPFC, and the self-control group had comparatively higher activation, in comparison to decisions in which self-control failed. Furthermore, vmPFC activity and DLPFC activity were inversely related in the decisions involving self-control, suggesting that DLPFC activity dampened vmPFC activity. Representations of the value of food items were modulated by the DLPFC. So, for example, in cases involving a liked but unhealthy option, DLPFC activity was involved in “ramping down” vmPFC activity, adjusting the goal value of the food item in light of health considerations. This was confirmed by functional connectivity analysis, which found that DLPFC was connected to the vmPFC during this task through the inferior frontal gyrus, a brain region activated in tasks involving working memory and goal maintenance. Thus, for those participants who successfully exercised self-control, health information was integrated into reward values of various food items. For those who did not exercise self- control, reward values represented only taste information. Thus, the DLPFC seems to interact with the vmPFC to allow self-control to be exerted in decision-making. The DLPFC also exerts influence over behavior in ways less dependent on conscious effort. It is well known that Pepsi consistently beats Coca-Cola in blind taste tests, but that Coke consistently outsells Pepsi (Gladwell 2005, 155-7). This has been dubbed the ‘Pepsi Paradox’ (Koenigs and Tranel 2008). One of the earliest fMRI papers in neuroeconomics investigated the neural correlates of the Pepsi Paradox (McClure et al. 2004). Behaviorally they replicated the paradoxical results: roughly equal numbers of participants stated that they liked Coke or Pepsi respectively. However, stated preference was not correlated with behavioral preference (Student’s t-test p = .46, two-tailed). Neuroimaging revealed that, in the absence of brand information, i.e. when judgments are based on sensory information, vmPFC activity predicts participants’ judgments. When Coke was paired with brand information (in this case, an image of a Coke can), there was

ratings had to pass two statistical tests indicating a stronger relation between health and decisions than taste and decisions (based on linear regression coefficient and R2 values). Individuals were included in the non-self-control group only if their pattern of responses met none of these conditions. This led to the exclusion of data from 15 participants who met the conditions for neither group.

116 also activity in the DLPFC and bilateral hippocampus, in contrast to blind delivery of Coke. In a similar comparison with Pepsi, brand information elicited no change in activation in comparison to blind delivery.9 This suggests that the DLPFC and hippocampus are associated with cultural influence on preferences. Interestingly, a recent study found that brand information had no effect on individuals with vmPFC damage (Koenigs and Tranel 2008). In a blind taste test, a majority of participants in each of a normal control group, a brain-damaged control group (with lesions to regions other than the vmPFC), and a vmPFC group preferred Pepsi to Coke (55%, 57%, and 63% for each group respectively). But in a condition in which they were given brand information, both the normal and brain-damaged control group preferred labeled Coke more than labeled Pepsi (60% versus 55% for the normal controls and 61% versus 52% in the brain-damaged control group). However, the vmPFC group shared no such contrast: 71% preferred labeled Pepsi, while only 39% preferred labeled Coke. This was significantly different from both control groups. Additionally, vmPFC patients showed more correlation in their preferences between the blind and labeled conditions than control groups. Thus, it seems the vmPFC is necessary to integrate information distinct from taste. This should not be surprising as vmPFC damage does not lead to dietary changes. This is in contrast to other disorders of the frontal and temporal lobes.10 Patients with vmPFC damage appear to make preference judgments on the basis of taste alone. Thus, it appears that the DLPFC is crucial in modulating OFC representations of reward.

9 It should be noted that vmPFC activity was not affected by brand knowledge. However, this finding should be taken with care as, in this and many other early fMRI studies, there is signal loss in the medial OFC region due to interference from the air-filled sinuses nearby. 10 For example, Klüver-Bucy Syndrome, resulting from damage to the amygdala and the adjacent temporal lobe leads to hyperorality and dietary changes (Kluver and Bucy 1939). For example, rhesus monkeys with this syndrome exhibited a tendency to orally examine objects and began eating animal-based foods like bacon, liver sausage and whitefish—foods that normal monkeys had never been observed to eat. Klüver-Bucy produces similar symptoms in humans (A.D. Baird et al. 2007, 1044). Frontotemporal Dementia (FTD) can also lead to dietary changes, including increased and indiscriminate eating (Mendez et al. 2008; Mendez, Licht, and J. Shapira 2008).

117 We’ve seen how the vmPFC functions to represent value and interacts with another emotional center, the amygdala and a more ‘cognitive’ region, the DLPFC. Indeed, interaction between vmPFC/OFC and amygdala and between dorsal PFC regions and emotional centers have been observed in a variety of emotion regulation tasks, some involving the use of conscious strategies (see Ochsner and Gross 2005 for a review). Obviously this is a selective and broad view of the functional organization of these systems, but it sheds light on the basic functions of the vmPFC.

6.2.3 Dissociations We have looked at how these systems function together. It is also important to note that it is possible to observe dissociations between these systems. Self-control is not always exercised successfully. Consider a dieter who claims to be trying to lose weight and who knows the value of his food choices, but persists in his eating habits or a foodie who craves spicy food, despite knowing that it will irritate her stomach ulcer. These are cases in which a cognitive goal is not reflected in one’s behavior, though the cognitive goal may be verbally expressed. In the Pepsi paradox, we have a case in which the value of a reward is being modulated unconsciously. People are not generally aware of the effect of brand information on the value of a cola drink, as reflected in their stated preferences. This appears to be subserved by modulation of the vmPFC by the DLPFC. More bizarre cases are possible. Anthony Dickinson, now a successful experimental psychologist relates a story from his student days (Dickinson and Balleine 2009). On a hot day in Palermo, Dickinson quenched his thirst with a fresh watermelon from a local market—a fruit he had never before eaten. Later that evening he overindulged in red wine, making himself quite sick. A few days later, he was out and found the same market, hoping to again quench his thirst. But this time, the taste of watermelon left him nauseated. As he later learned, what had happened in this case was that his wine-induced illness had generated an aversion, not to the wine, but to the novel taste of the watermelon. However, he was consciously unaware of the aversion until he had attempted to resample watermelon—at which point he became all too aware of it.

118 We can conjecture that the brain mechanisms responsible for the unconscious aversion in this case included the OFC or amygdala. This is unlike the other cases in which higher-level, explicit beliefs or desires fail to properly modulate value representations and affect behavior. In this case, the value representation of the watermelon had not properly interfaced with more ‘cognitive’ representations. Dickinson was now disposed to be disgusted by watermelon, but he would not know it until the novel taste was re-experienced. This highlights an important distinction between memory-based preference judgments and judgments of preferences based on what Fellows and Farah contrastingly describe as “an ongoing, dynamic assessment of relative value” (2007, 2673). I will call this dynamic evaluation. This later kind of judgment depends on evaluation in situ or as a result of simulation or imagination (sometimes called prospection (Gilbert and T.D. Wilson 2007; see also the intoduction to prospection and its problems in Gilbert 2006)). To see this difference consider the following example. When ordering at a restaurant I frequent, I may glance at the menu, recognize the seared ahi tuna as an entrée I have enjoyed recently and order that again. Indeed, I may not even look at the entire menu before deciding. In contrast, I may be in the mood to try something different or at a restaurant I have never been to before. In this case, I may need to engage in dynamic evaluation, considering which entrée I will likely enjoy most. This may involve imagining how the flavors of different options will strike me. Do I think I will enjoy a blue cheese-ale sauce with steak? Would mushroom-liver risotto be better? In the first case I rely on memory to make a decision; in the second case I engage in dynamic evaluation. What may be a bit more surprising is that the second case relies on my capacity for memory as well—though not just memory. Seeing this will take a bit of background. During the second half of the twentieth century, we came to understand that memory is not, in fact, a unitary phenomenon. Much of this understanding has come from the case studies of individuals who have suffered damage or removal of specific brain areas, for example the well-known patient H.M. and others (Corkin 2002; Schmolck et al. 2002; Rosenbaum et al. 2005; Tulving et al. 1988). In 1953, to treat severe epilepsy, H.M. underwent bilateral surgical resection of the medial temporal lobes, including most

119 of both hippocampi (Scoville and Milner 1957). This led to a dramatic dysfunction in memory. Specifically, H.M. acquired anterograde amnesia, the inability to form new declarative memories. Several years later it was discovered that despite H.M.’s amnesia, his ability to learn motor skills remained intact (Corkin 1968). Case studies like this eventually led to the conclusion that there is a distinction between declarative memory, which includes episodic memory (memory of events) and semantic memory (memory of facts), and other nondeclarative memory systems like procedural memory (memory of skills) and classical conditioning and that the neural bases of declarative memory are the hippocampi and medial temporal lobes (Shrager and L.R. Squire 2009). Clive Wearing, a musicologist who developed anterograde amnesia after a case of viral encephalitis, has a memory of less than 30 seconds (Sacks 2007, 188). He can still play piano—though he will claim he has never played a piece until he begins and says he remembers (Sacks 2007, 198). Wearing, like others with this kind of amnesia, has been said to live in a “permanent present” (Barbizet 1970, 33; as quoted in Tulving et al. 1988). Characteristically, he reports his experience as that of waking up; he says: “I’m conscious now…Never saw a human being before…for thirty years…It’s like death!” (Sacks 2007, 197). And, what has been discovered is that the hippocampus memory system, in addition to being necessary for the recall of facts and events (i.e. declarative memory), is also required for simulation, the ability to imagine future or possible outcomes (Schacter, Addis, and Buckner 2009). We are now in a position to see why dynamic evaluation depends on the hippocampal memory system. There are at least two distinct kinds of representations of value. One is remembered preferences. For example, when I order the seared ahi tuna, I depend on this kind of representation. Similarly, I simply remember that I like chocolate ice cream best—I don’t have to think about each flavor every time the Ice Cream Truck comes down my street. These are simply representations in declarative memory, subserved by the hippocampal system. In contrast, dynamic evaluation depends on the representations of value subserved by vmPFC. Dynamic evaluation functions in situ, in response to sensory experience, as in Dickinson’s tasting and re-tasting of watermelon. But it can also be a response to simulation. It is dynamic evaluation in response to simulation that is involved when I am

120 considering which entrée I would enjoy more, steak with blue cheese-ale sauce or liver and mushroom risotto. Thus, dynamic evaluation in response to simulation can be disrupted either by damage to the hippocampal memory system or damage to the vmPFC. In the first case, it interferes with the ability to simulate, though dynamic evaluation of sensory experiences will remain intact. In the second case, the ability to remember facts and events remains, as well as the ability to simulate, what is lacking is any emotional response to these simulations. This is exactly the profile possessed by vmPFC patients. Phineas Gage could learn and remember what had happened to him. E.V.R. can tell you about his meningioma and the events that followed its removal. As Damasio reports, what is particularly striking is that he “was able to recount the tragedy of his life with a detachment that was out of step with the magnitude of the events…He was not inhibiting the expression of internal emotional resonance or hushing inner turmoil. He simply did not have any turmoil to hush” (A.R. Damasio 1994, 44). What vmPFC patients lack is any emotional feeling about their situation. Anterograde amnesiacs simply have no idea. One such individual describes it as follows “It’s like swimming in the middle of a lake. There’s nothing there to hold you up or do anything with” (Tulving 1985, 4). Patients with vmPFC damage have no memory problems, they can remember facts and simulate the future. What they lack is the emotional processing that represents value in the vmPFC. It is precisely these representations that are useful for planning for the future. But such dynamic evaluation is also important in situ, for example during the IGT when people must keep track of changing reward contingencies and reverse their previously learned responses accordingly. Indeed, vmPFC patients exhibit dramatic instances of this dissociation between more ‘cognitive’ representations in declarative memory and more emotional representations in the vmPFC. Toward the end of a session of the IGT, some vmPFC patients are able to cognitively distinguish the good decks from the bad decks—but they continue to draw from the bad decks (Bechara et al. 1996; Bechara et al. 1997)! They are unable to use this declarative knowledge to guide their behavior. Similarly, in an interpersonal task involving a structured conversation with a stranger, vmPFC patients engage in socially inappropriate behavior, despite awareness of the social norms they are

121 violating (Beer et al. 2006). However, when they are shown a videotape, they are able to recognize their behavior as inappropriate. This is one of the most pervasive features of individuals with vmPFC. They fail to act appropriately despite knowing what the right thing to do is. E.V.R. showed normal performance on tasks designed to measure the ability to think of alternative options in a given situation, reflect on the future consequences of actions, engage in means-end reasoning, and foresee the outcome of social situations (Saver and A.R. Damasio 1991). Following his completion of one such task he is reported as saying: “And after all this, I still wouldn’t know what to do!” (A.R. Damasio 1994, 49). This combination of emotional dysfunction and preserved knowledge of social rules makes them a crucial test case for the role of emotion in moral judgment.

6.3 Moral judgment in vmPFC patients The basic dual process model predicts that patients with vmPFC damage should exhibit normal judgments in impersonal dilemmas, because these are not expected to engage emotions. Damage to vmPFC should selectively disrupt personal moral judgments that depend on emotional responses. That is, vmPFC patients should diverge in their responses to dilemmas that elicit strong emotional response in normal subjects, e.g. personal moral dilemmas. If, as claimed by linguisticism, emotions do not play a role in moral judgment then vmPFC patients should exhibit normal patterns of moral judgment. Current research supports the predictions of the dual process theory. However, it also provides a more detailed account of the role of emotion in moral judgment. Koenigs et al. (2007) tested six patients with damage to the vmPFC along with a normal control group and a control group with damage to other brain areas. They found that the judgments of vmPFC patients were comparable with both control groups except on personal moral dilemmas. These personal moral dilemmas involve actions that will cause direct personal harm—likely to engage emotional processing—but will also secure beneficial consequences. Patients with vmPFC damage were more likely than either control group to approve these actions, i.e. to offer the characteristically utilitarian response.

122 These results have been confirmed and extended by three additional studies (Ciaramelli et al. 2007; Moretto et al. 2010; Thomas, Croft, and Tranel 2011). Elisa Ciaramelli and colleagues (2007) found that vmPFC patients endorsed the utilitarian option in personal moral dilemmas more frequently than individuals in the control group. Reaction time data indicate that controls took longer to provide the utilitarian answer in personal moral dilemmas than in impersonal moral dilemmas, while vmPFC patients showed comparable RTs across conditions and responses. However, as discussed in a previous chapter (§3.4.1), some of these dilemmas elicited 100% or near 100% agreement in both the vmPFC patients and the control groups and were labeled low-conflict. Those dilemmas that did not exhibit such unanimity were labeled high-conflict. Interestingly, Koenigs et al. found that the vmPFC patients were significantly more likely to endorse the utilitarian option in high-conflict personal moral dilemmas (Koenigs et al. 2007, 910). This indicates that the emotional processing subserved by the vmPFC is crucial for normal responses in these kinds of moral dilemmas. Giovanna Moretto et al. (2010) also found that vmPFC patients are more likely to endorse the utilitarian option in high-conflict personal moral dilemmas than both a normal control group and a control group with non-frontal brain damage. Judgments on low-conflict personal moral, impersonal moral, and nonmoral dilemmas were comparable across all groups (2010, 1896). They also found comparable RT data. Both control groups had higher RTs when approving utilitarian options in personal, but not impersonal moral dilemmas, while vmPFC patients showed similar RTs for both personal and impersonal moral dilemmas, regardless of their responses. Additionally, Moretto et al. acquired recordings of SCRs throughout the task. While both control groups exhibited larger SCRs during contemplation of personal moral dilemmas prior to making a utilitarian judgment than during contemplation prior to a non-utilitarian judgment, the vmPFC group showed no difference in SCRs between utilitarian and non-utilitarian judgments in these cases. Thus, differences in moral judgment were correlated with abnormal physiological activity. This is similar to the physiological and behavioral results in the IGT, where a lack of anticipatory SCRs was correlated with abnormal performance. This physiological

123 response is correlated with emotional responses that, in normal subjects, contribute to moral judgment. As in the previous chapter (§5.5) ‘normal’ must be qualified. A similar relation between SCRs and judgments was found in the non-brain damaged control group. Moretto et al. found that normal participants who made more utilitarian judgments in personal moral dilemmas generated less SCRs prior to utilitarian judgment whereas participants who made fewer utilitarian judgments showed higher SCRs prior to utilitarian judgment. This pattern of results provides a strong case for the role of emotion in certain moral judgments. Patients with vmPFC damage are impaired on precisely the moral judgments which require what I have called dynamic evaluation. In many cases, however their moral judgments are normal. The reason for this is that these judgments rely on these patients’ intact declarative memory. Just as I can depend on my remembered preferences when ordering at my favorite restaurant, vmPFC patients can rely on moral rules that they have learned and stored in declarative memory. This account is supported by research on patients with Frontotemporal Dementia (FTD). FTD is a neurodegenerative condition that can affect the frontal or temporal lobes or both, and leads to antisocial behavior, problems with control, diminished emotion and loss of insight for their condition (Mendez et al. 2008). Despite this, FTD patients have preserved knowledge of moral rules and so closely resemble individuals with vmPFC damage (Mendez et al. 2005; Mendez 2006). Mendez et al. studied the standard trolley case and the footbridge case in individuals with FTD (2005). They found that a higher percentage of FTD patients judged that they would push the stranger onto the tracks in the footbridge dilemma (57.7%) than either a normal control group (19.2%) or a control group of patients with Alzheimer’s Disease (23.1%). FTD patients’ responses to the standard trolley dilemma were not different than the other two groups: most participants said that they would pull the switch. McKinnon et al. (2007) report that FTD patients sometimes provide justifications of their decisions in terms of “well known social dictums such as ‘I wouldn’t do it, because it is wrong to kill’” (McKinnon, Levine, and Moscovitch 2007,

124 168). Thus, in FTD we see a similar pattern of preserved moral knowledge in declarative memory with impaired decision-making due to deficits in emotional processes. Research on individuals with vmPFC damage reveals that emotion plays a crucial role in certain kinds of moral judgment: those involving dynamic evaluation. These cases are such that no previously learned moral rule straightforwardly applies. They are the high-conflict personal moral dilemmas in which individuals must weigh directly causing harm against beneficial consequences. In normal individuals this process will often involve simulation. For example, when considering whether it would be permissible to kill one of your fellow hostages in order to save yourself and the children in the Ecologists case (§3.4.1), you probably imagined the prospect of doing so in an attempt to weigh it against the costs of not doing so. It is unlikely that you have a rule stored in declarative memory which applies to just such cases. In contrast, it is more likely that you had memorized a rule that applies in the low- conflict (or easy) personal moral dilemmas. In one such case, you are an architect deciding whether to kill your despicable boss by pushing him off a building.11 Most normal adults have learned that, no matter how horrible a person is, it is not alright to kill him without some further justification (e.g. self-defense). The same thing is true of the infanticide case (§3.4.1), another low-conflict personal moral dilemma. Most of individuals in Western cultures have learned that it is not alright to abandon an infant simply because it is unwanted. (This is not the case in all cultures.12) Since a rule learned and stored in declarative memory applies in these cases, it is not necessary to engage dynamic evaluation and so vmPFC patients exhibit normal moral judgment in response to these dilemmas. Thus, the vmPFC appears to be crucial for dynamic evaluation in response to simulation. However, without normal emotional responses (as indicated by their abnormal SCRs) to the thought of causing harm, vmPFC patients’ judgments are driven more by the beneficial consequences and so they exhibit higher rates of characteristically utilitarian judgments than normal individuals with intact emotional responses.

11 Full text in Appendix A. 12 For some discussion, see Prinz (2007, 189-202).

125 6.4 Early onset vmPFC damage There is one more piece of the puzzle that deserves consideration: the role of emotions in moral development. In the previous chapter when discussing psychopathy I emphasized the role of development and particularly the ways in which psychopaths compensate for deficits in emotional learning. Damage to the vmPFC provides further support for the contention that emotion is crucial in moral development. This can be seen by considering the effects of early onset vmPFC damage. Early damage to the vmPFC is much more rare than adult onset damage. There are a handful of cases discussed in recent literature (S.W. Anderson et al. 1999; S.W. Anderson et al. 2000; S.W. Anderson et al. 2006) and some older case studies, for example Ackerly and Benton’s study of an individual with damage suffered at age 4 (1948). The pattern that emerges from these cases is that early onset damage produces similar deficits as discussed in adult onset damage. For example, such patients also perform poorly on the IGT. However, early onset damage is characterized by more severe behavioral problems than that resulting from later damage. These patients more closely resemble psychopaths, their behavior is frequently disruptive and often criminal, and involves lying, stealing, and verbal and physical assault. They engage in behavior that puts them in physical danger as well as financially and sexually risky behaviors and exhibit a lack of future planning. They present with severe emotional deficits, especially with respect to guilt, remorse and empathy. Most notable is that they appear to exhibit a lack of the social knowledge preserved in adult onset patients. Anderson et al. write, “the impairments largely reflect a failure to ever develop specific cognitive and behavioral competencies” (S.W. Anderson et al. 2000, 290; see also S.W. Anderson et al. 2006). The of Ackerly and Benton is less severe than other cases in some respects, but of their subject (35 years old at the time of writing), they note “One is struck with the childish simplicity and superficiality of his lying, stealing and running away and sex experiences” (Ackerly and H.L. Benton 1948, 498). In contrast to the normal performance on moral reasoning tasks of adult onset vmPFC damage, patients with early onset vmPFC damage are reported to be at the

126 ‘preconventional’ state of moral reasoning (S.W. Anderson et al. 1999).13 These results support the contention that the emotional processes subserved by the vmPFC are important during development. They appear to be necessary for the acquisition of social and moral knowledge stored in declarative memory. This corroborates the contention of the discussion of psychopathy, that intact emotional processing is an important part of normal moral development. Interestingly, however, early onset vmPFC patients do not appear to engage more ‘cognitive’ processes in order to compensate for their emotional deficits as psychopaths do. The reason for this is unknown, but it may be related to their lack of insight into their condition. The processes engaged to compensate likely require effort to engage; without an appreciation of their condition, early onset vmPFC patients may have no reason to engage such processes.14

6.5 Psychopathology and emotion in moral judgment: conclusions In this and the previous chapter, I have reviewed evidence from psychopathology. This evidence supports the claim that emotions play an important role both synchronically in some kinds of moral judgment and diachronically during moral development. This undermines claims (9) and (10) of linguisticism (§4.2) about the role of emotion in moral judgment. ‘Normal’ patterns of moral judgment depend on emotion in cases, which have been labeled high-conflict personal moral dilemmas, that require what I have called dynamic evaluation. This is an ability that requires an intact vmPFC. Individuals with vmPFC damage demonstrate abnormal patterns of moral judgment, more frequently choosing the characteristically utilitarian option.

13 Interestingly, Young et al. found that a majority of patients with vmPFC damage still exhibit the side-effect effect (2006). This involves the tendency of people to agree that a foreseen side-effect was brought about intentionally when that side-effect is bad, but disagree when that side-effect is good (Knobe 2003). Suggestively, cross-referencing demographic (Table 1) and experimental data (Table 4) reveals that the two exceptions suffered the earliest onset damage (during the first year and at age five). Perhaps early development is crucial for the mechanisms underlying the side-effect effect. I discuss this further in Chapter 9 (§9.2.1). 14 Alternatively, the rarity of this condition could result in a sample of individuals who all have brain damage more extensive than the vmPFC. Future research will help sort out these issues.

127 Additionally, we’ve seen that emotion plays an important role in moral development. Psychopathy, I’ve argued, is primarily a deficit in emotional learning. Psychopaths can partially compensate for these deficits by engaging in more ‘cognitive’ kinds of processing. However, this compensation does not produce representations of moral rules that are well connected to motivational systems. And so, while exhibiting normal moral judgment, psychopaths’ behavior often does not reflect their (more ‘cognitively’) represented knowledge of moral rules. In vmPFC patients, we saw that, while adult onset damage leaves moral and social knowledge intact, early onset damage disrupts the ability to acquire this knowledge in the first place. Thus, it appears that the emotional processing subserved by the vmPFC is necessary for the normal development of this moral knowledge, even if the vmPFC is not necessary once this knowledge has been acquired. In addition to undermining claims (9) and (10), the evidence reviewed in these chapters reveals three interrelated patterns that bear noting. Firstly, the deficit in emotional learning present in psychopaths and the results of early onset vmPFC damage highlight the importance of development in moral psychology. This suggests that there may be more than one developmental path to normal declarative knowledge of moral rules. Related to this, the second pattern is the existence of interesting dissociations in both pathological and ‘normal’ cases. Psychopaths exhibit knowledge of moral rules, but their behavior often does not conform to these rules, this can be explained by psychopaths’ deficits in emotional learning but normal declarative knowledge (§5.4.1). In vmPFC patients who have suffered damage after normal moral development we see the preservation of more ‘cognitive’ representations of moral and social rules in combination with emotional dysfunction and an inability to use this moral knowledge to guide behavior. In nonpathological individuals, similar dissociations are possible. Anthony Dickinson was not consciously aware of the aversion he had acquired to watermelon until he re-experienced it. Even in normal cases when the exercise of self-control fails, we saw evidence that representations of goal value in the vmPFC do not reflect our explicitly held beliefs and desires about food consumption. These cases help explain how, though

128 normal moral judgment and normal moral behavior often go together, they can come apart in certain circumstances. Specifically, they can come apart when there are ‘cognitive’ representations of moral rules or goals that do not appropriately influence emotional representations of value that are more closely connected to motivation and action. Also related to this, a third pattern involves interesting individual differences. In many cases, these variations blur the boundary between pathological populations and ‘normal’ populations. Here I’ve often put the word ‘normal’ in scare quotes to signify that there are significant individual differences within populations not generally considered pathological, for example male college students. These individual differences represent another important aspect of moral psychology, which any adequate theory will have to address. Each of these three related patterns deserves further consideration. Here, I simply draw attention to them. Later I return to drawing out some of the consequences of these patterns. Most importantly, they bear on some of the other claims of linguisticism as well as some of the philosophical issues to be discussed in Chapters 9 and 10. The most pressing issue brought up by linguisticism involves the role of causal- intentional psychology in moral judgment. I’ve already discussed how this is one area where the basic dual process model needs to be supplemented. Additionally, this issue is complicated by a recent finding: the moral judgments of vmPFC patients are relatively insensitive to the mental states of actors (Young, Bechara, et al. 2010). Individuals with vmPFC damage are more likely than normal people to let someone off the hook for intended harms that are unsuccessfully executed. In the next two chapters I discuss how causal and intentional representations contribute to moral judgment.

129

CHAPTER 7

CAUSAL COGNITION

In the last two chapters, I argued that emotion plays a role in some moral judgments. In particular, emotions are important in judging cases that involve a certain kind of dilemma: those that pit a putative moral violation against beneficial consequences. One example of this kind was the Footbridge case. What is particularly interesting about that case is that pushing the large man in front of a trolley to save five people is judged by a majority of participants to be impermissible and yet, in a similar case—the Trolley case—flipping a switch to divert a train toward one person in order to save five is judged by a majority of participants to be permissible. In Chapter 4 (§4.1), I discussed work by Nichols and Mallon (2006) that suggests that emotions cannot fully explain the asymmetry in judgments between these two cases. First, Nichols and Mallon replicated the asymmetry in a set of cases with little emotional impact, involving tea cups. Second, Nichols and Mallon also found that individuals distinguish prima facie moral violations, i.e. “weak impermissibility,” from “all-in permissibility” (2006, 534). Indeed, in certain cases, a majority of participants judged that, while an action would violate a moral rule, it was, all things considered, permissible. (Some of these were cases in which the cost of not violating the putative moral rule would be catastrophic.) Nichols and Mallon suggested that the representation of moral rules might play a role in explaining the Trolley/Footbridge asymmetry and judgments about weak impermissibility. This idea is developed further by proponents of linguisticism. On this view, people use a moral principle, the Doctrine of Double Effect, according to which harm intended as a means or an end is not permissible whereas it may be permissible to bring about that same harm unintentionally as a foreseen side-effect (Hauser 2006a; Cushman, Young, and Hauser 2006; Hauser et al. 2007; Mikhail 2007). Pushing the man in the

130 Footbridge case involves using him as a means and is thus judged impermissible. Killing one man by diverting the trolley in the Trolley case is a foreseen but unintended side- effect and is judged permissible. I provided some evidence against this interpretation and in favor of an alternative. Waldmann and Dietrich (2007) found that in another case, the Loop case, similar to the Trolley case, but where harm is brought about as a means, participants made judgments similar to those made in response to the Trolley case. This suggested that these results are not due to the use of a moral principle, but rather to the way these cases are mentally represented: by means of causal models. On this view, the asymmetry between the Footbridge and Trolley cases involves intervention myopia: the Trolley case focuses attention on the overall result of action: five people are saved, and one is killed; the Footbridge case focuses attention on the one individual who will be killed if he is pushed in front of the train, but will otherwise be fine. As noted, a crucial issue is whether the representation of a moral rule (e.g. DDE) plays a role in the etiology of these judgments or whether these judgments are merely in accordance with these rules. This issue is related to another of the claims I attributed to linguisticism in Chapter 4 (§4.2):

(6) Moral judgment derives from causal-intentional psychology.

A strong interpretation of this claim—that moral judgment derives from causal- intentional psychology exclusively—is undermined by the arguments of the last two chapters. A weaker interpretation is that causal-intentional psychology is important in moral judgment, but that other processes (e.g. affective processes) also play a role. I think this weaker claim is plausible. The differences between the Footbridge and the Trolley cases involve the causal details. Yet, these cases are judged differently. Obviously the way these differences are represented is important for moral judgment. There is a further issue that bears on the interpretation of (6). There is a more general dispute over the nature of causal cognition (discussed in §7.2.1): whether causal cognition is propositional or whether it depends on lower-level associative mechanisms. The view that people use some formulation of the DDE in making these judgments

131 depends on causal cognition being propositional. On the other hand, if these judgments derive from lower-level associative mechanisms, it is more likely that the etiology of these judgments does not involve the representation of the DDE and that these judgments are merely in accordance with it. To understand how causal cognition contributes to moral judgment, it is necessary to understand causal cognition more generally. That is the project of this chapter. In §7.1, I set the stage by discussing the varieties of causal cognition and reviewing evidence for the existence of two distinct memory systems that can contribute to causal cognition. In §7.2. I review research on causal cognition and argue that the two- system account offers a plausible explanation of this research. In the following chapter, I draw on this account to explain the role of causal cognition in moral judgment.

7.1 What is causal cognition? I take causal cognition1 to involve the acquisition and use of representations of the causal structure of the world. Two notes of clarification are in order. First, causal cognition, so identified, is very general and there are good reasons to believe that it is not a univocal capacity. As I discuss below, there are multiple neuropsychological systems that contribute to causal cognition. Given the importance of such a capacity for getting on in the world, this is not surprising. This is especially so for species like humans that make their living in the world largely through their ability to understand and manipulate their environment. Getting a grasp on this diversity is further complicated by the fact that causal cognition has been investigated from a variety of disciplinary perspectives by researchers using a variety of methodologies and terminology. Research in animal learning, cognitive and developmental psychology, computational modeling, and neuroscience has addressed this topic. These issues are hotly debated and no consensus has yet emerged.2 Some of

1 Though I use the term ‘causal cognition’, one should not take this to mean that I am committed to a cognitivist account of causal cognition. As part of the term ‘causal cognition’, I intend ‘cognition’ simply to indicate this as a kind of information processing and thus in accordance with the use of ‘cognitive’ as discussed earlier (§3.4.1). 2 See, for example, the recent target article by Mitchell et al. (2009) and the associated commentaries.

132 this conflict may be merely apparent, resulting from different researchers focusing on different aspects of causal cognition or different neuropsychological systems. Seemingly conflicting behavioral results may emerge from the interaction of multiple systems. Sorting through these issues is no simple task and the discussion in this chapter will necessarily be more speculative than that offered in the previous two chapters. The second clarification is that reference to “the causal structure of the world” might seem to indicate a commitment to causal realism, the view that causation depends on mind-independent features of reality over and above the existence of regularities. I mean to take no stand on this metaphysical issue. As Michael Strevens points out, “even if the world is Humean, in the sense that every fact can be captured by some purely statistical claim or other, it is a very particular Humean world: Its pattern of correlations is a very particular pattern, with very particular properties” (2007, 259). These particular properties include the asymmetry of causal relations (discussed further in §7.2.3). For example, whether a light bulb is illuminated might depend on the position of the switch to which it is connected. However, I cannot change the position of the switch by breaking the bulb, so that it is no longer illuminated. The relation between the switch and the illumination of the bulb has a certain asymmetry. By “causal structure of the world” I mean to refer to these “particular patterns,” whether or not these patterns are best explained by the metaphysical thesis of causal realism. Thus, causal cognition can be seen as the representation of these particular patterns or regularities and the use of these representations in further processing. Indeed, in recent accounts of causation and causal cognition, Woodward (2003) and Sloman (2005) have emphasized the importance of what they call invariance. Invariance refers to these patterns of regularities in the world. For example, things do not usually happen randomly; objects do not pop into and out of existence. Experience presents a world of objects, with fairly stable properties over time. For humans, the world is not simply populated by objects, it is also populated by agents. Indeed, Susan Carey (2009) has argued that objects and agents are two core domains in which humans have innate mechanisms that facilitate representation. She calls this core cognition. Core cognition has conceptual content that goes beyond perceptual representations. These representations are available to other processes, though their content is iconic

133 rather than semantic. Core cognition relies on what Carey calls “innate perceptual input analyzers,” that pick out entities in the world (e.g. middle-sized objects) and continue to do this throughout the life cycle; they are “domain-specific learning devises” (Carey 2009, 68-9). The hypothesis of core cognition is a kind of compromise between strong empiricist claims that human representations are built entirely from perceptual primitives (e.g. Quine 1960) and strong rationalist claims that all concepts are innate (e.g. Fodor 1975). The innate concepts are limited to domains that have remained fairly stable over evolutionary time and so natural selection offers a plausible explanation of these innate mechanisms. Carey makes a convincing case for core cognition in these domains. Part of the content in these domains involves causal representations. For example, when it comes to objects, there is well-known research on causal (Michotte 1963). Under the right conditions, people see certain kinds of events as involving causation.3 For example, consider two objects involved in a launching event: the first object, moving from the right, contacts the second object. The first object stops and the second object begins moving to the left. If the timing is right, people perceive the first object’s contact with the second as causing the motion of the second object. If the first object stops before it reaches the second, people will not perceive the subsequent motion of the second as being caused by the motion of the first. Only the right spatio-temporal configuration results in the perception of causation. Research indicates that even very young babies (as young as 6 months) perceive causation in these kinds of launching events (Carey 2009, chap. 6).4 There is also evidence that core agent cognition involves representations of causation. The fact that an entity moves on its own or possesses eyes or a face can lead young infants to represent that entity as an agent. Even geometric shapes whose motions seem to exhibit contingent interaction with the environment give rise to intentional

3 Michotte’s original studies depended on the reported perceptions of “observers,” likely Michotte and his friends and students. Subsequent studies have asked participants to judge whether, for example, the first object caused the second object to move. For further discussion, see Scholl and Tremoulet (2000). 4 Studies of causal perception in infants as well as the other results in studies with infants discussed below use looking time paradigms, a common methodology in developmental psychology. For further discussion of this methodology, see Carey (2009, 40-6).

134 interpretations of these movements, as in Heider and Simmel’s famous experiment (1944; see also Scholl and Tremoulet 2000). Thus, core cognition in the domain of objects and agents both involve causal representations. These representations of causation can be used in surprisingly sophisticated inferences. Saxe et al. (2007) found that infants less than a year old expect an agent (e.g. a hand or a puppet that had earlier been seen moving on its own) to be the source of the motion of an object previously observed to be inanimate (e.g. a bean bag). On the basis of research like this, Carey (2009) suggests that the concept of cause may be innate.5 In addition to representations of causation that emerge from core object and agent cognition, there is another important capacity that underlies the ability to extract causal information from statistical associations. There is evidence that this ability is distinct from core agent or object cognition. Schlottmann and Shanks (1992) modified a standard launching event experiment to include an additional element: a covariation between the movement of the second object and a color change of that object. Participants successfully identified the color changes as reliable indicators. They also indicated the perception of causation in single launching events when the spatio-temporal relations were correct, even though this was not a reliable indicator of the movement. This indicates that these two capacities (causal perception and causal inference from statistical patterns) are largely distinct and that the processes of causal perception are relatively encapsulated. This receives support from a study with split brain patients that found a double dissociation between causal perception, which depended on the right hemisphere of the brain, and causal inference based on statistical information, which was dependent upon the functioning of the left hemisphere (Roser et al. 2005). There is similar evidence that core agent cognition is partially distinct from subsequently developing capacities. Core agent cognition involves the capacity to represent things as agents, to represent agents’ goals and to follow the attention of others

5 It is important to distinguish Carey’s use of ‘concept’ from a more philosophical use. For Carey, concepts are mental representations that go beyond spatio-temporal primitives and have a “rich inferential role” (2009, 171). In contrast, in philosophical usage, concepts are the constituents of propositional representations. I use italics to denote concepts in the former sense and continue the convention of using all caps to refer to concepts in the latter sense. See also footnote 7, below.

135 (e.g. in gaze following). Many of these capacities are shared by primates (Tomasello, Call, and B. Hare 2003). However, the ability to represent others as having mental states with propositional content likely comes online much later. For example, it is not until after age three that children are able to pass the false-belief task and there is no evidence that primates share these more sophisticated Theory of Mind (ToM) capacities.6 However, there is evidence that core agent cognition and later-developing ToM capacities are not entirely distinct. Wellman and colleagues found that infant performance on a task requiring attention to intentional actions predicted performance on ToM tasks (e.g. the false-belief task) almost three years later, even when controlling for other factors (e.g. I.Q.) (Wellman et al. 2004; Wellman et al. 2008; see also M. Yamaguchi et al. 2009). One hypothesis is that the causal representations of core cognition function as inputs to other kinds of processing, including ToM. However, much work remains to be done. So we have evidence of causal representations in two domains of core cognition. At least partially distinct from this is a capacity to infer causation from statistical patterns in the world. For example, after observing that an allergic reaction nearly always follows the handling or eating of mangoes, I might come to the conclusion that the mangoes are causing the reaction. Below, I will argue that this capacity depends on associative learning mechanisms subserved by networks involving the basal ganglia, especially the striatum. Of course, my inference in the allergy case may also depend on my previous beliefs, e.g. that mangoes are the kind of thing that can cause allergic reactions. One form that these previous beliefs might take is an understanding of causal mechanisms. There is evidence that we have the ability to grasp causal mechanisms and to make inferences on the basis of these representations (Ahn et al. 1995). For example, my understanding of how a combustion engine works is almost entirely derived from textbook-style diagrams.

6 A standard false-belief task involves having a child watch a puppet place an object in a particular location. The puppet disappears out of sight and the object is moved to another location. Then the puppet returns and the child is asked where the puppet will look for the object. Three year-olds typically say that the puppet will look where the object is, rather than where it was left. They fail to represent the puppet’s false belief. See Wellman et al. (2001) for a meta-analysis of studies utilizing versions of the false-belief task.

136 It depends on rather abstract, functional understanding of parts like the crankshaft and the piston. These representations are not derived from any significant, first-hand experience with internal combustion engines. These kinds of causal representations seem to involve the construction of mental causal models. In line with the discussion of the previous chapter (§6.2.3), there is evidence that these kinds of processes depend on the hippocampus and medial temporal lobe (MTL). As I discuss in the next section (7.1.2), research indicates that these two systems, the basal ganglia system and the MTL system, are often engaged in parallel during learning and compete to influence behavior. Thus, there is a certain amount of redundancy. However, as we will see, these systems differ in important ways. Though the parallel activation and competition of these two systems can make sorting out these differences difficult, especially on the basis of behavioral evidence alone, neuroimaging and animal models are helping to shed light on these systems. Before I proceed to discuss this research, I want to highlight the role causal representations play in categorization and concept learning.

7.1.1 Concepts and causal cognition Ahn et al. (1995) found that people use information about mechanisms in causal judgments—indeed, they seem to prefer such information over information about statistical covariation when asked to make such judgments. In line with this, Ahn et al. (1995, 338) also point out the similarity between different theories of concepts and differences between covariation-based causal cognition and mechanism-based causal cognition. Understanding the importance of this requires a bit of background. There are three main theories about categorization and concept learning.7 Prototype and exemplar theories contend that concepts are, respectively, statistical representations

7 Much of this research is nicely summarized by Edouard Machery (2009). I draw on his account in this and the next paragraph. It is important to note that ‘concept’ in this literature is meant to refer to, as Machery says, “a specific class of bodies of knowledge, assumed to be used by default in the processes underlying higher cognitive competences” (2009, 50). This is distinct from common views of ‘concept’ in philosophy. It is important to note that on this view of concepts, they need not have the kind of semantic content required to constitute propositional attitudes. Thus, Machery argues that there is a mismatch between psychological and philosophical theories of concepts (2009, 32-7).

137 of the properties commonly possessed by members of a category or representations of the typical members of a category. However, they share a commitment to the view that categorization depends on computing the similarity of new instances to these representations (i.e. prototypes or exemplars). Thus, prototype and exemplar theories of concepts are similarity-based theories of concepts. In contrast, the theory theory of concepts holds that concepts are theories or are parts of theories and that concepts are unified by explanatory coherence, rather than similarity (Machery 2009, chap. 4). Of course, these characterizations are necessarily a bit vague because they describe broad approaches to theorizing about concepts, rather than specific accounts. Each approach has accumulated some amount of evidence in support of it. Machery has argued that, in fact, for each lexical term (e.g. ‘dog’), a person may have multiple concepts, each corresponding to a different theory of concepts. On this view, I may have an exemplar- concept capturing the kinds of dogs typical in my experience (skewed toward terriers and retrievers rather than Great Danes or Chinese crested dogs) as well as a theory-concept embodying my knowledge of dogs as a biological species, descended from Canine lupus, co-evolved with humans for between 10,000 and 100,000 years, and so on. Machery’s claim is both radical and controversial. Full evaluation must await further evidence. However, at present, I think a more modest claim is plausible. There is a wealth of evidence that supports the existence of similarity-based concepts and has played an important role in overturning the classical view of concepts (see, for example, Prinz 2002, 52-78; see also G.L. Murphy 2002). There is also increasing evidence that concepts go beyond similarity and can incorporate domain-specific knowledge (e.g. Keil 1989). Thus, I believe the existence of both similarity-based concepts and theory-based concepts is on firmer ground. Ahn et al. (1995) note the connections between (1) similarity-based concepts and covariation-based causal cognition and (2) theory-based concepts and mechanism-based causal cognition. Indeed, I think there is an interesting connection. This is important for our purposes in two ways. First, it is possible that similarity-based concepts and statistical covariation-based causal representations might share overlapping neural bases. The way we learn about categories on the basis of similarities and the way we learn about causal relations on the basis of covariation may depend on the same mechanisms. In the same

138 vein, theory-based concepts and causal models might also share overlapping neural bases. Though some research is taking up this idea (e.g. Waldmann et al. 2009; see also Danks 2007), much more is needed. Second, theory theories of concepts may reveal another way in which causal representations play a role in human cognition. There is accumulating evidence that causal factors play an important role in the development of theory-concepts. Theory theories of concepts emphasize how human categorization depends (increasingly throughout development) on underlying (and sometimes unseen) causal mechanisms. Frank Keil’s classic transformation experiments provide a nice example of how categorization can go beyond similarity (1989). He presented children with stories about various kinds of transformations of animals or artifacts and asked the children whether each item remained the same kind of thing. For example, one story involved a raccoon that was transformed by surgery, dye, the addition of a smell sac and behavioral changes so that it looked, smelled and acted like a skunk. Children as young as 7 years of age (and adults) maintain that the animal is still a raccoon. For more superficial transformations (e.g. costumes), children younger than 5 made similar judgments. In the case of artifacts, for example, a coffee pot that was modified to be a bird feeder, even young children agreed that the object was now a bird feeder. Work like this supports the idea that categorization depends (in part) on domain-specific theoretical considerations. More recently, research on folkbiology has shown that increased experience in a domain predicts that categorization will depend less on similarity and more on underlying causal factors. For example, Bailenson et al. (2002) examined categorization judgments of birds for both Western experts (US birders) and novices (US college students) and among the Itza´ Maya that live in the rainforest of Guatemala. Only the Western novices’ judgments showed the influence of similarity. The categorization judgments of both the Itza´ and the Western experts showed more influence of causal and ecological factors. Similar results were obtained in a category-based induction task with tree experts (Proffitt, Coley, and Medin 2000). Cross-cultural convergence as well as recent developmental evidence (e.g. R.F. Goldberg and Thompson-Schill 2009) suggest that this

139 is a deep-seated tendency in human cognition.8 Indeed, folkbiology shares some characteristics of core cognition. While this is another place where future work is greatly needed, it appears that causal representations play an important role in theory-based concepts. Thus, there is evidence of both similarity-based and theory-based concepts and a suggestive similarity between these kinds of concepts and covariation-based and mechanisms-based causal cognition, respectively. Further support for this comes from the importance of causal factors in theory-based concepts. In the following sections (§7.1.2 and §7.1.3), I review research on learning and memory. This research supports the existence of (at least) two distinct kinds of memory: procedural (or implicit) memory and declarative (or explicit) memory. Though theoretical dispute remains, there is increasing support for the existence of two distinct systems subserving these different kinds of memory: one system that depends upon the basal ganglia and another involving the MTL.9 This research provides support for the existence of these two systems, which I will argue explain covariation-based and mechanism-based causal cognition and may also shed light on similarity-based and theory-based concepts and categorization.

7.1.2 Sketching the two-systems account As noted in §6.2, the basal ganglia, especially areas of the striatum, receive projections from dopaminergic neurons in the midbrain that signal a reward-prediction error. This is exactly the kind of signal necessary for temporary-difference learning, a pervasive kind of learning in the brain.10 This kind of learning is gradual and error-based: learning only occurs when outcomes are contrary to expectations. It is capable of occurring automatically and without conscious awareness. Because of the central role of

8 For further discussion of some of these issues, see Atran and Medin (2008, chap. 4). 9 Part of the remaining resistance to the multi-system view may depend on disagreement about the individuation of ‘systems’. Here, I rely primarily on neural dissociation. For some further discussion on system individuation, see Machery (2009, 124). 10 Within this region, there is a significant amount of complexity. For a discussion of the cortico-striatal pathways in this region, see Seger (2006) and Langen et al. (2011). For a discussion of different learning rules implemented in this area, see Balleine et al. (2009). See also Frank et al. (2009), who develop a neurological model of the functioning within this region.

140 the basal ganglia, I will refer to this system as the basal ganglia system. The neural substrates of this system overlap or are closely connected with the neural substrates of what I referred to in the previous two chapters as emotional learning (§5.4, §6.2) and so this kind of learning is in part affective. This fits with the results of a well-known study that found that depression leads people to be “Sadder but wiser” (Alloy and Abramson 1979; for a recent review of subsequent research, see Allan, Siegel, and Hannah 2007). Alloy and Abramson found that depressed individuals were more accurate than nondepressed individuals on a contingency judgment task. Specifically, nondepressed individuals overestimated the extent to which outcomes depended on their own responses when those outcomes were desirable and underestimated the extent to which outcomes depended on their own responses when those outcomes were undesirable. Thus, the abnormal contingency judgments in depression, an affective disorder whose neurological bases include the medial and orbital PFC, striatum, and amygdala (L. Clark, Chamberlain, and B.J. Sahakian 2009), provides further support for the affective nature of this processing. Similarly, there is an influential perspective on human cognition, commonly traced back to William James, according to which “Thinking is for doing” (Fiske 1992). From this perspective, it would make sense that certain causal representations of the world are not value-neutral attempts to understand the structure of the environment. Rather, they are driven by purposes of organisms and closely connected to motivation. It is plausible that our ability to represent the causal structure of the world is deeply connected to our ability to value parts of it and to use this information to guide behavior, often under computational or temporal constraints. However, one consequence of this is a kind of representation that is comparatively less flexible. It depends more on responses to the environment than isomorphic representations. We can characterize the processing of the basal ganglia as affective, automatic, associative and likely unconscious. It can be seen as Type 1 processing. On the other hand, the processing handled by what I will call the MTL system, appears to be much more like Type 2 processing. It can facilitate rapid learning on the basis of observation alone (i.e. it is not error-based). It produces more flexible representations that can be applied to new situations. Indeed, the MTL may be the neural substrate of hypothesized

141 “Bayesian maps” (Gopnik et al. 2004) or similar kinds of map-like representations. However, it is also more dependent on conscious, controlled processing and is thus more sensitive to distraction.

7.1.3 Evidence from learning and memory research As discussed in the previous chapter (§6.2.3), the view that there are multiple memory systems was prompted by early lesion studies of patients from anterograde amnesia after sustaining damage to the MTL, including the hippocampus. This research revealed that the MTL subserves declarative memory (which includes episodic and semantic memory). However, such individuals retain the ability to learn motor skills (e.g. mirror drawing). They also show the effects of priming (Tulving and Schacter 1990). Indeed, it has been argued that there are a variety of memory systems that are preserved following MTL damage (Knowlton and Foerde 2008). In this section, I review evidence from learning and memory research that supports the two-system account sketched in the previous section. Knowlton et al. (1994) have shown that anterograde amnesiacs are relatively unimpaired in implicit learning tasks that do not depend on the MTL. Similar results have been obtained in studies of degenerative disorders of the basal ganglia like Parkinson’s Disease (PD) and Huntington’s Disease (HD) (M.J. Frank, Seeberger, and O'Reilly 2004; Shohamy et al. 2008; Poldrack and Foerde 2008). The phasic DA signals crucial for learning in the basal ganglia system depend on divergence from baseline firing rates (see §6.2). However, PD and HD do not affect all areas of the basal ganglia equally. Since there are functional differences (with respect to positive or negative feedback-based learning) across these regions, these patients present with a peculiar pattern of symptoms. Unmedicated individuals can learn on the basis of positive feedback, but not negative feedback. However, when patients are medicated with L-DOPA, a DA precursor that elevates baseline firing and facilitates normal functioning in the areas of the basal ganglia affected by the disease, the functioning of areas of the basal ganglia less affected by the disease are disrupted. Thus, medicated patients exhibit the opposite pattern of dysfunction. Thus, evidence from psychopathology provides some support for the existence of these distinct systems.

142 The existence of these two systems is also supported by neuroimaging evidence. Using a probabilistic classification task (PCT),11 Foerde et al. (2006) found that both the MTL system and basal ganglia system were activated. It appears that, at least in some cases, these two systems are activated in parallel and appear to compete to influence behavior. By introducing a secondary task (in this case counting the number of high- pitched tones in a series of alternating high- and low-pitched tones), Foerde et al. were able to disrupt learning in the MTL system, leaving learning in the basal ganglia system intact. Though task performance was unaffected (the functioning of either system appears sufficient for successful performance on the task), follow-up tests revealed that the representations of the MTL system could be applied to new contexts, while those of the basal ganglia system were far less flexible. Other research has revealed similar results, further supporting the characterization of these two systems sketched above (Foerde, Poldrack, and Knowlton 2007; for a review, see Poldrack and Foerde 2008). These results also converge with research from animal models. Packard (2009) reviews a variety of evidence from maze learning in rats, including pharmacological, genetic, and lesion studies. This research indicates that both the basal ganglia system and the MTL system can support successful performance in a plus-shaped maze task. Interestingly, the plus-maze allows researchers to test whether learning is response-based and the representations acquired are egocentric or whether animals engage in place learning and acquire allocentric representations. Using lesions, as well as both enhancing and disruptive pharmacological interventions on either the dorsolateral striatum or the hippocampus during both learning and testing, researchers have found that the hippocampus is necessary for allocentric, place learning, while the dorsolateral striatum is necessary for response learning. This research also found that, in normal rats, both systems appear to engage in learning, though performance appears to be dominated by the place learning. With extended training, however, there is a shift in performance to response learning.

11 This is a standard paradigm for studying implicit learning. One of the most popular versions is the weather prediction task. This involves predicting the weather on the basis of a set of four geometric figures. The figures and the weather are probabilistically related, so success requires integrating information over the course of training.

143 Similar temporal shifts have been found in humans as well (Shohamy et al. 2008). On a probabilistic categorization task, processing early in a series of trials may involve a simple strategy subserved by the MTL. With increased trials, performance approaches optimum as possessing shifts to the slower-learning, error-based basal ganglia system. If the task continues, processing may shift back to the MTL as it implements the learned (near) optimum strategy. Thus, it appears that the basal ganglia and the MTL systems are often engaged in parallel and compete to influence behavior (for further discussion, see reviews by Seger 2006; Poldrack and Foerde 2008; Shohamy et al. 2008). In this section, I’ve reviewed evidence bearing on the existence and properties of these two systems. I believe that both of these systems make important contributions to causal cognition and that we can understand much of the literature on causal cognition by adopting a two-system account of causal cognition. I turn to this in the next section. Indeed, it appears that some of the debate in the literature arises from behavioral effects that result from the interaction of these two systems.

7.2 A two-system account of causal cognition As Hume wrote: “It appears that, in single instances of the operation of bodies, we never can, by our utmost scrutiny, discover any thing but one event following another, without being able to comprehend any force or power by which the cause operates, or any connexion between it and its supposed effect” (Hume 1748, sec. 7.26). Core object cognition may provide us with an experience of causation; however, our capacity for causal cognition goes way beyond our perception of launching events. Humans are able to distinguish causation from spurious correlation, diagnostic from predictive information and we understand the difference between observation and intervention in inference; in short, we go “beyond the information given” (Waldmann, Hagmayer, and Blaisdell 2006). How is it that we are able to do this? I review research on causal cognition and argue that the two-system account offers the best explanation.

7.2.1 Causal cognition: a variety of views As indicated above, causal cognition has been investigated by a variety of different fields and subfields. For example, research in animal learning, cognitive

144 psychology, computational modeling, and neuroscience has all addressed this topic. These fields have given rise to several proposed accounts of causal cognition. One main division among these is between associative accounts and ‘cognitive’ accounts. Traditional associative accounts hold that people learn about causes and effects by utilizing the same mechanisms responsible for associative learning (e.g. Dickinson 1980; Shanks and Dickinson 1987). These accounts have emerged from research on conditioning. In contrast, other approaches have argued that causal cognition is more ‘cognitive’. Some researchers posit that causal representations are propositional and that causal cognition is an inferential process (e.g. De Houwer 2009; C.J. Mitchell, De Houwer, and Lovibond 2009). A related ‘cognitive’ approach holds that causal cognition is subserved by causal models incorporating structural features of causal relationships (e.g. Waldmann and Holyoak 1992; Waldmann, Hagmayer, and Blaisdell 2006). More computational accounts include the Power PC model (Cheng 1997) and the casual Bayes net approach (e.g. Glymour 2003; Gopnik et al. 2004). These various theories divide along several different issues. Related to the split between associative accounts and ‘cognitive’ accounts, are differences in the extent to which causal cognition depends on implicit, automatic processes. Associative accounts tend to emphasize automatic processing, while ‘cognitive’ accounts emphasize controlled processing. Running orthogonal to this is the issue of whether awareness is necessary for learning. Some supporters of associative accounts (e.g. Shanks 2007) argue that awareness is necessary for learning, as do proponents of ‘cognitive’ accounts. Lastly, a general issue involves the extent to which there is discontinuity between human and nonhuman animal causal cognition (e.g. Penn and Povinelli 2007). While the human capacity for causal cognition obviously goes beyond that of nonhuman animals, the extent to which human causal cognition depends on mechanisms homologous to nonhuman animals is disputed. Recently, research across these different fields has begun to be integrated. For example, as I mentioned in the previous chapter, work on animal learning has begun to be integrated with neuroscience. The finding that the midbrain dopamine system signals a reward prediction error is certainly one of the most impressive results of such interaction. Similarly, cognitive psychologists investigating causal models, hypothesized to be a

145 pervasive kind of mental representation, have begun attempting to integrate their work with research modeling causal cognition in terms of causal Bayes nets. Consensus on the nature of causal cognition has yet to emerge. I believe the two- system account of causal cognition can explain quite a bit of the research. I begin by discussing research from animal learning (§7.2.2). I then discuss cognitivist challenges to associative accounts (§7.2.3) and research on cues to causal structure (§7.2.4). I argue that the two-system account of causal cognition can explain these findings (§7.2.5). I then turn to the role of awareness in this kind of processing (§7.2.6).

7.2.2 Animal learning and causal cognition In a 1974 review of the literature on conditioning, William Brewer described the following research:

Experiment 6 in Hunter (1938) consists of producing finger withdrawal conditioning by saying “Lift your finger,” or “Don’t lift your finger.” If Ss [subjects] didn’t make a finger withdrawal response to “Lift your finger,” they were shocked. To insure objectivity the commands were presented by telephone. The Ss conditioned. (Brewer 1974, 14; as quoted in Shanks 2010, 274)

Not psychology’s finest moment. Nevertheless, there is extensive research on animal learning stretching back over one hundred years. This tradition includes early research on conditioning and reinforcement learning by I.P. Pavlov, E.L. Thorndike and B.F. Skinner and involves a variety of related phenomena studied by many different research programs using many different experimental paradigms.12 As we saw earlier (§5.4.1), Pavlovian conditioning involves the acquisition of an autonomic response (UR) to a previously neutral stimuli (CS), via experimentally pairing this stimuli with an affective stimuli (CS). We noted that the amygdala and OFC were especially important for such affective learning.

12 See Castro and Wasserman (2010), for a review of the experimental designs and primary results of research in this tradition.

146 (1911) investigated the effect of reinforcement or punishment on behavior. Thorndike’s work found that behaviors that are rewarded are more likely to be repeated, while actions that lead to punishment are less likely to be repeated. This is known as the Law of Effect. Importantly, Thorndike claimed that this process of learning was unconscious or implicit (Shanks 2010, 274). This research was extended by Skinner’s work on operant conditioning. He found that, for example, by placing a rat in a box with a lever and rewarding the rat when it pressed the lever, he could increase the frequency at which the rat pressed the lever. We briefly discussed this kind of learning in the previous chapter when we discussed reinforcement (§6.2) and reviewed research that the OFC is crucial for this kind of learning. Though these forms of learning can be distinguished, they often go together. If rats are rewarded for lever pressing only after hearing a tone, the tone may give rise to both the autonomic response of Pavlovian conditioning and the behavioral response of operant conditioning. Commonly, research on this kind of learning involves measuring how animals respond to various pairings between cues, e.g. CSs like tones or lights, and outcomes, which are often food rewards. These kinds of learning draw on a network of interconnected brain regions including the amygdala, vmPFC (including the OFC) and the basal ganglia (especially the striatum). In addition to sharing common neural substrates, these forms of learning share similar theoretical principles. These principles involve the conditions that govern the acquisition, extinction, generalization and discrimination of associations (Castro and Wasserman 2010). By experimentally pairing a bell with feeding, Pavlov’s dogs came to associate the bell with the food. Ultimately, the bell alone led to salivation. This is acquisition. If the bell becomes an unreliable predictor of the food, the association becomes weaker an ultimately stops. This is extinction.13 These kinds of learning also exhibit generalization and discrimination. For example, conditioning using a high-pitch tone leads to maximum CRs when tones of that pitch are used. However, similar stimuli, e.g. a low-pitch tone, also evoke the CR, though to a lesser extent. This is called

13 Following extinction, a process called spontaneous recovery is also possible. If the bell and the food are again paired, the associating can be reinstated without the need for relearning.

147 generalization. Similarly, if a high-pitch tone is reliably paired with food while a low- pitch tone is never so paired, a CR will only be produced following the high-pitch tone. This is called discrimination. The most important factors determining whether or not learning occurs are continuity and contingency: are the cue and outcome sufficiently temporally contiguous and does the cue reliably predict the outcome? To condition a rat to press a lever, the rat need not be rewarded every time the lever is pressed, but reward must be more likely directly following a lever press than not. If a cue is sufficiently predictive in comparison to another cue, learning can occur even after a long temporal delay.14 The most interesting phenomenon related to this last point is blocking. Since its discovery by Leon Kamin (1968), blocking has played a key role in debates over the nature of such learning processes (Shanks 2010). There are various kinds of blocking. The most basic form is called forward blocking. Suppose a tone (CS) is paired with an US like a food reward, establishing an association. Later a tone and a light (i.e. a compound CS) are paired with the same US. Crucially, CRs to the light are significantly weaker in these rats than control rats who were conditioned only using the compound CS. Because the US is fully predicted by the tone, when the tone and the light are paired with the US, the light provides no further information about the US. Thus, the association between the light and the US is ‘blocked’. The idea of contingency and continuity were formalized in the Rescoral-Wagner model of learning, a form of temporal-difference learning (Rescorla and Wagner 1972). This theory and its successors have been a mainstay of this literature for a long time.15 Because this research was done on nonhuman animals, it tended to accept the position of

14 Recall the story related in Chapter 6 (§6.2.3) about Anthony Dickinson’s watermelon aversion. Even though the wine was more temporally contiguous with Dickinson’s gastric malaise (and obviously the cause), the learned association involved the novel flavor (watermelon) because it better predicted the sickness. Indeed, at the time watermelon predicted illness with 100% accuracy. Similarly, in 1967, researchers at Florida State University announced that saccharin aversion could be induced in rats with x-rays delivered as much as 12 hours later (J.C. Smith and Roll 1967). In patients undergoing chemotherapy, aversion conditioning of novel flavors has been found even when treatment occurs after several hours (I.L. Bernstein 1985). 15 For a review of the influence of the Rescorla-Wagner model see Miller et al. (1995). For discussion of subsequent models of learning, see Niv and Montague (2009).

148 Thorndike that this kind of associative learning was unconscious and automatic. Additionally, as Shanks (2007) points out, this work fit well with the parallel distributed processing of connectionist models that were also becoming popular around the same time (e.g. McClelland and Rumelhart 1981). Generalization, for example, is a natural consequence of many models which utilize distributed representations (e.g. McClelland and Rumelhart 1985). It is important to point out that, in Brewer’s gloomy review of the conditioning literature, he was not questioning the body of results established by such research. Rather, his review focused on conditioning in humans and offered a cognitivist critique of behaviorist explanations of these results. In his view, such learning was due to “higher mental processes,” and so he concludes that “there is not and never has been any convincing evidence for unconscious, automatic mechanisms in the conditioning of adult human beings” (1974, 27). Indeed, Brewer parsimoniously (if rather implausibly) claims that the cognitivist theory can explain the results in nonhuman animals as well. By the beginning of the 1980s, work utilizing new formal models of learning in nonhuman animals had started to get picked up by researchers working on humans. It was suggested that animal and human learning might both depend on the same kinds of underlying processes (Alloy and Abramson 1979). This led to the discovery of blocking in humans, which supported this suggestion (Dickinson, Shanks, and Evenden 1984; for some retrospective discussion, see Shanks 2007). More relevant to our purposes, it was argued that causal cognition could be explained by these associative models (Shanks and Dickinson 1987; Dickinson and Burke 1996). The similarity between the principles of associative learning (contingency and continuity) and Hume’s account of causation was suggestive. In the Treatise, for example, Hume offers the following definition of ‘cause’: “An object precedent and contiguous to another, and where all the objects resembling the former are plac’d in a like relation of priority and contiguity to those objects, that resemble the latter” (1739, sec. I.iii.14).16

16 See also Hume’s discussion in the Enquiry (1748, sec. 7.29). For further discussion, see Garrett (1997, chap. 5).

149 This surge in interest in human associative learning also led to the discovery of backward blocking (Shanks 1985). Backward blocking is similar to forward blocking but involves conditioning on the compound CS first. A tone and light are paired with an US. Then, the tone alone is paired with the US. Following the pairing of only the tone with the US, CR to the light is ‘blocked’. Backward blocking is an instance of retrospective revaluation, involving changes in associations that can occur even when the CS (in this case, the light) is not present. This posed a problem for traditional associationist models, though subsequent associationist models have been developed to deal with retrospective revaluation (e.g. Dickinson and Burke 1996; Ghirlanda 2005). At the time, however, the nature of backward blocking naturally suggested a more cognitive account. The following kind of reasoning could explain backward blocking: when both the tone and light were present I got a reward, then I got a reward when only the tone was present, so it must be the tone that is responsible for the reward—the light must not be related. This naturally led to the development of more cognitive models. One important consideration that tends to favor cognitive accounts of causal cognition is that causal relations are asymmetrical while associative relations are symmetrical. To use a well-worn example: the height of the flagpole and the angle of the sun cause the length of the shadow. The length of the shadow is not a cause of the sun’s angle or the flagpole’s height. How could associative mechanisms possible capture causal cognition? Considerations like this led Michael Waldmann and Keith Holyoak (1992) to propose that people do not simply form associations based on covariation, but actually form causal models that represent the structure of underlying causal relations. This has led to a host of findings challenging traditional associationist accounts of causal cognition.

7.2.3 Cognitivist accounts of causal cognition In this section, I review evidence that suggests that causal cognition involves more ‘cognitive’ representations. There are roughly two kinds of cognitivist accounts. One kind argues that causal cognition involves causal models that represent structure in addition to correlation. Another kind of account argues that causal representations are propositional and that causal cognition involves inference. Though there are differences

150 between these two kinds of accounts, I discuss them together because they both challenge associative accounts. The most important considerations in favor of these accounts involve the representation of causal structure and strength. One important aspect of causal models is their structure. Consider an example from Waldmann (2000). Suppose we know that the presence of two substances (Substance 1 and Substance 2) in the blood stream are both correlated with Midosis, a (fictional) blood disease. Two simple causal models consistent with these correlational facts are (1) a common cause model: Midosis causes the presence of both substances and (2) a common effect model: both substances cause Midosis. Causal relationships have direction: causes cause effects and not vice versa. These causal models represent these directions. Waldmann tested whether people are sensitive to structure. He gave two groups of participants identical information about correlations. Participants first learn that Substance 1 is perfectly indicative of Midosis. They then learn that Substance 2 is also a perfect indicator of Midosis. However, in one condition, the instructions lead participants to treat Substances 1 and 2 as potential causes of the disease, while in the other condition the instructions lead participants to treat them as potential effects of the disease. Associative models make no distinction between cause and effect and the covariational information is the same in both conditions, so these kinds of models will predict no differences in causal learning or judgment between conditions. Since Substance 1 is perfectly correlated with the disease, associative models predict blocking of Substance 2 in both conditions. On the other hand, if people do form causal models that represent the structural difference between cause and effect, there should be differences across conditions. Specifically, the causal model theory predicts that blocking will occur only in the condition where the substances are treated as potential causes of a common effect. That is exactly what Waldmann found (2000). In the common effect condition, the redundant cue (the presence of Substance 2) received significantly lower ratings of predictiveness in comparison to the initial cue (the presence of Substance 1); in the common cause condition there was no blocking—there was no significant difference between the

151 predictiveness of the cues (see also Waldmann 2001; Waldmann and Holyoak 1992; Waldmann 1996 reviews evidence that people are sensitive to causal structure). Causal models do not just have directional structure. Individual causes can also have varying levels of strength or “causal power” (Cheng 1997). This idea has been integrated into formal models like Cheng’s Power PC model and causal Bayes net models (Glymour 2003; Gopnik et al. 2004). While these models may provide a good normative account, it is unclear to what extent they are descriptively adequate (Penn and Povinelli 2007). Indeed, Cheng claims that humans use a qualitative version of her model, not the quantitative version she presents (1997, 370). Some causal model theorists (e.g. Hagmayer et al. 2007) hold that such causal Bayes nets might underlie causal models. For our purposes, the formalism of Bayes nets is unnecessarily technical. I will continue to discuss these models in an informal way. The quantitative aspect of causes has been used to provide evidence for cognitivist theories. Consider a straightforward example of blocking. It is observed that eating sardines is correlated with an allergic reaction. It is then learned that eating sardines and pasta is also correlated with an allergic reaction. The predicted effect would be that the association between pasta and the allergic reaction would be blocked: the pasta provides no information beyond what is already expected from the association of the sardines and the reaction. However, this prediction is complicated if the outcome has multiple levels of expression. Consider a case where the allergic reaction can have varying levels of severity. If the allergic reaction is at maximal severity, then one cannot be sure that the redundant cue (pasta) is not associated with the reaction. Since the reaction is already at maximum severity, cognitive accounts of blocking would predict that contingency judgments would exhibit a kind of ceiling effect. In fact, such an effect has been demonstrated. De Houwer et al. (2002) found reduced blocking when an outcome is described as maximal in contrast to when it is described as less than maximal. Similar effects on blocking have been found by combining a standard blocking experiment with pre-training that encourages participants to think of the cues’ effects on the outcome as additive (Lovibond et al. 2003; Beckers et al. 2005). Consider the following example of additivity pre-training. It is first demonstrated that sardines cause

152 an allergic reaction, but sardines and pasta together cause a more severe reaction than sardines alone. This encourages participants to view each cue as additive with respect to the outcome. Following the pre-training, participants take part in a standard blocking experiment. They are told that strawberries are associated with an allergic reaction and that bananas and strawberries together are associated with an allergic reaction no more severe than strawberries alone. The additivity pre-training should enhance blocking. The pre-training raises the possibility that the bananas could increase the severity of the allergic reaction caused by strawberries alone. That this does not occur leads to blocking of the second cue. On the other hand, if pre-training does not demonstrate additivity (e.g. sardines and pasta each cause an allergic reaction individually and together they cause an allergic reaction of the same severity), then blocking should be reduced: bananas could still be associated with an allergic reaction. Indeed, Beckers et al. (2005) found that maximality affects blocking when controlling for additivity and additivity affects blocking when controlling for maximality. In addition to these results, there are other findings including higher-order retrospective revaluation (De Houwer and Beckers 2002a; 2002b). This involves an effect on the associative links of a particular cue in cases where the blocking involves other cues. Consider the following example. Participants learn that bananas and strawberries together cause an allergic reaction. They then learn that bananas and avocado together cause an allergic reaction. If strawberries alone are associated with an allergic reaction, we would expect the standard effect of blocking: no association between bananas and the allergic reaction. The second-order blocking effect mirrors the inference that if bananas are not a good predictor of allergic reactions, then avocado must be. So avocado is rated as a good predictor of allergic reactions. On the other hand if strawberries alone do not cause an allergic reaction, then we would not expect the second-order effect on judgments of avocado’s predictive value. That is exactly what de Houwer and Beckers found. Research in this vein is discovering a host of new findings (for a review, see Shanks 2010). Many of these findings seem to fit naturally with cognitive approaches since they invite an interpretation in terms of inferences from propositions. Indeed, in an

153 updating of the view put forth by Brewer, Mitchell et al. (2009) recently argued that human associative learning is propositional and that many of the effects just discussed are due to inferential processes. However, associative accounts have been modified to deal with many of these phenomena. For example, Melchers et al. (2004) argue that second-order blocking can be captured by a modified associative account. Similarly, a study by Livesey and Boakes (2004) provided evidence supporting a modified associative account of additivity pre- training. They first replicated the finding that additivity pre-training influenced blocking (experiment 1). However, rather than accepting that such results are due to the propositional nature of causal representations, they hypothesized that this effect was due to the fact that additivity pre-training encourages participants to treat cues as elemental rather than configural. That is, additivity encourages participants to represent compound cues (e.g. bananas and strawberries) as made up of two distinct elements. In contrast, configural representations of a compound cue would treat it as a single, novel element. They tested this using different graphic representations that encouraged either elemental or configural representations. In the experiment utilizing graphic representations conducive to elemental representation (experiment 2), blocking was observed to an extent consistent with their results utilizing additivity pre-training. In the experiment using graphic representations encouraging configurational representations (experiment 3), blocking was not observed. Thus, it appears that associative accounts can accommodate much of the research on blocking in associative learning. To this extent, I claim that the associative mechanisms of the basal ganglia system can explain these results. However, as I noted above, some judgments can be informed by prior knowledge, e.g. the fact that foods are the kinds of things that cause allergic reactions. Indeed, the reader might have noticed that in the discussion of maximality and additivity I described examples involving foods and allergic reactions. While these kinds of cases can be presented in a noncausal way, e.g. eating bananas is associated with having an allergic reaction, these cases naturally lend themselves to a causal interpretation in which the bananas cause the allergic reaction. Other interpretations are not intuitively plausible: it is unlikely that, for example, eating bananas and an allergic reaction each have a common cause or that an

154 allergic reaction could cause one to eat bananas. In the next section, I discuss additional research on causal cognition that seems to go beyond statistical covariation.

7.2.4 Cues to causal structure There is quite a bit of evidence that people utilize cues or pre-theoretic assumptions to infer causal structure (Lagnado et al. 2007). Lagnado et al. review a range of research indicating that people use the following kinds of cues to construct causal models: statistical relations, temporal order, intervention, and prior knowledge (2007, 157). This research indicates that people use temporal order, intervention or prior knowledge to generate a plausible causal model and then test this against statistical information. Temporal order refers to the fact that causes precede effects. So, for example, if you are a doctor and you learn that your patient ate strawberries and a short time later had an allergic reaction, you can rule out that the allergic reaction caused a craving for strawberries (which subsequently led to the eating)—though you cannot necessarily rule out the common cause explanation according to which some third event caused both a craving for strawberries and the allergic reaction. Similarly, temporal delay can affect the causal representations formed. If I eat strawberries and have an allergic reaction two days later, it is unlikely that the strawberries were the cause of the reaction, even if I haven’t eaten in the meantime. Intervention apparently provides a distinct cue to causal structure (Lagnado et al. 2007, 163). Indeed, intervention is what gives controlled experiments their epistemic power. Thus, it is not surprising that allowing people to intervene in a causal system can help them discover the causal structure. A doctor’s ability to give her patient strawberries and watch for the presence of an allergic reaction can help uncover the underlying causal structure. One of the most important cues to causal structure is prior knowledge. As noted, when we interpret the food consumption as a (potential) cause of the allergic reaction, this is based on our prior knowledge. Similarly, we know that infection causes fever and not vice versa. Our prior knowledge contributes to how we interpret correlational information. The evidence seems to indicate that people use prior knowledge (or

155 information from task instructions in experimental settings) to construct a causal model and then go on to test it against subsequent information, revising it if necessary (Lagnado et al. 2007, 165). Whereas traditional associative accounts predict symmetrical judgments regardless of whether cues or outcomes are described as causes and effects, alternative ‘cognitive’ approaches emphasize the prediction that judgments should exhibit asymmetry: blocking should occur when compound cues are described as potential causes, but not when they are described as potential effects. I’ve just reviewed evidence that supports the ‘cognitive’ causal model approach and indicates that people are sensitive to information about causal structure. Of course, there is further evidence of this asymmetry (e.g. Cobos, López, and Luque 2007; for further discussion see Shanks 2007; C.J. Mitchell, De Houwer, and Lovibond 2009). However, there are also cases in which the predicted asymmetry is not observed (e.g. Cobos et al. 2002; for further discussion see Shanks 2007). Thus, causal cues appear to exert selective influence on causal cognition. One key result in resolving this tension involves a study that found that adding one sentence to the experimental instructions could affect whether or not the asymmetry was observed (López, Cobos, and Caño 2005). López et al.’s experiments used a scenario involving an “electrical box” (2005, 1390). On one side are three switches with lights of different color above them. On the other side are two bulbs that light up as a result of the pattern of illumination of the colored lights, although the bulbs are never lit up together. Participants are asked to learn the causal relation between the colored lights and the bulbs. In the first set of studies (experiments 1A and 1B), participants received an in- depth questionnaire about the causal structure of the device (i.e. how the lights and switches were connected), either before or after participants’ judgments. The asymmetry was observed only in the condition in which the questionnaire was completed before the judgments. In a second study (experiment 2), participants in two conditions received instructions that were identical except that one set had an additional sentence that emphasized the relevance of the causal information. Only in the condition in which the

156 instructions included the additional sentence emphasizing the relevance of the causal information did the participants’ judgments exhibit asymmetry. These results suggest that the extent to which causal cues influence people’s judgments might depend on the extent to which they are aware of these cues. This indicates that cues to causal structure might depend in part on the MTL system. On the other hand, there is evidence that humans are capable of both model-free and model- based associative learning (Dayan and Niv 2008). Both varieties of associative learning depend on similar neural substrates, including areas of the striatum. Thus, almost all of the results discussed above may depend on the functioning of the basal ganglia system.

7.2.5 The two-system explanation Thus, there is much research, especially from the literature on animal learning, that fits comfortably with associative accounts of causal cognition. At the same time the influence of cues to causal structure on participants’ judgments may indicate a role for the MTL system. I think that the two-system account sketched above (§7.2) can explain these results. The basal ganglia system fits comfortably with the associative models of causal cognition. This is supported by recent neuroimaging studies correlating reward prediction error signals with causal cognition (D.C. Turner et al. 2004; Corlett et al. 2004; Tobler et al. 2006; Fenker et al. 2010; see J. Fugelsang and K.N. Dunbar 2009 for a review of neuroimaging research on causal cognition). Corlett et al. (2004) found right PFC and ventral striatum activation during retrospective revaluation. As the authors argue, this is exactly what associative accounts of causal cognition would predict. Similarly, Fenker et al. (2010) used a task involving causally and noncausally associated word pairs (e.g. ‘virus’ and ‘epidemic’ versus ‘ring’ and ‘emerald’) in which participants judged the relationship (causal or noncausal associative) between these pairs. They found that there was some overlap in judging causally and noncausally associated word pairs, but that causally related word pairs correlated with higher levels of activation in the OFC, amygdala, striatum, and SN/VTA. The MTL system is best suited to process cues to causal structure, since it is involved in the processing of more map-like (i.e. allocentric) representations. As

157 indicated above (§7.1.2) the basal ganglia and MTL systems seem to subserve Type 1 and Type 2 processing, respectively. The fact that the influence of causal cues on people’s judgments seems to depend on their appreciation of the significance of this information (López, Cobos, and Caño 2005) suggests that the MTL plays a role in mediating these effects. When such cues are not available or go unnoticed, judgments seem to derive from the basal ganglia system. Two other studies offer further support. In a standard blocking experiment, De Houwer and Beckers (2003) found that a secondary task did not disrupt blocking. Only when participants performed a highly demanding secondary task was blocking affected. This supports the view that blocking results from Type 1 processing of the basal ganglia system, which is not generally disrupted by cognitive load. Another suggestive result involves second-order conditioning (Karazinov and Boakes 2007). Second-order conditioning occurs when a cue becomes positively associated with an outcome simply by being paired with a reliable predictor of that outcome. Associative accounts uniquely predict the existence of second-order conditioning and in fact it was found when participants were forced to answer within 3 seconds. Thus, time constraints induced results predicted by associative accounts, supporting the view that these effects are due to Type 1 processing subserved by the basal ganglia system. Of course, there are further details to be worked out. Some of the updated associationist models posit particular kinds of representations (e.g. elemental versus configural). These kinds of representations might be due to structured input to these learning mechanisms (e.g. from core object cognition), systematic biases in the learning mechanisms, or the influence of the MTL system. Sorting through these possibilities requires much more work. Complicating this issues is the fact that these systems may often be engaged in parallel. This makes sorting out the contributions of each system on any particular task quite difficult. Nevertheless, I think there is a good case to be made that both of these systems contribute to causal cognition. Lastly, I think the two-system account can offer some explanation of the research on causal cognition in nonhuman animals. A recent review of the comparative literature concludes that neither traditional associationist accounts nor ‘cognitive’ or propositional

158 alternatives adequately explain the research (Penn and Povinelli 2007, 97). There is certainly evidence that nonhuman animals share some of the same capacities for causal cognition as humans. For example, a recent study found the existence of blocking—a tell- tale sign of error-based learning—in human categorization of images of natural objects (Soto and Wasserman 2010a). This mirrored results in pigeons using nearly identical stimuli and procedures (Soto and Wasserman 2010b). Thus, it appears that capacities subserved by the basal ganglia system are shared across many species. We also saw evidence that rats possess both egocentric and allocentric spatial representations, subserved by the basal ganglia and MTL systems, respectively. A recent study has found evidence of sophisticated causal cognition in rats. Blaisdell et al. (2006) found results that they argue provide evidence that rats employ causal models. This cuts against views according to which causal cognition in nonhuman animals is exclusively associative. While the interpretation of these results is highly contested (see Penn and Povinelli 2007, 105-6), the two-system account can make sense of these capacities in nonhuman animals, since the MTL system, which I have argued contributes to causal cognition in humans, may also contribute to more limited forms of causal cognition in nonhuman animals. At the same time, the two-system account makes sense of the fact that human causal cognition goes far beyond what nonhuman animals can do. Indeed, several (non-mutually exclusive) possibilities jump to mind: humans may have undergone significant development of the MTL system; they may have increased capacities due to language and cultural learning; or core cognition may provide more structured input to learning mechanisms. Needless to say, further work is needed to sort through these possibilities. However, the two-system account is both consistent with these results and can help generate hypotheses for future testing.

7.2.6 Awareness This brings us to one final issue: the role of awareness in causal learning and cognition. As noted in the previous chapter (§6.1), there is significant dispute about the role of awareness in learning. Some associative theorists argue that awareness is necessary for learning (e.g. Shanks 2007). This was also the view of Brewer, who argued that such learning was neither implicit nor unconscious, and instead depended on “higher

159 mental processes” (1974, 27). On the other hand, Thorndike and others have claimed that there are some forms of ‘implicit learning’ that do not require awareness. I think the two-system account offers a satisfying resolution to this dispute. In line with the idea that the basal ganglia system subserves Type 1 processes, I think that this system is capable of learning without awareness. However, the MTL system is more dependent on controlled processes, indicating that awareness may be necessary for the functioning of this system. Yet, as we saw, these two systems are often engaged in parallel. Thus, a learning process dependent on awareness will often be engaged at the same time that a learning process capable of functioning without awareness is. This makes it difficult to find evidence of learning without awareness, but it does not show that awareness is necessary for associative learning. Indeed, we should expect to find clear evidence of learning without awareness only in particular cases. In fact, this is exactly the pattern of results on learning and awareness we find. There has long been evidence that awareness is sufficient for learning, even in Pavlovian conditioning. Cook and Harris (1937) demonstrated that, in humans, SCRs can be generated following verbal instruction that a tone will be followed by a shock. This mirrors the SCRs generated after conditioning where the tone is actually paired with shock. Is there evidence that some forms of learning can take place without awareness? Several recent reviews have concluded that the evidence for learning without awareness is slim or nonexistent. Lovibond and Shanks (2002) argued that there is no solid evidence to contradict Brewer’s assessment that awareness is necessary for Pavlovian conditioning. Similarly, Shanks (2005) reviewed evidence for the existence of ‘implicit learning’ and found it lacking. A more recent review of awareness in learning largely concurs with the conclusion of the earlier reviews, though it highlights some results that could challenge this conclusion (Shanks 2010). I will discuss some of this research and explain how the two-system account can explain it. One prominent argument against the existence of learning without awareness is that measures of awareness used in studies have been inadequate to reach the conclusion that learning occurs without awareness. Consider, for example, the ‘Greenspoon effect’ (discussed in Shanks 2005). Greenspoon (1955) asked participants to spontaneously generate words. An experimenter reinforced plural nouns by saying “mmm-hmm.” This

160 resulted in increased numbers of plural nouns and yet, participants were unable to report on the reinforcement. However, subsequent research by Dulany (1961) found that participants in similar experiments responded to the verbal reinforcement by providing subsequent words in the same category, e.g. vegetables, and that they were able to report this. Obviously this is consistent with the results of Greenspoon’s study, since participants who produced more words of the same category after reinforcement would produce more plural nouns if that was what the experimenter reinforced. It does not, however, establish the reality of implicit learning. Greenspoon’s interpretation of his own results was due to his use of an inappropriate measure of awareness. A similar study in which participants were told that they would be playing the role of experimenter in a conditioning study, but in which the ‘participant’ was actually a confederate of the experimenter, did result in effects on the actual participants’ behavior without any reported awareness by the participants (Rosenfeld and Baer 1969). In fact, the same result was found when the confederate was replaced by tape recorded responses played over an intercom to the participant, who thought he was the experimenter talking to the ‘participant’ (Rosenfeld and Baer 1970). One suggestion is that the nature of the manipulation distracted participants from the verbal conditioning and resulted in their failure of awareness. Another interesting result involved conditioning of colored and flavored drinks (Dickinson and Brown 2007). Participants were presented with drinks that were differently flavored and differently colored and functioned as CSs. The drinks were either made pleasant by adding sugar, or unpleasant by adding Tween 20, a bitter-tasting chemical (polysorbate 20). Dickinson and Brown found that participants exhibited conditioning for both color and flavor, even though they were only aware of the color contingencies and could not recall the flavors that were paired with the Tween 20. Wardle et al. (2007) challenged these results. Rather than simply asking participants about flavor contingency, Wardle et al. had them re-taste flavors and judge how confident they were that the flavor had been paired with pleasant or unpleasant taste. The effect of unaware conditioning in the case of flavors observed by Dickinson and Brown disappeared. Subsequent analysis of their own data and reanalysis of the data of

161 Dickinson and Brown (2007) found conditioning only in the participants that were aware of the contingencies. Boakes has suggested that such results could be due to people’s difficulty in describing and identifying odors and flavors (Boakes 2009; see also Stevenson and Boakes 2004). In a similar study, Chambers et al. (2007) found evidence of conditioning without awareness using a novel caffeinated drink in participants who were identified as moderate caffeine users (consuming between 190 and 650 mg a day) and were deprived of caffeine. Caffeine deprived participants significantly preferred the taste of the caffeinated drink to the placebo. During debriefing, 3 (of 32) participants identified the purpose of the study, but only 1 of these correctly identified the placebo and caffeinated drinks. When informed of the purpose of the study, only 37% of subjects correctly identified the placebo and the caffeinated drinks. Excluding these participants produced the same pattern of results. This provides evidence that conditioning can occur without awareness. Unlike the color and flavor conditioning experiments just discussed, participants would not be expected to know which drinks were caffeinated, though in their caffeine deprived state, their judgments were influenced by the caffeination. Shanks (2010) suggests that the Perruchet effect is currently the clearest evidence in favor of implicit learning. Though these results are not definitive, the phenomenon is worth mentioning. Perruchet (1985; see also Perruchet, Cleeremans, and Destrebecqz 2006) used a conditioning paradigm in which the CS was reinforced 50% of the time. Following this training, participants were presented with a series of stimuli consisting of CS alone or CS paired with the US. Predictably, CS alone led to a decrease in the likelihood of a CR (in this case an eye blink) (consistent with extinction), while series of CS-US pairs led to the increase of CRs. Surprisingly, though, participants’ expectation of the US shifted in the opposite direction! Participants’ judgments of expectation demonstrated a Gambler’s fallacy-like effect—the more unreinforced CS were presented, the more they expected a reinforced CS. The fact that conditioned responses show a trend completely opposite to expectations is solid evidence of conditioning without awareness (for further discussion, see Shanks 2010, 289-90). Two recent studies utilizing techniques that render stimuli (consciously) imperceptible (masking and continuous flash suppression) appear to demonstrate learning

162 without awareness (Pessiglione et al. 2008; Seitz, D. Kim, and Watanabe 2009; see also Willingham, Salidis, and Gabrieli 2002 for an ingeniously designed fMRI experiment finding evidence of sequence learning without awareness). For example, Seitz et al. found that water delivery to participants who had been food and water deprived resulted in conditioned responses to a stimulus that was not consciously perceived. A similar study used “backward masking” of emotional stimuli (angry faces) and found evidence of learning in the form of SCRs and amygdala activity without conscious awareness of the stimuli (J. S. Morris, Ohman, and R.J. Dolan 1998). I think this pattern of results is best explained by the two-system account. The basal ganglia system functions automatically and without awareness, but the MTL system is activated in parallel. On this view, we would expect to observe that awareness and learning go together, even though the basal ganglia system is capable of learning in the absence of awareness. We would certainly not expect a dissociation between these two systems in run-of-the-mill cases like the following story related by B.F. Skinner. In an attempt to determine whether back rubbing could be used as a reinforcer, utilizing his three- or four-year-old daughter as test subject, Skinner reports: I waited until she lifted her foot slightly and then rubbed briefly. Almost immediately she lifted her foot again, and again I rubbed. Then she laughed. “What are you laughing at?” I said. “Every time I raise my foot you rub my back!” (Skinner 1982, 6; quoted in Greenwood 2009) How could one not notice this? Still, while difficult, on the two-system view it should still be possible to find evidence of learning without awareness. The evidence just reviewed is easily explained by the two-system view. Consider the results that most support the existence of learning without awareness. In the Rosenfeld and Baer (1969) study, participants’ attention is distracted by another task— trying to condition the confederate. As we saw earlier, a secondary task can interfere with the learning of the MTL system, resulting in learning in the basal ganglia system without awareness. The Dickinson and Brown (2007) study involved sensory modalities (like flavor and odor) for which participants’ ability to generate explicit descriptions is limited, thus minimizing the learning of the MTL system. Interestingly, the Wardle et al. (2007) study seemed to indicate that re-experiencing the flavors could prompt recall, suggesting

163 that the difficulty is in performance (e.g. connecting experienced flavors with flavor names during the testing phase) and that even in this case, the MTL system is engaged in learning. Several of the other studies utilized stimuli that were not consciously perceived, and thus provide the best evidence for a dissociation between the basal ganglia system and the MTL system (Pessiglione et al. 2008; Seitz, D. Kim, and Watanabe 2009; J. S. Morris, Ohman, and R.J. Dolan 1998; Chambers, Mobini, and Yeomans 2007). The Perruchet effect might hold out the best hope of supporting the two-system account since in this case, it appears to demonstrate the dissociation of the basal ganglia system and the MTL system. The basal ganglia system’s unconscious, error-based mechanisms adjust to the changing contingencies, while the MTL system drives participants judgments of expectation. Of course, demonstrating this effect likely requires precise experimental conditions, which could explain why replicating these results has been tricky. This is consistent with the work of Nisbett and Wilson (1977) and others, discussed earlier (§3.2 and §3.3.2), demonstrating the pervasive influence of unconscious processes on behavior (T.D. Wilson 2002; Bargh 2006). As noted, Nisbett and Wilson’s idea is that people do not have introspective access to certain processes, only to the output of such processes. People’s judgments that appear to be based on knowledge of these processes often involve rationalization based on ‘a priori’ theories. Thus, while we may often be right about the causes of our behavior, it is not because we have privileged access to the contents of our own minds, but because we have reliable theories about the kinds of things that are likely to serve as input to processes that drive behavior. Similarly, it makes sense of the fact that attempts to manipulate judgments can be undermined by being too obvious. Recall the failure of Schnall et al. (2008) to manipulate participants using the gooey creamed corn, collard greens and chocolate pudding mixture (§3.3.2). From the perspective of learning, we have seen that the basal ganglia system and the MTL system are often engaged in parallel. Thus, we will rarely clear evidence of learning without awareness. This fact itself may contribute to our ability to offer plausible explanations of our behavior, even when it derives from unconscious processes. Still, there may be cases where learning takes place in the basal ganglia system, but not in the

164 MTL system. In these cases, people’s explanations may come apart from the causes of their behavior. This is even more likely when we are not focused on monitoring our behavior.17 The two-system account also sheds light on an issue left unresolved in the previous chapter (§6.1) regarding the claim of Maia and McClelland (2004) that the Iowa Gambling Task does not involve learning without awareness. These experiments seem to reveal the same temporal shifts observed in experiments utilizing PCTs (§7.1.3). The basal ganglia system drives behavior as people utilize advantageous strategies, even while they remain unaware of what the advantageous strategy is. Toward the end of a series of trials this knowledge becomes explicit as performance shifts toward the MTL system. Since both the basal ganglia system and the MTL system are engaged in parallel, some ways of questioning participants may prompt them to become explicitly aware of the contingencies (e.g. that a certain deck tends to have high rewards, but even higher losses). This interpretation is consistent with a recent neuroimaging study on the IGT (N.S. Lawrence et al. 2009).

7.3 Conclusion In this chapter, I reviewed a wide variety of research and argued that it supported the existence of two distinct systems, both of which can contribute to causal cognition. These different systems have distinct properties. The MTL system involves more decoupled representations, which are (at least in part) dependent upon conscious awareness. On the other hand, the basal ganglia system involves affective, similarity- based processing and is capable of operating unconsciously. In the next chapter, I use this two-system account to shed light on the contributions of causal cognition to moral judgment.

17 Additionally, there may be cases where both systems are engaged but what is learned is different. This could provide one way of sorting out the role of these two systems in learning.

165

CHAPTER 8

CAUSAL COGNITION IN MORAL JUDGMENT

In the previous chapter I discussed several different kinds of processes that contribute to causal cognition. I focused especially on two systems. One system depends on the basal ganglia and involves automatic, unconscious, gradual, error-based learning. The other system involves controlled, conscious processing and depends on the medial temporal lobe (MTL). The representations subserved by this system were more decoupled, sometimes involving propositional content. I also discussed core cognition in two domains relevant to causal cognition: objects and agents. We are now in a position to see how causal cognition contributes to moral judgment. In fact, I think all of these processes can potentially contribute to moral judgment. I begin by offering an explanation of the asymmetry in judgments between the Trolley and Footbridge cases. The most developed alternative account of these results is put forward by proponents of linguisticism. So, in offering my own account of these results, I criticize this alternative and offer a critique of linguisticism more generally.

8.1 Against linguisticism There are three important issues to consider in understanding the contribution of causal cognition to moral judgment in any particular case: (1) the nature and content of the processing involved, (2) the source of the content of this processing, (3) the role of awareness in this processing. I begin by sketching how my own account and the linguisticist alternative address these three issues with respect to the Trolley/Footbridge asymmetry. As discussed at the beginning of Chapter 7 (see also §4.1), proponents of linguisticism explain this asymmetry by reference to people’s use of the Doctrine of Double Effect (DDE). Since, on this view, the representation of this moral rule figures in the etiology of these judgments, linguisticism seems committed to a propositional

166 account of causal cognition. However, as proponents of linguisticism point out, it is difficult to see how people could have learned such a rule, given that few people (mostly people in ethics classes) receive any instruction about rules like the DDE. Yet, most people’s judgments are in accordance with this rule. Thus, because of this poverty of the stimulus (discussed further below, §8.1.2), linguisticism maintains that the source of this content must be an innate moral grammar (Dwyer 1999; Mikhail 2008). Proponents of linguisticism are divided when it comes to the role of awareness. For example, Cushman et al. write “Some moral principles are available to conscious reflection—permitting but not guaranteeing a role for conscious reasoning—whereas others are better characterized by an intuitionist model” (2006, 1088). At the same time, Hauser (2006b) and Mikhail (2007) seem to suggest that this processing is unconscious. Hauser, for example, writes: “I suggest the possibility that our moral judgments are derived from unconscious, intuitive processes that operate over the casual-intentional structure of actions and their consequences” (2006b, 214). So proponents of linguisticism understand causal cognition to involve propositional, innate content, which may or may not be unconscious. In contrast, the account I have developed makes room both for low-level associative processes (subserved by the basal ganglia system) and propositional representations (subserved by the MTL system). Both of these systems are, in essence, learning mechanisms and, as such, this account rejects strong nativist claims about the content of these processes. And, as I argued in the previous chapter, the basal ganglia system is capable of functioning in the absence of awareness, while the MTL system depends more on conscious awareness. So the account I favor allows that both similarity- based associative processing and explicit, propositional representations contribute to moral judgment and thus allows that both conscious and unconscious processes could contribute to causal cognition in moral judgment. It also rejects strong nativist claims (though it allows for some innate content in core cognition, see §7.1).

8.1.1 A preliminary consideration against linguisticism At this point, the account I have sketched has an advantage over the competing linguisticist account. My account is far more flexible. The research I have been reviewing

167 throughout this dissertation has pointed to a significant amount of complexity and diversity in moral cognition. My account has the resources to deal with the range of phenomena we actually see. Consider the findings introduced in Chapter 4, which posed a problem for the basic dual process theory. They can be summarized as follows. First, we saw an asymmetry in judgments between the Trolley and Footbridge cases. Second, we saw that people distinguish between actions that violate a moral rule (‘weak impermissibility’) and those that are judged all-in permissible. This pattern of judgments was observed in both the Catastrophe case and the Tea Cups cases (Nichols and Mallon 2006). Third, we saw that few people refer to the DDE in order to justify pairs of judgments in accordance with it. We also saw that two other principles (the act/omission principle and the contact principle) are accessible for justification to varying degrees. It is my claim that linguisticism does not have the resources to account for these results. In part, this explains why proponents of linguisticism are divided on issues like the role of awareness in moral cognition. When asked to explain pairs of judgments, people seem to have different degrees of awareness of different principles. People were far more likely to cite the act/omission principle when asked to justify a pair of judgments they made in accordance with it, than to cite the DDE when asked to justify a pair of judgments exhibiting the Trolley/Footbridge asymmetry. Because the basal ganglia system can function unconsciously, while the MTL system depends more on consciousness, my account has the resources to explain these results. The view that multiple processes contribute to moral judgment makes room for multiple, (roughly) simultaneous moral assessments. We saw this in the study by Nichols and Mallon when people distinguish between ‘weak impermissibility’ and ‘all-in permissibility’. At the same time, it should be pointed out that while it is important to explain the Trolley/Footbridge asymmetry, it is also important not to lose sight of the fact that a substantial minority does not share these judgments. It is not immediately apparent how the moral grammar hypothesis can explain why some people’s judgments do not exhibit this asymmetry. In short, because it posits the existence of two systems which are often engaged in parallel and may interact to varying degrees in different cases and across different

168 individuals, my account has far more resources for explaining the range of phenomena in moral psychology. Of course, at this point the argument is fairly weak. To strengthen it, we would like to see how exactly this account explains particular results, not merely that it has the resources to do so. I offer a more detailed explanation in the following section (§8.1.3). It might be argued that this explanatory flexibility has been purchased too cheaply. Ordinarily, one might worry that the positing of two systems is ad hoc, intended simply to account for otherwise puzzling results. By adding enough parameters, any model can perfectly predict any possible set of . Yet the proliferation of parameters in such a case is clearly ad hoc. There may be reasons to engage in this exercise of curve fitting, but this would not provide evidence for the n-parameter model. It may be claimed that something similar is going on in the account I am offering. I do not think this charge holds water. The existence of these two systems was not posited to explain the research on moral judgment. Rather, in the previous chapter, I argued for the existence and properties of these two systems independently, drawing on a wide range of research (e.g. research on animal learning, categorization, and memory) including neuropsychological results. That this account was independently supported and has the resources to explain the diversity of results we see in research on moral judgment should count as a point in its favor. Before proceeding to a more detailed discussion of my account, I want to consider one additional challenge to my account.

8.1.2 The nativist challenge As indicated above, in contrast with linguisticism, the account I favor rejects strong nativism. While I cannot fully address this issue, I want to discuss one response to this kind of challenge offered by Kim Sterelny (2003) and to suggest that a similar response can be made in moral psychology. I do so by focusing on one of the most popular kinds of argument in this debate: the poverty of the stimulus argument. As hinted at above (§8.1), the basic structure of this argument is as follows: humans possess representations with a certain content, but it is impossible (or extremely unlikely) that such content could have been acquired through learning. Therefore, this content must be innate. The poverty of the stimulus (POS) argument is so called, because

169 it is claimed that stimuli available to an organism are not sufficient for it to acquire representations with the attributed content. In linguistics, for example, it has been argued that people’s linguistic judgments accord with certain principles of grammar and thus that people possess mental representations with this content. Yet, children are not explicitly taught these principles, nor is normal experience thought to be sufficient to recover these principles. Indeed, without explicit instruction, people have no access to these principles. Thus, it is claimed that these principles of grammar must be innate (Chomsky 1980). Proponents of linguisticism argue that moral judgments are relevantly similar. People make moral judgments which are in accordance with certain principles (e.g. the DDE). However, people are not explicitly taught such principles, nor is normal experience sufficient to recover these principles; so these moral principles must be innate. To attack a POS argument, we can (1) reject the attribution of representations with the kind of content specified, or (2) show that representations with such content could be acquired through learning. Of course, these are purely negative responses to this argument. They would not solve the problem that such arguments are intended to address. To do this, we would need to (1) correctly specify the content of the representations that explains the original phenomena (e.g. pairs of moral judgments consistent with the DDE) and (2) show that such content actually is acquired through learning. Indeed, on the negative side, I think proponents of POS arguments in moral psychology misrepresent the content of the representations at issue. To take the example of the DDE, I agree that people outside of philosophy classes are not explicitly taught this moral principle and I accept the results that indicate that people’s moral judgments are in accordance with this principle. However, as I discuss shortly, I claim that these judgments can be explained by lower-level associative processing in the basal ganglia system. Emerging research is revealing just how powerful the learning mechanisms instantiated by the basal ganglia system are.1 While much work remains to be done, I believe it is plausible that these learning mechanisms explain the acquisition of the necessary representations.

1 For example, utilizing simulations, Skyrms (2010) shows that complex signaling can emerge from simple reinforcement learning.

170 On the positive side, Kim Sterelny (2003) has argued that the appearance of innate, domain-specific cognition in folkbiology and theory of mind (ToM) can be explained by domain-general learning mechanisms. I think similar arguments can be constructed against linguisticism’s claim that there is an innate, domain-specific moral module. Nativists, Sterelny claims, overlook two important factors: perceptual tuning and scaffolded learning (2003, 222). He explains: “we have perceptual mechanisms that make the right aspects of behavior, voice, posture, and facial expression salient to us. These perceptual over time come to operate in a developmental environment that is the product of cumulative epistemic engineering” (2003, 222). The idea of perceptual tuning shares much in common with the idea suggested above (§7.2.5) that the output of innate core cognition might serve as input to a domain- general learning mechanisms. Having perceptual mechanisms already tuned to the right features of the environment simplifies the learning process. Core cognition posits innate mechanisms for gaze following and the detection of agents. If these processes serve as input to general learning mechanisms, then part of the problem of understanding others’ intentions is already solved. This kind of strategy undermines strong nativist claims about the existence of an innate, domain-specific ToM module; perceptual tuning and a general learning mechanisms can equally well explain the phenomena we observe.2 “Epistemic engineering” refers to the fact that the environment of development, of which social aspects form a large part, includes resources by which learning can be scaffolded. For example, language, narrative stories and the saturation of young humans’ environments by agents and interpretive practices all provide input that, when combined with perceptual tuning, can lead to the development of sophisticated ToM abilities. Thus, nativist claims that mature ToM depends on domain-specific innate content are undermined. I believe that similar considerations can be used to undermine nativist arguments in

2 Of course, the mechanisms underlying perceptual tuning may involve some innate content. However, we may have good independent reasons to posit such content (e.g. appearance of such capacities early in development). Thus, a POS argument would provide no reason to believe in any additional innate content. Moreover, as indicated above (§7.1), the kind of nativist claims made by core cognition are far weaker than the nativist claims made by proponents of POS arguments.

171 moral psychology. First, moral cognition often depends on aspects of ToM. Understanding the goals and intentions of agents is an important part of moral evaluation; so if Sterelny’s arguments are successful in undermining nativist claims in ToM, this will already go some way toward undermining these claims in moral psychology. Second, there is independent evidence that perceptual tuning mechanisms go beyond the goals of the agent and that young infants’ representations of agents’ goals include evaluative elements. Kuhlmeier et al. (2003), for example, habituated infants (12 months) to an animation showing a red circle attempting to climb a hill and either a blue square that attempted to hinder the circle or a yellow triangle that attempted to help the circle. Later, infants expected the red circle to approach the helper triangle. Similarly, infants prefer to interact with individuals previously observed helping others than individuals previously observed hindering (Hamlin, Wynn, and Bloom 2007). Thus, there is some support for perceptual tuning in moral psychology (for a review of related findings, see Wynne 2008). In addition, as I have already discussed, there is considerable support for the view that these domain-general learning mechanisms play an important role in moral development. The developmental patterns that emerged from the discussion of Chapters 5 and 6 fits with the functioning of the basal ganglia system. Dysfunction of certain brain areas from an early age (as in psychopathy and early onset vmPFC damage) leads to predictable patterns of dysfunction in moral judgment and behavior. The partially overlapping neural substrates of emotional learning and the basal ganglia system suggest that the basal ganglia system is important for normal moral development. I also discussed how moral education involves shaping children’s affective responses. Proponents of linguisticism rightly point out that children do not receive much explicit instruction in the moral domain. However, the learning mechanisms of the basal ganglia system do not depend on explicit instruction, but are involved in implicit, error-based learning. This kind of learning seems well suited to explain the pattern of moral development that we observe. Thus, I think that there are good reasons to think that the kind of argument Sterelny presents against strong nativist claims in ToM and folkbiology can be successfully applied in moral psychology against POS arguments. Of course, further

172 work is necessary, but I take it that I have at least sketched the outline of a response to the nativist challenge.

8.1.3 An explanation of the trolley/footbridge asymmetry I now want to offer an explanation for the Trolley/Footbridge asymmetry. I do not think that it involves an application of the DDE. Rather, I think that it results from the similarity-based, associative processing of the basal ganglia system. This idea is similar to the suggestion of Sloman and colleagues that “much of moral appraisal reflects the extent to which the causal model of the event being judged deviates from an idealized causal model” (Sloman, Fernbach, and Ewing 2009, 20). In categorization, we compare a novel object to previously stored representations (e.g. prototypes or exemplars) on the basis of the similarity of various features. As I argued in the previous chapter, similarity- based categorization likely depends upon the functioning of the basal ganglia system. Moral judgment depends on the same kind of process: we form causal representations of the situation under judgment and compare it to stored representations. This idea is consistent with the hypothesis of intervention myopia discussed earlier (§4.1) and it can be further developed by drawing from work on Theory (Kahneman and D.T. Miller 1986). Recall that, according to the intervention myopia hypothesis, in agent intervention cases (e.g. the Trolley case) people weigh the consequences of each outcome. Five deaths are worse than one, so most people judge it permissible to flip the switch. In patient intervention cases (e.g. the Footbridge case), the five people who will be killed in the absence of intervention are in the background. The focus is on the large man: the prospective patient of harm in the event of intervention. If you do nothing he will survive, but if you act he will die. Thus, more people judge that pushing the man is impermissible. Intervention myopia is likely an instance of a broader phenomenon discussed by Daniel Kahneman and Dale Miller. Drawing from research on exemplar models, Norm Theory claims that the processing of a stimulus makes salient certain alternatives and not others. For example, consider the following example (Kahneman and D.T. Miller 1986, 355):

173 Mr. C and Mr. D were scheduled to leave the airport on different flights at the same time. They traveled from town in the same limousine, were caught in a traffic jam, and arrived at the airport 30 minutes after the scheduled departure time of their flights. Mr. D is told that his flight left on time. Mr. C is told that his flight was delayed, and only left 5 minutes ago.

Thus, Mr. C and Mr. D are in nearly the same situation. However, when asked who will be more upset, nearly everyone (96%) agrees that Mr. C will be more upset. According to Norm Theory, the explanation for this is that, for Mr. C’s situation, a highly accessible alternative involves him just barely making the flight. Whereas, this alternative is not as salient in the case of Mr. D. Consider another example. The questions “Why did Carter lose the 1980 election?” and “Why did Reagan win the 1980 election?” ask for an explanation of one and the same event: the outcome of the 1980 election. However, the way these questions are asked focuses on either the faults of Carter or the strengths of Reagan (Kahneman and D.T. Miller 1986, 364). The probe can affect the considerations that come to mind and how the question is ultimately answered. Thus, according to Norm Theory, the alternatives we generate are often based on the stimuli itself. Kahneman and Miller argue that this is a pervasive phenomena in human cognition. Additionally, similar results have been found in behavioral economics, leading to the idea that preferences are not absolute or static but are constructed in situ and are thus context-sensitive (Lichtenstein and Slovic 2006; see also Schwarz 2007 for work in social psychology on attitude construction). Intervention myopia appears to be one more instance of this phenomenon: the Trolley and Footbridge cases generate divergent judgments because they render different alternatives more salient. This affects the comparison of alternatives and ultimately the judgments that are made. While some of the details remain to be worked out (e.g. the precise neural substrate), I believe it is very likely that this processing depends on these affective, associative mechanisms subserved in part by the basal ganglia. Importantly, this explanation does not involve the representation of any moral principle (e.g. the DDE) in the etiology of these judgments. Moreover, as indicated, these mechanisms are capable of functioning without awareness, explaining why people are

174 generally unable to come up with any justification of their judgments that exhibit the Trolley/Footbridge asymmetry. Using further variations of Trolley cases, Waldmann and Wiegmann (2010) found results that are not consistent with the DDE, but can be explained by their development of the intervention myopia hypothesis. Drawing on this, I believe we can extend this account to explain the distinction between weak permissibility and all-in permissibility exhibited both in the Tea Cups cases and in catastrophe cases. In various trolley cases, Waldmann and Wiegmann claim that two different contrasts are computed: “the global contrast between the number of victims in the presence and absence of the act, and an additional local contrast that compares the fates of the morally relevant target (i.e. threats, victims) of the proposed intervention in the presence versus absence of the act” (Waldmann and Wiegmann 2010, 2589). The local contrast can be seen as the representation of the action and salient alternatives, e.g. in the Footbridge case: push the man and he dies, do nothing and he lives. The global contrast represents a more abstract comparison of the situation and its alternatives. In the Trolley case if you do nothing, five will die, and if you flip the switch, only one will die. Norm theory suggests that the description of the case will affect the alternatives rendered salient. The local and global contrasts capture how this plays out in the Trolley/Footbridge asymmetry. The Footbridge case focuses attention on the local contrast, while the Trolley case draws attention to the global contrast. Now recall that Nichols and Mallon (2006) found evidence that in the Tea Cups cases, where emotional impact is minimized, participants still distinguish between Tea Cups Trolley and Tea Cups Footbridge. While people judge that the action is permissible, all things considered, in both, only in Tea Cups Footbridge do participants judge that the action violates a moral rule. Similarly, in cases where the consequences of not violating the moral rule (e.g. not pushing the large man onto the tracks) would be catastrophic, (e.g. would lead to millions of deaths), people’s judgments exhibit a similar pattern: they judge that the action (pushing the large man in front of the trolley) would violate a moral rule, but is, all things considered, permissible. I claim that weak impermissibility judgments are driven by associative processing (likely subserved by the basal ganglia system). In Tea Cups Trolley, with the focus on the

175 global contrast, people agree that the act is permissible and that no moral rule is violated. In Tea Cups Footbridge, people recognize that the action would violate a moral rule, but breaking a tea cup to save five is not emotionally salient and so the action is also judged all-in permissible. However, in the standard Footbridge case, this is not so. There is a conflict between the local and global contrast. To resolve this conflict, additional cognitive resources are recruited.3 In Chapter 6 (§6.3), I argued that cases like these, which pit the direct causing of harm against beneficial consequences, trigger a process of simulation that generates an emotional response. I called this dynamic evaluation in response to simulation. In the Footbridge case, the thought of pushing a man in front of a trolley produces a strong emotional reaction and so the action is generally judged all-in impermissible. Similarly, as Nichols and Mallon (2006) found, if the consequences of not acting are sufficiently bad, more people will overcome these emotional responses and judge that the action is permissible.4 Thus, we observe judgments of weak impermissibility and all- in permissibility. People recognize that the action will violate a moral rule, but judge that, all things considered, it is morally permissible. On my view, associative, affective processing computes the local and global contrasts. In some cases, like the standard Trolley case, the global contrast dominates. In other cases, e.g. ‘easy’ personal moral dilemmas like the Infanticide case, the local and global contrasts coincide. In cases like the Footbridge, there is a conflict between local and global contrast and additional cognitive resources are brought in. Norm theory helps explain why the Trolley and Footbridge cases render different alternatives salient. I’ve hypothesized that associative, affective processing lies at the bottom of this phenomena.

8.1.4 Some loose ends

3 As noted in Chapter 3 (§3.4.1), Greene et al. have suggested that the ACC may be responsible for detecting conflict and recruiting processes to resolve this conflict. 4 Some people (e.g. moral absolutists), do not override such these emotional responses. However, usually this also involves the influence of some explicitly held beliefs. In psychology, research on so-called protected values investigates how certain kinds of values are treated as inviolable. For some discussion, see Bartels and Medin (2007).

176 This discussion of causal cognition in moral judgment has attempted to integrate the claims of Chapter 5 and 6 on the role of emotion with the results of research on different kinds of trolley cases described in Chapter 4 and the account of causal cognition defended in Chapter 7. In addition to the synchronic role of emotion in moral judgment, I also identified a diachronic role for emotion in moral development. As I’ve suggested above (§8.1.2), this diachronic role for moral judgment is in part explained by the learning of the basal ganglia system. Here I want to discuss this issue a bit further. As I noted in Chapter 7 (§7.1.2) there is some overlap between the basal ganglia system and what I described in Chapters 5 and 6 as emotional learning. Thus, the functioning of the basal ganglia system is, at least in part, affective, though in may not involve a full-fledged emotional response. In this context it is important to note that vmPFC damage does not result in total lack of emotions. As we saw, vmPFC patients have problems with dynamic evaluation, response reversal and emotional control. They can exhibit normal emotional learning in some cases, even demonstrating a tendency toward explosive outbursts of anger. This suggests a picture where emotional learning depends on structures like the vmPFC, OFC, amygdala and the basal ganglia. Clearly vmPFC patients have some of these elements intact, probably those elements that are more closely connected to what I have been referring to as the basal ganglia system. Resolving further functional specialization within these systems must await further research, though it is important to note that there may well be some redundancy of function across these regions. I discuss affective processing further in Chapter 9 (§9.2.2). The MTL system involves more decoupled representations, which are not affective and are not directly connected to motivational structures. However, in Chapter 6 (§6.4), I did suggest that emotional learning might provide input to learning processes subserved by the MTL system. This helped explain the relatively normal moral judgments of adult onset vmPFC patients: moral knowledge stored in declarative memory (subserved by the MTL) survives damage to the vmPFC. As indicated by the important differences between vmPFC patients who sustained damage following normal moral development and those who suffered early onset vmPFC damage, intact emotional learning may be crucial for acquiring this explicit knowledge. However, it may also be the case that there are redundant processes supporting some of these judgments in normal

177 individuals. That is, normal individuals may possess both affective representations that can contribute to such judgments as well as more explicit, ‘cognitive’ representations in declarative memory that can contribute to judgments with the very same content. Indeed, there is evidence that commitment to deontology or is partially correlated with judgments on Trolley cases (Lombrozo 2009). Consequentialists judged trolley cases to be more permissible than deontologists.5 When the Trolley and Footbridge cases were seen together, consequentialists judged the Trolley case less permissible and the Footbridge case more permissible, than when these cases were judged individually. In addition, it appears that moral principles play a greater role in judging actions at greater temporal distance (Eyal, Liberman, and Trope 2008). The account I have offered in this section attempts to fill in some of the details of the kind of account of moral judgment I want to defend. In the next section, I want to mention some additional results and suggest that they may be explained along the same lines as the account of the Trolley/Footbridge asymmetry sketched above.

8.1.5 Causation, deviance and mental state I have discussed evidence that people’s moral judgments reflect an act/omission bias and may be sensitive to the degree of physical contact. There is also evidence bearing on the integration of actors’ mental states in moral judgment. I believe my account can be extended to explain these results as well.

Acts, omissions and causal deviance. The fact that harms brought about through action are more likely to be judged impermissible than harms brought about through inaction may be explicable along the same lines as the Trolley/Footbridge asymmetry. Consistent with Norm Theory, the representation of an action makes salient certain alternatives relative to the representation of an omission. Considering a harm brought about by an action will place the focus on the actor. When considering a case involving

5 Commitment to deontology or consequentialism was assessed by asking participants a series of questions about whether, for example, lying is (1) never permissible (deontological response), (2) permissible if it will bring about a greater net good (consequentialist response), or (3) obligatory if it will bring about a greater net good (strong consequentialist response).

178 an omission, focus may be shifted to the victim and other situational features instead, leading participants to judge an omission more leniently. A similar explanation may shed light on results in which harm is intended but brought about by a causally deviant path (Pizarro, Uhlmann, and Bloom 2003). In one case, for example, Zeke throws a knife at his enemy, Alan, intending to kill him. In the normal case he succeeds, while in the deviant case Alan is so alarmed that he has a heart attack, collapses and dies; the knife lands where he had been standing. Individuals discount blame in cases of deviance (relative to the normal case). Interestingly, Pizarro et al. (2003) also found that by asking participants to make rational judgments, this effect nearly disappears. This suggests that the judgments that discounted blame derive from Type 1 processes that can be overridden in certain circumstances. One factor that remains constant across these cases is the agents’ mental state. As suggested below, consciously focusing on the agents’ mental state may facilitate the overriding of these responses. Other research has attempted to sort through the effects of contact, personal force, distance and intention (Greene et al. 2009; J. Nagel and Waldmann 2010). The details in these cases are still sketchy, but more research may reveal a similar pattern explicable by associative mechanisms. In the next section, I discuss representations of mental state, an area where significantly more research has been done.

Representations of mental state. Recent research has revealed that the neural substrate of representations of mental state includes the right temporoparietal junction (rTPJ), left temporoparietal junction (lTPJ) and precuneus. These regions, especially the rTPJ, play a role similar to that of the MTL. This allows us to resolve the issue raised at the end of the Chapter 6: why do vmPFC patients fail to condemn intended but unsuccessful harms? As we saw above (§7.1), core agent cognition involves representations of the goals of agents. An additional capacity to represent more specific mental states comes online around age five. Before this time, however, children’s judgments of actions depend mostly on the consequences of those actions. Even after children acquire the ability to represent others’ mental states, it is difficult for them to integrate this

179 information into their moral assessments. Before age 7, children are likely to judge that actions that accidentally produced harm are worse than actions that were intended to cause harm but failed to do so (J.A. Baird and Astington 2004). Thus, the capacity to represent others’ mental states and the ability to integrate these representations in moral judgment have different developmental timeframes. A series of neuroimaging studies is shedding light on the neural basis of these more sophisticated ToM capacities. Work in nonmoral contexts has found selective activation of the rTPJ in tasks involving belief attribution (as opposed to representations of other aspects of agents and even representations of states like hunger) (Saxe and Powell 2006). Consistent with this, Young et al. (2007) found that moral vignettes depicting the mental states of agents selectively activated the rTPJ. This study and several of the studies discussed below use a similar design that allows the independent manipulation of factors, in this case: facts, beliefs and consequences. For example, one such vignette involves two friends touring a chemical factory. Grace goes to pour some coffee and her friend asks for sugar. There is some white powder, which is, in fact, either sugar or a toxic substance. It is in a container marked ‘sugar’ or ‘toxic’, causing Grace to form the belief that it is either sugar or a toxic substance. Finally, Grace puts it in her friend’s coffee which either has the consequence of causing her friend’s death or has no effect. Given this kind of design, Young et al. looked at four kinds of cases, varying along two dimensions: (1) whether Grace believes the white powder is sugar or a toxic substance and (2) whether her action causes the death of her friend or has no effect. They found activation of the rTPJ in response to each of these scenarios. However, they found markedly higher activation in the case in which Grace believes that the substance is toxic but her friend is fine, i.e. in cases of attempted but failed harm. Young et al. suggest that “the rTPJ response was selectively enhanced when subjects used information about protagonist’s beliefs to condemn the protagonist despite his failing to cause harm” (2007, 8238). Neutral intentions that accidentally brought about harm were not judged totally permissible and were accompanied by neural activation associated with cognitive conflict. A follow-up study correlated activation of the rTPJ with a tendency to mitigate

180 harm in cases of innocent intentions bringing about accidental harms. Individuals with higher rTPJ activation were more likely to exculpate agents for accidental harm on the basis of their innocent intentions (Young and Saxe 2009a). As noted above, the capacity to represent agents’ beliefs comes online before children are able to integrate those representations in moral judgment. Neuroimaging reveals that the rTPJ is involved both in the encoding of belief and in the integration of these beliefs into moral judgment (Young and Saxe 2008). Even in cases where explicit mental state information is not given in a scenario, rTPJ activation indicates that people make spontaneous attributions of mental states (Young and Saxe 2009b).6 Further support for the role of the rTPJ in integrating mental state representations in moral judgment comes from a study utilizing transcranial magnetic stimulation (TMS). TMS uses a powerful magnetic field to disrupt patterns of neural activation in regions of the cortex. Young et al. (2010) found that TMS to the rTPJ caused participants to judge attempted but failed harms more permissible than control groups (who had TMS to the right parietal cortex). This is exactly what would be expected if the activation of rTPJ observed in previous studies contributes to the integration of mental state information in moral judgment. Under normal conditions, actions intended to cause harm but which failed to do so prompt permissibility ratings on a par with ratings of actions intended to cause harm that succeeded, though the former judgment depends on additional processing in the rTPJ. When this activity is disrupted, people judge less harshly actions that are intended to harm but fail to do so. Interestingly, judgments of intended and successfully executed harms were not affected by TMS. One possibility is that mental state information is not integrated into moral judgment unless there is a conflict between the outcome and the represented mental state. Perhaps it is only when there is an inconsistency between the mental state of

6 One story involves child-care worker, Steve, who opens a package of beef, makes a meat loaf and serves it to the children. The participant is then told either that (1) the meat is fresh (moral-neutral), (2) the meat has become tainted through a small tear in the package (moral-negative), or (3) the meat was wrapped in plastic (nonmoral). They are then asked to judge the truth of a claim (e.g. “The meat is unsafe for consumption”). Participants in both the moral-neutral and moral-negative conditions exhibited higher rTPJ activation, despite the fact that they were not explicitly asked to make a moral judgment.

181 the actor and the outcome is the rTPJ recruited to resolve the conflict. This receives some support from reaction time data indicating the shortest reaction time in the intended harm condition, where mental state and outcome are consistent (Young et al. 2007; Young and Saxe 2009a). This idea is also supported by the phenomena of blame blocking (Cushman 2008). This involves cases similar to those involving causal deviance discussed above. Cushman (2008) used cases in which an actor’s attempt at harm fails, but the intended harm is brought about independently. For example, one case involves Smith and Brown, two runners a few days away from an important race. Smith holds a world record and is predicted to win. Brown hopes to eliminate Smith from the competition. Brown believes that Smith is allergic to poppy seeds, so when Smith is in the bathroom, Brown sprinkles poppy seeds on Smith’s salad. But Brown is wrong. Smith is allergic to hazelnuts, not poppy seeds. Smith returns and finishes the salad, which happens to have hazelnuts on it. Smith dies because of the hazelnuts. This case was opposed to an alternative in which Brown again attempts harm and is unsuccessful, but the harm is not brought about by some other means and Smith is fine. What is interesting about the results using these cases is that participants were less likely to assign blame to Brown when the intended harm was brought about coincidentally than when the harm was not brought about at all. That is, when asked “How much blame does Brown deserve?” participants judged that Brown deserved more blame (on a 7-point scale) in the case where he tries to bring about harm but fails than in the case where the harm results from some independent process. Thus, the fact that the harm was brought about by some independent causal path appeared to (at least partially) block participants’ attributions of blame to Brown (Cushman 2008). One hypothesis that fits with the neuroimaging evidence is that, when there is no inconsistency between mental state and outcome, additional processes focused on an agents’ mental state are not recruited. Thus, when the intended harm is coincidentally brought about, one process absolves Brown of causal responsibility, but because there is no inconsistency between mental state and outcome, no additional processes (e.g. those subserved by the rTPJ) are recruited and so Brown is not condemned for his attempted harm. Whereas, in the alternative case in which there is an inconsistency between mental

182 state and outcome, additional processing is recruited and supports judgments of blame for the intended harm. Further research is certainly needed here. We can go some way toward linking this research on the rTPJ with the account sketched above. As just mentioned, individuals with vmPFC damage judge intended but failed harms to be more morally permissible than controls (Young, Bechara, et al. 2010). This mirrors the results of studies of utilitarian judgment in vmPFC patients. Just as Koenigs et al. (2007; see also Ciaramelli et al. 2007; Moretto et al. 2010; Thomas, Croft, and Tranel 2011) found that vmPFC patients are more likely to judge an action causing harm permissible if there are net benefits, they also do not condemn individuals for intended but unsuccessful executed harms. In the former case, my explanation indicated that vmPFC patients, unlike normal participants, do not engage in dynamic evaluation in response to simulation. This process requires both simulation, subserved by the MTL and affective processing subserved by the vmPFC. These latter processes are disrupted in vmPFC patients. I think a similar explanation can be applied here. In cases of intended but failed harms, we need to represent the content of actors’ mental states. In contrast to other representations handled by the MTL, representations of mental states are localized in the rTPJ. However, to integrate these representations into moral judgment an evaluative response to these representations is also necessary. This was the crucial role played by the vmPFC and the reason that individuals with damage to this region make more utilitarian judgments: they experience no emotional response to simulations of causing direct harm (e.g. in the Footbridge case). In the case of the rTPJ, the vmPFC again provides a vital interface by which the evaluative response to representations of agents’ mental states is integrated into moral judgment. Without the evaluative processing of the vmPFC, these representations of mental state in the rTPJ can have no effect on exculpating or enhancing judgments of permissibility in moral judgment. Of course, further work is needed here as well, but I think this brief discussion highlights how the account sketched above can be extended to make sense of this research of representations of mental states. Thus, I think that an account along the lines of my explanation of the Trolley/Footbridge asymmetry has the potential to shed light on a variety of phenomena

183 related to moral judgment. It is likely that an explanation of these results will turn, not on the application of various kinds of moral principles, but on an understanding of the details of automatic, affective, associative processing, which is subserved in part by the basal ganglia system.

8.2 Conclusion In this chapter I have argued against some of the claims of linguisticism and offered my own explanation of a set of results involving variations of Trolley cases. I argued that these results can be explained by pervasive tendencies in human cognition that are thought to be subserved by affective, associative processing. I suggested that this account can be extended to explain a variety of other phenomena as well. In the next chapter, I want to draw together the research discussed so far and sketch a more general picture of moral psychology.

184

CHAPTER 9

MORAL PSYCHOLOGY: PUTTING THE PIECES TOGETHER

I have focused mostly on a set of puzzles that have arisen in recent moral psychology revolving around various kinds of trolley cases and the role of emotion in moral judgment. In the previous 4 chapters I developed an account that offers resolutions to these puzzles. Now I want to say something more general about moral psychology. At a general level, moral judgment depends on representations of actions, consequences and the mental states of actors, and the interrelations among these. Of course, the representation of these elements goes beyond moral psychology, since it includes action more generally. This is roughly the realm of folk psychology. Thus, we can see moral psychology as either a subset of or significantly overlapping with folk psychology. I think we can also see folk psychology itself as a subset of causal cognition, as I defined it in Chapter 7 (§7.1), since mental states are generally of interest in moral psychology insofar as they are causally related to actions. I have discussed evidence that processes of different kinds contribute to the representation of different aspects of moral situations. The processes involved vary across different kinds of moral judgment and across different individuals and have various interconnections with each other. In this chapter, I want to review some of these results (§9.2-9.3). I also want to offer some additional considerations against linguisticism and offer an overall assessment of the strength of the linguistic analogy (§9.4). I begin with a discussion of computationalism (§9.1).

9.1 Computationalism One way of thinking about how the brain represents mental states, actions, consequences and the relations between them is that such processing occurs in a fairly linear way (e.g. Mikhail 2002). We first represent the mental states of the actor, the

185 connection of these states to the action, and the connection of the action to its consequences, etc. All of this is fed into a ‘moral module’ whose output is a moral assessment of the situation. This is the kind of processing we might expect an engineer to use if she were designing a computer to make moral judgments. It is also the kind of picture that fits naturally with linguisticism. It is my contention that the brain’s representation of these elements is importantly different from this picture. I want to argue that the representational processes involved in moral psychology are more like a pastiche. Different elements are represented by different processes, distributed throughout the brain. Different processes have different properties (e.g. similarity vs. rule-based, conscious vs. unconscious processing, etc.) and these differences in properties very often influence the resulting moral judgments. There might be somewhat redundant representations. Other aspects might not be represented at all or such representations may not affect moral judgment. For example, we saw how TMS to the rTPJ did not disrupt moral judgment in cases of intended harm. It seems that moral judgments can be formed on the basis of limited information—even when more information is available. The interconnections among these processes may often be unexpected. This view of moral psychology is opposed to computationalism, which I identified in Chapter 4 (§4.2) as claim (8) of linguisticism. I see this view as exemplified in the work of John Mikhail; he writes: “an adequate scientific theory of moral cognition will often depend more on the computational problems that have to be solved than on the neuropsychological mechanisms in which those solutions are implemented” (Mikhail 2007, 143; see also Mikhail 2009). The discussion of the previous chapter shows that the neuropsychological implementation of a cognitive process does make a difference. I explained the Trolley/Footbridge asymmetry not in terms of the application of a moral rule (viz. the DDE) but in terms of the features of associative processing, likely subserved in part by the basal ganglia system. This processing involves representations that make salient some alternatives in certain circumstances but not others. From a computational standpoint, there is no reason that such processes should contribute to moral judgment. We can imagine (e.g. if we were designing a ‘moral computer’) utilizing completely different

186 kinds of representational processes, operating according to different principles which lack this feature. The representational processes actually involved have important consequences for the output of these processes. The pattern of moral judgments reliably observed is explained in part by features of the underlying neuropsychological processes. As I try to indicate throughout this chapter, I think this is a general feature of moral cognition. To be clear, in arguing against computationalism, I am not claiming that building computational models is not an important methodological tool in the cognitive sciences. Nor do I want to deny that cognition is, at base, computational. Rather, I want to argue that the details of these computational processes have explanatory value in moral psychology. Things can be clarified by looking at David Marr’s (1982) three-level view of information processing. On Marr’s view, information processing can be understood at three different levels: the computational level, which specifies the goal and strategic logic of the process; the algorithmic level, which specifies how the computation is to be implemented; and the implementational level, which specifies how the representation and algorithm are physically realized (1982, 227). Consider a computer program for doing your taxes. At the computational level, the problem is to take input about income, deductions etc. and to produce a tax return. At the algorithmic level, this involves implementing a particular computer program, though presumably many different algorithms could accomplish this same computational goal. Depending on the kind of computer you use to run the program, a slightly different implementation will be involved. We can see Marr’s three levels as capturing the features of a computational process at various levels of abstraction. As I see it, Mikhail is claiming that moral cognition can be understood entirely at the computational level. I claim that a full understanding of moral cognition will often require understanding what is going at the algorithmic or implementational level. One general reason for thinking this comes from what we know about systems built by natural selection. Natural selection, as François Jacob put it: does not work as an engineer works. It works like a tinkerer—a tinkerer who does not know exactly what he is going to produce but uses whatever he finds around

187 him whether it be pieces of string, fragments of wood, or old cardboards; in short it works like a tinkerer who uses everything at his disposal to produce some kind of workable object. (Jacob 1977, 1163) The brain is no exception. The particular cognitive solutions to adaptive problems that natural selection finds depend not only on the problems themselves, but on the materials available. Natural selection cannot build a computer program from scratch, it must begin with what is provided by previous evolution. It is only at the algorithmic and implementational levels that we can see how cognition has been shaped by the past. For example, as is well known, humans have a blind spot in their visual field. This is the result of a gap in photoreceptor cells where the optic nerve exits the retina. This is not generally noticeable in experience because subsequent processing ‘fills in’ the blind spot. However, there are engineering solutions to this problem that would not require such additional processing. The cells of the retina could project backward to the optic nerve, avoiding the need to pass through the retina. However, the current pathway is developmentally fixed. It forms a historical constraint that further evolution must work around. Natural selection is an optimizing force, but it is not absolutely so. It optimizes relative to the constraints imposed by the results of previous evolution. On this issue, Stephen Stich (2006) has distinguished between the view that morality is an elegant machine and the view that it is a kludge. The former view shares much in common with what I have called linguisticism: moral cognition depends on a moral grammar integrating innate moral rules and principles and has evolved for adaptive reasons. The opposing view denies these claims, holding that “many of the most important facts about the human moral capacity do not have an adaptive explanation” (2006, 184). While he admits that these two views are at opposite ends of a spectrum and the truth likely lies somewhere in the middle (though closer to the kludge view), I think opposing the elegant machine view to the kludge view obscures this issue. The picture of moral psychology as a pastiche is far from elegant, but elegance should not be opposed to adaptiveness. Work-around solutions may not be as elegant as the results of careful design, but they may nevertheless be as functional. Stich’s way of cutting up the pie fails to appreciate that, in biological organisms, optimization is relative to historical constraints. It fails to make room for the kind of view I favor, according to

188 which the mechanisms underlying moral cognition may be adaptive, even though they take the form of a pastiche resulting from a process of evolutionary tinkering. Furthermore, given the complexity of the world and the limits on human computational resources, a certain amount of inelegance may actually be adaptive. Recent formal results indicate that “conflict can emerge within a collective even when natural selection acts on the level of the collective only” (Livnat and Pippenger 2006, 3198). Given constraints on computation, internal conflict may provide an optimal solution to the problems faced by an organism. Pre-theoretically, we might think that optimal design would resolve internal conflict. These results challenge this by indicating that a certain kind of inelegance may itself be adaptive. While I take these considerations to provide some reasons to reject computationalism, I do not want to rely entirely on this general argument to support my view of moral psychology. In the next two sections, I discuss how research we have already discussed supports this view. I first discuss the different kinds of processes that we have identified as contributing to moral judgment (§9.2). I then turn to how this account makes sense of some of the patterns in moral psychology that we have previously identified (§9.3).

9.2 Multiple kinds of representations in moral psychology As I discussed in Chapter 6 (§6.2.3), case studies in the middle of the 20th century were crucial to the development of the contemporary view that, not only are there areas of the brain dedicated to memory, but that different areas handle different kinds of memory.1 I think the case studies by Damasio and others at the end of the 20th century have served a similar role in moral psychology. In this section, I want to review where we are now. I should begin by discussing what I mean by representation. I have used this term throughout the dissertation in an intuitive way and I believe that all of the processes I

1 For more discussion of this history, see Kandel (2006, chap. 8) and Danziger (2008, chap. 6).

189 discuss in this section possess representational content (though not necessarily in the same way).2 I want to say a bit more about what I take this to mean. There is perspective on mental representation, which stretches back to the work of Fred Dretske (1981), that draws on research in information theory. On this view, mental representations are states that convey information. If a state is reliably caused by another state, the former can be said to carry information about the latter. describes this kind of view as follows: “a mental representation is a mental state that is reliably caused by something and has been set in place by learning or evolution to detect that thing” (2002, 54). Skyrms (2010) offers a thorough discussion of information in the context of both learning and evolution. Indeed, he argues that the view of propositional content familiar to philosophers as functions from sets of possible worlds to truth values (e.g. R.C. Stalnaker 1984; D.K. Lewis 1986) is actually a special case of the information- theoretic account of content he offers. Skyrms’ work utilizes simulations of reinforcement learning that are precisely of the kind implemented by the basal ganglia system. This account of representation allows us to resolve an issue raised at the end of Chapter 3 (§3.4.1). Greene and colleagues (2004) indicated that the distinction between affect and cognition was on the verge of breaking down. On the one hand, some definitions of ‘cognition’ take it to involve some kind of information processing. Yet, as we have seen, affective processes seem to be involved in information processing just as much as traditionally ‘cognitive’ processes. This account of mental representation allows us to refer to cognition in the broader sense as information processing. Thus, affective processes (which I discuss at greater length below) will count as cognitive in this sense. We can then understand a narrower view of ‘cognition’ as involving a particular kind of information processing. Thus, in the previous chapter, we saw Brewer (1974, 27) speaking of “higher mental processes” and others offering propositional accounts of causal cognition. Related to these accounts, I discussed the kind of processing subserved by the MTL system as

2 There is an enormous literature on mental representation. Here I simply want to indicate the kind of view of representation that I endorse. I do not have space to discuss arguments for this account or common objections to this kind of view. For further discussion and defense see Dretske (1981; 1986), Prinz (2000; 2002), and Skyrms (2010).

190 involving decoupled representations. We can now identify ‘cognitive’ in the narrower sense with mental representations involved in these kinds of higher-level information processing, as having allocentric representations or involving propositional content. To distinguish this from cognition in general, we might call this latter kind of information processing reasoning. Importantly, the particular properties of different kinds of information processing will often be important for understanding how that kind of processing contributes to moral judgment. With this idea of representational content in mind, I turn to a discussion of the kinds of representational processes I’ve discussed. In Chapter 7, I argued that core cognition, the basal ganglia system, and the MTL system all contribute to causal cognition. In earlier chapters, I discussed various ways in which affective processing contributes to moral judgment. Here I say a little bit more about these processes, the different kinds of properties they possess and the interrelations among them.

9.2.1 Core cognition I argued that both core object cognition and core agent cognition can contribute to causal cognition. Each of these can be broken down further. For example, Carey suggests that in core agent cognition humans may have innate mechanisms both for detecting faces and eyes and for recognizing the presence of agents from patterns of motion, e.g. contingent interaction with the environment (2009, 187). Overall, core cognition is modular, unconscious and automatic. The representations of agency do not rise to the level of particular mental states, but represent agents as having ‘goals’ in a broader sense. Despite the rather low-level nature of core cognition, it continues to exert influence throughout life, even when additional representational resources come online that make core cognition redundant. For example, core object cognition involves a mid-level visual system based on spatio-temporal relations. This explains the appearance of motion in strings of blinking Christmas lights. A second system comes online later in development and privileges kind information. Under certain circumstances, the output of these systems will be different. The mid-level visual system privileges spatio-temporal relations, inferring apparent motion in a way which minimizes total movement, even if this involves a transformation in kind (e.g.

191 mouse to bird). The kind-based system, on the other hand, will infer movement that preserves kind information (e.g. mouse stays mouse, bird stays bird), even if this involves inferring apparent motion over a greater distance (Carey 2009, 72-3). Speculatively, it is possible that core agent cognition contributes to moral judgment by providing general representations of the goals of agents independently of the functioning of the rTPJ. One instance in which this may be so is in cases of the side- effect effect. As discussed above (Chap. 6, n. 14), this involves the tendency of people to agree that a foreseen side-effect was brought about intentionally when that side-effect is bad, but disagree when that side-effect is good. There is evidence that children as young as 4 reliably exhibit this effect (Leslie, Knobe, and A. Cohen 2006; Pellizzoni, Siegal, and Surian 2009). Yet, as we saw in the previous chapter (§8.1.5) this is before children can integrate information about mental state into moral judgment.3 Moreover, individuals with vmPFC damage exhibit the side-effect effect (Young et al. 2006). And, in the previous chapter, I argued that the representations of mental state in the rTPJ are likely integrated through the vmPFC. Thus, there is good evidence that the side-effect effect is not mediated by representations of mental state in the rTPJ. One hypothesis is that core agent cognition is responsible for attributions of intention in cases of the side-effect effect. This is consistent with recent discussions of the side-effect effect (Guglielmo, Monroe, and Malle 2009; Uttich and Lombrozo 2010), though further research is certainly needed here.

9.2.2 Affect Throughout the dissertation, I have discussed the various roles played by affective processes. Moral emotions and the CAD triad hypothesis were discussed in Chapter 2 in the context of various accounts of morality. In Chapter 3, I discussed the presumptive case for dual process theory, highlighting how manipulation of emotional state can

3 Interestingly, Leslie et al. (2006) explicitly asked children if the character cared about the side-effect, which was an explicit part of the story. The asymmetry was found only among those who correctly answered that the character did not. Pellizzoni et al. (2009) found that some four- and five-year olds exhibited the asymmetry even if the character was explicitly described as lacking knowledge about the side-effect. More research is needed to sort out these details.

192 influence moral judgment, especially when the manipulation is unnoticed. We also discussed research by Robert Zajonc and others on affective primacy, indicating the automatic influence of affect and its relative autonomy from ‘cognitive’ processes. In Chapter 5 I discussed the role of the amygdala and OFC in what I called emotional learning. This dovetailed with the discussion in Chapter 6 of the vmPFC and its central role in representing value. There we saw the importance of the midbrain dopamine (DA) system, which signals a reward-prediction error crucial for learning and projects to areas in the basal ganglia. In Chapter 7, I discussed evidence that the basal ganglia system makes important contributions to causal cognition and categorization. In the previous chapter I argued that this kind of processing helps explain the Trolley/Footbridge asymmetry. Here, I want to draw together this research. Throughout these discussions a number of themes have emerged. First, affective processing involves a widespread network of brain regions including: the amygdala, insula, OFC, vmPFC, basal ganglia (which includes striatum and NA), ACC and mPFC. Second, affective processing has a tight connection to motivation. Third, affective processing is crucially involved in learning. There is evidence that these mechanisms implement a particularly powerful kind of learning, temporal-difference learning, and that phasic DA signals play an indispensable role in this kind of learning. Fourth, we saw that affective processes can function in the absence of awareness—indeed, there is evidence that affect has a comparatively greater influence on behavior in the absence of awareness. Last, we saw that affective processing is connected to physiological responses. What emerges from this research is a picture of affect as a pervasive aspect of human psychology. Much affective processing does not involve a full emotional response (e.g. anger), but involves acquired responses to stimuli with evaluative content (e.g. the result of reinforcement learning). Affect seems to be well integrated with sensory processing. As we saw in Chapter 6, the OFC receives input from all sense modalities and is well placed to integrate sensory and evaluative information. This picture has much in common with core affect theory, a view which can be traced back to the work of . Core affect theory takes affect to be a psychologically primitive element of human psychology out of which other affective phenomena (e.g. emotions, moods) are

193 constructed (Russell 2003; Barrett and Bliss-Moreau 2009).4 On this view, humans are (almost) always in some state or other of core affect and these states can be characterized in terms of hedonic valence, i.e. some degree of pleasure or displeasure, and arousal, involving changes to the autonomic nervous system (ANS) and endocrine system. Barrett and Bliss-Moreau explain: core affect is realized by integrating incoming sensory information from the external world with homeostatic and interoceptive information from the body. The result is a mental state that can be used to safely navigate the world by predicting reward and threat, friend and foe. (Barrett and Bliss-Moreau 2009, 172) Core affect theory also endorses the view that core affect supports learning about the value of objects in the environment. Thus, core affect coincides quite well with the picture of affective processing that emerges form the research discussed above. I think we can tentatively accept core affect theory as a rough general theory of the affective realm. It offers an account of the central properties of affective processes. Core affect can be characterized primarily in terms of hedonic valence and arousal, depends on a broad set of neural substrates and these mechanisms implement a powerful kind of learning, and represents the value of objects (though these representations are not in propositional format). These representations are closely linked to motivation and do not depend on conscious awareness. As I have pointed out, this account fits nicely with the research already reviewed. I want to sketch how emotion is constructed (in part) out of core affect.5 This will allow us to make sense of the content of these affective processes, including emotions. Research on emotion is divided. One reason for this is because emotions include a number of components. These components are often taken to include subjective feeling, hedonic tone, physiological response, action tendencies, characteristic facial expressions,

4 I do not endorse all aspects of core affect theory. For example, core affect theorists have argued that discrete emotions (e.g. fear, anger, etc.) are not natural kinds (Barrett 2006). I am unconvinced, see Scarantino (2009) for a response to these arguments. 5 Here I cannot offer a full discussion of many of these points. The account I offer draws on work by Paul Griffiths (1997) and Jesse Prinz (2004). For further discussion see these books, as well as the references cited in the text. In addition, Fox (2008) offers a helpful review of much of the scientific research on emotion.

194 the modulation of attention and cognitive content. Different theories of emotion identify one or a few of these components as central (Prinz 2004, chap. 1). Two of the most popular kinds of theories are perceptual theories and cognitive theories. Perceptual theories derive from the work of William James (1884) and Carl Lange (1912), and have more recently been defended by Damasio (1994) and Prinz (2004). On these theories, as James explains, “bodily changes follow directly the PERCEPTION of the exciting fact, and…our feeling of the same changes as they occur IS the emotion” (1884, 189-90). Thus, the basic idea is that a stimulus triggers physiological changes and our perception of these bodily changes constitutes the emotion. It is not the case that our heart races because we are afraid, rather, we are afraid because our heart races. In contrast, cognitivist theories emphasize the content of emotions, arguing that emotions must be cognitive if they are to be rationally evaluable (e.g. Bedford 1956; Solomon 1973). One cognitivist view sees emotions as representing “core relational themes” (Lazarus 1991). These are representations of the organism-environment relationship that bear on the well-being of the organism. On this view, for example, anger represents an offense to me or my relations, fear represents immediate danger, sadness represents irrevocable loss, and so on. Thus, different theories of emotion privilege different components of emotion. At the same time, there are different kinds of emotions as well. I think we have good reason to distinguish between basic emotions and complex emotions (Griffiths 1997; 2003; 2004). Drawing on work by Paul Griffiths, we can characterize basic emotions as biologically-based affect programs, which are automatic, complex and coordinated responses to stimuli. When triggered, they unfold in a predictable way that includes physiological changes (including changes to the endocrine and autonomic nervous systems), and characteristic facial expressions. Evidence indicates that these facial expressions are, to some extent, culturally universal. Basic emotions are commonly taken to include anger, happiness, sadness, fear, surprise, and disgust, but there is some dispute about how many and how specific these basic emotions are.6

6 In his account of basic emotions, Griffiths draws heavily on the work of (1972; 1999). See Fox (2008, chap. 4) for further discussion.

195 On the other hand, many emotions do not seem to fit well with this account of affect programs. As Griffiths points out, “in many instances of guilt, envy, or jealousy the subject does not display the stereotypical pattern of physiological effects” (1997, 100). Complex emotions may build on core affect or basic emotions in various ways, incorporating more ‘cognitive’ elements. Griffiths explains: “suppose you are locked into a dysfunctional pattern of interaction with your spouse involving continual fault finding and put-downs” (2003, 238). Upon reflection, you may experience feelings of guilt and self-loathing, which seem to involve more ‘cognitive’ complexity. Complex emotions may also involve a blend of basic emotions. Prinz (2007), for example, argues that contempt is a blend of anger and disgust. In addition, it appears that some complex emotions are not culturally universal. The Japanese, for example, experience an emotion that involves a pleasurable sense of dependence on others, called ‘amae’ (Griffiths 1997; S. Yamaguchi 2004), although it may be that this emotion is experienced even in cultures that lack a word for it. Given the diversity of components of emotion and the different kinds of emotions, it is understandable that there has been much disagreement about the nature of emotion. I think that some unity can be brought to this. I follow Jesse Prinz (2004) in attempting a reconciliation of perceptual theories with cognitive theories. This reconciliation provides the key to understanding the content of these states. What follows draws heavily on Prinz’s view of the content of emotions. However, while Prinz takes emotions as basic to the affective realm, I take core affect to be basic. Recall that representations have their content in virtue of being set up by learning or evolution to detect that which they represent. Now core affect involves hedonic valence and arousal and, as we have seen, learning is central to core affect. The idea is that, through associative, temporal-difference learning states of core affect become associated with various rewards (or punishers). Thus, these states come to represent the reward value of that which causes these states. As I have argued (§7.2.6), this may occur without our awareness. Basic emotion involves the perception of these bodily changes. In line with perceptual theories of emotion, this explains the subjective feel of emotions. In addition to our perception of core affect, basic emotion involves changes in the skeletal-muscular

196 system, characteristic facial expressions and action tendencies. These elements are tightly connected, coordinated and automatic in basic emotion. However, whereas the content of core affect was largely dependent upon learning, the content of basic emotions (i.e. affect programs) is more strongly influenced by evolution. These affect programs have evolved as responses to specific problems faced by organisms. Seeing affect programs as species-specific, evolved responses to challenges posed by the environment allows us to make sense of both the cultural ubiquity of such responses and also their content. Drawing on work by Anthony Kenny (1963), Prinz distinguishes two kinds of intentional objects that emotions have. On the one hand, an emotional response has a particular object, that which triggers the emotion. On the other hand, an emotional response also has a formal object, that in virtue of which a particular thing triggers the emotion. In specifying the formal object of an emotion, Prinz draws on Lazarus’s (1991) account of core relational themes. For example, when one’s beloved pet dies, one experiences sadness. The death of the pet is the particular object since it is that which has triggered the emotion. The formal object of this experience is irrevocable loss, since sadness has been set up by evolution to be triggered by such loss. Thus, we can reconcile perceptual theories of emotion with cognitive theories. The perception of bodily changes represents core relational themes (Prinz 2004, chap. 3). While these representations may not be conceptually mediated (i.e. may not involve propositional content), they have content nonetheless. This also allows us to make sense of the distinction between basic and complex emotions. Basic emotions are constructed by evolution. The relational themes they are designed to detect are widespread problems faced by humans. Thus, they involved stereotyped, automatic and coordinated responses. Complex emotions are more dependent upon learning. Thus, though they are constructed out of basic emotion and core affect, they involve more cognitively complex responses to the environment and the relational themes they represent (i.e. their formal objects) are more specific to particular cultures. They exhibit regularity, but they do not involve the same coordinated and stereotyped responses as basic emotions. We may roughly characterize basic emotions as biological constructions and complex emotions as increasingly dependent upon cultural construction. Learning is necessary for both. While anger may be culturally universal,

197 which stimuli trigger anger is sensitive to learning and may vary from culture to culture. On the other hand, the nature of complex emotions themselves, and not just what triggers these emotions, depends upon cultural learning. Thus, all emotions are adaptive (i.e. either evolved or learned) responses to problems posed by the environment. For humans, an important part of this environment is social. As we might expect, many emotions serve important social functions. Robert Frank (1988) for example, argues that one function of emotions is to solve commitment problems. We can think about this issue in terms of dominance reasoning in situations of iterated competitive interaction. Suppose Smith and Jones regularly hunt in the same area. Now suppose that Smith has managed to kill a stag. Jones may try to use force to appropriate this stag for himself. In any given case, Smith may do worse to resist Jones’ force than simply to capitulate. Jones knows this: persistently challenging Smith will result in him consistently giving his resources to Jones. Smith has a commitment problem. Smith’s anger at Jones’s wrongful usurpation solves this problem. Anger has strong motivational properties—it will prompt Smith to take action against Jones. It also involves characteristic facial expressions that communicate Smith’s commitment to Jones. So, when Jones tries to take Smith’s stag, Smith becomes angry. This anger signals to Jones that Smith is motivationally primed to respond to continued attempts to take the stag. Moreover, it signals that Smith is committed to a response that goes beyond what would be rationally justified on the basis of the value of the stag alone. This signal is honest: the facial expression is genuinely connected to the motivational disposition and the former is difficult to fake. Anger, in this case, is a strategic response to a commitment problem. It allows Smith to act contrary to the reasoning that would have him consistently capitulate to Jones. Yet, since his anger is an honest signal, Jones will be discouraged from consistently challenging Smith, since he knows this will trigger anger and that the stag is not so valuable as to be worth fighting over. Thus, emotions like anger can be seen as serving important social functions.7

7 Of course, this idea is not without problems (see Sterelny 2003, 140-1 for further discussion; see also R.H. Frank 2001). Significant support for some aspects of this theory comes from research on social punishment (see, for example, Fehr, Fischbacher, and Gächter 2002).

198 More recently, drawing from ‘transactional’ accounts of emotion (e.g. B. Parkinson, Fischer, and Manstead 2004), Griffiths and Scarantio (2009), have tried to emphasize the importance of social context, the extent to which engagement with the environment is continuous throughout the unfolding of an emotion and need not be conceptually mediated, and the dependence of emotion on environmental scaffolding. All of this research fits comfortably with the account of affect being sketched. Indeed, earlier (§8.1.2), I indicated how scaffolded learning might play an important role in moral development. Similarly, we can make a bit more sense of the CAD triad hypothesis, discussed above (§2.1.3). We can see contempt, anger, and disgust as serving to respond to certain core relational themes. Indeed, recent research has started to trace the relation between ‘primitive forms of disgust’ in response to spoiled food or disease and ‘moral disgust’ (Chapman et al. 2009). While avoiding illness from contaminated food is an important problem faced by all cultures, various forms of ‘moral purity’ may be more or less important across cultures. Thus, we should expect that the basic emotion disgust will be ubiquitous while the more cognitively complex ‘moral disgust’ may exhibit significantly more variation across cultures. This fits well with the results of cultural psychology discussed in Chapter 2, especially with Haidt and Joseph’s account of the “psychological primitives” of morality (2007b). I’ve attempted to sketch an account of affect that fits well with the research discussed above. I have not offered an exhaustive defense of this account, though I have indicated authors who defend many of these claims. However, the fact that these independently argued for views coincide with the research on affective processing discussed here lends further support to this account. Before moving on, I want to mention some additional complexity. Above, I followed core affect theorists in the view that core affect could be characterized in terms of hedonic valence and arousal. Complicating this issue, is evidence that the processing of pleasure and reward, while closely connected, can actually come apart. It appears that the reward prediction error signal may sometimes come apart from the processing underlying pleasure (for a review of this research, see Kringelbach and Berridge 2009; see also Schroeder 2004). For example, Berridge and Robinson (1998) found that in rats,

199 certain kinds of lesions do not disrupt the characteristic facial expressions of pleasure and displeasure in response to sucrose solutions and quinine solutions, respectively. These lesions, however, do destroy rats’ tendency to drink sucrose solutions and avoid drinking quinine solutions. At the same time, electrical stimulation of the lateral hypothalamus has been shown to induce eating in rats (as well as other mammals, including humans) even though their facial expressions indicate displeasure (Berridge and Valenstein 1991). If the motivational aspects of reward processing can be accomplished without any subjective feelings of pleasure or displeasure, one might wonder what the purpose of such feelings is. Dickinson and Balleine (2009) suggest that pleasure functions as a ‘cognitive interface’. Whereas the learning instantiated by reward processing is relatively slow, experienced pleasure can facilitate more rapid, explicit forms of learning. Thus, hedonic valence can be decomposed further. It seems that arousal has multiple components as well. This helps resolve a common difficulty for perceptual theories of emotion. This difficulty is whether perceptual feedback from the body is really necessary to experience an emotion. James (1884) mentions this issue and discusses evidence that seems to cut against his own theory. Research in the 20th century, for example on emotional response in individuals with spinal cord injuries, has been mixed (Prinz 2004, 57-8). However, more recent neuroimaging data provides some resolution. Critchley et al. (2001) have shown that individuals with pure autonomic failure (PAF), a condition that affects the functioning of the peripheral nervous system and disrupts bodily feedback, do exhibit some emotional responses, though they are somewhat reduced. More importantly, fMRI results indicate that in addition to a loop through the peripheral nervous system, by which amygdala response triggers bodily changes that in turn result in insula activation and the experience of emotion, there is a cortical loop connecting the amygdala directly to the insula (Critchley, Mathias, and R.J. Dolan 2002). This is consistent with the research mentioned in Chapter 6 (footnote 6) that the amygdala-insula is important for emotional experiences (J.S. Morris 2002) and that the function of the insula is to integrate information from emotionally relevant processes including the amygdala, OFC, and ventromedial and dorsolateral prefrontal regions (Craig 2009).

200 The perceptual theory is correct that, normally, perceptual feedback is part of an emotional response. However, direct cortical connections suggest that such feedback in not absolutely necessary for such responses. It appears that cortical connections facilitate cortical aspects of arousal, even in individuals with autonomic dysfunction (or paralysis), which would otherwise disrupt such processing. Thus, while we may treat core affect as psychologically primitive, there is a considerable amount of complexity within these mechanisms. More importantly, in this section we have seen how different aspects of the affective realm can contribute to moral cognition.

9.2.3 The medial temporal lobe system In the above discussion of representation, I indicated that information processing may involve different formats. One high-level kind of information processing might involve the explicit application of principles. Such information processing we might call reasoning or rational thought. I think we have seen good reason to believe that the MTL system is crucial for this kind of high-level information processing. We saw evidence (§8.1.4) that commitment to utilitarianism or deontology has some influence on moral judgment, suggesting that people may sometimes use explicit principles and that psychological distance tends to encourage the use of such principles. The MTL system may also subserve nonpropositional, but more decoupled representations. As we saw above, there is evidence that the MTL handles allocentric, map-like representations. These kinds of representations appear to be involved in simulation, which is also important for moral judgment (§6.2.3). However, it is also important to point out that some cases that we might label reasoning involve a complex interplay of the MTL system and affective responses. Dynamic evaluation in response to simulation involves processing in both the MTL and the vmPFC. Similarly, in Chapter 6 (§6.4), I argued that the acquisition of explicit moral knowledge over the course of development may depend on intact affective processing. At the same time, however, there is some evidence that explicit principles can also modify affective processes. In Chapter 5 (§5.5), I discussed evidence that negative affective responses to meat in moral vegetarians is a result, rather than a cause, of a change in beliefs regarding the moral

201 status of meat eating. This illustrates how the processing of the MTL system influences affective processing and moral judgment.

9.2.4 The fundamental computational bias Given the complexity of these processes, it is difficult to say anything general about the relations between these processes. However, one prevalent tendency in human cognition that is relevant in this context is the fundamental computational bias. As discussed in Chapter 3 (§3.2), this is a general tendency toward lower-level, Type 1 processing. In the absence of something that would trigger more decoupled processing (e.g. cognitive conflict), people will generally rely on the intuitive response. This leads to judgments that often deviate from normative theories (Stanovich 2004, chap. 4). Of course, even this general trend is complicated by individual differences in this tendency, a disposition captured to some extent by Fredrick’s (2005) cognitive reflectance task.

9.3 Patterns revisited At the end of Chapter 6 (§6.5), I identified three general patterns in the research reviewed up to that point. I want to briefly discuss how the account sketched in this chapter helps shed light on these patterns as well as on the relation between judgment and justification discussed in Chapter 4. First, we saw evidence that there may be more than one path to ‘normal’ moral development. Learning is a central component of affective processing. Affective learning tends to be more gradual, but it does not depend on explicit instruction and involves representations closely linked to motivation. At the same time, the MTL system is involved in a different kind of learning. The MTL system learns quickly, through explicit instruction, though these representations are less well connected to motivation. This account helps shed light on the neuropsychological underpinnings of these different developmental pathways in moral psychology. Second, we saw interesting dissociations of function, some of which involved pathology. My account has plenty of room to explain such dissociations. Affect plays a crucial role, and dysfunction in affective areas (e.g. the vmPFC, amygdala) will disrupt moral judgment in different ways. At the same time, we saw that affect processing is

202 subserved by a broad neural basis. Thus, dysfunction of certain areas results in selective affective problems of the kind observed in vmPFC damage, without resulting in a total lack of emotion. We also saw how higher-level representations could be dissociated from affective representations. In Palermo, Anthony Dickinson had the explicit belief (subserved by the MTL system) that watermelons would be a delicious way to satisfy his thirst. This belief was likely formed on the basis of the experienced pleasure of eating the watermelon. But he had actually acquired an aversion (subserved by affective mechanisms) to watermelon that would make further enjoyment of the food impossible. This kind of dissociation between explicit beliefs and affective responses could also be behind some well-known phenomena in social psychology. Wilson and Gilbert (2000) review evidence for what they call miswanting. We often pursue ends that we think we will want, only to be dissatisfied when we actually achieve those ends. One experiment found that, although associate professors believed that tenure decisions would have significant effects on their happiness for several years after, Gilbert et al. (2002) found that those who actually achieved tenure were no more happy than those who had not. We may experience this in mundane cases as well. When looking at a double-frosted, cream-filled doughnut in the display case, we may anticipate far greater pleasure than we experience actually eating it. Related to this, it has also been shown that introspecting on the reasons for a choice can actually diminish our satisfaction with that choice (T.D. Wilson and Schooler 1991; T.D. Wilson et al. 1993). Our motivations for certain things are often driven by our explicit beliefs. Requiring decision makers to introspect on their reasons insures that this is the case. However, our satisfaction with a choice depends more upon the processing of reward and pleasure in affective regions. Thus, our explicit beliefs about what we want may come apart from our affective response to these things when we get them. This is not simply a result of the odd circumstances that precipitated the dissociation in the watermelon case (e.g. the fact that Dickinson’s first experience of watermelon coincided with an evening of heavy wine drinking), but rather it appears to be a quite common phenomena. Gilbert and Wilson discuss many of the causes of this (2000; see also T.D. Wilson and Gilbert 2003; Lichtenstein and Slovic 2006).

203 We have also seen a certain amount of redundancy in the representations that underlie moral judgment. Similar moral judgments may be subserved by both affective representations and explicit representations in the MTL system. Thus, individuals with damage to the vmPFC make judgments that are largely consistent with those of normal individuals, despite affective dysfunction. It is important to note, however, that the content of these different representations may be different, even if they generate similar moral judgments. This relates to the third general pattern: the existence of important individual differences. To take one example, in Chapter 4 (§4.1.1), I discussed evidence that people who report some training in philosophy were more likely to be able to justify pairs of their judgments that were in accordance with the DDE. As I suggested there and will discuss again shortly, there is reason to think that these justifications do not necessarily shed light on the etiology of these judgments. Nevertheless, it is possible that at least some individuals, who have been explicitly taught the DDE, actually are applying this principle in making these judgments. At the same time, I argued that, for many people, the Trolley/Footbridge asymmetry is explained by associative, affective processing that does not involve the representation of the DDE. Thus, we see that the same judgment may derive from distinct processes in different individuals and that these processes may involve mental representations with different kinds of representational content. Similarly, we saw (§5.5) evidence that on emotion recognition and moral judgment tasks, some individuals (even in nonpathological or noncriminal populations) depend on more ‘cognitive’ processing (as indicated by increased DLPFC activation and decreased amygdala activation) while control groups depended more on affective processing, even though the responses of these groups were indistinguishable. The DLPFC may play a role in applying rules in the process of making moral judgment, as opposed to the associative nature of affective processing. This is consistent with general theories of prefrontal function (e.g. E.K. Miller and J.D. Cohen 2001) and gains support from the role of the DLPFC in the exertion of self-control discussed in Chapter 6 (§6.2.2) as well as a recent neuroimaging study correlating differences in right DLPFC activation with performance on a moral judgment task (Prehn et al. 2008). Similarly, Moore et al. (2008) found that individuals who rate higher on a measure of working memory capacity

204 (WMC) were more likely to allow killing when the harm was inevitable (e.g. in cases like the Ecologists case). Thus, it appears that controlled reasoning may play a role in moral judgment in some individuals more than in others. Research discussed in Chapter 6 (§6.2.1 and §6.3) provides another example. We have seen how damage to the vmPFC leads people to make more utilitarian judgments in response to certain kinds of dilemmas (e.g. Koenigs et al. 2007). In addition, we have seen that individuals with a history of drug addiction (as well as some normal individuals) exhibit a behavioral and physiological profile similar to those with vmPFC damage, suggesting abnormal function of the vmPFC in these individuals, even in the absence of brain damage (Bechara 2005). If this is the case, we might expect such individuals to exhibit a similar tendency toward utilitarian moral judgments. At the same time, we have evidence that emotional responses can be overridden in order to issue in a utilitarian judgment. Dyed-in-the-wool utilitarians may simply disregard or override any emotional responses they have, in light of their explicitly held theoretical views (i.e. a commitment to the principle of utility). Indeed, Greene et al. (2008) found that individuals who made a higher percentage of utilitarian judgments had faster reaction times, suggesting that some individuals may use the principle of utility itself as a kind of heuristic in moral judgment. Thus, a tendency toward utilitarian moral judgment may be a result of below average emotional responses (as a result of vmPFC dysfunction) or, alternatively, a result of consistently overriding or ignoring normal emotional responses. One final example bears mentioning. In Chapter 3 (§3.3.2), I discussed research I have done indicating that performance on the cognitive reflectance task predicts the likelihood that people will judge that a case of consensual brother-sister incest is permissible. One interpretation of this result is that people who are more disposed to reflect on their responses end up rejecting their initial affective response, which would otherwise push them to judge the action to be impermissible. One last issue that has arisen throughout the dissertation is the relation between moral judgment and justification (see §3.3.2 and §4.1.1). I’ve suggested that there is some reason to think that justifications might draw on psychological mechanisms distinct from those that generate moral judgment. When moral justification purports to be a

205 genuine report of the processing that led to the judgment, but is not such a report it involves confabulation. If someone offers an argument intended to support the judgment but that does not purport to be a genuine report, it does not involve confabulation. If she endorses this argument, then, in the future, offering this justification will not involve confabulation either if the argument plays some role in her continuing to make such judgments.8 As we’ve seen, representations in the MTL system are consciously accessible while affective processing often functions unconsciously. This suggests that the MTL system plays a larger role in offering justifications and that the extent to which a judgment derives from this system indicates the extent to which a justification is likely to reflect the processes that actually generated that judgment. Some justifications may be purely a result of confabulation while in other cases the actual content of processing that led to the judgment may be accessible during justification. I think the right thing to say about the relationship between judgment and justification is that it varies both depending on the kind of case under judgment and across individuals. Confabulation may provide a plausible justification insofar as people are able to recover the salient difference(s) between two cases.9 For example, when asked to justify a pair of judgments according to which acts that cause harm are judged less permissible than omissions that cause harm, people may recognize the difference after the fact and offer it as a justification for their judgments. This justification may be plausible, but we should still recognize that it does involve confabulation. However, this may not indicate that something like the act/omission principle played a role in generating these judgments. In other cases, for example in Haidt’s moral dumbfounding cases (§3.3) and in Trolley/Footbridge pairs, many people fail to be able to offer any justification. While some individuals with philosophical training may explicitly apply the DDE and thus be able to offer it as a justification, it appears that in many of these cases, justification will involve confabulation. We have seen how the same judgment may derive from either the MTL system or from affective processing and that the content of these processes is not

8 I return to this issue in Chapter 10 (§10.5.2). 9 This is an example of the availability heuristic (see, for example, Kahneman 2003).

206 the same. The content of affective processing may not be the kind of thing that people would (upon reflection) endorse as actually justifying their judgments. There are other cases in which might be thought to provide a reliable guide to the cause of one’s judgment. This may be so if a judgment results from the explicit application of utilitarian principles. Similarly, in cases that involve simulation, introspection may also provide a reliable guide to the etiology of one’s judgment. For example, when thinking about the Ecologists case (§3.4.1), an individual may consider the fact that, if no action is taken, everyone will die anyway. As just mentioned, high working memory capacity is correlated with permissibility judgments in these cases, providing some support for the fact that conscious appreciation of this consideration and the application of cognitive control (likely subserved by the DLPFC) may lead to judgments of this kind and that the content of this processing may be accessible in justification. Thus, as with many other aspects of moral psychology, the relation between judgment and justification on this account is complicated by significant variation across cases and among individuals. This helps to explain the mixed results on this issue obtained by Cushman et al. (2006) and Hauser et al. (2007).

9.4 Taking stock: linguisticism I have tried to make a positive case for an account of moral psychology according to which (1) there are multiple kinds of representations that contribute to moral judgment, each of which have different properties and involve different kinds of content, (2) in some cases, associative, affective processing is responsible for certain patterns of moral judgment; (3) to the extent that (2) is true, justification will tend toward confabulation, and (4) there are important individual differences and developmental patterns. I hope that the picture I have sketched is both general enough to give the reader a sense of what an adequate theory of moral psychology would look like and at the same time specific enough for the reader to have some faith that this account is on the right track. If it is on the right track, it only underscores the necessity of the continued use of carefully controlled empirical investigation to sort out the complexity of moral psychology.

207 Here I want to return to the 10 claims attributed to linguisticism and state where we stand in light of the foregoing arguments. I repeat these claims here (for references, see §4.2):

(1) Moral processing and judgment are analogous to linguistic processing and judgment. (2) Moral psychology can adopt many of the methods and concepts of linguistics. (3) There is a specialized, domain-specific moral faculty, mechanism or organ. (4) There is a moral grammar or universal moral grammar. (5) Some forms of moral knowledge are innate (and can be so shown by a Poverty of Stimulus argument). (6) Moral judgment derives from causal-intentional psychology (7) The notion of moral competence plays a central role in moral psychology. (8) Moral cognition depends more on the computational structure than on the underlying neuropsychological mechanisms. (9) Emotion plays no (synchronic) role in moral judgment. (10) The primary function of emotion in moral psychology is in connecting moral judgment to action and the (diachronic) role of emotion in moral development is in attention modulation.

In the previous chapter, (§8.1) I offered some reasons to reject claims (3), (4) and (5), discussing how work that emphasizes perceptual tuning and scaffolded learning offers to account for results that suggest the existence of innate, domain-specific mechanisms. These considerations were not conclusive, but they do gain further support from the more thoroughly argued for claims about the importance of affective learning in moral psychology and the existence of multiple kinds of representations that contribute to moral judgment. Almost all of these representations were domain-general in nature. Of course, I did admit the existence of some innate representations in the form of core cognition and I admitted that there may be somewhat domain-specific neural mechanisms for representing the mental states of others. Nevertheless, I believe that the role of innate knowledge in moral judgment will be limited and that there is little evidence of any moral

208 domain-specific mechanisms along the lines of a moral grammar. This conclusion is supported both by neuroimaging results (Shenhav and Greene 2010) and research in cognitive psychology demonstrating that moral judgment is sensitive to the same kinds of biases as in other domains (Rai and Holyoak 2010; see also Tversky and Kahneman 1981). For similar reasons, above (§9.1) I provided some arguments against claim (8) (computationalism). While not conclusive, I drew on my explanation of the Trolley/Footbridge asymmetry to indicate how an understanding of this phenomenon requires an understanding of the underlying neural mechanisms. I hold that more detailed explanations of other aspects of moral psychology will further undermine computationalism. I think I have made a presumptive case against computationalism, though a definitive evaluation of it will depend on filling in more of the empirical details. In Chapter 7, I discussed in detail the nature of causal representations. I argued that several processes contribute to such representations, but that among them the basal ganglia system was especially important. I extended this idea in Chapter 8 (§8.1.3), discussing the role that causal representations play in certain kinds of moral judgments. However, as I pointed out (§8.1), linguisticism seems to construe causal cognition as propositional and the processing of the basal ganglia system does not involve propositional representations. Thus, this cuts against one interpretation of claim (6). Again, full evaluation of this claim must await further research on the extent to which nonpropositional, associative causal cognition explains other aspects of moral judgment, but I think there are good reasons to believe that further research will confirm this view. To reiterate the conclusion of Chapters 5 and 6, contra claims (9) and (10), we saw that affective processes play a role in moral judgments which involve dynamic evaluation in response to simulation and that affective learning plays a crucial role in moral development. These conclusions are underscored by the discussion of affect in this chapter (§9.2.2). Thus, I think we can confidently reject both these claims. I now turn to claim (7), that competence plays a central role in moral psychology. At first there is some difficulty getting a handle on this notion—Chomsky makes use of the notion as part of three different distinctions (Hacken 2007, 41-53). However, I think we can understand it well enough to make a preliminary assessment of this claim.

209 Edward Stein (1996, 51-7) distinguishes three different notions of competence, which he labels the idealization, mechanism, and knowledge views of competence. On these views, competence is, respectively, “linguistic behavior under ideal circumstances,” “the unfettered operation of the language organ,” or “a person’s knowledge of language” (E. Stein 1996, 53, 55). If we apply these distinctions to moral psychology, I think we will see that the importance of the notion of ‘competence’ stands or falls with claims (3) and (4). I have little doubt that ‘competence’ may sometimes play a role in a useful idealizations. I myself have made use of a similar notion in speaking non-formally of ‘normal’ individuals. This does not, however, make competence (or ‘normality’ in this sense) central to moral psychology. Thus, I think proponents of linguisticism have a more substantive notion of competence in mind, connected either to the idea of a moral mechanism or a moral grammar (and the knowledge represented therein). However, as we’ve seen there is reason to doubt the existence of both a domain- specific moral mechanism and a (universal) moral grammar. If the account of moral psychology that I have been sketching is correct, then there is no distinct moral mechanism nor any body of moral knowledge (e.g. moral principles) that all individuals share. There are multiple, somewhat redundant kinds of representations that contribute to moral judgment. This is complicated by significant individual differences. The complexity of moral psychology suggests that the notion of competence will not, in fact, be central to moral psychology. This leaves claims (1) and (2). In light of the arguments (some more conclusive than others) offered against the other elements of linguisticism, I think it is reasonable to be skeptical of the claim that there is a substantive analogy between linguistic processing and moral processing and that the methods and concepts of linguistic theory will be of great use in the study of moral psychology.10 On the basis of all the considerations

10 This statement should be qualified. In evaluating linguisticism I’ve assumed a certain view of linguistic processing that proponents of linguisticism themselves endorse. This is a roughly Chomskyan view (see, for example, Dwyer 2009; Hauser 2006b). If this view of linguistic processing turns out to be mistaken, then the grounds for analogy may be stronger. That would not, however, be a vindication of linguisticism as I have discussed it here.

210 discussed above, I think there are excellent reasons for rejecting linguisticism as I have construed it and for accepting a view of moral psychology along the lines that I have sketched.

9.5 Conclusion Thus concludes my discussion of the psychological mechanisms underlying moral judgment. In the next chapter, I return to issues in philosophical moral psychology. I address whether an improved understanding of these psychological mechanisms sheds any light on these issues.

211

CHAPTER 10

UNTANGLING MORAL PSYCHOLOGY

I began this dissertation with the hope that an examination of empirical research on moral judgment might be of use when addressing philosophical issues in moral psychology. In the intervening chapters I’ve reviewed a wealth of empirical research. This work largely vindicates the view Michael Stocker took of these issues over 30 years ago:

…motivation and evaluation do not stand in a simple and direct relation to each other, as so often supposed. Rather, they are interrelated in various and complex ways, and their interrelations are mediated by large arrays of complex psychic structures, such as mood, energy, and interest. Philosophical theories have ignored or misunderstood these structures and the corresponding all too common psychological phenomena. They have depicted the psyche, especially the interrelations between motivation and evaluation, as far too simple, far too unified, and far too rational. (1979)

In this chapter I return to these philosophical issues and discuss the ways the account of moral psychology developed in the previous chapters can contribute to disputes in philosophical moral psychology.

10.1 The moral problem Much of recent work in moral psychology has come out of the recognition that three independently plausible views are mutually incompatible (McNaughton 1988; M. Smith 1994):

212 I) The Humean Theory of Motivation: Belief and desire are distinct kinds of states and motivation requires both a desire and a relevant means-end belief.

II) Motivation internalism: There is an internal connection between moral judgment and motivation, such that moral judgments necessitate motivation to act in accordance with that judgment.

III) Moral : Moral judgments express beliefs that can be true or false.

One way of resolving this incompatibility is to reject one or another of these claims. Thus, noncognitivists reject (III), cognitivist internalists reject (I), and realist externalists reject (II). Others try to resolve this puzzle by modifying one or more of these claims. For example, Smith (1994) offers a qualified version of (II) and attempts to square it with (I) and (III). Two things should be noted. First, in rejecting one of these claims it would be nice to provide some explanation for its originally appearing plausible. Second, if we are to qualify one of these claims, we need to beware that we do not trivialize it in so doing. I believe that this is a helpful starting point, though ultimately, I will argue that each of these views, though plausible, requires revision or qualification. In §10.2 I offer an argument against motivation internalism. In the following section (§10.3) I develop a problem for Humean theories of motivation. In §10.4 I offer a resolution of these issues and in the process address noncognitivism. Finally, in §10.5 I consider two objections.

10.2 A neuropsychological argument against motivation internalism In this section I develop an argument against versions of (II): motivation internalism. I begin by characterizing the claim to be attacked. After briefly mentioning some previous challenges to this claim, I offer an argument whose premises draw on the empirical work discussed in earlier chapters. I then consider more restricted versions of motivation internalism.

213 10.2.1 Characterizing motivation internalism In this section I focus on the target of my argument. I do not claim that this is the only interesting internalist thesis, but I do give reasons for thinking it is the most interesting version in the vicinity. We can begin with the following characterization (Shafer-Landau 2003, 143):

(1) Necessarily, if an agent, A, makes a sincere moral judgment, A is motivated to act in accordance with it.

By way of refining this formulation, there are a few things to note. First, the condition that the judgment be sincere is intended to rule out cases of pretense or cases in which someone makes a judgment in an ‘inverted commas’ sense. I discuss cases like this in the next section (§10.2.2), but it may be thought that these cases do not involve genuine moral judgment and so the sincerity condition is superfluous. Emphasizing the sincerity of a moral judgment seems more like table pounding than a substantive component of an internalist thesis. Second, we should note that this motivation need not triumph. An agent may fail to act in accordance with her moral judgment, but there must be some motivation to do so. Third, formulations of internalism are sometimes put in terms of conceptual necessity. A recent formulation by Michael Smith is: “It is conceptually necessary that if an agent judges that she morally ought to φ in circumstances C, then either she is motivated to φ in C or she is practically irrational” (2008, 211). I resist these formulations. I am primarily interested in moral judgment and motivation, rather than the concept MORAL JUDGMENT. Thus, the relevant sense of necessity here is metaphysical necessity, not conceptual necessity. This claim can be understood as saying that there are no possible worlds in which an agent makes a moral judgment and is not motivated to act in accordance with it. Fourth, the formulation in (1) does not address the nature of the connection between moral judgment and motivation. As Mele (2003, 109; see also C.B. Miller 2007) points out, such formulations are consistent with the necessary connection being provided by features external to the judgment. Thus, they fail to address whether motivation is

214 “built into” moral judgments (Frankena 1958, 41). In a real sense, formulations along the lines of (1) are not formulations of internalism at all. We can offer a revised formulation:

(MI) It is metaphysically necessary that, for any agent A and action p, if she judges that she morally ought to p, then she has some motivation to p that derives solely from the judgment that she morally ought to p.

MI is the formulation of internalism I will focus on here. Externalism is the denial of this claim. One further distinction: MI is unrestricted (C.B. Miller 2007, 234); it applies to any agent insofar as she is able to make a moral judgment. Later, I will discuss restricted formulations of internalism (§10.2.4).

10.2.2 Telling stories: for and against internalism One of the most popular kinds of argument against internalism involves telling a story about an agent who sincerely makes a moral judgment and yet seems to have no motivation to act in accordance with that judgment. David Brink asserts that an amoralist is conceivable: “someone who recognizes certain considerations as moral considerations and yet remains unmoved by them” (1989, 27). Alfred Mele tells a story about an agent suffering from clinical depression who “retains certain of her beliefs about what she is morally required to do,” but nevertheless has no motivation to continue looking after a sick uncle—though she continues to believe that she morally ought to (1996, 733). Russ Shafer-Landau offers three additional stories about agents who, because of perceived futility, cowardice, or commitment to , endorse moral claims they have no motivation to act in accordance with (2003, 149-50).1 Having introduced these stories, this style of argument proceeds to claim that, if these stories are coherent—and they

1 Sigrun Svavarsdottir introduces a case about Patrick, an agent who recognizes an obligation to help a stranger, yet callously remains unmoved (Svavarsdottir 1999, 176- 77). She uses this case in a slightly different way than I consider in the text. She argues that the internalist must deny a potential explanation of Patrick’s story that is otherwise intelligible and thus that “the burden of argument is on those who insist on a more restrictive class of potential explanations” (2006, 165). I leave this argument to one side.

215 certainly seem so—then they represent genuine counter examples to internalism, for they show it is possible to make a sincere moral judgment without any corresponding motivation. A common response to this kind of argument is to deny that such cases are coherent or accurately described. R.M. Hare famously drew attention to the “inverted- commas use” of moral terms (1952, 124). Some have argued that the amoralist should be interpreted in this way, as making moral judgments in the inverted-commas sense (M. Smith 1994, 166-71). If so, these stories would pose no problems for internalism. This reply may be more or less plausible in different cases, but it bears pointing out that the opponent of internalism need only find one case for which this strategy fails. It seems implausible that every purported counterexample can be handled in this way.2 An alternative criticism of internalism focuses not on hypothetical counterexamples, but on actual counterexamples. Adina Roskies has argued that individuals with damage to the ventromedial prefrontal cortex (vmPFC) exhibit normal declarative knowledge and reasoning in the moral domain, yet often lack motivation in “ethically charged situations” (2003, 57). Her arguments have sparked some debate.3 It strikes me that both parties in this dispute have been telling stories about how things work when someone makes a moral judgment. Without some further reason to believe one of the stories or another, I say they have merely been telling stories. I want to offer an argument that sidesteps the issue of counterexamples.

10.2.3 A new argument against internalism As we have seen, there are distinct neuropsychogical bases for moral judgment. ‘Cognitive’ representations in declarative memory can give rise to moral judgments. This is dramatically illustrated in individuals with affective dysfunction in conditions like psychopathy and following vmPFC damage. However, we have seen this in

2 There are, of course, other responses available to the internalist. The argument offered below (§10.2.3), however, does not depend on counterexamples, so I leave the issue of internalist responses to these counterexamples to one side. 3 See, for example, the exchange between Roskies (2008), Michael Smith (2008), and Jeanette Kennett and Cordelia Fine (2008) in volume 3 of Sinnott-Armstrong (2008). I discuss some of the same issues that arise there. However, I do so from a different argumentative perspective.

216 nonpathological individuals as well (§5.5). We have also seen that affective representations can give rise to moral judgments. We saw dramatic illustrations of this in experiments involving the manipulation of disgust (e.g. by hypnosis or novelty fart spray). Neuroimaging evidence indicates the role of affective processing even in the absence of such manipulations. The neuropsychological bases of these representations are distinct. Declarative memory is subserved by the medial temporal lobe, including the hippocampus, while affective representations involve processing in broad range interconnected brain regions including the amygdala, vmPFC/OFC, mPFC/ACC and areas of the basal ganglia. Moreover, only affective representations are directly linked to motivation. We saw that the influence of explicit beliefs on behavior was mediated by prefrontal activation modulating such affective representations of value (§6.2.2) and that such integration was often disrupted in patients with vmPFC damage. Thus, since moral judgments can derive from processes not directly linked to motivation, there is no metaphysically necessary connection between moral judgment and motivation. We can put the argument a bit more formally as follows:

1) Both affective representations and ‘cognitive’ representations are sufficient for moral judgment. 2) Only affective representations are directly linked to motivation. 3) Moral judgment can derive from processes not directly linked to motivation. 4) Therefore, there is no metaphysically necessary connection between moral judgment and motivation.

This formulation requires a bit of spelling out with respect to how I mean to use the terms ‘sufficient’, ‘derive’, and the notion of ‘directly linked’. The nature of processing in the brain can often make it tricky to interpret claims about sufficiency or linkage. In his discussion of neural regions sufficient for the experience of pleasure, Schroeder warns: “After all, no part of the brain is sufficient for an experience of pleasure if surgically removed from the organism and stimulated in vitro” (2004, 77). Similarly, we must be careful when speaking of mental representations

217 instantiated by neural processes being sufficient for a judgment. The idea behind (1) is simply that, given a certain representation, nothing besides the mechanisms that facilitate expression are necessary. Thus, (1) claims that there are multiple processes that provide sufficient input to the process of forming a moral judgment. To take a simple example, judgments about whether a fruit is present may derive from perceptions in various sense modalities (e.g. smell, taste, vision), given any such perceptions, nothing more than what is involved in making the judgment itself is required. When such a representation serves as input to a processes of judgment formation, we can speak of the judgment as deriving from that representational process. A similar idea stands behind the idea of being ‘directly linked’. Given the intricate connections in the brain, things can be more or less directly linked. The sense in which affective representations are directly linked to motivation is that these representations project by direct connections to premotor regions that ultimately give rise to behavior. Thus, these regions are a crucial part of a common path to action. Representations in declarative memory must be mediated by prefrontal activity in order to influence processes projecting to these premotor regions. Lastly, by way of discussing a possible objection, I want to clarify the role played by empirical evidence in this argument. An internalist may insist that, for all I have said, it is possible that cognitive processes, which are not directly linked to motivation, can necessitate motivation.4 From the outset this objection seems weak because it claims only that it is possible that motivation can arise from cognitive processes not directly linked to motivation. Internalism is a much stronger claim, viz. that in cases where moral judgment derives from such cognitive processes it is necessary that these cognitive processes not

4 The spirit of this objection derives from Mele’s discussion of what he calls the Cognitive Engine Theory (2003, 92-101). The target of that discussion is whether such cognitive engine theorists can support the claim that “in ordinary circumstances, justificatory reasoning produces motivation in actual human agents independently of antecedent motivation” (Mele 2003, 96). Mele argues that such a view is at best psychologically implausible. In the text I develop a response along these same lines by challenging theorists sympathetic to the above claim to meet my argument on its own terms by providing more than a merely skeptical response to the empirical research I review.

218 directly linked to motivation give rise to motivation. Leaving this point aside, there is a more general response. The strategy I have adopted here is to review a large body of scientific research and to use this to support certain general claims about the neuropsychological structure of the mechanisms underlying moral judgment and motivation. And from these claims to draw the conclusion that internalism is false. Of course, conclusions derived from empirical research are not deductively valid. Thus, arguments that depend upon conclusions so derived are always subject to skeptical concerns or alternative interpretations. However, it looks like the internalist objection here is merely skeptical. The internalist claims, for all that you have argued it is possible that cognitive processes not directly linked to motivation can necessitate motivation. Right. Here is another hypothesis that is possible given everything that I know: Barack Obama is really a Martian marionette connected by cleverly disguised radio signals to a supercomputer on Mars. Without some independent evidence for the hypothesis, I am entitled to treat the internalist objection as akin to the Martian marionette hypothesis, that is, as merely a skeptical hypothesis or an unsubstantiated alternative. Moreover, substantiating the criticism requires not only providing some independent motivation for thinking that cognitive processes not directly linked to motivation can necessitate motivation, but also doing so in a way that explains why none of the research I have reviewed provides any support for this view. This requires serious engagement with the empirical literature. Until internalists have attempted to meet this challenge, this objection can be dismissed as merely skeptical. This argument has several advantages over other arguments against internalism. First, while it draws on psychopathological evidence to support the premises, these premises are about the structure of moral processing in normal individuals. Thus, the argument does not depend on contested claims about whether or not, for example, psychopaths or vmPFC patients are moral agents. Moreover, it does not depend on the interpretation of actual or hypothetical counterexamples. Obviously the conclusion entails that such counterexamples are possible, but it does not rely on either the details of such cases or our responses to such

219 cases. Indeed, it is logically consistent with the conclusion that in the actual world, moral judgments are always accompanied by some motivation to act in accordance with them. I think this last claim is in fact false. For illustrative purposes, it may be helpful to introduce such a case. (Skeptical readers may construe the case in a possible world howsoever nearby they choose.) An easily imagined case involves a reasoned change in view. Above (§9.2.3), I have discussed cases of moral vegetarianism. I argued that affective responses often follow from changes in explicit beliefs. For example, imagine an agent Petra, after reading the arguments of Peter Singer, becomes convinced of the of meat eating. Immediately after conversion she will endorse various moral judgments and yet she may feel no motivation to act in accordance with them. The development of corresponding affective responses may take some time and effort to establish. It may take a significant amount of self control on Petra’s part to act in accordance with her judgments. It may be maintained that, in this case, Petra surely will have some motivation to act in accordance with her judgment, even if only because she is motivated to exercise self control in order to act in accordance with her considered judgments.5 There are several things to say in response to this kind of concern. First, even granting that Petra has some motivation in this case, it does not follow (except perhaps by stipulation) that this will be so in all cases. It may be argued that making a moral judgment requires that one is motivated to bring it about that one has some motivation to act in accordance with this judgment. Petra’s failing to bring it about that she has motivation to act in accordance with her moral judgment could then be taken to undermine the claim that she has made a sincere judgment. Bracketing the concern discussed above (§10.2.1) about the use of ‘sincere’ in this context, this response still misses the mark. The motivation to act in accordance with the judgment would fail to derive solely from the judgment itself. On the proposed view, the only motivation that would follow from the judgment alone would be the motivation to bring it about that one has motivation to act in accordance with the judgment. This is simply a different motivation from that specified by MI.

5 This worry has been most forcefully pushed by Piers Rawling (personal communication).

220 Moreover, there are good reasons to resist such a view. Petra may have good reasons not to exercise self control. Suppose Petra’s reading of Peter Singer’s arguments happens to occur on her way to visit her aging grandparents who have prepared a delicious baked ham for her visit. Her abstinence from the meal would be devastating to them. Thus, Petra may have very good reasons not to exercise self control in order to bring it about that she has motivation to act in accordance with her judgment until after her familial visit. We should not doubt the sincerity of her moral judgment on that account. While she might still have some motivation to bring it about that she has motivation to act in accordance with her judgment, nothing guarantees that she will actually have motivation to act in accordance with her judgment.

10.2.4 Qualifying internalism The line of discussion in the previous section suggests a way of qualifying the internalist thesis. Perhaps a ‘real’ moral judgment ought to require both an explicit representation and the corresponding affective response. This qualification appears to be somewhat ad hoc. If an independent reason for this qualification can be convincingly developed, there are two reasons to resist it nonetheless. First, just as Petra had good reasons not to exercise self control in bringing about motivation to act in accordance with her judgment until after her family dinner, there may be general reasons not to do so. Suppose that Stan has become convinced of the truth of some form of utilitarianism. As a result, he makes a set of moral judgments, for example: “I am morally obligated to send most of my income to the distant needy,” or “In the event that a runaway trolley will inevitably kill five innocent railway workers and a large man is around that could be pushed in front of the trolley in order save the workers, it is morally required to do so.” After conversion, Stan may find himself with no motivation to act in accordance with this latter judgment. He also knows that in the event that he ever finds himself in such a situation he will not be able to summon the self control to act quickly enough to perform the required action. He knows his only option is to engage in a long process of modifying his affective responses so that, in the unlikely event that he ever meets up with a runaway trolley while strolling across a footbridge with a large man he will be able to immediately

221 act to save the five workers. It seems terribly implausible to require such a condition on Stan in order that his judgment in the footbridge case to count as a ‘real’ moral judgment. A second consideration also speaks against such a strong condition on ‘real’ moral judgment. We might (rightly) think that many normal children have a natural aversion to harming animals. Children may be so disinclined to cause suffering in animals despite having no explicit beliefs about the matter. They may nevertheless affirm that so acting is morally right. There seems no good reason to deny that this child has succeeded in making a moral judgment in this case. I think there are similar cases in adults.6 In many of our interactions with close friends and partners, we immediately perceive the morally relevant features of the situation. For example, a close friend’s tone of voice or facial expression may communicate that they are feeling vulnerable or disheartened. Without thought we may immediately act to offer security or comfort. While, upon reflection, we might come to explicitly formulate the relevant moral principle and thereby acquire a representation in declarative memory with that content, such a process should not be required for us to sincerely judge that, in such a case, we did the right thing, even if we remain unable to say precisely wherein the rightness consisted. Thus, I think we have good reason to resist this qualification of the internalist thesis; moral judgment should not require that explicit beliefs and affective responses necessarily be coordinated in every case. There may be other ways of trying to qualify MI. Michael Smith, for example, offers the following characterization of internalism: “If an agent judges it right to do something then she is motivated accordingly, at least absent practical irrationality” (2004, 285). We may consider a restricted version of MI that applies only to practically rational agents:

(MI-R) It is metaphysically necessary that, for any practically rational agent A and action p, if she judges that she morally ought to p, then she has some motivation to p that derives solely from the judgment that she morally ought to p.

6 The following case is inspired by a case used by Josh Shepherd (2009) in a discussion of a distinct issue.

222

The most important point to make about any restricted formulation of internalism is that, to avoid trivializing the view, the proponent must provide an independent account of the restricted class. For example, Smith must (and does) provide an independent specification of practical rationality. I think restrictions of this kind are subject to the matching problem. At base the problem derives from the complexity of moral psychology itself, and especially the ways in which explicit beliefs and affective responses have been shown to come apart. As I have emphasized (e.g. §9.3) such dissociations are not limited to pathological cases, but are quite common. Significant individual differences reveal that moral judgment depends on affect and ‘cognition’ to different extents across individuals. The matching problem arises in the following way. Given the complexity of human moral psychology, it is overwhelmingly implausible that any account of practical rationality will succeed in categorizing as irrational all the cases in which affective responses and explicit beliefs come apart. There seems to be a mismatch between the grain of accounts of practical rationality and the grain at which we must explain cases in which a moral judgment is not accompanied by the appropriate motivation. I think the matching problem generalizes to any proposed restriction that can be independently specified, e.g. normal agents, virtuous agents, etc. (see C.B. Miller 2007 for further discussion of various ways of restricting the internalist thesis). If we relax this last condition to allow somewhat jury-rigged restrictions to the internalist thesis, we find an equally devastating problem: these restricted formulations perniciously trivialize the thesis of internalism. Suppose, following the qualification suggested at the beginning of this section, we simply define virtuous agents as those agents in which moral judgment derives from both an explicit representation and a corresponding affective response. While this allows the restricted internalist thesis, it does great violence to the notion of virtue. The virtuous agent is stuck engaging in precisely the kind of pointless training with which Stan is saddled and is required to engage in endless reflection making explicit all the moral commitments that constitute normal affective responses. This account makes the virtuous agent into a kind of

223 overintellectualizing fetishist. This is neither a desirable ideal nor a good account of the virtuous agent.

10.2.5 Explaining the connection between moral judgment and motivation In arguing against internalism, it would be nice to offer a diagnosis of the plausibility of the claim. I do so here in part by explaining the connection between motivation and moral judgment. I take up this issue again later (§10.4). One thing to note is that on the account I have developed, the relation between moral judgment and motivation can be very complicated. This raises a problem for internalist views which hold that the relation is simple or straightforward. My account allows that sometimes the connection between moral judgment and motivation is quite similar to internalist explanations. Some moral judgments do derive from affective processes that have direct motivational connections. Additionally, as we have seen, affective mechanisms are often engaged in parallel with more explicit cognition. Thus, affective responses and explicit beliefs may be sufficiently coordinated that a moral judgment that depends on such explicit beliefs may be accompanied by the appropriate motivation. Throughout the process of moral socialization, children acquire both affective representations connected to motivation as well as declarative moral knowledge. One way this works is through explicit generalizations based on these affective responses. This is a kind of bootstrapping of declarative moral knowledge. This happens in the same way that, for example, we might learn that fruit can be refreshing on a hot day. We are not explicitly taught this, but after enough experience with fruit on hot days, we come to have beliefs of this kind. Another way that we acquire new declarative moral knowledge is through explicit instruction in moral rules, for example, as part of parental discipline.7 When agents mature they may come to new declarative moral knowledge through reason, argument and discussion. This sheds light on the plausibility of internalist views. John McDowell, for example, has argued that the virtuous agent sees actions as required as a result of perceiving the situation she is in. He writes: “a conception of how things are suffices on

7 For more on the role of discipline in moral socialization, see work by Hoffman (1990; 1994; 2000), Kochanska (1995; 1997) and Kochanska et al. (2007).

224 its own to show us the favourable light in which the action appeared” (1978, 22). Surely this is account is sometimes correct. We do say things like “I just saw that the stranger needed help and I acted.” Moral action need not always be mediated by moral judgment. Sometimes our affective mechanisms respond immediately to our environment. However, when I think of my promise to attend a curriculum committee meeting this afternoon, I have no experience of the sparkling glory of doing my duty. I think of the two hours of agonizing monotony and how to get up the motivation to go. Moreover, McDowell says: “In moral upbringing what one learns is not to behave on conformity with rules of conduct, but to see situations in a special light, as constituting reasons for action” (1978, 21). This is a central part of human moral experience. McDowell’s mistake is to generalize from this central case to all cases. On the other hand, externalist accounts that endorse a broadly Humean account of moral psychology, often have difficulty accommodating this aspect of moral experience.

10.2.6 Fetishism McDowell’s comment about the virtuous agent raises the issue of the connection between moral judgment and motivation in the exemplary agent. Michael Smith has argued for internalism on the basis that externalist views cannot offer a plausible account of the connection in the “good and strong willed person” (M. Smith 1994, 72). It is worth considering how the view sketched above can respond to this challenge. Smith asks, when the “good and strong-willed” person undergoes a reasoned change in moral view, what explains her change in motivation (M. Smith 1994, 72)? On the internalist view, things are simple: a corresponding motivation is the direct result of the newly formed belief. On externalist views, Smith argues, the only explanation for the reliable connection between judgment and motivation is a generic desire to do the right thing, whatever that is (i.e. a desire de dicto, rather than de re). This, according to Smith, makes moral virtue a fetish. Good people care about, for example, the welfare of their friends and family for their own sake. But the externalist has it that the good and strong- willed person’s motivations come about, not because of a “non-derivative concern” for her family’s welfare, but due to a generic desire to do the right thing (M. Smith 1994, 74).

225 The view sketched here has the resources to offer a different answer.8 What explains the connection between moral judgment and motivation in the good and strong- willed person? I say: it depends on how far removed the new resolution is from one’s current set of motivations. For example, suppose I believe it is wrong to eat meat. Then, while reading PETA literature, I become convinced that it is wrong to eat eggs and dairy as well. What explains the connection between this new judgment and motivation is that I now see that the motivations that I already have apply in this new case. Suppose, on the other hand, that I am a committed omnivore. If I become convinced that eating meat is wrong, it is unlikely that a motivation to act accordingly will immediately arise. It will take some effort, the exercise of a certain amount of self- control to be motivated and ultimately to act in accordance with my new moral judgment. Recall the research discussed earlier (§9.2.3) indicating that, in moral vegetarians, changes in affective response are a result, rather than a cause, of a change in beliefs regarding the moral status of eating meat. Smith stipulates that the case involves a good and strong-willed person. This seems to involve two distinct assumptions. Yet, the internalist answer he offers depends only on the agent being strong-willed (1994, 72). We may ask what contribution these stipulations are supposed to make, since on Smith’s account the motivation to act in accordance with a newly formed moral judgment will follow in any moral agent. Smith is perhaps attempting to appeal to externalists who will require these assumptions in order to accept the plausibility of the case. However, this stipulation obscures the issue of explaining the complexity of the relation between moral judgment and motivation in the agent who is less than ideal. The internalist account stumbles over precisely these cases. At the same time, Smith saddles his opponent with the view that being good is a matter of having a generic desire to do the right thing (1994, 74). He fails to consider an alternative account of the connection between moral judgment and motivation in the good

8 Mele (2003, 123-5) offers a similar response to Smith’s problem of fetishism from the perspective of a broadly Humean account. He argues that, for an agent who is not “morally perfect,” a generic desire to do what she morally ought to do can provide motivation to fulfill her obligations, when intrinsic desires do not provide sufficient motivation (2003, 125). He also argues that a morally perfect agent may believe that he is morally required to perform an action and do so, without “acting even partly on the basis of” that belief (2003, 124). See, also Dreier (2000) for further discussion.

226 and strong-willed agent along the following lines. First, the agent is good and so already possesses a host of motivations such that any newly formed moral belief is likely to already find a motivation that applies. Second, the agent is strong-willed, so in cases where a motivation to act in accordance with a new belief is not already present, she can successfully exercise self-control in bringing about motivation to act accordingly. The internalist’s explanation is far too simple, even for the good and strong-willed person. On this account, the emergence of motivation seems almost magical. This does not seem to fit our experience at all. The farther a new resolution is from our current set of motivations, the harder it is likely to be and the more likely we are to fail. My account offers an explanation for the difficulties we face in generating motivation to act in accordance with our moral judgments. We often find ourselves in situations where we are tempted to act contrary to our moral beliefs. In these cases, we may sometimes fall back on a generic motivation to do the right thing or to be a good person. Blindly following rules because they are the rules is moral fetishism. Using our commitment to general principles to motivate us to do what we believe is right is an important strategy we have for motivating ourselves to act in accordance with our moral judgments. The cases that philosophers have offered as counterexamples to internalism all involve instances in which motivation and moral judgment may come apart: cowardice, callousness, affective disorders, theoretical commitments, and so on. My account explains how it is that moral judgment and motivation can come apart in cases like these. Thus, I conclude that the unrestricted internalist claim (MI) is false and that proposed restrictions of the thesis run up against the matching problem or concerns about trivialization. As we will see in the next section, the debate over internalism is closely related to debates about Humean theories of motivation.

10.3 A neuropsychological problem for Humean theories of motivation There are many ways of developing a theory of motivation along the lines of that offered by Hume.9 Thus, I begin by stating the view that is the focus of discussion here

9 Though, there is some question as to how much Hume himself would have endorsed Humean views (for some discussion, see Railton 2005).

227 (§10.3.1). In the following sections (§10.3.2), I raise a problem for Humean theories of motivation and extend this discussion by raising an issue which challenges both Humean and anti-Humean theories (§10.3.3).

10.3.1 Characterizing Humean theories of motivation We can distinguish two central components of Humean theories of motivation.10 First, beliefs and desires are distinct kinds of state. Beliefs are cognitive, in the sense that they aim to represent how the world is and can be true or false. Desires, on the other hand, are conative states, motivating action. Desires provide the motivational impetus for action while beliefs provide information about the means to satisfy them. As David McNaughton put it: “Desires without beliefs are blind; beliefs without desires are inert” (1988, 22). From this, the second component of Humean theories naturally follows: motivation depends on a belief-desire pair. For example, my going to the pub yesterday evening depended on a desire to have a good beer and a belief that by going to the pub I could purchase a good beer. Moreover, belief on its own is not sufficient to produce behavior and so reason will always be slave to the passions. Our satisfying our desires may be guided by our beliefs, using reason to determine the best means to such satisfaction, but without desire, reason is impotent. Anti-Humean theories reject this view of motivation. Some reject the view that desires are rationally inscrutable (e.g. M. Smith 1994). Others hold that belief alone is sometimes sufficient to motivate action (e.g. T. Nagel 1970; McDowell 1978; McNaughton 1988). Such theorists sometimes claim that, when we see that someone is motivated in a certain way, we ascribe a relevant desire to them. However, this does not show that the desire itself played a role in producing motivation. The motivation might just as well have been produced by belief alone. How should we see Humean theories in the light of the account of moral psychology offered above? We might think that this account provides some support for Humean theories. Let us begin with belief. The decoupled representations of the MTL system fit naturally with the Humean’s account of belief. These are consciously

10 For further discussion, see Smith (1994, chap. 4) and McNaughton (1988, 20-2).

228 accessible and many of them have propositional format (though there are more map-like representations whose representational format involves a more spatial isomorphism to the world). Additionally, these representations aim to represent the world and lack close connections to motivation. Thus, these representations seem to closely fit with the account of beliefs provided by the Humean theory. What about desires? Timothy Schroeder (2004) draws on neuropsychological research to critique standard theories of desire and to argue that desire is best captured in terms of reward processing. On this view, he explains: “To have an intrinsic desire that P is to use the capacity to perceptually or cognitively represent that P to constitute P as a reward” (2004, 131). Schroeder’s arguments for this account of desire nicely complement the discussion of these reward processing mechanisms offered here (see §6.2 and §9.2.2). Core affective mechanisms are largely responsible for the processing of reward. And, as we have seen, these mechanisms are directly linked to motivation. There is, however, one problem with Schroeder’s account. As Paul Thagard (2006) has pointed out, on Schroeder’s view desires are propositional attitudes. But, the reward systems Schroeder discusses are largely preserved across animal species and it is very unlikely that many nonhuman animals have the capacity to represent propositions. Even in humans, it can be awkward to paraphrase desires in propositional terms. For example, my desire for a good beer must be paraphrased as follows to fit into the propositional attitude account: Chris desires that he have a good beer. The problem, as Thagard puts it, is that “Animals desire food, not that they should have food. Humans are usually the same” (2006, 152). We may further support this claim by considering how sometimes even articulating our desires in language does violence to them. Our language is often too specific to accurately represent the content of our desires. Some of our desires may in fact be quite specific: I may have a craving for a steak with blue cheese-ale sauce at a specific restaurant. But often our desires are much less specific: I order the chocolate-espresso layer cake, but really anything chocolaty would be equally satisfying. In the circumstances of ordering at a restaurant, the more general desire for something chocolaty gives rise to a more specific desire for the chocolate-espresso layer cake

229 because this is the most expedient way to satisfy the original desire. Nevertheless, the original desire itself did not have such a specific content. To put this another way, sometimes our language makes obligatory a fineness of grain which is simply not present in the content of the states we attempt to express in our use of language. Sometimes we may express such desires by using more abstract concepts, like CHOCOLATY, SWEET or SALTY. This should not necessarily lead us to believe that the content of the underlying affective processes themselves is propositional in format (see §9.2.2). Schroeder provides firm support for a reward theory of desire, but examination of the mechanisms underlying reward processing indicates that these representations will often not involve content in propositional format. Thus, we should resist his formulation in terms of propositional attitudes. How then should we make sense of desire? To retain the spirit of Schroeder’s account, we should simply drop this aspect of his account. Desires are simply representations that are so constituted to represent their object as a reward. This account of desire shares all the appeal of Schroeder’s account without presupposing content in a particular format (i.e. propositional). Thus, we can accept that desire, at base, depends on reward processing. It is important to draw a distinction between desire, on this account, and other kinds of representations. Recall the distinction between memory-based preferences and dynamic evaluation in response to simulation (§6.2.3). In ordering at my favorite restaurant I may rely on my memory of my favorite entrée or I may imagine different options in order to choose. As I argued, both these processes require the MTL system. However, dynamic evaluation in response to simulation involves affective mechanisms as well. Indeed, our capacity for simulation allows us to access these affective representations in the absence of environmental stimuli—even to predict our responses to things we have never experienced (cf. Gilbert and T.D. Wilson 2007). This capacity often facilitates working for unseen goals; we may find ourselves imagining the end product of our work in order to sustain the motivation to keep working. There is an important distinction here. On the one hand we have affective representations. These can be triggered by both perception and simulation and insofar as they represent things as rewards, they fit most naturally with the reward theory of desire.

230 On the other hand, we have more ‘cognitive’ states subserved by the MTL that involve remembered preferences. These representations are propositional in format and depend, to some extent, on conscious recollection. These representations may serve as the basis for modulation of affective representations of value during the exercise of self control discussed in Chapter 6 (§6.2.3). For example, when we find ourselves drawn to that double-frosted, cream-filled doughnut, we may remember the last time we indulged and how much less enjoyable it was than we had anticipated. Indeed, it is likely that experienced pleasure serves as input to these processes in accordance with Dickinson and Balleine’s (2009) “cognitive interface” hypothesis of the function of experienced pleasure (discussed in §9.2.2). To distinguish these representations from the affective representations involved in reward-processing, I will call these cognitive desires. I call these states cognitive desires to reflect the facts of common usage: our verbally reported preferences will often depend on these representations. Since we naturally think of such reports as expressing our desires, these states have some claim to be called ‘desires’. However, the relation between such cognitive desires and reward processing can come apart. Cognitive desires are less directly connected to motivation. This strains the reward theory of desire and so—just as, strictly speaking, a rubber duck is not a duck—strictly speaking, cognitive desires are not desires. On the view sketched above, cognitive desires are actually beliefs: they are subserved by the MTL system and they lack a direct connection to motivation. Cognitive desires are beliefs about what we want or like. Thus, we should resist identifying them with desires. However, since our cognitive desires very often align quite closely with our desires, it does little harm in ordinary contexts to speak loosely. Still, it will be helpful to look at cases where these two kinds of representations come apart. The case of Anthony Dickinson discussed above (§6.2.3), involved a dissociation between a cognitive desire for watermelon and an unconsciously acquired aversion to watermelon (likely subserved by reward processing mechanisms). In the previous chapter (§9.3), I discussed the miswanting. Sometimes this phenomenon results from a divergence between anticipated pleasure in response to simulation (i.e. dynamic evaluation in response to simulation) and actually experienced pleasure. This is often due to failures of imagination, i.e. the way we simulate events may not reflect how they will

231 actually occur (Gilbert and T.D. Wilson 2007). Moreover, dynamic evaluation in response to simulation may also be misleading because, just as we are not always aware of the aspect of the environment that causes an emotional response, we are not always aware of the aspect of a simulation that causes an emotional response. A similar phenomenon may occur when there is a dissociation between memory- based preferences and actually experienced pleasure. We may often find that things are far less enjoyable than we remember them being.11 This may especially be the case when we subject our cognitive desires to critical scrutiny. As we saw above, introspection can undermine satisfaction with one’s choices (e.g. T.D. Wilson and Schooler 1991). In addition, in Chapter 9 (§9.2.2), I discussed how, though they often go together, pleasure and reward can come apart. In some cases of addiction, for example, an addict may desire a drug, despite having a cognitive desire not to take a drug. However, an addict may have an increasing desire (i.e. ‘wanting’) for a drug, even if subjective pleasure does not increase or even if it decreases (T.E. Robinson and Berridge 1993; T.E. Robinson and Berridge 2003). Moreover, research indicates that humans will continue to work for drugs (e.g. methamphetamine, morphine) even when the doses are so low as to cause no change in subjective feeling (Lamb et al. 1991; C.L. Hart et al. 2001; see also Berridge and Kringelbach 2008). Thus, desire can come apart from both cognitive desire and experienced pleasure. Having introduced Humean theories of motivation and connected them to the earlier discussion, we can see that such theories have a certain plausibility. They seem to fit well with many ordinary cases. Sometimes we acquire (or recognize) a desire and are then faced with the problem of how to satisfy that desire. Similarly, we may often feel as if our desires are wholly independent from the descriptive facts. Some desires are wholly

11 This may occur in at least two ways. My failure to enjoy the steak with blue cheese-ale sauce that I remembered enjoying so much a month ago may be due to variations in the quality of the entrée (due to different chefs, ingredients, etc.). More interesting for our purposes, it may also be due to the fact that the memory of my enjoyment of the steak may have been influenced by extraneous things, good wine or good conversation, for example. Thus, the entrée may be objectively as pleasing, my surprise is a result of an inflated value. Similarly, tastes may change over time. If, after a long period of ordering on the basis of remembered preferences, I try something new, I may find that I like the mushroom-liver risotto much more than the steak with blue cheese-ale sauce. My cognitive desire has come apart from actual desire.

232 inexplicable: I have no idea why I like chocolate. I just do. And the world could be exactly the same and I could dislike chocolate. My desires seem to be all about me. In the moral domain, cases in which we exercise self control to act in accordance with our moral judgments also seem to fit well with Humean theories. In the next section, I will raise a problem for Humean theories.

10.3.2 A problem for Humean theories Provided we distinguish desires from cognitive desires, the Humean theory seems to fit nicely with the account sketched so far. The Humean theory claims that beliefs and desires are distinct kinds of states. We’ve identified beliefs with ‘cognitive’ representations in the MTL and desires with affective representations involving reward processing. However, there are some affective processes that have both cognitive content and direct connections to motivation. This raises a serious problem for Humean theories. Consider, for example, the case of fear. On the view sketched in the previous chapter (§9.2.2), fear is a basic emotion. The formal object of fear is immediate danger. In a particular case, my seeing a snake in the woods, for example, the snake will be the particular object of my fear. When a fear response is triggered by an oddly-shaped stick mistaken for a snake, my fear response misrepresents the world. Thus, such affective representations are cognitive in the sense discussed above with respect to beliefs: they aim to represent the world. However, they also have motivational properties: they prompt action aimed at escaping or defusing the danger. These kinds of affective representations present a series of counterexamples to Humean theories, which hold that beliefs and desires are distinct kinds of states. The Humean will perhaps resist these counterexamples claiming that emotions are hybrid states. Fear, for example, may consist of a belief, a desire, and an affective response. This runs counter to the account of emotion sketched above (§9.2.2), according to which fear is a state which at once represents the snake as being dangerous and has certain motivational properties. In this context, the hybrid theory seems ad hoc, while the account I offer is independently motivated, in part by empirical research on emotion. In the previous chapter, I argued that basic emotions like fear are evolutionary constructions from core affect (themselves characterized in terms of hedonic valence and arousal) in

233 combination with other features including characteristic facial expression and action tendencies. Each basic emotion is a tightly coordinated response to certain core relational themes. They are importantly different from explicitly held beliefs: they do not involve propositional content. A hybrid theory may be plausible for what I discussed in the previous chapter as complex emotions. Jealousy, for example, may plausibly seen as involving beliefs with propositional content as components. One reason for this is that complex emotions are less coordinated. In any particular case of jealousy, we may well be able to imagine someone sharing the same kind of cognitive state without experiencing the emotion. One husband may, for example, feel jealous on apprehending his wife’s flirtations with another man. Another husband may similarly apprehend his wife’s flirtations but feel no jealously because he no longer cares about the marriage. Because basic emotions are tightly coordinated responses triggered by aspects of the environment and this fact is what gives them the content that they have, it does not make sense to think about basic emotions as being decomposable into beliefs, desires, and affective responses. It is the basic emotion itself that has representational content and not any component of it. When it comes to complex emotions that are partially constituted by cognitive judgments or beliefs with propositional content, we can separate this cognitive content because these propositional representations acquire their content in a different way—not from being caused in the particular way that they are, but by having the particular conceptual components that they in fact have. Thus, the kind of content that basic emotions have and the way that these states acquire this content is importantly different from the content of judgments or beliefs with propositional content. This objection suggests another way the Humean might seek to escape such counterexamples. A Humean theory might be modified so as to require that beliefs have cognitive content in propositional format. This would allow them to exclude such affective representations and maintain that beliefs and desires are distinct kinds of states. However, this new theory faces other problems. This theory is incomplete; it can no longer be thought of as a complete theory of motivation, since it fails to account for the contribution of these affective mechanisms. It thus gives up the second aspect of the theory, that motivation requires a belief-desire pair. Thus, careful attention to the nature

234 of affective processing presents a serious problem for Humean theories of motivation. Lest anti-Humean theorist rejoice, in the next section I want to discuss cases that raise issues for both Humean and anti-Humean. In the following section, I suggest a way to resolve these issues.

10.3.3 A related worry: affective disorders In §10.2, I offered a neuropsychological argument against internalism. Here, I extend this discussion to show how internalism combined with either Humean theories or anti-Humean theories gives rise to an additional series of problem. This will help us to see how to resolve the debate between Humean and anti-Humean theories. Insofar as both Humean and anti-Humean theories of motivation try to capture an internal connection between moral judgment and motivation, clinical depression raises problems for both (J.R. Roberts 2001). Indeed, it is likely that other affective disorders (e.g. kleptomania, obsessive-compulsive disorder) present similar difficulties. Here, I discuss Roberts’ argument, which focuses on depression. What raises an issue for these views is that depression is characterized by diminished motivation together with an intact “evaluative outlook” (J.R. Roberts 2001, 45). Indeed, depressives seem to suffer all the more for continuing to value things they have no motivation to pursue. Standard Humean theories will capture diminished motivation in terms of a loss of desire. However, insofar these theories treat valuing as a form of desiring (e.g. D.K. Lewis 1989) in order to preserve the connection between value and motivation, they cannot explain how the loss of desire is consistent with the preservation of an individual’s values. Anti-Humean theories of motivation stumble over depression in a different way. These theories claim that belief is sufficient for motivation. If they construe valuing as a species of belief, then anti-Humean theories cannot readily accommodate the persistence of these values without the appropriate motivation. Thus, Roberts claims that depression presents a problem for both standard Humean and anti-Humean theories of motivation. As we saw above (§7.1.2) depression involves dysfunction of brain areas involved in affective processing, including the mPFC, OFC and amygdala. Thus each of these theories seems to stumble over adequately capturing the phenomenon of affective

235 dysfunction. I do not present these criticisms as definitive. Both Humeans and anti- Humeans may have responses to their respective problems. However, if I am right, these problems and the worry raised in the previous section point to a different solution. It is time to cut the Gordian knot.

10.4 Dissolving the moral problem At this point, we have seen problems that apply to all of the major contenders in philosophical moral psychology:

(1) Noncognitivist Humean internalism (2) Cognitivist Humean externalism (3) Cognitivist Humean internalism (4) Cognitivist anti-Humean internalism

I believe most of the views actually defended in contemporary moral psychology can be fit, comfortably or otherwise, into one of these four categories. Each of these views faces at least one of the problems faced above. In §10.2, I offered a neuropsychological argument against internalism and in the following section (§10.3) I raised a problem for Humean theories. On the other hand, we have spent little time discussing noncognitivism. However, many of these view are either Humean or internalist or both (e.g. Blackburn 1984, 186-9) and so face the problems discussed above. I will return to the issue of noncognitivism below (§10.4.2). For now I want to concentrate on the debate within cognitivist theories. There is one important exception: Cognitivist anti-Humean externalism. I leave this off the list because, as I have set up these issues this view is primarily defined negatively, including those theorists who deny internalism and reject Humean theories of motivation. This makes this label relatively uninformative. Evaluating these views will require looking at the positive account. The view that I develop would fall into this category (as would others, e.g. Shafer-Landau 2003). However, my account does not simply deny these claims (i.e. internalism and Humeanism) but rather shows how each of these theories is partially correct.

236

10.4.1 Humean externalism, anti-Humean internalism I think we can see the beginning of an explanation for the debate in moral psychology. First, note that Humean theories of motivation fit most comfortably with externalist accounts. Indeed, the claims that belief and desire are distinct and that both are required for motivation (at least together with cognitivism) entail externalism. Similarly, the anti-Humean view that beliefs can themselves be motivational fits most comfortably with internalism. Now, if we conjoin these theses respectively, we can begin to see why each has been plausible. Humean externalism and anti-Humean internalism are both plausible views, just about different phenomena. Anti-Humean internalism roughly captures the functioning of affective mechanisms. These mechanisms have both representational content as well as direct motivational connections. To take an example from above, when we see a close friend’s expression of vulnerability, we act immediately to offer comfort. This action is not mediated by an explicit judgment. Our perception of the situation leads directly to action. Appropriately trained affective responses facilitate this capacity, which seems to stand behind much of normal moral behavior. On the other hand, Humean externalism seems to capture other common cases of moral decision making, for example, when we consider the best punishment for a student caught plagiarizing or how to confront a rude coworker. In these cases, we have antecedent motivation, e.g. to handle the plagiarist professionally and appropriately. We then engage in some reflection to determine how we can best do this. Thus, in these cases motivation really does seem to require a belief-desire pair. Moreover, our conscious reflection will be facilitated by explicit beliefs subserved by the MTL. On their own, these representations will be impotent without some affective state to provide motivation. In these cases, Humean externalism plausibly describes how explicit beliefs and affective desires combine to produce motivation. Humean externalism also seems to capture the processes of reflection on our explicit beliefs or consideration of others’ arguments. To return to another example, suppose we are considering the permissibility of meat eating. When we become convinced that some of our previously held beliefs jointly entail that meat eating is

237 impermissible, this will not on its own provide motivation to act accordingly, even if we remain committed to these beliefs.12 We will often need to exert self-control in order to act on these judgments. Thus, the account of moral psychology I’ve defended above makes sense of these disputes in moral psychology. Humean externalist theories and anti-Humean internalist theories both appear plausible because they each capture different aspects of moral psychology. Nevertheless, neither of these theories presents a general theory of moral psychology. Our moral beliefs and cognitive desires subserved by the MTL system are explicit and sensitive to argument. But, they require the addition of some motivational state to influence behavior. This will draw on either our previously held motivations or will require the exercise of self-control to generate such motivation. Our affective representations have both motivational and representational aspects, but alone, they are limited. These representations are not sensitive to argument directly. Modifying our affective responses takes a certain kind of practice. Simulation, rehearsal and the exercise of self control are important for training these affective mechanisms. In addition to shedding light on the persistence of these disputes in moral psychology, this account also resolves the problems that arise from affective disorders. Depression, as we have seen, involves a dysfunction of affective mechanisms. Humean theories construe this as a lack of desire. This wrongly identifies our evaluative outlook with these desires and renders such theories unable to capture the persistence of evaluative beliefs in depression. Anti-Humean theories rightly identify the kind of valuing preserved in depression as a species of belief. The problem for this account lies in thinking that these kinds of cognitive representations (i.e. beliefs) can be intrinsically motivating. Affective disorders raise problems because they often involve a dissociation between affective mechanisms and more ‘cognitive’ ones (i.e. evaluative beliefs and cognitive desires). The kleptomaniac has an irresistible urge to steal, yet maintains that

12 In some cases, drawing such a conclusion might lead us to reconsider our previously held beliefs. However, the fact that we draw such an inference (e.g. that meat eating is impermissible) and yet have no motivation to act accordingly does not suffice to show that we do not remain committed to these beliefs.

238 stealing is wrong. Similarly, accounts of the difference between the willing and the unwilling addict that draw on the notion of ‘identification’ with one’s desires (e.g. Frankfurt 1971) could be developed further by drawing on this account of moral psychology.

10.4.2 Whither Noncognitivism? Noncognitivist may feel like their view has gotten short shrift. This is in part true. However, I think there are several considerations that justify this brisk treatment. To start with, noncognitivism seems to be motivated by what might be considered overly austere metaphysical and epistemological commitments. According to Simon Blackburn, for example, his view “intends to ask no more from the world than what we know is already there—ordinary features of things on the basis of which we make decisions about them, like or dislike them, fear them and avoid them, desire them and seek them out” (1984, 182). Two considerations lead us to question the role of such commitments in motivating noncognitivism. First, if we are willing to entertain the existence of biological and psychological properties (as Blackburn seems happy to) then why not moral properties as well? Such properties seem to involve only a level of abstraction slightly above psychological properties. Indeed, so-called Cornell Realists have argued that moral properties are a kind of natural property (e.g. Boyd 1988; Sturgeon 1988; see also A. Miller 2003). We ordinarily speak of actions being right and wrong just as much as we speak of things being desirable or likeable. To affirm the one while denying the other seems to hold moral properties to a different standard. Second, and related to this last point, there have been attempts to account for normative notions partly in terms of people’s responses (e.g. Johnston 1989; J. Gert 2009). This approach often draws on analogy with other secondary qualities like color. Plausible accounts define color in terms of a disposition to look a certain way to certain observers under certain conditions. Response-dependent accounts offer a similar account of normative properties. If moral properties, for example, can be captured in part by what our normal responses under certain conditions are—facts which noncognitivists certainly do not deny—then one wonders what motivation for noncognitivism remains.

239 Thus, even if a noncognitivist insists on her metaphysical and epistemological scruples, it is not clear that all versions of cognitivist realism need run too far afoul of these commitments. With an improved descriptive understanding of moral psychology, I believe much of the motivation for noncognitivism is lost. Moreover, noncognitivists often try to capture the realist-sounding talk of ordinary language. Blackburn, for example, writes: “If a projectivist can…earn the right to think in terms of an ethical truth and ethical features of things, then he can talk of us responding to them and responding ethically…” (1993, 198). If such a project succeeds, it may often be unclear what separates noncognitivism from realism after all (cf. D.K. Lewis 2005). Either way, these issues go far beyond the scope of this dissertation.13 However, there is another reason noncognitivism has not figured more in the discussion. To make sense of the distinction between cognitivism and noncognitivism, we must see noncognitivists as denying something that cognitivists affirm. In light of the account of moral psychology I have defended, it is increasingly difficult to see the denial of cognitivism as plausible. The more interesting debate in the vicinity is about the role played by representations whose content is in propositional format and those in which it is nonpropositional. This is a debate within cognitivism. A standard notion of ‘cognitive’ involves representations that ‘aim at’ truth or that can be true or false. However, on the account of emotion sketched above (§9.2.2), emotions have cognitive content on this view.14 Thus, noncognitivists cannot simply depend on moral judgments inheriting their noncognitive content from the emotions they are said to express. Cognitivists can accept that moral judgments sometimes derive from emotions. In §9.2, I discussed another notion of cognitivism. There, I distinguished between ‘cognitive’ as involving higher-level information processing. We should, however, resist calling lower-level information processing noncognitive. Similarly, while we might think of representations in propositional format as more ‘cognitive’, it would be a mistake to consider nonpropositional representations to be ‘noncognitive’. Thus, I think

13 For some discussion see Huemer (2005, 42-5). 14 Recall, for example, the problem for Humean theories of motivation discussed above (§10.3.2). For further discussion of noncognitivism about emotion, in light of empirical research, see Starkey (2007) and Scarantino (2010).

240 noncognitivism loses much appeal since it seems difficult to see noncognitivists as denying something that cognitivists would accept. The more interesting distinction is between processes with different kinds of cognitive content. I think it is time to move past the debate between noncognitivists and cognitivists and to focus on understanding how these different kinds of content contribute to moral judgment.

10.4.3 Making sense of complexity At this point, I’ve illustrated how the account of moral psychology defended in previous chapters sheds light on philosophical issues. I have used this account to criticize several common views in philosophical moral psychology. In addition, I have argued that this account actually offers an explanation of the appeal of other views. Humean externalism and anti-Humean internalism are both plausible views of certain phenomena, but not all of moral psychology. Each of several traditional views glosses over the complexity of moral psychology, a theme I have stressed ad nauseum. One response to the claim that moral judgment can derive from multiple psychological mechanisms is to argue that, while this may be true, ‘real’ moral judgments only derive from one of these mechanisms. I think this is implausible. The redundant, overlapping and dissociable systems are part and parcel of our ability to be moral agents. This somewhat pastiche-like collection of mechanisms is in fact adaptive. On the one hand, explicit beliefs are sensitive to reasoning. On the other hand, the affective responses provide the ability to quickly and (usually) appropriately respond to moral features of our environments. Moreover, emotion may allow us to overcome certain commitment problems that arise in social groups. Consider, for example, how the prevalence of dominance reasoning in Prisoner’s Dilemma situations would undermine social cooperation. Human moral psychology is not always perfect. However, the complex, sometimes fragmented nature of moral psychology is not merely an accident either. Disruption of parts of these mechanisms have dramatic and undesirable effects. Thus, privileging some mechanism in the generation of moral judgment does not seem a very plausible response to this complexity.

241

10.5 Agency and Rationalism In this section, I deal with two criticisms of the kind of approach taken here. Philip Gerrans and Jeanette Kennett (2010) argue that sentimentalist accounts that draw on neuroscientific research do not adequately capture certain aspects of moral agency. While this criticism is well taken, I believe there are problems with Gerrans and Kennett’s account of agency. I criticize their account and use this discussion to address how my account deals with the issue of agency. Terry Horgan and Mark Timmons (2007) have defended rationalism against views which argue that moral reasoning is largely confabulation. In addressing this criticism, I discuss my view with respect to rationalism.

10.5.1 Moral judgment and moral agency Gerrans and Kennett argue against what they label neurosentimentalism: views that draw on neuroscientific work to support the claim that “moral judgment depends essentially on tacit affective processes” (2010, 586). The major problem with these views is that moral decision making, like decision making in general, is the “the exercise of a capacity for agency” (2010, 585). Thus, there is an agential condition that neurosentimentalism and related views fail to meet. Gerrans and Kennett criticize neurosentimentalist accounts, as well as related rationalist externalist accounts according to which judgments deriving from reasoning have no internal connection to motivation. While these views appear to be supported by recent empirical research, they offer an alternative interpretation of this research that addresses their concerns about agency. I begin by discussing their criticisms and move on to their positive account. It is important to note that my view is not exactly a version of neurosentimentalism. Views like Jesse Prinz’s (Prinz 2006; Prinz 2007) are closer to their target. Prinz claims that “emotions are both necessary and sufficient for moral judgment” (2006, 32). I have argued that affect is certainly important, but I would not say that in any particular case it is necessary. I hold that both affect and explicit reasoning contribute to moral judgment, which combines elements of sentimentalism and rational externalism.

242 Even though my view is not the intended target of Gerrans and Kennett’s criticism, I draw on the same neuroscientific research they discuss and it is worth discussing how my account deals with the issue of moral agency. While I think that some of Gerrans and Kennett’s criticisms fail, my goal is less to defend neurosentimentalism than it is to show how my own account deals with the issue of agency. Gerrans and Kennett point out how empirical research, almost all of which I have discussed in earlier chapters, is often used to support both sentimentalist and rationalist externalist accounts. They claim that such accounts do not meet an agential constraint: moral agency requires an agent to view herself as a “temporally extended entity” (2010, 588). Neurosentimentalism offers a synchronic account that simply cannot capture this aspect of moral agency. Thus, Gerrans and Kennett present a dilemma for neurosentimentalism, claiming that such accounts either “divorce moral (and practical) judgment from moral (and practical) agency” (2010, 586) or hold that “amnesics and ventromedial patients can be moral agents” (2010, 588). Neither horn, they claim, is attractive. The first thing to say is that, as we have seen, moral judgment does come apart from moral agency. In many cases we must exercise self control in order to do what we judge we morally ought to do and I have discussed some of the neural mechanisms that might underlie such processes. Thus, my account makes sense of the connection between moral judgment and agency, which is often very tenuous. Allowing moral judgment and moral agency to come apart to some extent is more plausible than Gerrans and Kennett seem to allow. When considering the second horn of the dilemma, it is important to note that the argument against internalism offered above did not depend on claiming that vmPFC patients are moral agents. They argue that neurosentimentalism is committed to counting amnensics and vmPFC patients as moral agents. This, they argue is problematic. It may well be the case the such views are so committed and that this is a problem. However, there is a concern with Gerrans and Kennett’s own view of agency. As I discuss below, depending upon how we interpret Gerrans and Kennett’s account it either sets the standard for moral agency unreasonably high or it too does not clearly rule out vmPFC

243 patients as moral agents. In fact, I think this issue points to a deeper problem with Gerrans and Kennett’s account. A preliminary issue that is worth noting is that Gerrans and Kennett do not appear to make any distinction between agency per se and moral agency. One might think that it is possible to be an agent and yet to lack certain capacities necessary for moral agency. Perhaps certain nonhuman animals can be agents without being capable of moral agency. Thus, it might be thought that the conditions for moral agency go beyond the conditions for agency. Making such a distinction might render Gerrans and Kennett’s account more plausible if construed as an account of distinctively moral agency. I return to this issue below. Gerrans and Kennett argue that vmPFC damage undermines the capacity for mental time travel or what I have referred to above (§6.2.3) as simulation. In doing so, they draw on precisely the same research I discussed in Chapter 6. There is an important point of disagreement here over the interpretation of that research. Gerrans and Kennett claim that “It seems likely that the normal case of decision-making involves mental time travel” and that “a patient with ventromedial damage cannot perform mental time travel” (2010, 600). I argued that the vmPFC is not crucial for simulation per se, but for emotional response to such simulation and the integration of this information in decision making. I will not repeat my previous arguments, but it will be helpful to look at the evidence Gerrans and Kennett cite. Gerrans and Kennett cite a case study (Levine et al. 1999) of an individual with “amnesia and ventromedial damage with deficient mental time travel…and decision making deficits…which indicate a loss of agency” (2010, 600). Apart from the difficulty of drawing conclusions from a single case, the subject of this study, M.L., is reported to have retrograde amnesia (i.e. impaired memory of events previous to the injury) and partially preserved anterograde memory (Levine et al. 1998). A more recent discussion of M.L.’s case suggests a dissociation between semantic and episodic memory and that deficits in mental time travel might, in fact, be due to M.L.’s retrograde amnesia (Levine et al. 2009). This case does not seem to provide very compelling evidence that the vmPFC is necessary for mental time travel. M.L. presents with a puzzling set of clinical

244 symptoms. This does not make it an ideal case from which to garner support for Gerrans and Kennett’s theoretical conclusions. Gerrans and Kennett also cite a study (Gutbrod et al. 2006) indicating poor performance on the IGT in amnesia patients. As discussed at the end of §7.2.6, several processes are involved in the IGT (see N.S. Lawrence et al. 2009). That explicit memory (subserved by the MTL) is required for performance on the IGT should not be completely surprising. It does not, however, provide evidence that vmPFC patients also have a deficit in mental time travel. Rather, it simply indicates what we already know: that performance on the IGT does not depend only on the vmPFC. Thus, I think there are good reasons to reject Gerrans and Kennett’s interpretation. This is not mere quibbling. The interpretation offered by Gerrans and Kennett allows them to argue that the agency of vmPFC patients is much more impaired than on the interpretation I offer. They go on to claim: “the capacity for mental time travel is arguably required for any demand to be normative for a person—for them to have reasons whose force is independent of immediate stimulus bound responses.” They continue: “the process of becoming an agent capable of engaging in this normative domain [morality] just is the process of learning to transcend the present moment, both cognitively and behaviourally” (2010, 602). On the view I defend, being able to engage in mental time travel is distinct from being able to respond emotionally to such simulations. Individuals with vmPFC damage do not seem to have any problem identifying themselves with a temporally existing entity. What they lack is emotional responses to simulation that would facilitate more appropriate action on the basis of such simulation. Rejecting Gerrans and Kennett’s interpretation of the empirical research makes their broader claims far less plausible. Still, we can see them as offering an account according to which moral agency requires the ability to see oneself as a temporally extended entity and to take certain considerations as normative. I will refer to these as reflective capacities. These are not sufficient for moral agency but they are necessary and thus are a “prerequisite for moral judgement” (2010, 606). On the other hand, Gerrans and Kennett claim, views like neurosentimentalism “leave the moral agent out of the picture” (2010, 606).

245 There are two ways of interpreting this agential constraint. First, it might be thought that every moral judgment involves the exercise of these reflective capacities. This would seem to place undue burden on moral agents: few humans would meet this standard. Most ordinary cases of moral judgment would not involve the exercise of such abilities. And passages like the following tell against this interpretation: “There are many cases of moral judgement which are not well captured by the synchronic focus of most experimental work on moral cognition” (2010, 607). This quote suggests that these are moral judgments, but only insofar as they proceed from an agent who has the ability to engage in reflective capacities. This second way of interpreting the claim seems more plausible. However, this claim is actually very weak. Merely having the ability to engage such reflective capacities is consistent with one never actually doing so or doing so very rarely or doing so only in certain cases. However, this does not seem to rule out any but those cases involving the most extreme pathology. As I’ve argued vmPFC patients have the capacity to engage in simulation; they can make sense of themselves as temporally existing entities. It is also important to recall that vmPFC patients are not completely impaired in the affective realm. As noted, while they do not exhibit a tendency toward instrumental aggression, they do often have difficulty with emotional regulation. Thus, individuals with vmPFC damage are able to engage in simulation and have some emotional responses that might allow them to take certain considerations as normative. They appear to meet the conditions for moral agency after all. Suppose Gerrans and Kennett respond by arguing for a stronger constraint on when moral agents must exercise their reflective capacities. Other considerations tell against strengthening this requirement. Earlier (§9.3, see also §6.2.1 and §6.3), I discussed evidence that some ‘normal’ people exhibit a similar physiological and behavioral profile as vmPFC patients. If we follow Gerrans and Kennett in raising the standards of moral agency so as to exclude individuals with vmPFC patients must we also exclude such ‘normal’ individuals? The problem is not simply that Gerrans and Kennett set the bar too high. The more basic problem is that, given the existence of individual differences, there is no straightforward way to construct a standard such that pathological cases will be excluded

246 without thereby also excluding some apparently normal people. This is not to say there are not clear cases of moral agency and clear cases where such agency is lacking, only that the boundary between them is none too clear. This problem points to a resolution. Ultimately, I think the problems with Gerrans and Kennett’s view arise from a set of related commitments. First, they seem to treat moral agency as one half of a relatively clear distinction. It may plausibly be thought that agency can be a matter of degree and, while there may be a lower threshold, there are varying degrees above this and that more robust forms of agency are of some theoretical interest. Al Mele (2003, chap. 10), for example, claims that to be an agent is simply to be a being that acts. On such a view, we may consider the ability to act as necessary and sufficient for basic agency. Many nonhuman animals may be agents in this basic sense, though they may lack more robust varieties of agency. We may then identify further conditions as constitutive of more robust forms of agency. Moral agency, for example, may involve the capacity to take certain considerations as normative. However, it may also require certain social skills. These might involve being able to easily read others’ facial expressions and respond appropriately. Individuals with autism may lack some of these capacities. Thus, they may not meet the conditions for what we might call social agency. To the extent that such social agency is required for moral agency, such individuals might exhibit impaired moral agency. Nevertheless, such individuals certainly seem to meet the conditions for basic agency and may well be able to engage in some forms of moral agency. Thus, moral agency may be a complex capacity that may be partially impaired. The reflective capacities that Gerrans and Kennett highlight in their account of agency require the ability to conceive of oneself as a temporally existing entity. This seems to involve the capacity for self representation that goes far beyond what would normally be considered necessary for agency and perhaps even too high for other more robust forms of agency, including limited expressions of moral agency (in small children, for example). A second commitment of Gerrans and Kennett seems to be a desire to ‘locate’ agency in the vmPFC. They seem to want there to be one area where a person’s sense of herself as a temporally existing entity occurs together with her ability to take certain

247 considerations as normative. I’ve argued that this is not borne out by the empirical research. However, I think that Gerrans and Kennett’s misinterpretation might actually be a result of a deeper tendency to want to see ‘agency’ as a kind of unified faculty rather than as dependent upon a set of interrelated but distinguishable capacities. The capacity for simulation is largely distinct from the capacity to respond emotionally to such simulations. Disruption of either of these capacities may impair (various kinds of) agency without destroying it altogether. And each of these capacities may be partially disrupted. When this occurs, forms of agency that depend on such capacities may be impaired without being totally absent. Moreover, individual differences in normal people compound these worries by indicating that there are differences in the way (moral) agency is expressed across different individuals. Some people may find it easy to do the right thing, others may find they need to exercise more self control. Some people may focus on the emotional aspects of a situation to facilitate right action. Others may find themselves focusing on moral principles and the desire to avoid in order to do the right thing. While it is important for moral psychological theories to account for agency, they must do so in a way that accommodates the existence of diversity that is revealed throughout the domain. Thus, I think that Gerrans and Kennett’s account of agency fails to appreciate that agency can be a matter of degree, that there can be multiple varieties of agency, that each of these varieties of agency may depend on several interrelated brain regions, and that there are likely important differences in these varieties of agency across individuals. Moral agency is complex. Only some of the time do we engage in the kind of reflection that Gerrans and Kennett seem to single out. Often, the exercise of our moral agency is much more fluid. Gerrans and Kennett’s account of agency leaves out the sense in which moral agency involves appropriate response to our environment, especially the social aspects of our environment. We might think that social agency is importantly related or partially constitutive of moral agency. These social capacities are underwritten by precisely the “tacit affective mechanisms” that Gerrans and Kennett argue cannot capture important aspects of moral agency. Moreover, I think Gerrans and Kennett fail to give these affective mechanisms their due. They fail to appreciate the resources that neurosentimentalists have at their

248 disposal. These affective mechanisms are, as I have emphasized, very powerful learning mechanisms. Thus, the exercise of them is not merely a synchronic affair. They have been shaped over the entire course of development and these influences are expressed when such affective responses are triggered. While these mechanisms do not necessarily involve the exercise of any reflective capacities, the shape that these affective responses take on over the course of development is (at least in part) constitutive of the diachronic self that is expressed in agency, especially the kind of social agency that seems just as important for moral agency as the reflective capacities that Gerrans and Kennett highlight. My account does not privilege either the affective mechanisms or processes of explicit reasoning. Rather, it argues that both are likely necessary for ‘normal’ moral agency. However, various forms of agency can occur under conditions in which some of these aspects are partially or fully impaired. Agency is not an all or nothing status. In phylogeny and ontogeny, as well as with the onset of onset of degenerative conditions or focal brain damage we see various forms of agency emerge or erode gradually (diachronically or synchronically across different cases). My account begins to shed light on the ways in which elements of (various kinds of) agency can come apart and the pathologies that can result from the warping of agency. It highlights how these elements work together in normal individuals and how compensation for dysfunction can occur in certain cases of pathology. Far from leaving the agent out of the picture, the account I offer pulls back the curtain to reveal the very mechanisms that are constitutive of agency. Thus, I think Gerrans and Kennett have, to some extent, overstated their criticisms of neurosentimentalism. To the extent that Gerrans and Kennett seek to revitalize rationalist internalism, I think their view is implausible. To the extent that they seek to correct for neurosentimentalist accounts that overemphasize emotion or fail to emphasize the important interconnections between reasoning and emotion in moral agency, I agree. Indeed, my account does precisely this.

10.5.2 Morphological rationalism Like the criticisms in the previous section, Horgan and Timmons (2007) discuss conclusions that have been drawn from empirical research and argue that some of these

249 claims should be resisted. Their discussion picks up on one of the points discussed at the end of the last section concerning the power of affective mechanisms. Horgan and Timmons do not question whether the function of such mechanisms are sufficient for moral agency. Rather, they claim that the ubiquity of such mechanisms in moral judgment does not undermine . As I discussed above, there is evidence that the mechanisms that produce moral judgments are often automatic, affective mechanisms and the justification of these judgments often involves confabulation. Horgan and Timmons focus on these issues involving rationalism and confabulation. As in the previous section, it is important to note that my account is not the direct target of their criticism. I do make room for the role of reasoning in some moral judgments. Intuitionist views like Jonathan Haidt’s (Haidt 2001; Haidt and Bjorkland 2008) are the more direct target of their criticism. Nevertheless, my view does hold that confabulation in moral justification is common. Additionally, it is worth discussing how my account stands with respect to rationalism. Horgan and Timmons argue that we should resist concluding from the empirical research that moral justification (i.e. “ moral reason-giving”) often involves confabulation (2007, 279). They develop a view called morphological rationalism. On this view, moral principles are “morphologically embodied” in the structure of our affective mechanisms and, as such, moral judgments that derive from such mechanisms involve these moral principles, albeit in an unconscious, automatic way, via what Horgan and Timmons call “proceduralization” (2007, 279). They claim that morphological rationalism is not only possible, but plausible. Indeed, they offer an argument from the “non-jarring” phenomenology of moral judgment in favor of morphological rationalism (2007, 290). The issues raised by Horgan and Timmons are tricky. I cannot hope to settle them conclusively here. However, I think they help raise exactly the right questions about how we should proceed in light of the empirical research. On their account, to morphologically possess a moral principle, an individual must be disposed to “undergo transitions in cognition, from certain input to moral judgments as output, such that (1) these transitions systematically and non-accidentally conform with the moral principle, (2) this systematic conformity results from the person’s

250 persisting psychological structure… and (3) those cognitive transitions typically result from this persisting structure without the mediation of a tokening representation of the principle” (2007, 285). They believe that this account makes it quite plausible that moral principles are possessed morphologically. In particular, they draw on Chomsky’s account of grammatical competence. We’ve already seen some issues with interpreting Chomsky (§9.4). It appears that they are making use of the ‘knowledge view’ of competence; they write: “According to Chomsky, people’s judgments of grammaticality are to be explained by their possession of grammatical rules—rules which individuals may not be able to explicitly formulate, but to which their competent grammatical judgments conform” (2007, 287). In agreement with other research discussed above (e.g. Hauser 2006a) Horgan and Timmons claim that moral psychology fits well with this model. Lastly, they present an argument based on the “non-jarring character” of moral phenomenology (2007, 290). They base this argument on what they call the “maxim of default competence-based explanation,” according to which All else equal, a theoretical explanation of a pervasive, population-wide psychological phenomenon will be more adequate to the extent that (1) it explains the phenomenon as the product of cognitive competence rather than as a performance error, and (2) it avoids ascribing some deep-seated, population-wide, error-tendency to the cognitive architecture that subserves competence itself. (2007, 289) Views like Haidt’s according to which what people take to be giving reasons for their judgments is actually post-hoc confabulation run afoul of this maxim. And since morphological rationalism offers a competing interpretation that does not invoke widespread performance error, it is to be preferred over intuitionism. Consider, for example, ’s (1977) case of seeing a group of hoodlums setting fire to a cat. When we observe such a situation we immediately conclude that it is wrong. When we cite reasons for this judgment, for example the wrongness of cruelty, Horgan and Timmons claim: “One’s citing such reasons for this judgment is experienced as getting at and hence as fitting the original judgment” (2007, 292). We do not experience the reason giving in this case as jarring as one might expect if such a judgment were really a confabulation and as Haidt (2001) reports in cases of moral

251 dumbfounding (§3.3). Morphological rationalism explains why our explanation of such judgments does not strike us as jarring. A crucial issue here is the morphological embodiment of moral principles. From my previous discussion of the role of moral principles in the etiology of moral judgment (Chapter 8), I think we can see that morphological rationalism may be plausible in some cases. We saw evidence that utilitarian principles can be automatized. Presumably this involves the application of reasoning over time to shape these automatic responses. In cases where reasoning figures in the development of automatic responses, morphological rationalism holds some plausibility. However, I also argued that the content of processing in affective mechanisms does not generally involve the use of moral principles. Processing dependent upon the basal ganglia depends more on similarity-based processing. The Trolley/Footbridge asymmetry, for example, was explained in terms of the processing details of this system (§8.1.3). These results raise problems for Horgan and Timmons’ view. First, it is not clear that certain judgments actually conform perfectly with certain principles. This threatens to undermine condition (1) of morphological embodiment. As noted, Waldmann and Wiegmann (2010) argue that judgments in some Trolley case variations do not in fact conform to the DDE. Second, while morphological embodiment does not require the tokening of the relevant moral principle, it is not always clear that these principles are nonaccidentally embodied in psychological structures. For example, to the extent that intuitive judgments do conform to the DDE, it seems to be purely accidental. The DDE does not figure consciously or unconsciously in the development of the psychological mechanisms underlying these responses. Rather, these judgments derive from features of these underlying psychological mechanisms. That said, I think that we can see a principle as morphologically embodied if it plays a role in the development of the relevant responses. For example, a parent may shape a child’s affective tendencies in a way that causes them to conform to a certain moral principle and in this way insure that the conformity is nonaccidental. This highlights the importance of understanding the social context of human moral psychology and development. We may not need to explicitly learn a principle as long as that principle

252 plays some role in the practice that leads to our acquisition of the affective or intuitive responses which conform with that principle. Of course, explicit representations of moral principles may play a role in the development of these responses even in the individual case. This might be the case with people who have internalized certain utilitarian principles or in moral vegetarians who have cultivated the appropriate affective responses. In such cases, these responses can be seen as meeting the conditions for morphological embodiment. The important point of this discussion is that establishing morphological rationalism must be done on a case by case basis that pays close attention to the relevant empirical details. I do not think that morphological rationalism can be argued for on a wholesale basis. Thus, I turn to Horgan and Timmons’ positive argument. The most serious problem with their positive claim is that it depends upon an analogy with linguistics and, as I have argued (§9.4), this analogy is not very good. The maxim of default competence-based explanation depends on the distinction between competence and performance that we have good reason to think does not map on to the moral domain very well. A question that goes beyond the linguisticist framework is: is confabulation an error? To address this question, it is important to make two distinctions. The first distinction is between different kinds of error (for further discussion, see Gigerenzer 2005). On the one hand, there are what I will call good errors. Good errors are those that are built into our cognitive capacities in some way. They arise from features that generally facilitate cognition, but can break down in certain cases. Certain perceptual illusions are examples of good errors. For example, the fact that the shading of circles can affect whether we perceive them as convex or concave reflects assumptions about the environment, in this case that light always comes from above. Good errors tend to arise only in response to stimuli that rarely or never occur in nature. Thus, they exhibit the extent to which our visual systems are adapted to our actual environment. On the other hand there are bad errors which arise simply from too little care or thought. These might be thought to derive from cognitive limitations or a mismatch between a cognitive capacity and the environment. Now these are tricky issues, and it may be disputed whether a particular error (e.g. the gambler’s fallacy) is one kind of

253 error or the other. Nevertheless, it is important to make the distinction between errors that arise when the assumptions built into an adaptive mechanism are violated and those that arise from other conditions. The second distinction involves two different ways of thinking about confabulation. On the one hand we may define weak confabulation as what occurs simply when the mechanism that produces the justification of a judgment does so independently of the mechanism that produced the judgment. As discussed earlier (§9.3), when this occurs we may still often produce a plausible or correct explanation, either because of parallel activation of these two mechanisms or because we recover the salient features of the situation. On the other hand we may think about strong confabulation as weak confabulation in which the explanation is in fact mistaken. Cases of split-brain patients and arguably cases of moral dumbfounding involve strong confabulation. Drawing on these two distinctions we can return to our initial question. It appears that weak confabulation may not involve error of either kind. Moreover, it is arguable that even strong confabulation involves only good errors. Weak confabulation results from the division of labor between mechanisms that underlie judgment and those that underlie explanation. However, there is not necessarily a reason to believe that strong confabulation does not simply arise from special cases. The function of judgment and explanation mechanisms may in fact be adaptive. One last issue involves the question of whether we have reason to believe that (either strong or weak) confabulation need be experienced as jarring. Certainly there is no evidence that split-brain patients, which provide the most dramatic illustration of strong confabulation, experience any jarringness. Indeed, a jarring phenomenology may well undermine the confabulation. Experiencing the explanation as jarring might lead one to reject the explanation. However, if the arguments above have been on the right track, there is no reason to expect that either weak or strong confabulation will necessarily be jarring. We do not generally experience visual illusions as jarring, even when we are aware of them as illusions. Thus, while the phenomenology of moral judgment is an important consideration to take into consideration, in this case I cannot see that it provides support one way or another.

254 Thus, I think the prospects for rationalism must be addressed on a case by case basis. Consider, for example, our intuitive judgments in the Trolley and Footbridge cases. These judgments have been used to support a general moral principle, the DDE, but I have argued that they derive from the structure of underlying associative, affective mechanisms. At this point we can ask, on the basis of this information, whether we ought to endorse this principle and thus vindicate these intuitive judgments, or whether we ought to reject these intuitive judgments. Proceeding in the normal way, arguments can be given on either side, debate and discussion will ensue and in this way we can hope to answer such questions. After subjecting these intuitions to rational scrutiny we will then shape our intuitions (or our children’s intuitions) in ways consistent with the output of this rational debate. In such cases, morphological rationalism will be vindicated. This discussion highlights the relation between descriptive and normative approaches to morality and the contribution that careful attention to empirical research can make to issues in . The normative status of our intuitive judgments in response to, for example, the Trolley and Footbridge cases has been a matter of considerable debate. Drawing on empirical research I have raised an issue with respect to the normative status of these judgments. Apart from the fact that this issue just could not have arisen without attention to the empirical research I do not think we can hope to resolve such issues without continued attention to the relevant empirical research. In this way, empirical research will prove a crucial tool for future work in ethics and moral psychology.

10.6 Conclusion: science and ethics In his “Lecture on Ethics,” Ludwig Wittgenstein expressed his skepticism that ethics can be a science in the following way: “I can only describe my feeling by the metaphor, that, if a man could write a book on Ethics which really was a book on Ethics, this book would, with an explosion, destroy all the other books in the world” (1965, 7). Hyperbole notwithstanding, I think that many philosophers remain skeptical that science is relevant to ethics. I have certainly not intended for this dissertation to destroy all the other books in the world. I do not think that ethics will become a part of science, but I also do not think

255 that science is irrelevant to ethics. In this chapter, I have argued that careful attention to empirical research not only helps us resolve certain disputes in moral psychology, but also contributes to an explanation of how these debate arose in the first place. As we have seen in the previous section (§10.5.2), empirical research can raise normative issues that we simply have no access to from the armchair. Needless to say, in raising these issues, empirical work does not offer a resolution to them. This must be done in the normal way, by engaging in rational discussion, but it must of course be done from a perspective informed by the relevant scientific work. In light of this, continuing to address issues in ethics and moral psychology in ignorance of empirical research is simply bad philosophy. In pursuing a project like this, it would be a mistake to claim to have reached any definitive conclusions. Rather, the work in this dissertation has just begun to sketch out the possibilities. That said, I think that the basics of the account of moral psychology I have offered will stand up well, though I have no doubt it will be subject to serious revisions. More important than these details, I think I have shown that such an approach is worth continuing to pursue. Eschewing the traditional a priori methodology in ethics offers the prospect of renewal in ethics. As Darwin might have put it: in the future I see open fields for far more important researches.

256

APPENDIX A

MORAL VIGNETTES

1) Heinz A woman was near death from a special kind of cancer. There was one drug that the doctors thought might save her. It was a form of radium that a druggist in the same town had recently discovered. The drug was expensive to make, but the druggist was charging ten times what the drug cost him to produce. He paid $200 for the radium and charged $2,000 for a small dose of the drug. The sick woman's husband, Heinz, went to everyone he knew to borrow the money, but he could only get together about $1,000 which is half of what it cost. He told the druggist that his wife was dying and asked him to sell it cheaper or let him pay later. But the druggist said: "No, I discovered the drug and I'm going to make money from it." So Heinz got desperate and broke into the man's store to steal the drug for his wife.

Should Heinz have broken into the store to steal the drug for his wife? Why or why not? (Kohlberg 1981)

2) Consensual Sibling Incest Julie and mark are brother and sister. They are traveling together in France on summer vacation from college. One night they are saying alone in a cabin near the beach. They decide that it would be interesting and fun if they tried making love. At the very least it would be a new experience for each of them. Julie was already taking birth control pills, but Mark uses a condom too, just to be safe. They both enjoy making love, but they decide not to do it again. They keep that night as a special secret, which makes them feel even closer to each other. What do you think about that? Was it OK for them to make love? (Haidt 2001)

257

3) Standard Trolley You are at the wheel of a runaway trolley quickly approaching a fork in the tracks. On the tracks extending to the left is a group of five railway workmen. On the tracks extending to the right is a single railway workman. If you do nothing the trolley will proceed to the left, causing the deaths of the five workmen. The only way to avoid the deaths of these workmen is to hit a switch on your dashboard that will cause the trolley to proceed to the right, causing the death of the single workman. Is it appropriate for you to hit the switch in order to avoid the deaths of the five workmen? (Greene 2004; derived from Foot 1967)

4) Standard Footbridge A runaway trolley is heading down the tracks toward five workmen who will be killed if the trolley proceeds on its present course. You are on a footbridge over the tracks, in between the approaching trolley and the five workmen. Next to you on this footbridge is a stranger who happens to be very large. The only way to save the lives of the five workmen is to push this stranger off the bridge and onto the tracks below where his large body will stop the trolley. The stranger will die if you do this, but the five workmen will be saved. Is it appropriate for you to push the stranger on to the tracks in order to save the five workmen? (Greene 2004; derived from Thomson 1985)

5) Ecologists You are part of a group of ecologists who live in a remote stretch of jungle. The entire group, which includes eight children, has been taken hostage by a group of paramilitary terrorists. One of the terrorists takes a liking to you. He informs you that his leader intends to kill you and the rest of the hostages the following morning.

258 He is willing to help you and the children escape, but as an act of good faith he wants you to kill one of your fellow hostages whom he does not like. If you refuse his offer all the hostages including the children and yourself will die. If you accept his offer then the others will die in the morning but you and the eight children will escape. Is it appropriate for you to kill one of your fellow hostages in order to escape from the terrorists and save the lives of the eight children? (Greene 2004)

6) Infanticide You are a fifteen-year-old girl who has become pregnant. By wearing loose clothing and deliberately putting on weight you have managed to keep your pregnancy a secret. One day, while at school, your water breaks. You run to the girls locker room and hide for several hours while you deliver the baby. You are sure that you are not prepared to care for this baby. You think to yourself that it would be such a relief to simply clean up the mess you’ve made in the locker room, wrap the baby in some towels, throw the baby in the dumpster behind the school, and act as if nothing had ever happened. Is it appropriate for you to throw your baby in the dumpster in order to move on with your life? (Greene 2004)

7) Crying Baby Enemy soldiers have taken over your village. They have orders to kill all remaining civilians. You and some of your townspeople have sought refuge in the cellar of a large house. Outside you hear the voices of soldiers who have come to search the house for valuables. Your baby begins to cry loudly. You cover his mouth to block the sound. If you remove your hand from his mouth his crying will summon the attention of the soldiers who will kill you, your child, and the others hiding out in the cellar. To save yourself and the others you must smother your child to death. Is it appropriate for you to smother your child in order to save yourself and the other townspeople? (Greene 2004)

259

8) Architect You are a young architect visiting one of your construction sites with your boss. Your boss is a despicable individual who makes everyone around him miserable including you. It occurs to you that if you were to push him off of the building you are inspecting he would fall to his death and everyone would think it was an accident. Is it appropriate for you to push your boss off of the building in order to get him out of your life? (Greene 2004)

9) Tea Cups Trolley When Billy’s mother leaves the house one day, she says “you are forbidden from breaking any of the teacups that are on the counter.” Later that morning, Billy starts up his train set and goes to make a snack. When he returns, he finds that his 18 month old sister Ann has taken several of the teacups and placed them on the train tracks. Billy sees that if the train continues on its present course, it will run through and break five cups. Billy cannot get to the cups or to the off-switch in time, but he can reach a lever, which will divert the train to a side track. There is only one cup on the side track. He knows that the only way to save the five cups is to divert the train to the side track, which will break the cup on the side track. Billy proceeds to pull the lever and the train is diverted down the side track, breaking one of the cups. (Nichols and Mallon 2006)

10) Tea Cups Footbridge When Susie’s mother leaves the house one day, she says “you are forbidden from breaking any of the teacups that are on the counter.” While Susie is playing in her bedroom, her 18 month old brother Fred has taken down several of the teacups and he has also turned on a mechanical toy truck, which is about to crush 5 of the cups. As Fred leaves the room, Susie walks in and sees that the truck is about to wreck the cups. She is standing next to the counter with the remaining teacups and she realizes that the only way to stop the truck in time is by

260 throwing one of the teacups at the truck, which will break the cup she throws. Susie is in fact an excellent thrower and knows that if she throws the teacup at the truck she will save the five cups. Susie proceeds to throw the teacup, which breaks that cup, but it stops the truck and saves the five other teacups. (Nichols and Mallon 2006)

11) Catastrophic Footbridge A train is transporting an extremely dangerous artificially produced virus to a safe disposal site. The virus is profoundly contagious and nearly always leads to the death of the victim within a matter of weeks. If the virus were to be released into the atmosphere, billions of people would die from it. Indeed, there is a chance that it will wipe out more than half of the human population. Jonas is one of the scientists who was responsible for ensuring that the virus would be destroyed, and he is watching the train from a footbridge. As the train is approaching he sees through his binoculars that there is a powerful bomb planted on the tracks ahead, and there is no way for him to communicate with the train operators to get them to stop the train in time. If the train passes over the bomb, it will explode and the virus will be released into the environment with catastrophic consequences. There is a large stranger looking over the footbridge next to Jonas. Jonas knows that the stranger has nothing to do with the bomb, but the only way to stop the train from hitting the bomb is to push this stranger over the railing. For unlike Jonas’s body, the stranger’s body is big enough that it will bring the train to a halt, although this will kill the stranger. Jonas proceeds to push the stranger over the railing, which kills the stranger, but it prevents the explosion and saves billions of people from dying from the virus. (Nichols and Mallon 2006)

12) Chicken A man goes to the supermarket once a week and buys a dead chicken. But before cooking the chicken, he has sexual intercourse with it. Then he cooks it and eats it. (Haidt, Koller, and M.G. Dias 1993)

261

13) Flag A woman is cleaning out her closet, and she finds her old [American or Brazilian] flag. She doesn’t want the flag anymore so she cuts it up into pieces and uses the rags to clean her bathroom. (Haidt, Koller, and M.G. Dias 1993)

14) Dog A family’s dog was killed by a car in front of their house. They had heard that dog meat was delicious, so they cut up the dog’s body and cooked it and ate it for dinner. (Haidt, Koller, and M.G. Dias 1993)

262

APPENDIX B

TWO- AND FOUR-FACTOR MODELS OF PSYCHOPATHY

Item Two-factor model Four-factor model 1 Glibness/superficial charm 1 1 2 Grandiose sense of self-worth 1 1 3 Need for stimulation 2 3 4 Pathological lying 1 1 5 Conning/manipulative 1 1 6 Lack of remorse or guilt 1 2 7 Shallow affect 1 2 8 Callous/lack of empathy 1 2 9 Parasitic lifestyle 2 3 10 Poor behavioral controls 2 4 11 Promiscuous sexual behavior – – 12 Early behavioral problems 2 4 13 Lack of realistic, long-term goals 2 3 14 Impulsivity 2 3 15 Irresponsibility 2 3 16 Failure to accept responsibility 1 2 17 Many marital relationships – – 18 Juvenile delinquency 2 4 19 Revocation of conditional release 2 4 20 Criminal versatility – 4

Two-factor model: Factor 1: Interpersonal/Affective; Factor 2: Social Deviance Four-factor model: Factor 1: Interpersonal; Factor 2: Affective; Factor 3: Lifestyle; Factor 4: Antisocial (adapted from Kiehl et al. 2001)

263

REFERENCES

Ackerly, S.S., and H.L. Benton. 1948. Report of case of bilateral frontal lobe defect. Research Publications--Association for Research in Nervous and Mental Disease 27: 479-504.

Adolphs, R., D. Tranel, and A.R. Damasio. 1998. The human amygdala in social judgment. Nature 393: 470-474.

Adolphs, R., D. Tranel, H. Damasio, and A.R. Damasio. 1995. Fear and the human amygdala. Journal of Neuroscience 15: 5879-5891.

Ahn, W.K., C.W. Kalish, D.L. Medin, and S.A. Gelman. 1995. The role of covariation versus mechanism information in causal attribution. Cognition 54: 299-352.

Algoe, S.B., and J. Haidt. 2009. Witnessing excellence in action: the ‘other-praising’ emotions of elevation, gratitude, and admiration. The journal of 4: 105-127.

Allan, L.G., S. Siegel, and S. Hannah. 2007. The sad truth about depressive realism. The Quarterly Journal of 60: 482.

Alloy, L.B., and L.Y. Abramson. 1979. Judgment of contingency in depressed and nondepressed students: sadder but wiser? Journal of Experimental Psychology. General 108: 441-485.

American Psychiatric Association. 1994. Diagnostic and Statistical Manual of Mental Disorders. 4th ed. Washington, D.C.: American Psychiatric Association.

Anderson, S.W., J. Barrash, A. Bechara, and D. Tranel. 2006. Impairments of Emotion and Real-World Complex Behavior Following Childhood- or Adult-Onset Damage to Ventromedial Prefrontal Cortex. Journal of the International Neuropsychological Society 12: 224-235.

Anderson, S.W., A. Bechara, H. Damasio, D. Tranel, and A.R. Damasio. 1999. Impairment of social and moral behavior related to early damage in human prefrontal cortex. Nature Neuroscience 2: 1032-1037.

Anderson, S.W., H. Damasio, D. Tranel, and A.R. Damasio. 2000. Long-Term Sequelae of Prefrontal Cortex Damage Acquired in Early Childhood. Developmental 18: 281-296.

Aniskiewicz, A.S. 1979. Autonomic components of vicarious conditioning and

264 psychopathy. Journal of 35: 60-67.

Atran, S., and D.L. Medin. 2008. The Native Mind and the Cultural Construction of Nature. Cambridge, MA: MIT Press.

Babiak, P., and R.D. Hare. 2006. Snakes in Suits: When Psychopaths Go to Work. New York: HarperBusiness.

Bailenson, J.N., M.S. Shum, S. Atran, D.L. Medin, and J.D. Coley. 2002. A bird's eye view: biological categorization and reasoning within and across cultures. Cognition 84: 1-53.

Baird, A.A., J.A. Fugelsang, and C. Bennett. 2005. "What were you thinking?": An fMRI study of adolescent decision making. Poster presented at the annual meeting of the Society, New York.

Baird, A.D., S.J. Wilson, P.F. Bladin, M.M. Saling, and D.C. Reutens. 2007. Neurological control of human sexual behaviour: insights from lesion studies. Journal of Neurology, Neurosurgery & Psychiatry 78: 1042 -1049.

Baird, J.A., and J.W. Astington. 2004. The role of mental state understanding in the development of moral cognition and moral action. New Directions for Child and Adolescent Development, no. 103: 37-49.

Balleine, B.W., N.D. Daw, and J.P. O'Doherty. 2009. Multiple Forms of Value Learning and the Functions of Dopamine. In Neuroeconomics: Decision Making and the Brain, ed. P.W. Glimcher, C. Camerer, R.A. Poldrack, and E. Fehr, 367-87. New York: Academic Press.

Barbizet, J. 1970. Human memory and its pathology. San Francisco, CA: W. H. Freeman.

Bargh, J.A. 2006. Social Psychology and the Unconscious: The Automaticity of Higher Mental Processes. Philadelphia: Psychology Press.

Barrett, L.F. 2006. Solving the Emotion Paradox: Categorization and the Experience of Emotion. Personality and Social Psychology Review 10: 20 -46.

Barrett, L.F., and E. Bliss-Moreau. 2009. Affect as Psychological Primitive. In Advances in Experimental Social Psychology, ed. M.P. Zanna, 41:167-218. Burlington, VT: Academic Press.

Bartels, D.M., and D.L. Medin. 2007. Are Morally Motivated Decision Makers Insensitive to the Consequences of Their Choices? Psychological Science 18: 24 - 28.

Baumeister, R.F., A.M. Stillwell, and T.F. Heatherton. 1994. Guilt: an interpersonal

265 approach. Psychological Bulletin 115: 243-267.

Baumeister, R.F., and K.D. Vohs. 2003. Social Psychology Articles From the 1980s and 1990s: Some New Classics and Overlooked Gems. Psychological Inquiry 14: 193–195.

Baxter, M.G., and E.A. Murray. 2002. The amygdala and reward. Nature Reviews Neuroscience 3: 563-573.

Bechara, A. 2005. Decision making, impulse control and loss of willpower to resist drugs: a neurocognitive perspective. Nature Neuroscience 8: 1458-1463.

Bechara, A., and A.R. Damasio. 2005. The somatic marker hypothesis: A neural theory of economic decision. Games and Economic Behavior 52: 336–372.

Bechara, A., A.R. Damasio, H. Damasio, and S.W. Anderson. 1994. Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50: 7-15.

Bechara, A., H. Damasio, A.R. Damasio, and G.P. Lee. 1999. Different Contributions of the Human Amygdala and Ventromedial Prefrontal Cortex to Decision-Making. Journal of Neuroscience 19: 5473-5481.

Bechara, A., H. Damasio, D. Tranel, and A.R. Damasio. 1997. Deciding Advantageously Before Knowing the Advantageous Strategy. Science 275: 1293-1295.

Bechara, A., S. Dolan, N. Denburg, A. Hindes, S.W. Anderson, and P.E. Nathan. 2001. Decision-making deficits, linked to a dysfunctional ventromedial prefrontal cortex, revealed in alcohol and stimulant abusers. Neuropsychologia 39: 376-389.

Bechara, A., D. Tranel, H. Damasio, R.. Adolphs, C. Rockland, and A.R. Damasio. 1995. Double dissociation of conditioning and declarative knowledge relative to the amygdala and hippocampus in humans. Science 269: 1115-1118.

Bechara, A., D. Tranel, H. Damasio, and A.R. Damasio. 1996. Failure to Respond Autonomically to Anticipated Future Outcomes Following Damage to Prefrontal Cortex. Cerebral Cortex 6: 215-225.

Beckers, T., J. De Houwer, O. Pineño, and R.R. Miller. 2005. Outcome Additivity and Outcome Maximality Influence Cue Competition in Human Causal Learning. Journal of Experimental Psychology: Learning, Memory, and Cognition 31: 238- 249.

Bedford, E. 1956. Emotions. Proceedings of the Aristotelian Society 57: 281-304.

Beer, J.S., O.P. John, D. Scabini, and R.T. Knight. 2006. Orbitofrontal Cortex and Social

266 Behavior: Integrating Self-monitoring and Emotion-Cognition Interactions. Journal of Cognitive Neuroscience 18: 871-879.

Berkowitz, L. 1993. Aggression: Its Causes, Consequences, and Control. Philadelphia: Temple University Press.

Bernstein, I.L. 1985. Learned Food Aversions in the Progression of Cancer and Its Treatment. Annals of the New York Academy of Sciences 443: 365-380.

Berridge, K.C., and M.L. Kringelbach. 2008. of pleasure: reward in humans and animals. 199: 457-480.

Berridge, K.C., and T.E. Robinson. 1998. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Research. Brain Research Reviews 28: 309-369.

Berridge, K.C., and E.S. Valenstein. 1991. What Psychological Process Mediates Feeding Evoked by Electrical Stimulation of the Lateral Hypothalamus? 105: 3-14.

Birbaumer, N., R. Veit, M. Lotze, M. Erb, C. Hermann, W. Grodd, and H. Flor. 2005. Deficient Fear Conditioning in Psychopathy: A Functional Magnetic Resonance Imaging Study. Archives of General Psychiatry 62: 799-805.

Blackburn, S. 1984. Spreading the Word: Groundings in the Philosophy of Language. Oxford: Oxford University Press.

———. 1985. Errors and the Phenomenology of Value. In Morality and Objectivity: a tribute to J.L. Mackie, ed. T. Honderich. London: Routledge & Kegan Paul.

———. 1993. Essays in quasi-realism. Oxford: Oxford University Press.

Blair, R.J. 1995. A cognitive developmental approach to morality: investigating the psychopath. Cognition 57: 1-29.

———. 1997. Moral reasoning and the child with psychopathic tendencies. Personality and Individual Differences 22: 731-739.

———. 2001. Neurocognitive models of aggression, the antisocial personality disorders, and psychopathy. Journal of Neurology, Neurosurgery, and Psychiatry 71: 727- 731.

———. 2007. The amygdala and ventromedial prefrontal cortex in morality and psychopathy. Trends in Cognitive Sciences 11: 387-392.

———. 2010. Psychopathy, frustration, and reactive aggression: The role of

267 ventromedial prefrontal cortex. British Journal of Psychology 101: 383-399.

Blair, R.J., E. Colledge, and D. G. V. Mitchell. 2001. Somatic Markers and Response Reversal: Is There Orbitofrontal Cortex Dysfunction in Boys with Psychopathic Tendencies? Journal of Abnormal Child Psychology 29: 499-511.

Blair, R.J., L. Jones, F. Clark, and M. Smith. 1997. The psychopathic individual: A lack of responsiveness to distress cues? 34: 192-198.

Blair, R.J., D.G.V. Mitchell, and K.S. Blair. 2005. The Psychopath: Emotion and the Brain. Malden, MA: Wiley-Blackwell.

Blair, R.J., D.G.V. Mitchell, A. Leonard, S. Budhani, K. S. Peschardt, and C. Newman. 2004. Passive avoidance learning in individuals with psychopathy: modulation by reward but not by punishment. Personality and Individual Differences 37: 1179- 1192.

Blair, R.J., D.G.V. Mitchell, R.A. Richell, S. Kelly, A. Leonard, C. Newman, and S.K. Scott. 2002. Turning a Deaf Ear to Fear: Impaired Recognition of Vocal Affect in Psychopathic Individuals. Journal of 111: 682-686.

Blaisdell, A.P., K. Sawa, K.J. Leising, and M.R. Waldmann. 2006. Causal Reasoning in Rats. Science 311: 1020-1022.

Boakes, R.A. 2009. Learning Without Thinking. Behavioral and Brain Sciences 32: 202- 203.

Botvinick, M.M., T.S. Braver, D.M. Barch, C.S. Carter, and J.D. Cohen. 2001. Conflict monitoring and cognitive control. 108: 624-652.

Boyd, R. 1988. How to be a moral realist. In Essays on , ed. G. Sayre- McCord, 181-228. Ithaca, NY: Cornell University Press.

Braddon-Mitchell, D., and R. Nola. 2009. Conceptual Analysis and Philosophical Naturalism. Cambridge, MA: The MIT Press.

Bradley, A.C. 1904. Shakespearean tragedy. New York: Macmillan and Co.

Brandt, R.B. 1954. Hopi Ethics: A Theoretical Analysis. Chicago: University of Chicago Press.

———. 1979. A Theory of the Good and Right. Oxford: Oxford University Press.

Brewer, W.F. 1974. There is no convincing evidence for operant or classical conditioning in adult humans. In Cognition and the Symbolic Processes, ed. W.B. Weimer and D.S. Palermo, 1-42. Hillsdale, NJ: Lawrence Erlbaum Associates.

268

Brink, D.O. 1989. Moral Realism and the Foundations of Ethics. Cambridge: Cambridge University Press.

Buchanan, T.W., D. Tranel, and R. Adolphs. 2009. The Human Amygdala in Social Function. In The Human Amygdala, ed. P.J. Whalen and E.A. Phelps, 289-318. New York: Guilford Press.

Budhani, S., R.A. Richell, and R.J. Blair. 2006. Impaired reversal but intact acquisition: Probabilistic response reversal deficits in adult individuals with psychopathy. Journal of abnormal psychology 115: 552–558.

Caldwell, M.F., D.J. McCormick, D. Umstead, and G.J. Van Rybroek. 2007. Evidence of Treatment Progress and Therapeutic Outcomes Among Adolescents With Psychopathic Features. Criminal Justice and Behavior 34: 573-587.

Camille, N., G. Coricelli, J. Sallet, P. Pradat-Diehl, J.-R. Duhamel, and A. Sirigu. 2004. The Involvement of the Orbitofrontal Cortex in the Experience of Regret. Science 304: 1167-1170.

Campbell, R. 2006. The Problem of Moral Judgment. In Engaged Philosophy: Essays in Honour of David Braybrooke, ed. S. Sherwin and P. Schotch, 249-270. Toronto: University of Toronto Press.

———. 2007. What is Moral Judgment? The Journal of Philosophy 104: 321-349.

Cardinal, R.N., J.A. Parkinson, J. Hall, and B.J. Everitt. 2002. Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neuroscience & Biobehavioral Reviews 26: 321-352.

Carey, S. 2009. The Origin of Concepts. Oxford: Oxford University Press.

Carter, C.S., and V. van Veen. 2007. Anterior cingulate cortex and conflict detection: an update of theory and data. Cognitive, Affective & Behavioral Neuroscience 7: 367-379.

Castro, L., and E.A. Wasserman. 2010. Animal learning. Wiley Interdisciplinary Reviews: Cognitive Science 1: 89-98.

Chaiken, S., and Y. Trope. 1999. Dual-Process Theories in Social Psychology. New York: The Guilford Press.

Chambers, L., S. Mobini, and M.R. Yeomans. 2007. Caffeine deprivation state modulates expression of acquired liking for caffeine-paired flavours. The Quarterly Journal of Experimental Psychology 60: 1356.

269 Chapman, H.A., D.A. Kim, J.M. Susskind, and A.K. Anderson. 2009. In Bad Taste: Evidence for the Oral Origins of Moral Disgust. Science 323: 1222 -1226.

Cheng, P.W. 1997. From Covariation to Causation: A Causal Power Theory. Psychological Review 104: 367-405.

Chomsky, N. 1980. Rules and representations. New York: Columbia University Press.

Ciaramelli, E., M. Muccioli, E. Làdavas, and G. di Pellegrino. 2007. Selective deficit in personal moral judgment following damage to ventromedial prefrontal cortex. Social cognitive and affective neuroscience 2: 84-92.

Cima, M., F. Tonnaer, and M.D. Hauser. 2010. Psychopaths know right from wrong but don't care. Social Cognitive and Affective Neuroscience 5: 59-67.

Clark, L., S.R. Chamberlain, and B.J. Sahakian. 2009. Neurocognitive Mechanisms in Depression: Implications for Treatment. Annual Review of Neuroscience 32: 57- 74.

Cobos, P.L., F.J. López, A. Caño, J. Almaraz, and D.R. Shanks. 2002. Mechanisms of Predictive and Diagnostic Causal Induction. Journal of Experimental Psychology: Animal Behavior Processes 28: 331-346.

Cobos, P.L., F.J. López, and D. Luque. 2007. Interference between cues of the same outcome depends on the causal interpretation of the events. The Quarterly Journal of Experimental Psychology 60: 369.

Coccaro, E.F., M.S. McCloskey, D.A. Fitzgerald, and K.L. Phan. 2007. Amygdala and orbitofrontal reactivity to social threat in individuals with impulsive aggression. Biological Psychiatry 62: 168-178.

Cook, S.W., and R.E. Harris. 1937. The verbal conditioning of the galvanic skin reflex. Journal of Experimental Psychology 21: 202-210.

Copp, D. 2001. Realist-Expressivism: A neglected option for moral realism. and Policy 18: 1-43.

Corkin, S. 1968. Acquisition of motor skill after bilateral medial temporal-lobe excision. Neuropsychologia 6: 255-265.

———. 2002. What's new with the amnesic patient H.M.? Nature Reviews Neuroscience 3: 153-160.

Corlett, P.R., M.R.F. Aitken, A. Dickinson, D.R. Shanks, G.D. Honey, R.A.E. Honey, T.W. Robbins, E.T. Bullmore, and P.C. Fletcher. 2004. Prediction error during retrospective revaluation of causal associations in humans: fMRI evidence in

270 favor of an associative model of learning. Neuron 44: 877-888.

Craig, A.D. 2009. How do you feel--now? The anterior insula and human awareness. Nature Reviews Neuroscience 10: 59-70.

Critchley, H.D., C.J. Mathias, and R.J. Dolan. 2001. Neuroanatomical basis for first- and second-order representations of bodily states. Nature Reviews Neuroscience 4: 207-212.

———. 2002. Fear Conditioning in Humans: The Influence of Awareness and Autonomic Arousal on Functional Neuroanatomy. Neuron 33: 653-663.

Cushman, F. 2008. Crime and punishment: Distinguishing the roles of causal and intentional analyses in moral judgment. Cognition 108: 353-380.

Cushman, F., L. Young, and M.D. Hauser. 2006. The Role of Conscious Reasoning and Intuition in Moral Judgment: Testing Three Principles of Harm. Psychological Science 17: 1082-1089.

Damasio, A.R. 1994. Descartes' Error. New York: Putnam.

Damasio, A.R., D. Tranel, and H. Damasio. 1990. Individuals with sociopathic behavior caused by frontal damage fail to respond autonomically to social stimuli. Behavioural Brain Research 41: 81-94.

Daniels, N. 1979. Wide Reflective Equilibrium and Theory Acceptance in Ethics. The Journal of Philosophy 76: 256-282.

Danks, D. 2007. Theory Unification and Graphical Models of Human Categorization. In Causal Learning: Psychology, Philosophy, and Computation, ed. A. Gopnik and L. Schulz, 173-189. Oxford: Oxford University Press.

Danziger, K. 2008. Marking the Mind: A History of Memory. Cambridge, MA: Cambridge University Press.

Darwall, S., A. Gibbard, and P. Railton. 1992. Toward Fin de siècle Ethics: Some Trends. The Philosophical Review 101: 115-189.

Darwin, C. 1859. On the Origin of Species. London: John Murray.

Dayan, P., and Y. Niv. 2008. Reinforcement learning: the good, the bad and the ugly. Current Opinion in Neurobiology 18: 185-196.

De Houwer, J. 2009. The propositional approach to associative learning as an alternative for association formation models. Learning & Behavior 37: 1-20.

271 De Houwer, J., and T. Beckers. 2002a. Higher-order retrospective revaluation in human causal learning. The Quarterly Journal of Experimental Psychology Section B: Comparative and 55: 137.

———. 2002b. Second-Order Backward Blocking and Unovershadowing in Human Causal Learning. Experimental Psychology 49: 27-33.

———. 2003. Secondary task difficulty modulates forward blocking in human contingency learning. The Quarterly Journal of Experimental Psychology Section B: Comparative and Physiological Psychology 56: 345.

De Houwer, J., T. Beckers, and S. Glautier. 2002. Outcome and cue properties modulate blocking. The Quarterly Journal of Experimental Psychology Section A: Human Experimental Psychology 55: 965.

Denis, L., and E.N. Zalta. 2008. Kant and Hume on Morality. In Stanford Encyclopedia of Philosophy. http://plato.stanford.edu/entries/kant-hume-morality/.

Dias, R., T. W. Robbins, and A. C. Roberts. 1996. Dissociation in prefrontal cortex of affective and attentional shifts. Nature 380: 69-72.

Dickinson, A. 1980. Contemporary animal learning theory. New York: Cambridge University Press.

Dickinson, A., and B.W. Balleine. 2009. Hedonics: The Cognitive-Motivational Interface. In Pleasures of the Brain, ed. M.L. Kringelbach and K.C. Berridge, 74- 84. Oxford: Oxford University Press.

Dickinson, A., and K.J. Brown. 2007. Flavor-evaluative conditioning is unaffected by contingency knowledge during training with color-flavor compounds. Learning & Behavior 35: 36-42.

Dickinson, A., and J. Burke. 1996. Within compound Associations Mediate the Retrospective Revaluation of Causality Judgements. The Quarterly Journal of Experimental Psychology Section B: Comparative and Physiological Psychology 49: 60-80.

Dickinson, A., D.R. Shanks, and J. Evenden. 1984. Judgement of act-outcome contingency: The role of selective attribution. The Quarterly Journal of Experimental Psychology Section A: Human Experimental Psychology 36: 29.

Doris, J.M. 2002. Lack of Character: Personality and Moral Behavior. Cambridge: Cambridge University Press.

Doris, J.M., and S. Stich. 2006. Moral Psychology: Empirical Approaches. In Stanford Encyclopedia of Philosophy, ed. E.N. Zalta.

272 http://plato.stanford.edu/entries/moral-psych-emp/.

Dowden, B. 1993. Logical Reasoning. Belmont, CA: Wadsworth Publishing Company.

Dreier, J. 2000. Dispositions and Fetishes: Externalist Models of Moral Motivation. Philosophy and Phenomenological Research 61: 619-638.

Dretske, F.I. 1981. Knowledge and the Flow of Information. Cambridge, MA: MIT Press.

———. 1986. Misrepresentation. In Belief: Form, Content and Function, ed. R.J. Bogdan. Oxford: Oxford University Press.

Dulany, D.E. 1961. Hypotheses and habits in verbal "operant conditioning.'. Journal of Abnormal and Social Psychology 63: 251-263.

Dunn, B.D., T. Dalgleish, and A.D. Lawrence. 2006. The somatic marker hypothesis: A critical evaluation. Neuroscience & Biobehavioral Reviews 30: 239-271.

Dwyer, S. 1999. Moral Competence. In Philosophy and Linguistics, ed. K. Murasugi and R. Stainton. Boulder, CO: Westview Press.

———. 2009. Moral Dumbfounding and the Linguistic Analogy: Methodological Implications for the Study of Moral Judgment. Mind & Language 24: 274-296.

Dwyer, S., and M.D. Hauser. 2008. Dupoux and Jacob's moral instincts: throwing out the baby, the bathwater and the bathtub. Trends in Cognitive Sciences 12: 1-2.

Dwyer, S., B. Huebner, and M.D. Hauser. 2010. The Linguistic Analogy: Motivations, Results, and Speculations. Topics in Cognitive Science 2: 486-510.

Edel, M., and A. Edel. 1959. Anthropology and Ethics. Springfield, IL: Charles C. Thomas.

Ekman, P. 1972. Emotion in the Human Face. New York: Pergamon Press.

———. 1999. Basic Emotions. In Handbook of Cognition and Emotion, ed. T. Dalgleish and M.J. Power. Chichester: John Wiley & Sons.

Evans, J.St.B.T. 2009. How many dual-process theories do we need? One, two, or many? In , ed. J.St.B.T. Evans and K. Frankish. Oxford: Oxford University Press.

Evans, J.St.B.T., and K. Frankish. 2009. In Two Minds: Dual Processes and Beyond. Oxford: Oxford University Press.

Evans, J.St.B.T., and P. Wason. 1976. Rationalization in a reasoning task. British Jornal of Psychology 67: 479-86.

273

Everitt, B.J., R.N. Cardinal, J.A. Parkinson, and T.W. Robbins. 2003. Appetitive behavior: impact of amygdala-dependent mechanisms of emotional learning. Annals of the New York Academy of Sciences 985: 233.

Eyal, T., N. Liberman, and Y. Trope. 2008. Judging near and distant virtue and vice. Journal of Experimental Social Psychology 44: 1204-1209.

Fehr, E., U. Fischbacher, and S. Gächter. 2002. Strong reciprocity, human cooperation, and the enforcement of social norms. Human Nature 13: 1-25.

Fellows, L.K., and M.J. Farah. 2003. Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. Brain 126: 1830- 1837.

———. 2005a. Different Underlying Impairments in Decision-making Following Ventromedial and Dorsolateral Frontal Lobe Damage in Humans. Cerebral Cortex 15: 58-63.

———. 2005b. Dissociable elements of human foresight: a role for the ventromedial frontal lobes in framing the future, but not in discounting future rewards. Neuropsychologia 43: 1214-1221.

———. 2007. The Role of Ventromedial Prefrontal Cortex in Decision Making: Judgment under Uncertainty or Judgment Per Se? Cerebral Cortex 17: 2669 - 2674.

Fenker, D.B., M.A. Schoenfeld, M.R. Waldmann, H. Schuetze, H.-J. Heinze, and E. Duezel. 2010. “Virus and Epidemic”: Causal Knowledge Activates Prediction Error Circuitry. Journal of Cognitive Neuroscience 22: 2151-2163.

Fessler, D.M.T., A.P. Arguello, J.M. Mekdara, and R. Macias. 2003. Disgust sensitivity and meat consumption: a test of an emotivist account of moral vegetarianism. Appetite 41: 31-41.

Finger, E.C., A.A. Marsh, D.G.V. Mitchell, M.E. Reid, C. Sims, S. Budhani, D.S. Kosson, et al. 2008. Abnormal Ventromedial Prefrontal Cortex Function in Children With Psychopathic Traits During Reversal Learning. Archives of General Psychiatry 65: 586-594.

Fiske, S.T. 1992. Thinking is for doing: portraits of social cognition from daguerreotype to laserphoto. Journal of Personality and Social Psychology 63: 877-889.

FitzGerald, T.H., B. Seymour, and R.J. Dolan. 2009. The Role of Human Orbitofrontal Cortex in Value Comparison for Incommensurable Objects. Journal of Neuroscience 29: 8388-8395.

274

Flor, H., N. Birbaumer, C. Hermann, S. Ziegler, and C.J. Patrick. 2002. Aversive Pavlovian Conditioning in Psychopaths: Peripheral and Central Correlates. Psychophysiology 39: 505-518.

Fodor, J.A. 1975. Language of Thought. Cambridge, MA: Harvard University Press.

———. 1983. The modularity of mind. Cambridge, MA: MIT Press.

Foerde, K., B.J. Knowlton, and R.A. Poldrack. 2006. Modulation of competing memory systems by distraction. Proceedings of the National Academy of Sciences 103: 11778 -11783.

Foerde, K., R.A. Poldrack, and B.J. Knowlton. 2007. Secondary-task effects on classification learning. Memory & Cognition 35: 864-874.

Foot, P. 1967. The problem of abortion and the doctrine of double effect. Oxford Review 5: 5-15.

———. 1995. Morality as a System of Hypothetical Imperatives. In 20th Century Ethical Theory, ed. S.M. Cahn and J.G. Haber. Englewood Cliffs, NJ: Prentice Hall.

———. 2002. Virtues and Vices: And Other Essays in Moral Philosophy. Oxford: Oxford University Press.

Forth, A.E., D.S. Kosson, and R.D. Hare. 2003. Hare psychopathy checklist: youth version. Toronto: Multi-Health Systems.

Fox, E. 2008. Emotion science: cognitive and neuroscientific approaches to understanding human emotions. New York: Palgrave Macmillan.

Frank, M.J., M.X. Cohen, and A.G. Sanfey. 2009. Multiple Systems in Decision Making. Current Directions in Psychological Science 18: 73 -77.

Frank, M.J., L.C. Seeberger, and R.C. O'Reilly. 2004. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306: 1940-1943.

Frank, R.H. 1988. Passions Within Reasons. New York: W. W. Norton & Company.

———. 2001. Cooperation through emotional commitment. Evolution and the capacity for commitment 3: 57–76.

Frankena, W. 1958. Obligation and motivation in recent moral philosophy. In Essays in moral philosophy, ed. A.I. Melden, 40–81. Seattle: University of Washington Press.

275 ———. 1965. Ethics. Englewood Cliffs, NJ: Prentice Hall.

Frankfurt, H.G. 1971. Freedom of the Will and the Concept of a Person. The Journal of Philosophy 68: 5-20.

Frederick, S. 2005. Cognitive Reflection and Decision Making. The Journal of Economic Perspectives 19: 25-42.

Fugelsang, J., and K.N. Dunbar. 2009. Brain-based mechanisms underlying causal reasoning. In Neural Correlates of Thinking, ed. E. Kraft, B. Gulyás, and E. Pöppel. Berlin: Springer.

Gailliot, M.T., B. Michelle Peruche, E.A. Plant, and R.F. Baumeister. 2009. Stereotypes and prejudice in the blood: Sucrose drinks reduce prejudice and stereotyping. Journal of Experimental Social Psychology 45: 288-290.

Garrett, D. 1997. Cognition and Commitment in Hume's Philosophy. Oxford: Oxford University Press.

Gazzaniga, M.S. 1998. The Mind's Past. Berkeley, CA: University of California Press.

Geach, P.T. 1965. Assertion. The Philosophical Review 74: 449-465.

Gerrans, P., and J. Kennett. 2010. Neurosentimentalism and Moral Agency. Mind 119: 585 -614.

Gert, B. 2005. Morality: Its Nature and Justification. Oxford: Oxford University Press.

———. 2008. The Definition of Morality. In Stanford Encyclopedia of Philosophy, ed. E.N. Zalta. http://plato.stanford.edu/entries/morality-definition/.

Gert, J. 2009. Response-Dependence and Normative Bedrock. Philosophy and Phenomenological Research 79: 718-742.

Ghirlanda, S. 2005. Retrospective revaluation as simple associative learning. Journal of Experimental Psychology. Animal Behavior Processes 31: 107-111.

Gigerenzer, G. 2005. I Think, Therefore I Err. Social Research 72: 1-24.

Gilbert, D.T. 2006. Stumbling on Happiness. New York: Knopf.

Gilbert, D.T., E.C. Pinel, T.D. Wilson, S.J. Blumberg, and T.P. Wheatley. 2002. Durability bias in . In Heuristics and Biases: The psychology of intuitive judgment, ed. T. Gilovich, D.W. Griffin, and D. Kahneman, 292-312. New York: Cambridge University Press.

276 Gilbert, D.T., R. Tafarodi, and P. Malone. 1993. You Can't Not Believe Everything You Read. Jouranl of Personality and Social Psychology 65: 221-233.

Gilbert, D.T., and T.D. Wilson. 2000. Miswanting. In Thinking and feeling: The role of affect in social cognition, ed. J. Forgas, 178-197. Cambridge: Cambridge University Press.

———. 2007. Prospection: experiencing the future. Science 317: 1351-1354.

Gladwell, M. 2005. Blink: The Power of Thinking Without Thinking. New York: Little, Brown and Company.

Glenn, A.L., A. Raine, and R.A. Schug. 2009. The neural correlates of moral decision- making in psychopathy. Molecular Psychiatry 14: 5-6.

Glenn, A.L., A. Raine, R.A. Schug, L. Young, and M.D. Hauser. 2009. Increased DLPFC activity during moral decision-making in psychopathy. Molecular Psychiatry 14: 909-911.

Glimcher, P.W., C. Camerer, R.A. Poldrack, and E. Fehr. 2009. Neuroeconomics: Decision Making and the Brain. New York: Academic Press.

Glymour, C. 2003. Learning, prediction and causal Bayes nets. Trends in Cognitive Sciences 7: 43-48.

Goldberg, R.F., and S.L. Thompson-Schill. 2009. Developmental “Roots” in Mature Biological Knowledge. Psychological Science 20: 480 -487.

Gopnik, A., C. Glymour, D.M. Sobel, L. Schulz, T. Kushnir, and D. Danks. 2004. A Theory of Causal Learning in Children: Causal Maps and Bayes Nets. Psychological Review 111: 3-32.

Gordon, H.L., A.A. Baird, and A. End. 2004. Functional differences among those high and low on a trait measure of psychopathy. Biological Psychiatry 56: 516-521.

Grant, S., C. Contoreggi, and E.D. London. 2000. Drug abusers show impaired performance in a laboratory test of decision making. Neuropsychologia 38: 1180- 1187.

Greene, J.D. 2004. Neuron - Supplemental Data. http://www.cell.com/neuron/supplemental/S0896-6273(04)00634-8.

———. 2008. The Secret Joke of Kant's Soul. In Moral Psychology, ed. W. Sinnott- Armstrong. Vol. 3. Cambridge, MA: MIT Press.

———. 2009. Dual-process morality and the personal/impersonal distinction: A reply to

277 McGuire, Langdon, Coltheart, and Mackenzie. Journal of Experimental Social Psychology 45: 581-584.

Greene, J.D., F. Cushman, K.E. Stewart, K. Lowenberg, L.E. Nystrom, and J.D. Cohen. 2009. Pushing moral buttons: The interaction between personal force and intention in moral judgment. Cognition 111: 364-371.

Greene, J.D., S.A. Morelli, K. Lowenberg, L.E. Nystrom, and J.D. Cohen. 2008. Cognitive load selectively interferes with utilitarian moral judgment. Cognition 107: 1144-1154.

Greene, J.D., L.E. Nystrom, A.D. Engell, J.M. Darley, and J.D. Cohen. 2004. The Neural Bases of Cognitive Conflict and Control in Moral Judgment. Neuron 44: 389-400.

Greene, J.D., R.B. Sommerville, L.E. Nystrom, J.M. Darley, and J.D. Cohen. 2001. An fMRI Investigation of Emotional Engagement in Moral Judgment. Science 293: 2105-2108.

Greenspoon, J. 1955. The Reinforcing Effect of Two Spoken Sounds on the Frequency of Two Responses. The American Journal of Psychology 68: 409-416.

Greenwood, J.D. 2009. Cognition, Consciousness, and the Cognitive Revolution. Behavioral and Brain Sciences 32: 209-210.

Gretton, H.M., R.D. Hare, and R.E.H. Catchpole. 2004. Psychopathy and offending from adolescence to adulthood: A 10-year follow-up. Journal of Consulting and Clinical Psychology 72: 636-645.

Griffiths, P.E. 1997. What Emotions Really Are: The Problem of Psychological Categories. Chicago: University of Chicago Press.

———. 2002. What is Innateness? The Monist 85: 70-85.

———. 2003. Basic Emotions, Complex Emotions, Machiavellian Emotions. Royal Institute of Philosophy Supplements 52: 39-67.

———. 2004. Is emotion a natural kind? In Thinking about feeling, 233–249.

Griffiths, P.E., and A. Scarantino. 2009. Emotions in the wild: The situated perspective on emotion. In Cambridge handbook of situated cognition, ed. P. Robbins and M. Aydede, 437-453. Cambridge: Cambridge University Press.

Grusec, J.E., and J.J. Goodnow. 1994. Impact of parental discipline methods on the child's internalization of values: A reconceptualization of current points of view. Developmental Psychology 30: 4-19.

278 Guglielmo, S., A.E. Monroe, and B.F Malle. 2009. At the Heart of Morality Lies Folk Psychology. Inquiry: An Interdisciplinary Journal of Philosophy 52: 449.

Gutbrod, K., C. Krouzel, H. Hofer, R. Müri, W. Perrig, and R. Ptak. 2006. Decision- making in amnesia: do advantageous decisions require conscious knowledge of previous behavioural choices? Neuropsychologia 44: 1315-1324.

Hacken, P.T. 2007. Chomskyan Linguistics and Its Competitors. Oakville, CT: Equinox Publishing.

Hagmayer, Y., S.A. Sloman, D.A. Lagnado, and M.R. Waldmann. 2007. Causal Reasoning Through Intervention. In Causal Learning: Psychology, Philosophy, and Computation, ed. A. Gopnik and L. Schulz, 86-100. Oxford: Oxford University Press.

Haidt, J. 2001. The emotional dog and its rational tail: a social intuitionist approach to moral judgment. Psychological Review 108: 814-834.

———. 2003. The Moral Emotions. In Handbook of Affective Sciences, ed. R.J. Davidson, K.R. Scherer, and H.H. Goldsmith, 852-870. Oxford: Oxford University Press.

———. 2007. The New Synthesis in Moral Psychology. Science 316: 998 -1002.

———. 2008. Morality. Perspectives on Psychological Science 3: 65-72.

Haidt, J., and F. Bjorkland. 2008. Social Intuitionists Answer Six Questions about Moral Psychology. In Moral Psychology, ed. W. Sinnott-Armstrong, 2:181-218. Cambridge, MA: MIT Press.

Haidt, J., F. Bjorkland, and S. Murphy. 2000. Moral dumbfounding: When intuition finds no reason. University of Virginia: Unpublished manuscript.

Haidt, J., and M.A. Hersh. 2001. Sexual Morality: The Cultures and Emotions of Conservatives and Liberals. Journal of Applied Social Psychology 31: 191-221.

Haidt, J., and C. Joseph. 2004. Intuitive ethics: how innately prepared intuitions generate culturally variable cirtues. Daedalus 133: 55-66.

———. 2007a. When Morality Opposes Justice: Conservatives Have Moral Intuitions that Liberals may not Recognize. Social Justice Research 20: 98-116.

———. 2007b. The Moral Mind: How Five Sets of Innate Intuitions Guide the Development of Many Culture-Specific Virtues, and Perhaps Even Modules. In The Innate Mind, ed. P. Carruthers, S. Laurence, and S. Stich, 3 Foundations and the Future: 367-391. Oxford: Oxford University Press.

279

Haidt, J., S.H. Koller, and M.G. Dias. 1993. Affect, culture, and morality, or is it wrong to eat your dog? Journal of Personality and Social Psychology 65: 613-628.

Hamlin, J.K., K. Wynn, and P. Bloom. 2007. Social evaluation by preverbal infants. Nature 450: 557-559.

Hare, R.D. 1991. The Hare Psychopathy Checklist -- Revised. Toronto: Multi-Health Systems.

———. 1993. Without : The Disturbing World of the Psychopaths Among Us. New York: The Guilford Press.

———. 1996. Psychopathy: A Clinical Construct Whose Time Has Come. Criminal Justice and Behavior 23: 25-54.

———. 2003. The Hare Psychopathy Checklist --Revised. 2nd ed. Tornto: Multi-Health Systems.

Hare, R.D., S.D. Hart, and T.J. Harpur. 1991. Psychopathy and the DSM-IV criteria for antisocial personality disorder. Journal of Abnormal Psychology 100: 391-98.

Hare, R.M. 1952. The Language of Morals. Oxford: Clarendon Press.

Hare, T.A., C. Camerer, and A. Rangel. 2009. Self-Control in Decision-Making Involves Modulation of the vmPFC Valuation System. Science 324: 646-648.

Harlow, J. 1868. Recovery from the passage of an iron bar through the head. Publications of the Massachusetts Medical Society 2: 327-347.

Harman, G. 1977. The Nature of Morality: An Introduction to Ethics. Oxford: Oxford University Press.

———. 1999. Moral Philosophy and Linguistics. In Proceedings of the 20th World Congress of Philosophy, I: Ethics, ed. K. Brinkman, 107-115. Bowling Green, OH: Philosophy Documentation Center.

Harris, G.T., and M.E. Rice. 2006. Handbook of Psychopathy. In Handbook of Psychopathy, ed. C. J. Patrick, 555-572. New York: The Guilford Press.

Hart, C.L., A.S. Ward, M. Haney, R.W. Foltin, and M.W. Fischman. 2001. Methamphetamine self-administration by humans. Psychopharmacology 157: 75- 81.

Hauser, M.D. 2006a. Moral Minds: How Nature Designed Our Universal Sense of Right and Wrong. New York: HarperCollins.

280

———. 2006b. The liver and the moral organ. Soc Cogn Affect Neurosci 1: 214-220.

Hauser, M.D., F. Cushman, L. Young, R.K. Jin, and J. Mikhail. 2007. A Dissociation Between Moral Judgments and Justifications. Mind & Language 22: 1-21.

Hauser, M.D., L. Young, and F. Cushman. 2008. Reviving Rawls' Linguistic Analogy: Operative Principles and the Causal Structure of Moral Actions. In Moral Psychology, ed. W. Sinnott-Armstrong, 2:107-144. Cambridge, MA: MIT Press.

Heider, F., and M. Simmel. 1944. An Experimental Study of Apparent Behavior. The American Journal of Psychology 57: 243-259.

Heine, B. 1985. The Mountain People: Some Notes on the Ik of North-Eastern Uganda. Africa: Journal of the International African Institute 55: 3-16.

Hoffman, M.L. 1990. Empathy and justice motivation. Motivation and Emotion 14: 151- 172.

———. 1994. Discipline and Internalization. Developmental Psychology 30: 26-28.

———. 2000. Empathy and Moral Development: Implications for caring and justice. Cambridge: Cambridge University Press.

Horgan, T., and M. Timmons. 2007. Morphological Rationalism and the Psychology of Moral Judgment. Ethical Theory and Moral Practice 10: 279-295.

House, T.H., and W.L. Milligan. 1976. Autonomic responses to modeled distress in prison psychopaths. Journal of Personality and Social Psychology 34: 556-560.

Huebner, B., S. Dwyer, and M.D. Hauser. 2009. The role of emotion in moral psychology. Trends in Cognitive Sciences 13: 1-6.

Huemer, M. 2005. . New York: Palgrave Macmillan.

Hume, D. 1739. A Treatise of Human Nature. Ed. D.F. Norton and M.J. Norton. Oxford: Oxford University Press.

———. 1748. An Enquiry concerning Human Understanding. Ed. T. Beauchamp. Oxford: Oxford University Press.

Intrator, J., R.D. Hare, P. Stritzke, K. Brichtswein, D. Dorfman, T. Harpur, D. Bernstein, et al. 1997. A brain imaging (single photon emission computerized tomography) study of semantic and affective processing in psychopaths. Biological Psychiatry 42: 96-103.

281 Jackson, F. 1998. From Metaphysics to Ethics: A Defence of Conceptual Analysis. Oxford: Oxford University Press.

Jackson, F., and P. Pettit. 1995. Moral Functionalism and Moral Motivation. The Philosophical Quarterly 45: 20-40.

Jacob, F. 1977. Evolution and tinkering. Science 196: 1161 -1166.

Jacobson-Widding, Anita. 1997. 'I lied, I farted, I stole...' Dignity and Morality in African discourses on personhood. In The Ethnography of Moralities, ed. Signe Howell, 48-73. London: Routledge.

James, W. 1884. What is an Emotion? Mind 9: 188-205.

———. 1890. The Principles of Psychology, Vol. 2. New York: Henry Holt and Company.

Johns, J.H., and H.C. Quay. 1962. The effect of social reward on verbal conditioning in psychopathic and neurotic military offenders. Journal of Consulting Psychology 26: 217-220.

Johnston, M. 1989. Dispositional theories of value. Proceedings of the Aristotelian Society 63: 139–74.

Joyce, R. 2001. The Myth of Morality. Cambridge: Cambridge University Press.

———. 2006. The . Cambridge, MA: MIT Press.

Kable, J.W., and P.W. Glimcher. 2007. The neural correlates of subjective value during intertemporal choice. Nature Neuroscience 10: 1625-1633.

———. 2009. The neurobiology of decision: consensus and controversy. Neuron 63: 733-745.

Kahneman, D. 2000. A Psychological Point of View: Violations of Rational Rules as a Diagnostic of Mental Processes. Behavioral and Brain Sciences 23: 681-683.

———. 2003. A perspective on judgment and choice: mapping bounded rationality. The American Psychologist 58: 697-720.

Kahneman, D., and D.T. Miller. 1986. Norm Theory: Comparing Reality to Its Alternatives. Psychological Review 93: 136-153.

Kahneman, D., and C.R. Sunstein. 2005. Cognitive Psychology of Moral Intuitions. In Neurobiology of Human Values, ed. J.-P. Changeux, A.R. Damasio, S. Wolf, and C. Yves, 91-105. Berlin: Springer.

282

Kahneman, D., and A. Tversky, eds. 1982. Judgment under uncertainty: Heuristics and biases. Cambridge: Cambridge University Press.

Kahneman, D., and A. Tversky. 1982. On the study of statistical intuitions. In Judgment under uncertainty: Heuristics and biases, ed. D. Kahneman and A. Tversky, 493- 508. Cambridge: Cambridge University Press.

Kamin, L.J. 1968. "Attention-like" processes in classical conditioning. In Miami Symposium on the Prediction of Behavior: Aversive Stimulation, ed. M.R. Jones, 9-31. Coral Cables, FL: University of Miami Press.

Kandel, E.R. 2006. In search of memory: the emergence of a new science of mind. New York: W. W. Norton & Company.

Kant, I. 1799. The metaphysic of morals, divided into metaphysical elements of law and of ethics. Vol. 2. 2 vols. London [i.e. Hamburg]: printed for the translator; and sold by William Richardson.

Karazinov, D.M., and R.A. Boakes. 2007. Second-order conditioning in human predictive judgements when there is little time to think. The Quarterly Journal of Experimental Psychology 60: 448.

Keil, F.C. 1989. Concepts, Kinds, and Cognitive Development. Cambridge, MA: MIT Press.

Kelly, D., and S. Stich. 2007. Two Theories About the Cognitive Architecture Underlying Morality. In The Innate Mind, ed. P. Carruthers, S. Laurence, and S. Stich, 3:348-366. Oxford: Oxford University Press.

Kelly, D., S. Stich, K.J. Haley, S.J. Eng, and D.M.T. Fessler. 2007. Harm, Affect, and the Moral/Conventional Distinction. Mind & Language 22: 117-131.

Kennedy, D.P., J. Glascher, J.M. Tyszka, and R. Adolphs. 2009. Personal space regulation by the human amygdala. Nature Neuroscience 12: 1226-1227.

Kennett, J., and C. Fine. 2008. Internalism and the Evidence from Psychopaths and "Acquired Sociopaths". In Moral Psychology, ed. W. Sinnott-Armstrong, 3:173- 90. Cambridge, MA: MIT Press.

Kenny, A.J.P. 1963. Action, Emotion and Will. Oxford: Clarendon Press.

Kiehl, K.A. 2006. A cognitive neuroscience perspective on psychopathy: Evidence for paralimbic system dysfunction. Psychiatry Research 142: 107-128.

Kiehl, K.A., R.D. Hare, J.J. McDonald, and J. Brink. 1999. Semantic and Affective

283 Processing in Psychopaths: An Event-Related Potential (ERP) Study. Psychophysiology 36: 765-774.

Kiehl, K.A., A.M. Smith, R.D. Hare, A. Mendrek, B.B. Forster, J. Brink, and P.F. Liddle. 2001. Limbic abnormalities in affective processing by criminal psychopaths as revealed by functional magnetic resonance imaging. Biological Psychiatry 50: 677-684.

Kluckhohn, Clyde. 1953. Universal Categories of Culture. In Anthropology Today, ed. A Kroeber. Chicago: University of Chicago Press.

Kluver, H., and P.C. Bucy. 1939. Preliminary Analysis of Functions of the Temporal Lobes in Monkeys. Archives of Neurology and Psychiatry 42: 979-1000.

Knobe, J. 2003. Intentional action and side effects in ordinary language. Analysis 63: 190-194.

Knowlton, B.J., and K. Foerde. 2008. Neural Representations of Nondeclarative Memories. Current Directions in Psychological Science 17: 107 -111.

Knowlton, B.J., L.R. Squire, and M.A. Gluck. 1994. Probabilistic classification learning in amnesia. Learning & Memory 1: 106-120.

Kochanska, G. 1995. Children's Temperament, Mothers' Discipline, and Security of Attachment: Multiple Pathways to Emerging Internalization. Child Development 66: 597-615.

———. 1997. Mutually Responsive Orientation between Mothers and Their Young Children: Implications for Early Socialization. Child Development 68: 94-112.

———. 1998. Mother-Child Relationship, Child Fearfulness, and Emerging Attachment: A Short-Term Longitudinal Study. Developmental Psychology 34: 480-490.

Kochanska, G., N. Aksan, and M.E. Joy. 2007. Children's Fearfulness as a Moderator of Parenting in Early Socialization: Two Longitudinal Studies. Developmental Psychology 43: 222-237.

Kochanska, G., D.R. Forman, N. Aksan, and S.B. Dunbar. 2005. Pathways to conscience: early mother-child mutually responsive orientation and children's moral emotion, conduct, and cognition. Journal of Child Psychology and Psychiatry 46: 19-34.

Koenigs, M., and D. Tranel. 2007. Irrational Economic Decision-Making after Ventromedial Prefrontal Damage: Evidence from the Ultimatum Game. Journal of Neuroscience 27: 951-956.

———. 2008. Prefrontal cortex damage abolishes brand-cued changes in cola preference.

284 Social Cognitive and Affective Neuroscience 3: 1-6.

Koenigs, M., L. Young, R. Adolphs, D. Tranel, F. Cushman, M.D. Hauser, and A.R. Damasio. 2007. Damage to the prefrontal cortex increases utilitarian moral judgements. Nature 446: 908-911.

Kohlberg, L. 1969. Stage and Sequence: The Cognitive-Developmental Approach to Socialization. In Handbook of Socialization Theory and Research, ed. D. Goslin, 347-480. Chicago: Rand McNally and Company.

———. 1981. Essays on moral development. San Francisco, CA: Harper & Row.

Koonz, Claudia. 2003. The Nazi Conscience. Cambridge, MA: Belknap Press of Harvard University Press.

Krajbich, I., R. Adolphs, D. Tranel, N.L. Denburg, and C.F. Camerer. 2009. Economic Games Quantify Diminished Sense of Guilt in Patients with Damage to the Prefrontal Cortex. Journal of Neuroscience 29: 2188-2192.

Kringelbach, M.L. 2005. The human orbitofrontal cortex: linking reward to hedonic experience. Nature Reviews Neuroscience 6: 691-702.

Kringelbach, M.L., and K.C. Berridge. 2009. Towards a functional neuroanatomy of pleasure and happiness. Trends in Cognitive Sciences 13: 479-487.

Kringelbach, M.L., A. Lehtonen, A. Squire, A.G. Harvey, M.G. Craske, I.E. Holliday, A.L. Green, et al. 2008. A specific and rapid neural signature for parental instinct. PloS One 3: e1664.

Kringelbach, M.L., and E.T. Rolls. 2004. The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology. Progress in Neurobiology 72: 341-372.

Kuhlmeier, V., K. Wynn, and P. Bloom. 2003. Attribution of Dispositional States by 12- Month-Olds. Psychological Science 14: 402 -408.

LaBar, K.S., J.E. LeDoux, D.D. Spencer, and E.A. Phelps. 1995. Impaired fear conditioning following unilateral temporal lobectomy in humans. Journal of Neuroscience 15: 6846-6855.

Lagnado, D.A., M.R. Waldmann, Y. Hagmayer, and S.A. Sloman. 2007. Beyond Covariation: Cues to Causal Structure. In Causal Learning: Psychology, Philosophy, and Computation, ed. A. Gopnik and L. Schulz, 154-172. Oxford: Oxford University Press.

Laidlaw, J. 2002. For An Anthropology Of Ethics And Freedom. Journal of the Royal

285 Anthropological Institute 8: 311.

Lamb, R.J., K.L. Preston, C.W. Schindler, R.A. Meisch, F. Davis, J.L. Katz, J.E. Henningfield, and S.R. Goldberg. 1991. The reinforcing and subjective effects of morphine in post-addicts: a dose-response study. Journal of Pharmacology and Experimental Therapeutics 259: 1165 -1173.

Lange, C.G. 1912. The mechanism of the emotions. In The classical psychologists: selections illustrating psychology from Anaxagoras to Wundt, ed. B. Rand. New York: Houghton Mifflin company.

Langen, M., S. Durston, M.J.H. Kas, H. van Engeland, and W.G. Staal. 2011. The neurobiology of repetitive behavior: ...and men. Neuroscience & Biobehavioral Reviews 35: 356-365.

Lawrence, N.S., F. Jollant, O. O'Daly, F. Zelaya, and M.L. Phillips. 2009. Distinct Roles of Prefrontal Cortical Subregions in the Iowa Gambling Task. Cerebral Cortex 19: 1134-1143.

Lazarus, R.S. 1991. Emotion and . Oxford: Oxford University Press.

Ledoux, J.E. 1996. The Emotional Brain: The Mysterious Underpinnings of Emotional Life. New York: Simon & Schuster.

Lee, S.W.S., and N. Schwarz. 2010. Washing Away Postdecisional Dissonance. Science 328: 709.

Leslie, A.M., J. Knobe, and A. Cohen. 2006. Acting Intentionally and the Side-Effect Effect: Theory of Mind and Moral Judgment. Psychological Science 17: 421-427.

Levesque, H.J. 1986. Making believers out of computers. Artificial Intelligence 30: 81- 108.

Levine, B., S.E. Black, R. Cabeza, M. Sinden, A.R. Mcintosh, J.P. Toth, E. Tulving, and D.T. Stuss. 1998. Episodic memory and the self in a case of isolated retrograde amnesia. Brain 121: 1951 -1973.

Levine, B., M. Freedman, D. Dawson, S. Black, and D.T. Stuss. 1999. Ventral frontal contribution to self-regulation: Convergence of episodic memory and inhibition. Neurocase: The Neural Basis of Cognition 5: 263.

Levine, B., E. Svoboda, G.R. Turner, M. Mandic, and A. Mackey. 2009. Behavioral and functional neuroanatomical correlates of anterograde autobiographical memory in isolated retrograde amnesic patient M.L. Neuropsychologia 47: 2188-2196.

Lewis, D.K. 1970. How to Define Theoretical Terms. The Journal of Philosophy 67: 427-

286 446.

———. 1972. Psychophysical and theoretical identifications. Australasian Journal of Philosophy 50, no. 3: 249.

———. 1976. The Paradoxes of Time Travel. American Philosophical Quarterly 13: 145-152.

———. 1986. On the Plurality of Worlds. Malden, MA: Blackwell Publishers.

———. 1989. Dispositional Theories of Value. Proceedings of the Aristotelian Society Supplementary Volume 63: 113-37.

———. 2005. Quasi-realism is fictionalism. In Fictionalism in metaphysics, ed. M.E. Kalderon, 314–21. Oxford: Oxford University Press.

Lichtenstein, S., and P. Slovic. 2006. The Construction of Preference. Cambridge, MA: Cambridge University Press.

Livesey, E.J., and R.A. Boakes. 2004. Outcome additivity, elemental processing and blocking in human causality judgements. The Quarterly Journal of Experimental Psychology Section B: Comparative and Physiological Psychology 57: 361.

Livnat, A., and N. Pippenger. 2006. An optimal brain can be composed of conflicting agents. Proceedings of the National Academy of Sciences of the United States of America 103: 3198 -3202.

Lombrozo, T. 2009. The Role of Moral Commitments in Moral Judgment. Cognitive Science 33: 273-286.

Lovibond, P.F., S.-L. Been, C. J. Mitchell, M.E. Bouton, and R. Frohardt. 2003. Forward and backward blocking of causal judgment is enhanced by additivity of effect magnitude. Memory & Cognition 31: 133-142.

Lovibond, P.F., and D.R. Shanks. 2002. The role of awareness in Pavlovian conditioning: empirical evidence and theoretical implications. Journal of Experimental Psychology. Animal Behavior Processes 28: 3-26.

López, F.J., P.L. Cobos, and A. Caño. 2005. Associative and causal reasoning accounts of causal induction: Symmetries and asymmetries in predictive and diagnostic inferences. Memory & Cognition 33: 1388-1398.

Lycan, W. 2009. Serious Metaphysics: Frank Jackson's Defense of Conceptual Analysis. In Mind, Ethics, and Conditionals: Themes from the Philosophy of Frank Jackson, ed. I. Ravenscroft, 61-84. Oxford: Oxford University Press.

287 Lykken, D.T. 1957. A study of anxiety in the sociopathic personality. Journal of Abnormal and Social Psychology 55: 6-10.

Lynam, D.R., A. Caspi, T.E. Moffitt, R. Loeber, and M. Stouthamer-Loeber. 2007. Longitudinal Evidence That Psychopathy Scores in Early Adolescence Predict Adult Psychopathy. Journal of Abnormal Psychology 116: 155-165.

Lynam, D.R., R. Loeber, and M. Stouthamer-Loeber. 2008. The Stability of Psychopathy From Adolescence Into Adulthood. Criminal Justice and Behavior 35: 228 -243.

Lynam, D.R., D.J. Miller, D. Vachon, R. Loeber, and M. Stouthamer-Loeber. 2009. Psychopathy in Adolescence Predicts Official Reports of Offending in Adulthood. Youth Violence and Juvenile Justice 7: 189 -207.

MacBeath, A. 1952. Experiments in Living: A Study of the Nature and Foundation of Ethics or Morals in the Light of Recent Work in Social Anthropology. London: Macmillan.

Maccoby, E.E. 1992. The role of parents in the socialization of children: An historical overview. Developmental Psychology 28: 1006-1017.

Machery, E. 2009. Doing without Concepts. Oxford: Oxford University Press.

MacIntyre, A. 2007. After Virtue: A Study in Moral Theory. 3rd ed. Notre Dame, IN: University of Notre Dame Press.

Mackie, J. 1977. Ethics: Inventing Right and Wrong. New York: Penguin Books.

Maia, T.V., and J.L. McClelland. 2004. A Reexamination of the Evidence for the Somatic Marker Hypothesis: What Participants Really Know in the Iowa Gambling Task. Proceedings of the National Academy of Sciences of the United States of America 101: 16075-16080.

Marr, D. 1982. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. New York: W. H. Freeman.

Marsh, A.A., and N. Ambady. 2007. The influence of the fear facial expression on prosocial responding. Cognition and Emotion 21: 225-247.

Marsh, A.A., E.C. Finger, D.G.V. Mitchell, M.E. Reid, C. Sims, D.S. Kosson, K.E. Towbin, E. Leibenluft, D.S. Pine, and R.J. Dolan. 2008. Reduced Amygdala Response to Fearful Expressions in Children and Adolescents With Callous- Unemotional Traits and Disruptive Behavior Disorders. American Journal of Psychiatry 165: 712-720.

Masicampo, E.J., and R.F. Baumeister. 2008. Toward a Physiology of Dual-Process

288 Reasoning and Judgment: Lemonade, Willpower, and Expensive Rule-Based Analysis. Psychological Science 19: 255-260.

Mayr, E. 2005. 80 Years of watching the evolutionary scenery. Journal of Genetics 84: 91-93.

McClelland, J.L., and D.E. Rumelhart. 1981. An Interactive Activation Model of Context Effects in Letter Perception, Part 1. An account of basic findings. Psychological Review 88: 375-407.

———. 1985. Distributed memory and the representation of general and specific information. Journal of Experimental Psychology. General 114: 159-197.

McClure, S.M., J. Li, D. Tomlin, K.S. Cypert, L.M. Montague, and P.R. Montague. 2004. Neural Correlates of Behavioral Preference for Culturally Familiar Drinks. Neuron 44: 379-387.

McDowell, J. 1978. Are Moral Requirements Hypothetical Imperatives? Proceedings of the Aristotelian Society, Supplementary Volume 52: 13-29.

McGuire, J., R. Langdon, M. Coltheart, and C. Mackenzie. 2009. A reanalysis of the personal/impersonal distinction in moral psychology research. Journal of Experimental Social Psychology 45: 577-580.

McKinnon, M.C., B. Levine, and M. Moscovitch. 2007. Domain general contributions to social reasoning: The perspective from cognitive neuroscience. In Integrating the mind: domain general versus domain specific processes in higher cognition, ed. M.J. Roberts, 153-178. New York: Psychology Press.

McNaughton, D. 1988. Moral Vision: An Introduction to Ethics. Malden, MA: Blackwell Publishers.

Melchers, K.G., H. Lachnit, and D.R. Shanks. 2004. Within-compound associations in retrospective revaluation and in direct learning: a challenge for comparator theory. The Quarterly Journal of Experimental Psychology. B, Comparative and Physiological Psychology 57: 25-53.

Mele, A.R. 1996. Internalist Moral Cognitivism and Listlessness. Ethics 106: 727-753.

———. 2003. Motivation and Agency. New York: Oxford University Press.

Mendez, M.F. 2006. What frontotemporal dementia reveals about the neurobiological basis of morality. Medical Hypotheses 67: 411-418.

Mendez, M.F., E. Anderson, and J. Shapira. 2005. An Investigation of Moral Judgment in Frontotemporal Dementia. Cognitive and Behavioral Neurology 18: 193-197.

289

Mendez, M.F., A.K. Chen, J.S. Shapira, and B.L. Miller. 2005. Acquired sociopathy and frontotemporal dementia. Dementia and Geriatric Cognitive Disorders 20: 99- 104.

Mendez, M.F., E.C. Lauterbach, S.M. Sampson, and ANPA Committee on Research. 2008. An Evidence-Based Review of the Psychopathology of Frontotemporal Dementia: A Report of the ANPA Committee on Research. Journal of Neuropychiatry and Clinical Neuroscience 20: 130-149.

Mendez, M.F., E.A. Licht, and J. Shapira. 2008. Changes in Dietary or Eating Behavior in Frontotemporal Dementia Versus Alzheimer's Disease. American Journal of Alzheimer's Disease and Other Dementias 23: 280 -285.

Michotte, A. 1963. The perception of causality. Trans. T.R. Miles and E. Miles. New York: Basic Books.

Mikhail, J. 2002. Aspects of the theory of moral cognition: Investigating intuitive knowledge of the prohibition of intentional battery and the principle of double effect. Washington, DC: Georgetown University Law Center.

———. 2007. Universal Moral Grammar: Theory, Evidence, and the Future. Trends in Cognitive Sciences 11: 143-52.

———. 2008. The Poverty of the Moral Stimulus. In Moral Psychology, ed. W. Sinnott- Armstrong. Vol. 1. Cambridge, MA: MIT Press.

———. 2009. Moral Grammar and Intuitive Jurisprudence: A Formal Model of Unconscious Moral and Legal Knowledge. In Psychology of Learning and Motivation: Moral Judgment and Decision Making, ed. D.M. Bartels, C.W. Bauman, L.J. Skitka, and D.L. Medin, 50:27-100. San Diego, CA: Elsevier.

Mikhail, J., C. Sorrentino, and E. Spelke. 1998. Toward a Universal Moral Grammar. Proceedings of the Twentieth Annual Conference of the Cognitive Science Society: 1250.

Miller, A. 2003. An Introduction to Contemporary Metaethics. Malden, MA: Polity.

Miller, C.B. 2007. Motivational internalism. Philosophical Studies 139: 233-255.

Miller, E.K., and J.D. Cohen. 2001. An Integrative theory of Prefrontal Cortex Function. Annual Review of Neuroscience 24: 167.

Miller, L., R. Murphy, and A. Buss. 1981. Consciousness of body: Private and public. Journal of Peronsonality and Social Psychology 41: 397-406.

290 Miller, R.R., R.C. Barnet, and N.J. Grahame. 1995. Assessment of the Rescorla-Wagner model. Psychological Bulletin 117: 363-386.

Millon, T. 1995. Disorders of Personality: DSM-IV and Beyond. 2nd ed. New York: Wiley.

Mineka, S., M. Davidson, M. Cook, and R. Keir. 1984. Observational conditioning of snake fear in rhesus monkeys. Journal of Abnormal Psychology 93: 355-372.

Mitchell, C.J., J. De Houwer, and P.F. Lovibond. 2009. The Propositional Nature of Human Associative Learning. Behavioral and Brain Sciences 32: 183-198.

Mitchell, D. G. V., E. Colledge, A. Leonard, and R.J. Blair. 2002. Risky decisions and response reversal: is there evidence of orbitofrontal cortex dysfunction in psychopathic individuals? Neuropsychologia 40: 2013-2022.

Mitchell, J.P., J. Schirmer, D.L. Ames, and D.T. Gilbert. 2010. Medial Prefrontal Cortex Predicts Intertemporal Choice. Journal of Cognitive Neuroscience: 1-10.

Moore, A.B., B.A Clark, and M.J. Kane. 2008. Who Shalt Not Kill? Individual Differences in Working Memory Capacity, Executive Control, and Moral Judgment. Psychological Science 19: 549-557.

Moretto, G., E. Ladavas, F. Mattioli, and G. di Pellegrino. 2010. A Psychophysiological Investigation of Moral Judgment after Ventromedial Prefrontal Damage. Journal of Cognitive Neuroscience: 1888-1899.

Morris, J. S., A. Ohman, and R.J. Dolan. 1998. Conscious and unconscious emotional learning in the human amygdala. Nature 393: 467-470.

Morris, J.S. 2002. How do you feel? Trends in Cognitive Sciences 6: 317-319.

Murphy, G.L. 2002. The Big Book of Concepts. Cambridge, MA: The MIT Press.

Murphy, J.M. 1976. Psychiatric Labeling in Cross-Cultural Perspective. Science 191: 1019-1028.

Nado, J., D. Kelly, and S. Stich. 2009. Moral Judgment. In Routledge Companion to , ed. J. Symons and P. Calvo. New York: Routledge.

Nagel, J., and M.R. Waldmann. 2010. Deconfounding Distance Effects in Moral Reasoning. In Proceedings of the Thirty-Second Annual Conference of the Cognitive Science Society, ed. S. Ohlsson and R. Catrambone. Austin, TX: Cognitive Science Society.

Nagel, T. 1970. The Possibility of . Princeton, NJ: Princeton University Press.

291

Newman, J. P. 1998. Psychopathic behavior: An information processing perspective. In Psychopathy: Theory, research and implications for society, ed. D.J. Cooke, A.E. Forth, and R.D. Hare, 81–104. Dordrecht: Kluwer Academic Publishers.

Newman, J.P., C.M. Patterson, and D.S. Kosson. 1987. Response Perseveration in Psychopaths. Journal of Abnormal Psychology 96: 145-148.

Newman, J.P., and W.A. Schmitt. 1998. Passive Avoidance in Psychopathic Offenders: A Replication and Extension. Journal of Abnormal Psychology 107: 527-532.

Nichols, S. 2002. Norms with feeling: towards a psychological account of moral judgment. Cognition 84: 221-236.

———. 2004a. Sentimental Rules: On the Natural Foundations of Moral Judgment. Oxford: Oxford University Press.

———. 2004b. After objectivity: an empirical study of moral judgment. Philosophical Psychology 17: 3-26.

Nichols, S., and R. Mallon. 2006. Moral dilemmas and moral rules. Cognition 100: 530- 542.

Nisan, M. 1987. Moral Norms and Social Conventions: A Cross-Cultural Comparison. Developmental Psychology 23: 719-725.

Nisbett, R.E., and T.D. Wilson. 1977. Telling more than we can know: Verbal reports on mental processes. Psychological Review 84: 231-259.

Niv, Y., and P.R. Montague. 2009. Theoretical and empirical studies of learning. In Neuroeconomics: Decision Making and the Brain, ed. P.W. Glimcher, C. Camerer, R.A. Poldrack, and E. Fehr, 367-387. New York: Academic Press.

Nucci, L.P. 2001. Education in the Moral Domain. Cambridge: Cambridge University Press.

Nucci, L.P., and E. Turiel. 1978. Social Interactions and the Development of Social Concepts in Preschool Children. Child Development 49: 400-407.

Ochsner, K.N., and J.J. Gross. 2005. The cognitive control of emotion. Trends in Cognitive Sciences 9: 242-249.

Ohman, A. 2009. Human Fear Conditioning and the Amygdala. In The Human Amygdala, ed. P.J. Whalen and E.A. Phelps, 118-154. New York: Guilford Press.

Oppenheimer, D. 2004. Spontaneous Discounting of Availability in Frequency Judgment

292 Tasks. Psychological Science 15: 100-105.

Oxford, M., T.A. Cavell, and J.N. Hughes. 2003. Callous/Unemotional Traits Moderate the Relation Between Ineffective Parenting and Child Externalizing Problems: A Partial Replication and Extension. Journal of Clinical Child & Adolescent Psychology 32: 577-585.

Packard, M.G. 2009. Exhumed from thought: basal ganglia and response learning in the plus-maze. Behavioural Brain Research 199: 24-31.

Padoa-Schioppa, C., and J.A. Assad. 2006. Neurons in the orbitofrontal cortex encode economic value. Nature 441: 223-226.

———. 2008. The representation of economic value in the orbitofrontal cortex is invariant for changes of menu. Nature Neuroscience 11: 95-102.

Parkin, R. 1985. Anthropology of Evil. Oxford: Wiley-Blackwell.

Parkinson, B., A.H. Fischer, and A.S.R. Manstead. 2004. Emotion in Social Relations: Cultural, Group, and Interpersonal Processes. Psychology Press.

Pellizzoni, S., M. Siegal, and L. Surian. 2009. Foreknowledge, Caring, and the Side- Effect Effect in Young Children. Developmental Psychology 45: 289-295.

Penn, D.C., and D.J. Povinelli. 2007. Causal Cognition in Human and Nonhuman Animals: A Comparative, Critical Review. Annual Review of Psychology 58: 97- 118.

Perruchet, P. 1985. A pitfall for the expectancy theory of human eyelid conditioning. The Pavlovian Journal of Biological Science 20: 163-170.

Perruchet, P., A. Cleeremans, and A. Destrebecqz. 2006. Dissociating the effects of automatic activation and explicit expectancy on reaction times in a simple associative learning task. Journal of Experimental Psychology. Learning, Memory, and Cognition 32: 955-965.

Persaud, N., P. McLeod, and A. Cowey. 2007. Post-decision wagering objectively measures awareness. Nature Neuroscience 10: 257-261.

Pessiglione, M., P. Petrovic, J. Daunizeau, S. Palminteri, R.J. Dolan, and C.D. Frith. 2008. Subliminal Instrumental Conditioning Demonstrated in the Human Brain. Neuron 59: 561-567.

Phelps, E.A. 2006. Emotion and Cognition: Insights from Studies of the Human Amygdala. Annual Review of Psychology 57: 27-53.

293 Piaget, J. 1932. The Moral Judgment of the Child. London: Kegan Paul, Trench, Truber and CO.

Pinel, P. 1806. A treatise on insanity. Sheffield: W. Todd.

Pizarro, D.A., E. Uhlmann, and P. Bloom. 2003. Causal deviance and the attribution of . Journal of Experimental Social Psychology 39: 653-660.

Poldrack, R.A., and K. Foerde. 2008. Category learning and the memory systems debate. Neuroscience and Biobehavioral Reviews 32: 197-205.

Prehn, K., I. Wartenburger, K. Mériau, C. Scheibe, O. Goodenough, A. Villringer, E. van der Meer, and H.R. Heekeren. 2008. Individual differences in moral judgment competence influence neural correlates of socio-normative judgments. Social Cognitive and Affective Neuroscience 3: 33-46.

Prinz, J.J. 2000. The Duality of Content. Philosophical Studies 100: 1-34.

———. 2002. Furnishing the Mind: Concepts and Their Perceptual Basis. Cambridge, MA: MIT Press.

———. 2004. Gut Reactions: A Perceptual Theory of Emotion. Oxford: Oxford University Press.

———. 2006. The Emotional Basis of Moral Judgments. Philosophical Explorations 9: 29-43.

———. 2007. The Emotional Construction of Morals. Oxford: Oxford University Press.

Proffitt, J.B., J.D. Coley, and D.L. Medin. 2000. Expertise and category-based induction. Journal of Experimental Psychology. Learning, Memory, and Cognition 26: 811- 828.

Pronin, E., C.Y. Olivola, and K.A. Kennedy. 2008. Doing Unto Future As You Would Do Unto Others: Psychological Distance and Decision Making. Personality and Social Psychology Bulletin 34: 224 -236.

Quine, W.v.O. 1960. Word and Object. Cambridge, MA: MIT Press.

Rai, T.S., and K.J. Holyoak. 2010. Moral Principles or Consumer Preferences? Alternative Framings of the . Cognitive Science 34: 311-321.

Railton, P. 2005. Humean Theory of Practical Rationality. In The Oxford Handbook of Ethical Theory, ed. David Copp, 265-281. Oxford: Oxford University Press.

Rawls, J. 1971. A Theory of Justice. Cambridge, MA: Belknap Press.

294

Reimann, M., and A. Bechara. 2010. The somatic marker framework as a neurological theory of decision-making: Review, conceptual comparisons, and future neuroeconomics research. Journal of Economic Psychology 31: 767-776.

Rescorla, R.A., and A.W. Wagner. 1972. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In Classical Conditioning II: Current Research and Theory, ed. A.H. Black and W.F. Prokasy, 64-99. New York: Appleton-Century-Crofts.

Richerson, P.J., and R. Boyd. 2005. Not by genes alone: how culture transformed human evolution. University of Chicago Press.

Ridge, M. 2006. Ecumenical Expressivism: Finessing Frege. Ethics 116: 302-336.

Roberts, J.R. 2001. Mental Illness, Motivation and Moral Commitment. The Philosophical Quarterly 51: 41-59.

Robinson, D. 2009. Moral Functionalism, Ethical Quasi-Relativism, and the Canberra Plan. In Conceptual Analysis and Philosophical Naturalism, ed. D. Braddon- Mitchell and R. Nola, 315-48. Cambridge, MA: MIT Press.

Robinson, T.E., and K.C. Berridge. 1993. The neural basis of drug craving: An incentive- sensitization theory of addiction. Brain Research Reviews 18: 247-291.

———. 2003. Addiction. Annual Review of Psychology 54: 25-53.

Rolls, E.T. 2005. Emotion Explained. Oxford: Oxford University Press.

Rolls, E.T., J. Hornak, E. Wade, and J. McGrath. 1994. Emotion-related learning in patients with social and emotional changes associated with frontal lobe damage. Journal of Neurology, Neurosurgery, and Psychiatry 57: 1518-1524.

Rosenbaum, R.S., S. Köhler, D.L. Schacter, M. Moscovitch, R. Westmacott, S.E. Black, F. Gao, and E. Tulving. 2005. The case of K.C.: contributions of a memory- impaired person to memory theory. Neuropsychologia 43: 989-1021.

Rosenfeld, H., and D.M. Baer. 1969. Unnoticed verbal conditioning of an aware experimenter by a more aware subject: The double-agent effect. Psychological Review 76: 425-432.

———. 1970. Unbiased and unnoticed verbal conditioning: the double agent robot procedure. Journal of the Experimental Analysis of Behavior 14: 99-105.

Roser, M.E., J.A. Fugelsang, K.N. Dunbar, P.M. Corballis, and M.S. Gazzaniga. 2005. Dissociating Processes Supporting Causal Perception and Causal Inference in the

295 Brain. Neuropsychology 19: 591-602.

Roskies, A. 2003. Are ethical judgments intrinsically motivational? Lessons from "acquired sociopathy". Philosophical Psychology 16: 51-66.

———. 2008. Internalism and the Evidence from Pathology. In Moral Psychology, ed. W. Sinnott-Armstrong, 3:191-206. Cambridge, MA: MIT Press.

Rozin, P, M. Markwith, and C. Stoess. 1997. Moralization and Becoming a Vegetarian: The Transformation of Preferences into Values and the Recruitment of Disgust. Psychological Science 8: 67-73.

Rozin, P., L. Lowery, S. Imada, and J. Haidt. 1999. The CAD triad hypothesis: a mapping between three moral emotions (contempt, anger, disgust) and three moral codes (community, autonomy, divinity). Journal of Personality and Social Psychology 76: 574-586.

Rozin, P., L. Millman, and C. Nemeroff. 1986. Operation of the laws of sympathetic magic indisgust and other domains. Jouranl of Personality and Social Psychology 50: 703-712.

Russell, J.A. 2003. Core Affect and the Psychological Construction of Emotion. Psychological Review 110: 145-172.

Sacks, O. 2007. Musicophilia: Tales of Music and the Brain. New York: Knopf.

Saddoris, M.P., M. Gallagher, and G. Schoenbaum. 2005. Rapid Associative Encoding in Basolateral Amygdala Depends on Connections with Orbitofrontal Cortex. Neuron 46: 321-331.

Salekin, R.R., R. Rogers, and K.W. Sewell. 1997. Construct validity of psychopathy in a female offender sample: a multitrait-multimethod evaluation. Journal of Abnormal Psychology 106: 576-85.

Sanfey, A.G., J.K. Rilling, J.A. Aronson, L.E. Nystrom, and J.D. Cohen. 2003. The Neural Basis of Economic Decision-Making in the Ultimatum Game. Science 300: 1755-1758.

Saver, J.L., and A.R. Damasio. 1991. Preserved access and processing of social knowledge in a patient with acquired sociopathy due to ventromedial frontal damage. Neuropsychologia 29: 1241-1249.

Saxe, R., and L.J. Powell. 2006. It's the Thought That Counts. Psychological Science 17: 692 -699.

Saxe, R., T. Tzelnic, and S. Carey. 2007. Knowing who dunnit: Infants identify the causal

296 agent in an unseen causal interaction. Developmental Psychology 43: 149-158.

Scanlon, T. M. 1998. What We Owe to Each Other. Cambridge, MA: Harvard University Press.

Scarantino, A. 2009. Core Affect and Natural Affective Kinds. Philosophy of Science 76: 940-957.

———. 2010. Insights and Blindspots of the Cognitivist Theory of Emotions. The British Journal for the Philosophy of Science 61: 729 -768.

Schachter, S., and J. Singer. 1962. Cognitive, social, and physiological determinants of emotional state. Psychological Review 69: 379-399.

Schacter, D.L., D.R. Addis, and R.L. Buckner. 2009. Constructive Memory and the Simulation of Future Events. In The Cognitive Neurosciences, ed. M.S. Gazzaniga, 751-762. 4th ed. Cambridge, MA: MIT Press.

Schlottmann, A., and D.R. Shanks. 1992. Evidence for a distinction between judged and perceived causality. The Quarterly Journal of Experimental Psychology. A, Human Experimental Psychology 44: 321-342.

Schmolck, H., E.A. Kensinger, S. Corkin, and L.R. Squire. 2002. Semantic knowledge in patient H.M. and other patients with bilateral medial and lateral temporal lobe lesions. Hippocampus 12: 520-533.

Schnall, S., J. Benton, and S. Harvey. 2008. With a Clean Conscience: Cleanliness Reduces the Severity of Moral Judgments. Psychological Science 19: 1219-1222.

Schnall, S., J. Haidt, G.L. Clore, and A. Jordan. 2008. Disgust as embodied moral judgment. Personality and Social Psychology Bulletin 34: 1096-1109.

Schoenbaum, G., M.R. Roesch, and T.A. Stalnaker. 2006. Orbitofrontal cortex, decision- making and drug addiction. Trends in neurosciences 29: 116-124.

Scholl, B.J., and P.D. Tremoulet. 2000. Perceptual causality and animacy. Trends in Cognitive Sciences 4: 299-309.

Schroeder, T. 2004. Three Faces of Desire. Oxford: Oxford University Press.

Schultz, W., P. Dayan, and P.R. Montague. 1997. A Neural Substrate of Prediction and Reward. Science 275: 1593-1599.

Schwarz, N. 2007. Attitude Construction: Evaluation in Context. Social Cognition 25: 638-656.

297 Schwarz, N., and G.L. Clore. 1983. Mood, misattribution, and judgments of well-being: Informative and directive funcitons of affective states. Journal of Peronsonality and Social Psychology 45: 513-23.

Scoville, W.B., and B. Milner. 1957. Loss of Recent Memory after Bilateral Hippocampal Lesions. Journal of Neurology, Neurosurgery, and Psychiatry 20: 11-21.

Seger, C.A. 2006. The Basal Ganglia in Human Learning. The Neuroscientist 12: 285 - 290.

Seitz, A.R., D. Kim, and T. Watanabe. 2009. Rewards Evoke Learning of Unconsciously Processed Visual Stimuli in Adult Humans. Neuron 61: 700-707.

Shafer-Landau, R. 2003. Moral Realism: A Defence. Oxford: Oxford University Press.

Shanks, D.R. 1985. Forward and backward blocking in human contingency judgement. The Quarterly Journal of Experimental Psychology Section B: Comparative and Physiological Psychology 37: 1-21.

———. 2005. Implicit learning. In Handbook of Cognition, ed. K. Lamberts and R. Goldstone, 202-220. London: Sage.

———. 2007. Associationism and cognition: Human contingency learning at 25. Quarterly Journal of Experimental Psychology 60: 291-309.

———. 2010. Learning: From Association to Cognition. Annual Review of Psychology 61: 273-301.

Shanks, D.R., and A. Dickinson. 1987. Associative Accounts of Causality Judgment. In The Psychology of Learning and Motiavtion, ed. G.H. Bower, 21:229-261. Academic Press.

Shenhav, A., and J.D. Greene. 2010. Moral Judgments Recruit Domain-General Valuation Mechanisms to Integrate Representations of Probability and Magnitude. Neuron 67: 667-677.

Shepherd, J. 2009. The integrative view of moral reasons for action. Unpublished manuscript. Florida State University.

Shiv, B., G. Loewenstein, and A. Bechara. 2005. The dark side of emotion in decision- making: When individuals with decreased emotional reactions make more advantageous decisions. Cognitive Brain Research 23: 85-92.

Shiv, B., G. Loewenstein, A. Bechara, H. Damasio, and A.R. Damasio. 2005. Investment Behavior and the Negative Side of Emotion. Psychological Science 16: 435-439.

298

Shohamy, D., C.E. Myers, J. Kalanithi, and M.A. Gluck. 2008. Basal ganglia and Dopamine Contributions to Probabilistic Category Learning. Neuroscience and biobehavioral reviews 32: 219-236.

Shrager, Y., and L.R. Squire. 2009. Medial Temporal Lobe Function and Human Memory. In The Cognitive Neurosciences, ed. M.S. Gazzaniga, 675-690. 4th ed. Cambridge, MA: MIT Press.

Shweder, R., M. Mahapatra, and J. Miller. 1987. Culture and Moral Development. In The emergence of morality in young children, ed. J. Kagan and S. Lamb. Chicago: University of Chicago Press.

Shweder, R., N. Much, M. Mahapatra, and L. Park. 1997. The "Big Three" of Morality (Autonomy, Community, Divinity) and the "Big Three" Explanations of Suffering. In Morality and Health, ed. A. Brandt and P. Rozin, 119-172. New York: Routledge.

Silvers, J.A., and J. Haidt. 2008. Moral elevation can induce nursing. Emotion 8: 291- 295.

Singer, P. 1975. Animal Liberation: A New Ethics For Our Treatment of Animals. New York: New York Review.

Sinnott-Armstrong, W. 2006. Moral Skepticisms. Oxford: Oxford University Press. ———. 2008. Moral Psychology. 3 vols. Cambridge, MA: MIT Press.

Skeem, J.L., and D.J. Cooke. 2010. Is criminal behavior a central component of psychopathy? Conceptual directions for resolving the debate. Psychological Assessment 22: 433-445.

Skinner, B.F. 1982. Contrived reinforcement. The Behavior Analyst 5: 3.

Skyrms, B. 2010. Signals: Evolution, Learning, and Information. Oxford: Oxford University Press.

Sloman, S.A. 1996. The Empirical Case for Two Systems of Reasoning. Psychological Bulletin 119: 33-22.

———. 2005. Causal Models: How People Think about the World and Its Alternatives. Oxford: Oxford University Press.

Sloman, S.A., P.M. Fernbach, and S. Ewing. 2009. Causal Models: The Representational Infrastructure for Moral Judgment. In The Psychology of Learning and Motivation, ed. B.H. Ross, 50:1-26. San Diego, CA: Academic Press.

299 Slote, M. 2005. Moral Sentimentalism and Moral Psychology. In Oxford Handbook of Ethical Theory, ed. D. Copp, 219-239. Oxford: Oxford University Press.

Smetana, J. 1993. Understanding of social rules. In The development of social cognition: The child as psychologist, ed. M. Bennett, 111-41. New York: Guilford Press.

Smith, B., and R. Casati. 1994. Naive physics: An essay in ontology. Philosophical Psychology 7: 225–244.

Smith, J.C., and D.L. Roll. 1967. Trace conditioning with X-rays as an aversive stimulus. Psychonomic Science 9: 11–12.

Smith, M. 1994. The Moral Problem. Malden, MA: Wiley-Blackwell.

———. 2004. Ethics and the A Priori: Selected Essays on Moral Psychology and Meta- Ethics. Cambridge: Cambridge University Press.

———. 2008. The Truth about Internalism. In Moral Psychology, ed. W. Sinnott- Armstrong, 3:207-15. Cambridge, MA: MIT Press.

Solomon, R.C. 1973. Emotions and Choice. The Review of Metaphysics 27: 20-41.

Soto, F.A., and E.A. Wasserman. 2010a. Missing the Forest for the Trees. Psychological Science 21: 1510 -1517.

———. 2010b. Error-Driven Learning in Visual Categorization and Object Recognition: A Common-Elements Model. Psychological Review 117: 349-381.

Specter, M. 1999. The Dangerous Philosopher. The New Yorker.

Stalnaker, R.C. 1984. Inquiry. Cambridge, MA: The MIT Press.

Stanovich, K.E. 1999. Who Is Rational?: Studies of individual Differences in Reasoning. Mahwah, NJ: Lawrence Erlbaum.

———. 2004. The Robot's Rebellion: Finding Meaning in the Age of Darwin. Chicaho: University of Chicago Press.

———. 2009a. Distinguishing the reflective, algorithmic, and autonomous minds: Is it time for a tri-process theory? In In Two Minds: Dual Processes and Beyond, ed. J.St.B.T. Evans and K. Frankish. Oxford: Oxford University Press.

———. 2009b. What Intelligence Tests Miss: The Psychology of Rational Thought. Yale University Press.

Stanovich, K.E., M.E. Toplak, and R.F. West. 2008. The development of rational

300 thought: a taxonomy of heuristics and biases. Advances in Child Development and Behavior 36: 251-285.

Starkey, C. 2007. Manipulating emotion: the best evidence for non-cognitivism in the light of proper function. Analysis 67: 230 -237.

Stein, E. 1996. Without Good Reason: The Rationality Debate in Philosophy and Cognitive Science. Oxfrod: Oxford University Press.

Sterelny, K. 2003. Thought in a Hostile World: The Evolution of Human Cognition. Malden, MA: Wiley-Blackwell.

Stevenson, R.J., and R.A. Boakes. 2004. Sweet and Sour Smells: Learned Synesthesia Between the Senses of Taste and Smell. In The Handbook of Multisensory Processes, ed. G.A. Calvert, C. Spence, and B.E. Stein, 69-84. Cambridge, MA: MIT Press.

Stich, S. 1990. The Fragmentation of Reason. The MIT Press.

———. 2006. Is Morality an Elegant Machine or a Kludge? Journal of Cognition & Culture 6: 181-189.

———. 2007. Jean Nicod Lectures. http://www.institutnicod.org/lectures2007_outline.htm.

Stocker, M. 1979. Desiring the Bad: An Essay in Moral Psychology. The Journal of Philosophy 76: 738-753.

Strevens, M. 2007. Beyond Covariation: Cues to Causal Structure. In Causal Learning: Psychology, Philosophy, and Computation, ed. A. Gopnik and L. Schulz, 245- 260. Oxford: Oxford University Press.

Strohminger, N., R.L. Lewis, and D.E. Meyer. 2009. What Would Seinfeld Do? Divergent Effects of Different Positive Emotions on Moral Judgment.

Sturgeon, N. 1988. Moral Explanations. In Essays on Moral Realism, ed. Geoffrey Sayre- McCord, 229-55. Ithaca, NY: Cornell University Press.

Sutton, J. 2004. Are Concepts Mental Representations or Abstracta? Philosophy and Phenomenological Research 68: 89-108.

Svavarsdottir, S. 1999. Moral Cognitivism and Motivation. The Philosophical Review 108: 161-219.

———. 2006. How do moral judgments motivate? In Contemporary Debates in Moral Theory, ed. J. Dreier, 163-81. Malden, MA: Wiley-Blackwell.

301

Taylor, P. 1978. On Taking the Moral Point of View. Midwest Studies In Philosophy 3: 35-61.

Tetlock, P.E., O.V. Kristel, S.B. Elson, M.C. Green, and J.S. Lerner. 2000. The psychology of the unthinkable: taboo trade-offs, forbidden base rates, and heretical counterfactuals. Journal of Personality and Social Psychology 78: 853- 870.

Thagard, P. 2006. Desires Are Not Propositional Attitudes. Dialogue: Canadian Philosophical Review/Revue canadienne de philosophie 45: 151-156.

Thomas, B.C., K.E. Croft, and D. Tranel. 2011. Harming Kin to Save Strangers: Further Evidence for Abnormally Utilitarian Moral Judgments after Ventromedial Prefrontal Damage. Journal of Cognitive Neuroscience: 1-11.

Thomson, J.J. 1985. The Trolley Problem. The Yale Law Journal 94: 1395-1415.

Thorndike, E.L. 1911. Animal intelligence: experimental studies. New York: The Macmillan company.

Tobler, P.N., J.P. O'Doherty, R.J. Dolan, and W. Schultz. 2006. Human Neural Learning Depends on Reward Prediction Errors in the Blocking Paradigm. Journal of 95: 301-310.

Tomasello, M., J. Call, and B. Hare. 2003. Chimpanzees understand psychological states - the question is which ones and to what extent. Trends in Cognitive Sciences 7: 153-156.

Tulving, E. 1985. Memory and consciousness. Canadian Psychology 26: 1-12.

Tulving, E., and D.L. Schacter. 1990. Priming and human memory systems. Science 247: 301-306.

Tulving, E., D.L. Schacter, D.R. McLachlan, and M. Moscovitch. 1988. Priming of semantic autobiographical knowledge: A case study of retrograde amnesia. Brain and Cognition 8: 3-20.

Turiel, E. 1983. The Development of Social Knowledge: Morality and Convention. Cambridge: Cambridge University Press.

Turiel, E., M. Killen, and C. Helwig. 1987. Morality: Its structure, functions, and vagaries. In The emergence of morality in young children, ed. J. Kagan and S. Lamb. Chicago: University of Chicago Press.

Turnbull, C. 1972. The Mountain People. New York: Simon & Schuster.

302

Turner, D.C., M.R.F. Aitken, D.R. McLachlan, Barbara J. Sahakian, Trevor W. Robbins, Christian Schwarzbauer, and Paul C. Fletcher. 2004. The Role of the Lateral Frontal Cortex in Causal Associative Learning: Exploring Preventative and Super-learning. Cerebral Cortex 14: 872 -880.

Tversky, A., and D. Kahneman. 1981. The framing of decisions and the psychology of choice. Science 211: 453 -458.

Uttich, K., and T. Lombrozo. 2010. Norms inform mental state ascriptions: A rational explanation for the side-effect effect. Cognition 116: 87-100.

Valdesolo, P., and D. DeSteno. 2006. Manipulations of Emotional Context Shape Moral Judgment. Psychological Science 17: 476-477.

Waldmann, M.R. 1996. Knowledge-based causal induction. In Psychology of Learning and Motivation, ed. D.R. Shanks, K.J. Holyoake, and D.L. Medin, 34:27-88. San Diego: Academic Press.

———. 2000. Competition among causes but not effects in predictive and diagnostic learning. Journal of Experimental Psychology. Learning, Memory, and Cognition 26: 53-76.

———. 2001. Predictive versus diagnostic causal learning: Evidence from an overshadowing paradigm. Psychonomic Bulletin & Review 8: 600-608.

Waldmann, M.R., and J.H. Dieterich. 2007. Throwing a Bomb on a Person Versus Throwing a Person on a Bomb: Intervention Myopia in Moral Intuitions. Psychological Science 18: 247-253.

Waldmann, M.R., Y. Hagmayer, and A.P. Blaisdell. 2006. Beyond the Information Given: Causal Models in Learning and Reasoning. Current Directions in Psychological Science 15: 307-311.

Waldmann, M.R., and K.J. Holyoak. 1992. Predictive and diagnostic learning within causal models: asymmetries in cue competition. Journal of Experimental Psychology. General 121: 222-236.

Waldmann, M.R., B. Meder, M. Sydow, and Y. Hagmayer. 2009. The tight coupling between category and causal learning. Cognitive Processing 11: 143-158.

Waldmann, M.R., and A. Wiegmann. 2010. A double causal contrast theory of moral intuitions in trolley dilemmas. In Proceedings of the Thrity-Second Annual Conference of the Cognitive Science Society, ed. S. Ohlsson and R. Catrambone, 2589-2594. Austin, TX: Cognitive Science Society.

303 Wallis, J.D. 2007. Orbitofrontal Cortex and Its Contribution to Decision-Making. Annual Review of Neuroscience 30: 31-56.

Wardle, S.G., C.J. Mitchell, and P.F. Lovibond. 2007. Flavor evaluative conditioning and contingency awareness. Learning & Behavior 35: 233-241.

Weber, S., U. Habel, K. Amunts, and F. Schneider. 2008. Structural brain abnormalities in psychopaths - a review. Behavioral Sciences & the Law 26: 7-28.

Wellman, H.M., D. Cross, and J. Watson. 2001. Meta-Analysis of Theory-of-Mind Development: The Truth about False Belief. Child Development 72: 655-684.

Wellman, H.M., S. Lopez-Duran, J. LaBounty, and B. Hamilton. 2008. Infant attention to intentional action predicts preschool theory of mind. Developmental Psychology 44: 618-623.

Wellman, H.M., A.T. Phillips, S. Dunphy-Lelii, and N. Lalonde. 2004. Infant social attention predicts preschool social cognition. Developmental Science 7: 283-288.

West, F. 1978. Iago the Psychopath. South Atlantic Bulletin 43: 27-35.

Wheatley, T., and J. Haidt. 2005. Hypnotic Disgust Makes Moral Judgments More Severe. Psychological Science 16: 780-784.

Wicker, B., C. Keysers, J. Plailly, J.P. Royet, V. Gallese, and G. Rizzolatti. 2003. Both of us disgusted in My insula: the common neural basis of seeing and feeling disgust. Neuron 40: 655-664.

Williams, B.A. 1973. A Critique of Utilitarianism. In Utilitarianism: for and against, ed. J. J. C. Smart and B.A. Williams. Cambridge: Cambridge University Press.

———. 2006. Ethics and the Limits of Philosophy. Oxford: Routledge.

Willingham, D.B., J. Salidis, and J.D.E. Gabrieli. 2002. Direct Comparison of Neural Systems Mediating Conscious and Unconscious Skill Learning. Journal of Neurophysiology 88: 1451-1460.

Wilson, C. 2004. Moral Animals: Ideals and Constraints in Moral Theory. Oxford: Oxford University Press.

Wilson, E.O. 1975. Sociobiology: The New Synthesis. Cambridge, MA: Harvard University Press.

Wilson, T.D. 2002. Strangers to Ourselves: Discovering the Adaptive Unconscious. Cambridge, MA: Belknap Press.

304 Wilson, T.D., and D.T. Gilbert. 2003. Affective Forecasting. In Advances in Experimental Social Psychology, ed. M.P. Zanna and J.M. Olson, 35:345-411. Burlington, VT: Academic Press.

Wilson, T.D., D.J. Lisle, J.W. Schooler, S.D. Hodges, K.J. Klaaren, and S.J. LaFleur. 1993. Introspecting about Reasons can Reduce Post-Choice Satisfaction. Personality and Social Psychology Bulletin 19: 331 -339.

Wilson, T.D., and J.W. Schooler. 1991. Thinking Too Much: Introspection Can Reduce the Quality of Preferences and Decisions. Journal of Personality and Social Psychology 60: 181-192.

Wittgenstein, L. 1965. A Lecture on Ethics. The Philosophical Review 74: 3-12.

Woodward, J. 2003. Making Things Happen: A Theory of Causal Explanation. Oxford: Oxford University Press.

Wootton, J.M., P.J. Frick, K.K. Shelton, and P. Silverthorn. 1997. Ineffective Parenting and Childhood Conduct Problems: The Moderating Role of Callous-Unemotional Traits. Journal of Consulting and Clinical Psychology 65: 301-308.

Wynne, K. 2008. Some Innate Foundations of Social and Moral Cognition. In The Innate Mind, ed. P. Carruthers, S. Laurence, and S. Stich, 3:330-348. Oxford: Oxford University Press.

Yamaguchi, M., V. Kuhlmeier, K. Wynn, and K. vanMarle. 2009. Continuity in social cognition from infancy to childhood. Developmental Science 12: 746-752.

Yamaguchi, S. 2004. Further Clarifications of the Concept of Amae in Relation to Dependence and Attachment. Human Development 47: 28-33.

Yang, Y., A. Raine, T. Lencz, S. Bihrle, L. LaCasse, and P. Colletti. 2005. Volume reduction in prefrontal gray matter in unsuccessful criminal psychopaths. Biological Psychiatry 57: 1103-1108.

Young, L., A. Bechara, D. Tranel, H. Damasio, M.D. Hauser, and A.R. Damasio. 2010. Damage to Ventromedial Prefrontal Cortex Impairs Judgment of Harmful Intent. Neuron 65: 845-851.

Young, L., J.A. Camprodon, M.D. Hauser, A. Pascual-Leone, and R. Saxe. 2010. Disruption of the right temporoparietal junction with transcranial magnetic stimulation reduces the role of beliefs in moral judgments. Proceedings of the National Academy of Sciences of the United States of America 107: 6753-6758.

Young, L., F. Cushman, R. Adolphs, D. Tranel, and M.D. Hauser. 2006. Does emotion

305 mediate the relationship between an action's moral status and its intentional status? Neuropsychological evidence. Journal of Cognition and Culture 6: 291- 304.

Young, L., F. Cushman, M.D. Hauser, and R. Saxe. 2007. The Neural Basis of the Interaction between Theory of Mind and Moral Judgment. Proceedings of the National Academy of Sciences of the United States of America 104: 8235-8240.

Young, L., and M. Koenigs. 2007. Investigating emotion in moral cognition: a review of evidence from functional neuroimaging and neuropsychology. British Medical Bulletin 84: 69-79.

Young, L., and R. Saxe. 2008. The neural basis of belief encoding and integration in moral judgment. NeuroImage 40: 1912-1920.

———. 2009a. Innocent intentions: A correlation between forgiveness for accidental harm and neural activity. Neuropsychologia 47: 2065-2072.

———. 2009b. An fMRI Investigation of Spontaneous Mental State Inference for Moral Judgment. Journal of Cognitive Neuroscience 21: 1396-1405.

Zajonc, R. 1980. Feeling and thinking: Preferences need no inferences. American Psychologist 35: 151-175.

———. 1984. On the Primacy of Affect. American Psychologist 39: 117-123.

Zhong, C.-B., and K. Liljenquist. 2006. Washing Away Your Sins: Threatened Morality and Physical Cleansing. Science 313: 1451-1452.

Zhong, C.-B., B. Strejcek, and N. Sivanathan. 2010. A clean self can render harsh moral judgment. Journal of Experimental Social Psychology 46: 859-62.

306

BIOGRAPHICAL SKETCH

Christopher Zarpentine

Christopher Zarpentine grew up in upstate New York. He attended Ithaca College, receiving his B.A. in 2003 with majors in both Philosophy and Music. He entered graduate school at Florida State University in 2004, where he received his M.A. in Philosophy in 2006. He also completed an M.A. in the History and Philosophy of Science program. During the 2009-2010 academic year he held a fellowship in Integrated Training in Biology and Society through the National Science Foundation.

307