Brain Potentials Associated with Expected and Unexpected Good and Bad Outcomes
Total Page:16
File Type:pdf, Size:1020Kb
Psychophysiology, 42 (2005), 161–170. Blackwell Publishing Inc. Printed in the USA. Copyright r 2005 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2005.00278.x Brain potentials associated with expected and unexpected good and bad outcomes GREG HAJCAK,a CLAY B. HOLROYD,b JASON S. MOSER,a and ROBERT F. SIMONSa aDepartment of Psychology, University of Delaware, Newark, Delaware, USA bDepartment of Psychology, University of Victoria, Victoria, British Columbia, Canada Abstract The error-related negativity (ERN) is an event-related brain potential observed when subjects receive feedback in- dicating errors or monetary losses. Evidence suggests that the ERN is larger for unexpected negative feedback. The P300 has also been shown to be enhanced for unexpected feedback, but does not appear to be sensitive to feedback valence. The present study evaluated the role of expectations on the ERN and P300 in two experiments that ma- nipulated the probability of negative feedback (25%, 50%, or 75%) on a trial-by-trial basis in experiment 1, and by varying the frequency of positive and negative feedback across blocks of trials in experiment 2. In both experiments, P300 amplitude was larger for unexpected feedback; however, the ERN was equally large for expected and unexpected negative feedback. These results are discussed in terms of the potential role of expectations in processing errors and negative feedback. Descriptors: Expectations, Feedback, Event-related brain potential, Error-related negativity, Ne, P300, Reinforce- ment learning, Response monitoring A number of recent event-related brain potential (ERP) studies ing the presentation of negative feedback (cf. Ruchsow, Grothe, have focused on neural activity related to errors and negative Spitzer, & Kiefer, 2002; see Nieuwenhuis, Holroyd, Mol, & feedback. In particular, the response-locked ERP at fronto-cen- Coles, 2004, for a review). Miltner et al. found that when subjects tral recording sites is characterized by a negative deflection that received negative feedback regarding the accuracy of their per- begins around the time of an erroneous response and peaks ap- formance, the ERP following negative feedback was character- proximately 50 ms later (Falkenstein, Hohnsbein, Hoormann, & ized by a negative deflection at fronto-central recording sites with Blanke, 1991; Falkenstein, Hoormann, Christ, & Hohnsbein, a peak latency of approximately 250 ms. Like the response ERN, 2000; Gehring, Coles, Meyer, & Donchin, 1990; Gehring, Goss, this feedback ERN was source localized to near the ACC (Milt- Coles, Meyer, & Donchin, 1993). This error-related negativity ner et al., 1997). In considering the topographical and morpho- (ERN or Ne) has been observed across various stimulus and re- logical similarity of this feedback ERN and the response ERN, sponse modalities, and consequently appears to reflect the activity Miltner et al. proposed that both ERNs might represent the of a generic response monitoring system (Bernstein, Scheffers, & activity of a single error detection system. Coles, 1995; Falkenstein et al., 1991; Holroyd, Dien, & Coles, More recently, Holroyd and Coles (2002) argued that both 1998; Nieuwenhuis, Ridderinkhof, Blom, Band, & Kok, 2001). the response ERN and feedback ERN reflect the activity of a Studies utilizing whole-head ERP recording systems and source- reinforcement learning system that continually evaluates ongo- localization software have consistently indicated that the ERN is ing events against expected outcomes. This reinforcement learn- generated by a single source in the anterior cingulate cortex ing theory of the ERN is predicated on previous research (ACC; Dehaene, Posner, & Tucker, 1994; Holroyd et al., 1998). implicating the basal ganglia and the midbrain dopamine system Following early response-locked ERN studies, Miltner, Bra- in reward prediction and reinforcement learning. According un, and Coles (1997) reported an ERN-like component follow- to this previous research (Barto, 1995; Montague, Dayan, & Sejnowski, 1996; for review, see Schultz, 2002), the basal ganglia evaluate ongoing events and predict whether future events will be This research was supported in part by National Institutes of Mental favorable or unfavorable. When the basal ganglia revise their Health (NIMH) predoctoral fellowship MH069047 (G.H.), NIMH predictions for the better or for the worse, they induce a phasic postdoctoral fellowship MH63550 (C.B.H.), and NIMH grant increase or decrease, respectively, in the activity of midbrain do- MH62196. Portions of this article were presented at the 44th annual meeting of the Society for Psychophysiological Research, Santa Fe, New pamine neurons. These phasic increases and decreases in dopa- Mexico, October, 2004. mine activity indicate that ongoing events are ‘‘better than Address reprint requests to: Greg Hajcak, Department of Psychology, expected’’ or ‘‘worse than expected,’’ respectively. University of Delaware, Newark, DE 19716, USA. E-mail: hajcak@ The reinforcement learning theory of the ERN extends this psych.udel.edu. theoretical framework by proposing that the impact of the 161 162 G. Hajcak et al. dopamine signals on motor-related areas of the ACC modulates ticipants were presented with a cue indicating that they had either the amplitude of the ERN, such that phasic decreases in dopa- a 25%, 50%, or 75% chance of receiving a financial reward on mine activity (indicating that ongoing events are worse than ex- that trial. We predicted that the cues would induce expectancies pected) are associated with large ERNs, and phasic increases in of future rewards, and that negative outcomes would elicit the dopamine activity (indicating that ongoing events are better than largest feedback ERNs on those trials in which rewards were expected) are associated with small ERNs (Holroyd & Coles, most expected. 2002; see also Holroyd, 2004). According to the theory, then, the To verify the success of this manipulation, we also measured response ERN and the feedback ERN are both elicited by the the amplitude of the P300, a component of the ERP that is sen- first (unpredicted) indication that an unfavorable event is occur- sitive to expectancies (Courchesne, Hillyard, & Courchesne, ring or is about to occur, the former by error responses and the 1977; Duncan-Johnson & Donchin, 1977; Johnson & Donchin, latter by undesired outcomes. The theory further holds that the 1980). We reasoned that if strong expectancies were induced in dopamine signals are used by the motor-related areas in the ACC the participants by the stimulus cues, then these expectancies to improve performance on the task at hand according to prin- would be reflected in the amplitude of the P300. Furthermore, ciples of reinforcement learning. based on a recent study by Yeung and Sanfey (2004), who found Although the reinforcement learning theory of the ERN pro- equally large P300s following equiprobable negative and positive poses that the ERN is associated with events that are worse than feedback in a gambling experiment, we predicted that the P300 expected, the theory is nonspecific about what actually consti- would not differ for positive and negative feedback. tutes an ‘‘unfavorable’’ outcome. Thus, for example, the theory does not distinguish between financial rewards and punishments on the one hand (as indicated by ‘‘utilitarian’’ feedback) and correct trials and error trials on the other (as indicated by ‘‘per- EXPERIMENT 1 formance’’ feedback). Instead, the theory focuses on the rein- In this experiment, participants performed a guessing task similar forcing properties that these outcomes share in common to that used by Holroyd et al. (2003): Participants were instructed (Holroyd, Coles, & Nieuwenhuis, 2002). For example, in a re- to guess which of four doors hid a prize; however, unlike the cent experiment Nieuwenhuis, Yeung, Holroyd, Schurger, and Holroyd et al. study, at the start of each trial participants re- Cohen (2004) found that when feedback stimuli conveyed both ceived a cue indicating how many of the doors (1, 2, or 3) hid a utilitarian and performance information, the feedback ERN prize. Following each response, participants were presented with could reflect either dimension of the information, depending on feedback indicating whether or not their guess was accurate. which aspect of the feedback was emphasized. The authors ar- Consistent with the cue, the probability of negative feedback was gued that their findings indicated that ‘‘In the context of this 75%, 50%, and 25%, respectively, and thus the overall prob- [reinforcement learning] theory, utilitarian and performance as- ability of negative feedback was 50%. We reasoned that partic- pects of feedback are functionally equivalent: both evaluate out- ipants would tend to expect negative feedback on one-cue trials comes along a good–bad dimension, and hence both can elicit an and expect positive feedback on three-cue trials. If the ERN is ERN’’ (p. 745, cf. Gehring & Willoughby, 2002). sensitive to expectations, as predicted by the reinforcement Recently, we have begun to explore the conditions by which learning theory of the ERN, then negative feedback on three-cue external outcomes are categorized by the system as favorable or trials should be associated with a larger ERN relative to negative unfavorable. In a recent experiment, for example, Holroyd, Lar- feedback on one-cue trials and the ERN to negative feedback on sen, and Cohen (2004) found that the ERN was sensitive to the two-cue trials should be intermediate.