Error and Deviance Processing in Implicit and Explicit Sequence Learning
Total Page:16
File Type:pdf, Size:1020Kb
Error and Deviance Processing in Implicit and Explicit Sequence Learning Nicola K. Ferdinand, Axel Mecklinger, and Jutta Kray Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/20/4/629/1759430/jocn.2008.20046.pdf by guest on 18 May 2021 Abstract & In this experiment, we examined the extent to which error- in the course of the experiment, albeit faster for explicit than driven learning may operate under implicit learning condi- implicit learners. This observation supports the view that de- tions. We compared error monitoring in a sequence learning viant events acquire the status of perceived errors during ex- task in which stimuli consisted of regular, irregular, or random plicit and implicit learning, and thus, an N2b is generated sequences. Subjects were either informed (explicit condition) resembling the ERN/Ne to committed errors. While performing or not informed (implicit condition) about the existence of the the task, expectations about upcoming events are generated, sequence. For both conditions, reaction times were faster to compared to the actual events, and evaluated on the dimen- stimuli from regular sequences than from random sequences, sion ‘‘better or worse than expected.’’ The accuracy of this thus supporting the view that sequence learning occurs irre- process improves with learning, as shown by a gradual increase spective of learning condition. Response-locked event-related in N2b amplitude as a function of learning. Additionally, a P3b, potentials (ERPs) showed a pronounced ERN/Ne, thereby sig- which is thought to mirror conscious processing of deviant naling the detection of committed errors. Deviant stimuli from stimuli and is related to updating of working memory repre- irregular sequences elicited an N2b component that developed sentations, was found for explicit learners only. & INTRODUCTION ated in the anterior cingulate cortex (ACC; e.g., Ullsperger Thorndike (1911/1970) described in his law of effect that & von Cramon, 2001). Furthermore, the ERN/Ne is sen- actions followed by positive reinforcement are more likely sitive to the degree of an error (Bernstein, Scheffers, & to be repeated in the future, whereas behavior that is Coles, 1995) and influenced by its subjective significance followed by negative outcomes is less likely to recur. This (Hajcak, Moser, Yeung, & Simons, 2005; Gehring et al., implies that behavior is evaluated in the light of potential 1993). It is also elicited by error observation and by consequences, and nonreward events (i.e., errors) need feedback that signals an error was committed (Miltner, to be detected in order for reinforcement learning to take Braun, & Coles, 1997), thus emphasizing the flexibility of place. In short, humans have to monitor their perfor- the underlying error processing system. The ERN/Ne is mance in order to detect and correct errors, and this often followed by a positivity to errors (Pe), a positive detection process allows them to successfully adapt their wave with centro-parietal distribution. behavior to changing environmental demands. A neural system that plays a crucial role in reinforce- Of the abovementioned monitoring processes, the ment learning is the mesencephalic dopamine system. detection of committed errors is thought to be mirrored Schultz (2002) and Schultz, Dayan, and Montague (1997) by the error-related negativity (ERN) or error negativity recorded spike activity from mesencephalic dopamine (Ne), an event-related potential (ERP) component elic- cells in conditioning experiments with monkeys. They ited around the time an error is made (Gehring, Goss, found that the presentation of a reward elicits a phasic Coles, Meyer, & Donchin, 1993; Falkenstein, Hohnsbein, response in the dopamine neurons. This observation is Hoormann, & Blanke, 1990). The ERN/Ne can be ob- consistent with the hypothesis that the mesencephalic served in simple reaction time tasks (e.g., Gehring et al., dopamine system codes for the hedonic aspects of 1993; Falkenstein et al., 1990) as well as in recognition reward. During learning, the dopaminergic signal ‘‘prop- memory tasks (Nessler & Mecklinger, 2003). ERN/Ne on- agates back in time’’ from the time the reward is de- set coincides with response initiation and peaks roughly livered to when the conditioned stimulus is presented. 80 msec afterward. Its topographical maximum lies over Thus, the mesencephalic dopamine system can also fronto-central brain regions, and is thought to be gener- become active in anticipation of a forthcoming reward. Moreover, when a reward is not given, the mesence- phalic dopamine neurons decrease their firing rate at the Saarland University, Saarbru¨cken, Germany time the reward would normally have been delivered. D 2008 Massachusetts Institute of Technology Journal of Cognitive Neuroscience 20:4, pp. 629–642 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2008.20046 by guest on 28 September 2021 Dopaminergic activity also falls below baseline when the that the human brain learns by evaluating the results of monkey is presented with a stimulus that predicts pun- our actions, and that this learning is driven by reward- ishment. On the basis of these results, Schultz and col- related information carried to the ACC. However, these leagues have proposed that the dopamine neurons are conclusions were derived from explicit learning experi- sensitive to changes in the prediction of the hedonistic ments such as probabilistic learning studies. At present, ‘‘value’’ of ongoing events: A positive dopamine signal is it is unclear whether dopamine-induced modulation of elicited when an event proves better than predicted, and ERN/Ne amplitude, as proposed by the reinforcement a negative one when an event proves worse. Moreover, learning theory, can be generalized to other learning con- they suggested that the phasic responses seen in dopa- ditions, for example, implicit learning. The main goal of mine neurons might serve as error signals used for ad- the present study is to investigate the role of error moni- Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/20/4/629/1759430/jocn.2008.20046.pdf by guest on 18 May 2021 justing the associative strength of stimuli and responses toring in implicit learning. More specifically, we examined in neural areas that receive input from the mesencephalic whether the detection of committed and perceived errors dopamine system (e.g., the prefrontal cortex). (nonreward events) and their implication for learning re- In a model recently proposed by Holroyd and Coles quire an intention to learn, or can occur without aware- (2002), ERN/Ne amplitude has been linked to the mesen- ness of the to-be-learned materials. cephalic dopamine system. More specifically, this model So far there are only a few ERN/Ne studies that have assumes that when participants commit errors in reaction distinguished between explicit and implicit error monitor- time tasks, the mesencephalic dopamine system conveys ing, namely, monitoring for noticed and unnoticed errors. reinforcement learning signals to the frontal cortex. If the One reason for this might be that error monitoring outcome of an event is better than expected (and thus processes have mostly been investigated with overt re- the executed action implies reward), the result is a phasic sponse errors, often defined as inappropriate button dopamine burst. If the outcome is worse than expected presses, which people are typically aware of. Nieuwenhuis, (thus implying nonreward or punishment), the result is Ridderinkhof, Blom, Band, and Kok (2001) used a a dip in phasic dopamine. The ERN/Ne is presumably different approach to examine whether an ERN/Ne and generated by disinhibiting the apical dendrites of motor Pe can be observed after response errors that subjects neurons in the ACC when the dopamine reinforcement are unaware of. They had their subjects perform an anti- signal is lacking. These error signals are used to train the saccade task, that is, subjects were instructed to gener- ACC, ensuring that control over the motor system will be ate a saccade to the opposite side of a peripheral onset released to a motor controller that is best suited for the cue. This task elicited many incorrect reflex-like sac- task at hand. The response conflict hypothesis offers an cades that people were not aware of. Nieuwenhuis et al. alternative explanation for ERN/Ne generation. According (2001) found that irrespective of error awareness, erro- to this theory, the ACC plays an important role in moni- neous saccades were followed by an ERN/Ne, whereas toring for the occurrence of conflict during response the Pe was more pronounced for conscious than for selection (e.g., Botvinick, Braver, Barch, Carter, & Cohen, unconscious errors. This means that error monitoring as 2001). Response conflict occurs whenever two or more reflected in the ERN/Ne seems to take place even when incompatible response tendencies are simultaneously ac- participants are not aware of their errors. tive. Upon detection of such conflict, the ACC conveys a Capitalizing on these findings, in the present study feedback signal to the brain areas involved in the execu- we examined whether errors subjects are unaware of tion of control, informing these areas that executive con- are used as feedback signals that mediate learning pro- trol processes must be more strongly engaged. cesses. For this purpose, we used a sequence learning To test their model, Holroyd and Coles examined the paradigm with deviant stimuli inserted into an otherwise