EVENT-RELATED POTENTIALS DURING LEARNING AND RECOGNITION OF
COMPLEX PlCTURES
Maria Luisa Arrnilio
A thesis submitted in conformity with the requirements
for the degree of Master's of Arts
Graduate Department of Psychology
University of Toronto
O Copyright by Maria Luisa Armilio, 1997. National Library Bibliothèque nationale l*l of Canada du Canada Acquisitions and Acquisitions et Bibliographie Services services bibliographiques 395 Wellington Street 395, rue Wellington Ottawa ON K1A ON4 ûttawaON K1A ON4 Canada Canada
The author has granted a non- L'auteur a accordé une licence non exclusive licence allowing the exclusive permettant à la National Library of Canada to Bibliothèque nationale du Canada de reproduce, loan, distribute or sell reproduire, prêter, distribuer ou copies of this thesis in microfonn, vendre des copies de cette thèse sous paper or electronic formats. la foxme de microfiche/film, de reproduction sur papier ou sur format électronique.
The author retains ownership of the L'auteur conserve la prcpnété du copyright in this thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substantial extracts fiom it Ni Ia thèse ni des extraits substantiels may be printed or otherwise de celle-ci ne doivent être imprimés reproduced without the author's ou autrement reproduits sans son permission. autorisation. ERPs and Picture Memory
Armilio, Maria Luisa Department of Psychology University of Toronto
Event-related potentials during leaniing and recognition of complex pictures. Master's Degree, 1997.
Event-related potentials (Ems) were recorded while subjects learned a large set of complex, coloured pictures. Recognition memory was examined within the sarne day and after a 24 hour hg. Memory performance decreased From 88% to 65% over 24 hours. The ERP wavefoms showed a prominent parietal-occipital P 100-N 150-P240 complex that was the same in leaming and recognition. A centro-parietal P650 wave was Iarger during recognition than during learning for both the new and old pictures. During learning the pictures elicited a sustained positive potential mavimally recorded in the occipital regions. Finally. in the recognition condition, new pictures elicited an N400 wave over Frontal electrodes while old pictures elicited an earlier centroparietal P650 than novel pictures. Learning is most clearly associated with a sustained occipital positivity. recognition with a parietal positive wave. and novelty-detection with a frontal negative wave. ERPs and Picture Memory
Acknowlegements I would like to thank my thesis advisor, Dr. Terence Picton. for his help throughout the preparation of this thesis as well as Dr. Fergus Craik for his suggestions in the design of the expenment and preparation of the written work. Many thanks to Brigitte Boucher and
Vincent Choi for their technical expertise, and to those who volunteered their time to participate in the experiment. My research was supported by a University of Toronto Open Fellowship and a Men's Service Group Studentship from Baycrest Centre for Geriatric Care in Toronto.
iii ERPs and Picture Memory
Table of Contents
Page
List of Figures ...... v
Introduction...... 1 Method . Participants...... Stimuli...... Procedure ...... Results...... Discussion ...... ERPs and Picture Memory
List of Figures
Figure 1 : Black and white exarnples of the complex, coloured stimuli used in the Experiment. Figure 2: Schema of the procedure employed in the Expenment. Figure 3: Cornparison of grand mean ERP waveforms for new correct and old correct
pictures on Day 1 and Day 2. Figure 4: P 100-N150-P240/P350 complex - Potentials consistently evoked by pictures. Figure 5: New vs. Old - The N450 wave. Figure 6: New vs. Old - The P650 wave. Figure 7: Leaming vs. Recognition - The sustained potential. ERPs and Picture Memory 1
Event-related potentiais daring learning and recognition of complex pictures
When the psychological study of memory began in the laboratories of Hermann Ebbinghaus in the late 1800's, the centuries-old philosophicai speculation about the mental process called 'memory' became a topic of scientific investigation. In the past 100 years psychoiogists probing the workings of memory have been trying to determine how information gets into memory and how this information is later retrieved. Current opinion suggests that memory consists of many complex systems (Tulving & Schacter,
1990). Information cmbe stored in such systems for durations fiom a fraction of second up to an entire lifetime. The capacity of these memory systems can be as limited as a tiny buffer store or as extensive as the long term memory store that appears to surpass the largest and most powerful cornputers.
Whereas hurnans have severe limitations when stonng and retrieving verbal information such as letters, words. nurnbers and sentences, they have an impressive ability to remember complex meaningfùl stimulus configurations such as pictures of people. places and things. Although memory for pictures is extremely enduring and accurate. the physiological bais of this specialized picture memory process has been relatively unexplored. The present study examined the event-related potentials (ERPs) recorded while participants learned and recognized a large set of complex, coloured pictures. More specifically, it addressed the fünction of detecting novel pictures to the hurnan memory process. ERPs and Picture Memory 2
Noveltv Detection Any complex system must be efficient in order to be successful. Since it is uneconomical to store redundant information, it is necessary to decide what incoming information is already in storage and what is not. information that has not yet been experienced should not be overlooked. Thus, novelty detection seems to play a cntical role in deciding what needs to be responded to and stored (Tulving, Markowitsch, Kapur.
Habib & Houle, 1994a; Tulving & Kroll. 1995; Tulving, Markowitsch, Craik, Habib &
Houle, 1996). Two paramount questions arise: ( 1 ) What areas of the brain are active during novelty detection? and (2) How quickly cmnovelty be determined?
The brain imaging technique. positron emission tomography (PET). has successfdly identified areas of the brain that are active dunng novelty detection. Several recent PET studies by Tulving and his colleagues have indicated wide-spread novelty- assessment networks in the brain (Tulving et al.. 1994a; Tulving & Kroll, 1995; Tulving et al., 1996). During recognition tasks involving both verbal and nonverbal stimuli. the most prominent novelty activations occur in cortical and subcortical regions of the right expanded limbic systern - hippocarnpal formation. parahippocampal gyrus. retrosplenial cortex, thalamus. subcallosal area. the border between cortical areas IO and 33 as weil as anterior and inferior cingulate cortex, putamen and media1 prefrontal cortex - and additional activity in various areas of the temporal and temporoparietal lobes bilaterally. These observations have led to the 'novelty-encoding hypothesis'. Specific novelty- assessment networks function to influence whether the encoding of on-line information into long-term memory will take place. Tulving et al. (1996) hypothesize that the encoding of incoming information depends on the novelty of the 'on-line' information. Novelty detection may be accomplished by the neuronal networks in the limbic/temporal regions. These novelty-detecting neurons then activate the lefi-frontal cortical areas that ERPs and Picture Memory 3 have been shown to be involved in the encoding of novel information for episodic memory (Tulving et al, 1994a).
While the good spatial resolution of PET can provide information regarding the neuroanatomical areas where novelty detection occurs. it is unabte to answer the second question. that is. how quickly does novelty detection occur? In PET. regional cerebral blood flow is measured for 60 seconds. Thus. data collected from a PET scan provides a composite image of al1 neural activity during that time Me.While information about the neural interactions between different regions can be extracted from PET data through decomposition of interregional covariances of activity to yield a 'network analysis' (e.g..
McIntosh & Godez-Lima 1994), it is impossible to extrapolate back from average blood flow to the time-course of neuronal activity. Thus. while PET can provide answers to the question 'where' novelty detection occurs in the brain. the 'when' question remains unanswered.
A usefül tool for addressing the temporal sequence of the processes in novelty detection. and memory in general. is the event-reiated potential or ERP. As the neurons of the human brain process information in cognitive tasks such as leaming and remembering they generate electrical fields. When large numbers of neurons are synchronously active and when the individual fields of these active neurons are similarly oriented, these electrical fields may be recorded from the scalp. These ERPs are capable of indexing both the timing of cognitive events and the component structure of these processes. In addition. multichannel ERP recordings may indicate the location of the activity involved, although with less accurate anatomical resolution than techniques such as PET and functional magnetic resonance imaging (W).Studies using ERPs have demonstrated changes in timing and component structure associated with memory but ERPs and Picture Memory 4 have not yet answered the question as to where these processes occur in the brain in relation to time and components.
A type of novelty detection that has been reported in the ERP literature concems the onenting response. The tasks involve the delivery of a novel stimulus that is irrelevant to the ongoing task. For exarnple. the typical auditory oddball task consists of frequently occurring standard or non-target stimuli and infiequently occumng targets to which participants are required to provide a response. Generally. the standards and target are perceptually similar (e.g.. two tones of different frequencies). Voluntary detection of the target in such a task elicits a prominent positivity over parietal sites at a latency of 300 - 600 ms. referred to as the P3b. However. infiequent stimuli that are substantially different fiorn the standard and target stimuli (e.g.. a complex sound such as a do$ bark). evoke a positive potential. referred to as the P3a, which occurs at a latency of 250 - 550 ms and with maximum amplitude over Frontocentra1 regions (Courchesne. Hillyard. &
Galambos. 1975; Knight. 1984; Yamaguchi & Knight. 199 1 a: Knight. 1996).
Lesion data recorded from neurological patients has provided evidence for the involvement of multiple brain regions in the generation of the P3a. While unilateral prefrontal damage does not affect the P3b potentids to target stimuli. this damage eliminates the frontocentral P3a potential to unexpected novel stimuli (auditory: Knight,
1984; somatosensory: Yamaguchi & Knight, 199 1 b). Similarly. damage to the posterior hippocarnpal region does not change the P3b response to target stimuli. but reduces the frontocentral P3a response to auditory or somatosensory novel stimuli (Knight, t 996). In a visual task. Knight (submitted) found that prefrontal, parietal and temporal-parietal lesions al1 significantly reduced the amplitude of the P3a while leaving the P3b relatively unaffected. Thus. novelty-related P3a potentials index brain activity in a cortico-lirnbic ERPs and Picnire Memory 5 orienting system involving widespread. multimodd interconnections of association cortex (Knight, submitted).
The novelty-related brain networks proposed by Knight and his colleagues and Tulving and his colleagues are very sirnilar. However. the function that each proposes for novelty detection differ in one critical aspect - relevance to the task. While both the recognition memory paradigm (used by Tulving) and the novelty oddball paradigm (used by Knight) involve orientation to novel stimuli. only the recognition memory paradigm involves orientation to stimuli relevant to the task at hand (Le.. the occurence of a novel picture in the recognition test requires a response). Stimuli such as a dog barks are irrelevant to the oddball task. This distinction in task relevance makes it dificult to make direct comparisons. In addition. the novelty oddball studies do not address how stimulus novelty is important to normal memory function.
Fabiani and her colleagues take the approach to novelty that is both task relevant and related to memory (Donchin & Fabiani. 199 1 : Fabiani. Karis & Donchin. 1990:
Karis, Fabiani & Donchin. 1984). These studies were based on the "von Restorff" or isolation effect described by von Restorff in 1933 (cited in Karis et al.. 1984). This procedure involves one or more 'isolate' items that deviate in some way (e.g.. colour. size) 60m the other items in a series of study items. Enhanced learning of the isolate item(s) typically occurs such that the probability that it will be recalled increases (e-g.. Cimbalo.
1978; Wallace, 1965). Fabiani and colleagues (Fabiani. Karis & Donchin. 1990; Karis. Fabiani & Donchin, 1984) presented participants with short lists of visually presented words, some of which were presented in a different size from the rest of the items in the series. They found that al1 participants showed larger amplitude P300s to isolated items and recognized the isolated items faster than non-isolated items in recognition tests. ERPs and Picture Memory 6
More specifically, participants employing rote-leaming mnemonic strategies showed a higher von Restorff effect than those employing more elaborative strategies. This separation extended to the ERP results such that oniy the participants using the rote-type strategy showed larger P3OOs to words during initial presentation that were subsequently recalled. Thus, Donchin and Fabiani (199 1) suggested that. in the absence of elaborative processing. the relative distinctiveness of the physical attributes of words plays a part in how well an item is leamed and, therefore, can explain the recall advantage of the isolated words. They Merhypothesized that the amplitude of the P300 is proportional to the subjective distinctiveness of the item evoking it.
While the isolates used in this paradigrn may be relevant to the memory task. the fact that they are physically deviant relative to the other items to be studied does not make these items truly novel. While they may br different from the other stimuli in the experiment, they can be considered quite familiar. even aftsr one or two have been seen.
Furthemore. Rugg (1995) contends that there "seems no reason to assume that. wifhin a sample of physically distinctive items. relative distinctiveness is the rnost important determinant of recall probability"(p. 140).
Complex pictures have several advantages over these and other stimuli in memory/novelty paradigms. Pictures. and in particular complex pictures, are more easily presented as truly novel items than words. which have usually been expenenced before. Pictures cm also be processed quickly. Although sentences also serve as good novel stimuli (e.g.. Tulving et al.. 1994b), they require considerably more time to present and process. Also important to the averaging process in ERP studies. large numbers of pictures can be presented and memory for these pictures is more accurate and durable over time than for verbal stimuli. ERPs and Picture Memory 7
Picture Memorv
Supenor memory for picmes compared to words is a well-established phenornenon. Presenting 600 black and white photographs for 5 s each in a yesho continuous recognition paradigm. Nickerson (1 965) found that recognition accuracy for old picîures decreased frorn 97% with a lag of 40 intervening items to 87% with a lag of
200 items. A follow-up study by Nickerson (1 968) reveaied that when pictures. seen only once before. were presented 1 or 360 days later. the accuncy of correctly recognizing the old picture decreased fiom 76% to 32%. Shepard ( 1967) employed a two-alternative forced choice (2-AFC) task to compare rnemory performance for several hundred words. sentences and coloured pictures. In addition. rnemory preformance for pictures was also tested at delays of 2 hours. 3. 7 and 120 days. Although participants showed good performance in the irnmediate task for words and sentences (88% and 89% correct. respectively). their performance was particularly impressive Cor the pictonal stimuli (100% correct). The supenonty of picture memory continued at delays of 2 hours. 3. 7 and 120 days with accuracies of 100%. 92%. 87% and 58%. respectively. Even after a delay of 1 week. memory for pictonal stimuli was equivalent to that found for the verbal materials tested at the shortest delay. Perhaps the most irnpressive demonstration of the vast memory for pictures by humans was provided by Standing. Conezio and Haber (1970). In this study. participants were shown 2.560 complex photographs for a duration of 10 s each and recognition memory was tested using a 2-AFC task on 4 successive days. Performance exceeded 90% accuracy even after a delay of 3 days. Thus. large nurnbers of pictures, each viewed for only a few seconds. are retained extremely well.
Pictorial encoding involves. in part, the establishment of an increasingly detailed memory representation of a picture over time (Intraub. 1980). Potter and Levy ( 1969), varying the presentation time for magazine pictures from 2000 ms to 125 ms per picture. ERPs and Picture Memory 8 studied the probability of recognition in a yesho recognition paradigm as function of presentation rate of pictures on first presentation. They found that performance drops dramatically when the duration of each picture is reduced. The probabilities of correct responses to old picnires viewed for a duration of 2000 ms and 125 ms were 93% and 16%, respectively. Potter and Lew (1 969) also found that while performance depended neither on the time taken by the entire sequence nor on the presentation time of the imrnediately preceding picnire at encoding, the probability of recognition depended upon the time each picture was in view. This limitation on encoding time may relate to the nurnber of different eye fixations that can be made during the presentation of a picture (Lohs. 1972). Thus. at longer presentations durations. participants could continually scan the picture while encoding additional details (Loflus. 1972: Lofius & Bell. 1975).
I ntraub ( 1980) gained further insight into pictorial encoding by exarnining the effect that stimulus off time has on recognition memory. She found that recognition performance was not much affected by reduction of stimulus duration fiom 5000 ms to 1 IO ms at encoding. Accuracy was 96% correct when stimuli were displayed for a duration of 5000 rns at encoding with no interstimulus interval (ISI) and dropped to 84% correct when stimuli were presented for only 110 ms with an ISI of 4890 ms. Decreases in performance were much more prominent when stimulus duration was held constant and the total study time (i.e.. stimulus duration plus [SI) was reduced. Thus, increasing the time between the study items led not only to the retention of more pictures but also to the storage of more information per picture. The results of Intraub (1980). therefore. suggest that the encoding of visual detail is not confined to the duration of the stimulus.
There are no known capacity limitations to picture memory. The bounds of picture memory are. therefore, potentially limitless and retention of pictorial information ERPs and Pichire Memory 9 over several days is easily accomplished, even with very bnef exposure. In addition. when either stimulus duration or stimulus off time is decreased. less complete visuospatial information about a picture is retained.
ERPs and Memorv In the past ten years. many studies have explored the ERP correlates of encoding and retrieval of information in long term memory. For the most part. these ERP studies of memory have focused on verbal tasks. The standard means of investigating memory processes with ERPs is achieved by comparing waveforms evoked by items presented on the tkst and subsequent occasion. ERPs elicited by old items are compared to those elicited by new items either when the assigned task does not require participants to discriminate these (Le.. indirect tests) or during explicit recognition tasks (Le.. direct tests). The characteristic outcome of these cornparisons is a widespread positivity in the ERPs waveforms for repeated items relative to new items. This "ERP repetition effect" or "oldnew effect" starts at approximately 350 ms afier stimulus onset and lasts for approximately 300 rns. This effect is extremely robust in tasks involving word tasks. Many studies have found waveforms elicited by correctly recognized old words diverge from waveforms elicited by correctly recognized new words hundreds of milliseconds before overt measures of reaction tirne are obtained (Friedman, 1990; Johnson.
Pfefferbaurn & Kopell. 1985; Karis et al., 1984; Neville. Kutas. Chesney & Schmidt.
1986; Rugg & Doyle, 1992; Smith & Halgren. 1989). Likewise, many incidental memory tasks require participants to detect and respond to "target" items interspersed arnong non-target. a proportion which are repetitions of previously presented items; these studies have also found potent repetition effects for both imrnediate and delayed repetitions (Bentin & Peled, 1990; Rugg, 1985. 1987; Rugg & Nagy. 1987). The results fiom both direct and indirect studies are similar enough that the associated ERP effects ERPs and Picture Memory 10 likely reflect a common process such as successful retrieval from rnemory (Van Petten & Senkfor, 1996).
However. Smith and Halgren ( 1989) suggested that the repetition effect is the combination of two spatially and temporally distinct components. They used a recognition test for visually presented words with normal participants and with patients that had undergone either a left- or nght-sided anterior temporal lobectomy. While the normal participants and the nght-sided patients showed repetition effects. their lefi-sided patients did not. Furthennore. these effects resulted fiom the modulation of two components. An earlier component in the range of 250 to 450 ms was attenuated in ERPs elicited by old words and is probably related to the N400 component that is sensitive to semantic context (Kutas & HiIlyard. 1980). A subsequent long-lastinç positive wave. probably related to the P300. was greater in amplitude to ERPs elicited by old words. Smith and Halgren (1989) proposed that the repetition effect is composed of an overlap of both these early and late components. The lefi-lesioned patients showed only a mild deficit in performance in the absence of ERP repetition eEects. This observation led Smith and Halgren (1989) to propose that the ERP repetition effects observed with the normal and right-lesioned patients reflected processes specific to recollection. Since the left-lesioned patients were presurnably relying on differences in fmiliarity to make recognition judgments, these differences are not reflected in the ERP differences. They concluded that both the early and late effects are found only when recognition judgments are based on recollection.
nie results of Rugg and Nagy (1989) contradict this hypothesis by Smith and Halgren (1 989). but do support the existence of two componrnts in the repetition effect. Using the time over which items must be remembered as a variable to which the ERPs and Picnire Memory 1 1
components may be sensitive, they tested normal participants in a continuous recognition task. They found that the early and late differences in the ERP waveforms elicited by the
first and subsequent presentation of the words were unaffecten by whether the second presentatiori occurred after 6 or 19 intervening items. Recognition rnernory for the words presented in the continuous recognition task was tested after a 35 to 40 minute delay. Although participants' performance was well above chance level. there were no differences in the ERPs until about 500 ms. sometime after the NJOO peak. Thus. the early effects were no longer apparent even though the late effect rernained. While these results support the assertion by Smith and Halgreh ( 1989) that repetition effects are the result of at least two cornponents. Rugg and Nagy (1 989) conclude that modulation of the N400 regio~of the ERP waveform is not a necessary precursor to correct recognition judgements. Instead. they argued that early effects were govemed more by the study-test interval.
ERl' studies have memory have focused on verbal tasks to such a great extent that the ERP repetition effect is ofien referred to as the 'word repetition effect' (e.g.. Smith & Halgren. 1989). However. some recent ERP studies have started to explore mernory for nonverbal stimuli. For example. Chao. Neilson-Bohlman and Knight (1995) found that auditory metpory for environmental noises could &O be dissociated into distinct processes. îlieir results showed that an N400 wiis generated by the initial presentation and long-delay repetition of the sounds. and that a P300 was larger in amplitude and longer in latency when sounds were repeated at either short or long deiays. Van Petten and Senkfor (1 996) examined the late positive component of the ERP repetition effect using novel visual patterns and words. While words elicited the typical ERP repetition effect upon second presentation during an incidental repetition. the repeated patterns showed linle sign of an N400 and elicited a slow hegative shifl with an occipital ERPs and Picture Memory 12 maximum instead of the greater positivity found with repeated words. While accuracy was greater for words than the patterns during a recognition memory task and words and patterns showed equivalent overall amplitudes for the late positive recognition effect. old patterns also elicited a posterior negative shifi. Van Petten and Senkfor ( 1996) suggested that a late positive component is elicited by patterns as it is by words but that it is partiaily cancelled by a postenor slow negative shift that appears not to be linked with either perceptual priming or conscious recollection.
While the strength of the picture superiority ef3ect is very impressive. the electrophysiological basis of this specialized mernory process has been relatively unexplored. Only a handful of studies have employed picture stimuli. A few of these studies have looked at recognition for simple pictures (Friedman. 1990). compared recognition memory for pictures and words (Noldy. Stelmack & Campbell. 1990) and recognition memory for faces (e.g.,Smith & Halgren. 1987).
Noldy et al. (1 990) tested recognition memory for simple line-drawings and the names of these drawings after either an incidental or intentional learning task.
Recordings were taken from four midline sites (Fz. Cz, Pz. Oz) and two lateral sites (CL C6). Recognition performance was greater for pictures and when the learning task was intentional. Comparisons across ERP waveforms elicited by words and pictures revealed differences in a N4501P650 cornplex. In general. N450 amplitudes were larger for words than for pictures at both fronto-central and posterior sites. At acquisition. this N450 difference was significant only in the intentional learning condition. This difference was also observed during recognition memory for intentional items (both new and old items) and incidental items (old items only). Noldy et al. (1 990) argued that these results supported the hypothesis that the N400 is linguistic in nature and does not just reflect the ERPs and Picture Memory 13 processing of any complex stimulus (Stuss. Sarazin. Leech & Picton. 1983: Stuss. Picton
& Cem. 1986). For the P600. however, greater amplitudes were found for pictures than for words for both types of leaming conditions at acquisition and during recognition for old items, from both the intentional and incidentai learning conditions. and for new items fiom the intentional learning condition. Inasmuch as pictures were remembered better than words, Noldy et al. ( 1990) argued that the larger P600 for pictures than for words may be evidence for the association of the P600 and better memory.
Furthemore. Noldy et al. (1990) revealed the charactenstic ERP repetition effects previous observed with verbal stimuli through comparisons within stimulus types. For pictures in the incidental learning condition set. N450 amplitude for new items was larger than for acquisition and old items. For words in the intentional learning condition set condition. N450 amplitude was greater in amplitude for new items during recognition than at acquisition. For both pictures and words. the P600 was greater for old items than new items. However. there were no significant differences between the leming conditions for this late positive component during either the acquisition or recognition phases.
Using a continuous recognition paradigm. Friedman ( 1990) tested rnemory for line drawings of cornmon objects at lags of 2. 8 or 32 stimuli. Recordings were taken fiom 5 midline sites (Fpz, Fz, Cz, Pz and Oz). Significant effects oflag were not found on any of the behavioural measures (except reaction time), nor on any of the ERP data. Friedman (1990) suggests that pictures. unlike verbal stimuli. are not held in a limited- capacity pnmary rnemory buffer with other pictures. When ERPs were averaged across the various lags. however. two repetition effects were revealed. The first was a greater negativity maximal at frontal electrodes in ERP waveforms for new items that began at ERPs and Picnire Memory 14
300 ms and lasted approximately 300 rns. An N300 wave was modulated by the oldhew distinction such that this component was larger to new than to old items with this difference being most significant at the frontal sites. Participants were able to discriminate the difference between old and new picnires by 300 ms. and this component retlected the registration of this discrimination. While the amplitude of the P600 for old items was significantly greater than new items. it was interpreted as reduced negative activity in the ERPs to old items.
The second type of activity consisted of a greater negativity. maximal at Cz and Pz. in the ERPs elicited by old items that began 600 ms post-stimulus onset and continued on until the end of the recording epoch. It was suggested ththis effect was mediated by a 'positive slow wave' that is activated to presentation of new items. Thus. the late negative component in the ERP wavefoms to old items is due to the absence of this slow positive wave which unrnasks the negativity. The oldhew memory effects on the ERP waveform were not very robust in this study. and Friedman (1990) attributes this outcome to the reduction of elaborative processing that may have occurred For encoding of the pictures as cornpared to words.
Both of the above studies show an ERP repetition effect with picture stimuli. However, the stimuli in both tasks were easy to narne and. therefore, might have been processed in a similar manner to words: the line drawings may have been verbally labelled. This issue bnngs about two concems. First. the strong picture rnemory effect has been show with complex stimuli (e-g.. Intraub. 1980; Nickerson. 1965: Potter &
Levy. 1969; Shepard. 1965; Standing et al.. 1970). Second. the encoding and retrieval processes that take place when looking at the complex world are likely to involve both semantic interpretation and registration of detail (Mandler & Johnson, 1976). Despite the ERPs and Picture Memory 15 advantages of working with simple drawings, important aspects of normal visual processing may be lost in experimental paradigms that use such simple stimuli.
A study looking at ERPs to farniliar and unfamiliar faces sheds some light on the above concems. Smith and Halgren (1987) measured activity at F3, F4, C3. C4, T5 and T6 in addition to the midline sites of Fz. CL PZ and Oz to presentation of black and white slides of male faces. They found that ERPs to non-repeated and repeated faces began to diverge about 400 ms after stimulus offset such that this component was larger in amplitude to non-repeated faces. Notably. this N445 peak was larger in amplitude to novel faces at posterior sites and. in particular. over the right posterior temporal lobe. The researchers descnbed their result as consistent with the specialization of this area for integrating and remembering complex visuospatial information in general. and faces in particular (Milner. 1968). Following the N445. repeated faces elicited a larger amplitude P520 than unrepeated faces with this difference beinp maximal at central and parietal sites. Smith and Halgren (1987) suggest that the P520 may reflect the cornpletion of stimulus classification or task closüre. In addition, the modulation of the N445 and P520 was independent. A negative-going parietal slow wave, more negative for repeated faces. followed the P510 and continued to approximately 1O00 ms post-stimulus onset. Thus. classification of facial stimuli elicits similar repetition effects to those observed with memory activity in Ianguage processing.
Rugg, Doyle. Thomas, Perrett and Harries (briefly reported in Rugg & Doyle. 1994) investigated repetition of two types of stimuli that had similar physical content but differed in their representational content. One set of stimuli consisted of pictures that depicted a wide variety of scenes and objects that had not been previously seen by participants. The other stimuli were subjected to distortion by the addition of spurious ERPs and Picture Memory 16 high spatial frequencies. Participants were required to respond whenever a picture was repeated immediately. The trials of interest were those nontarget repetitions that were repeated between 5 and 9 items afier first presentation. Rugg et al. found robust repetition effects with the normal pictures. but no such effect with the distorted stimuli. Thus, manipulation of 'meaningfulness' appears to affect the ERP repetition e ffect. They concluded that. although nomal pictures can be verbaily encoded (i.e.. named). there is little scope for 'unitiùngf the representation of distorted pictures. That is, the distorted pictures could not be identified by a code (semantic. phonological, lexical. etc..) that uniquely represented such items within some domain of processing. This concurs with their hypothesis that a necessary condition for the repetition effect is that repeating items must possess some minimal representational content or structure in order to be encoded.
A recent study by Rugg and his colleagues (Rugg. Saordi &: Doyle. in press) has supported their hypothesis that any item that can be presented in a unitized fashion should be capable of evoking this effect regardless of whether the representation includes semantic information. Using line drawings of objects that could or could not exist in 3- dimensional space and unstructured visual patterns. they found that only the 3-D objects provided the repetition effect. While repetition of the unstmctured visual patterns had rather little effect on the ERPs. first presentation of the 3-D objects showed more positive ERP waveforms in general compared to the unstmctured patterns. There were morphological differences between the two types of stimuli. especially at earlier latencies at posterior sites. Rugg et al. (in press) suggest that these difierences retlect the differential processing accorded the two types of stimuli on the basis of systematic differences in low level sensory attributes, such as texture and spatial fiequency. However. in contrat to al1 previous studies. the ERP repetition effect to the 3-D objects appeared as a sizeable, reliable, negative shifi relative to ERPs elicited by first ERPs and Picture Memory 17 presentations. Early modulation of the waveform showed a postenor maximum and the subsequent modulation showed a frontal distribution. Rugg et al (in press) rule out explmations that the task was more dificult or that the negative shift reflects the differential development of a contingent negative variation. instead. they suggest that this negative shift reflects genuine differences in the activity of repetition sensitive structures.
The above review of memory studies using nonverbal stimuli suggests that not al1 aspects of the repetition effect are linguistic in nature. While simple line-drawings do not rule out the linguistic nature of these effects. studies such those performed by Smith and
Halgren (1 987) and Rugg et al. (reported in Rugg & Doyle. 1994) have contributed evidence to support this view. Although even simplified line-drawing representations of the world cm be considered enormously complex. much more information can be retained from complex pictures. For example. Mandler and Johnson ( 1976) showed that is is possible to characterize and investigate the kinds of information that are encoded and retained from complex scenes. The negative repetition effect observed by Rugg et al. (in press) implies that there is something inherently important to stimuli that have meaningful representations. be they simple line drawings or complex pictures. that is reflected in the ERP repetition effect. While Rugg et al. (reported in Rugg & Doyle.
1994) have demonstrated robust repetition effects. ihey do not pr: the durability of picture memory to the test with interitem intervals of only 9 items. Rugg and Nagy
( 1989) showed that ERP differences were unaffected by lags as great as 19 intervening items and Friedman (1990) showed that a lag of 32 items with line-drawings did not affect ERP differences. Based on the extraordinary durability and capacity of picture memory. it would be expected that ERP differences should endure substantially longer lags as well. ERPs and Picnire Memory 18
The Present Studv The present study examined recognition memory, and more specifically novelty detection for cornplex pictures. Novelty detection was examined in two different time frames ( I ) within the same day, and (2) afier a 24 hour deiay and a large number of sites were recorded from (48 sites). The expected outcornes of this study are that the decision as to whether as picture is novel would be rapid. based on the latency of a negativity in the ERP waveform and the source of this activity would localized within the temporal 10 bes. ERPs and Picture Memory 19
Method
Participants
Ten normal adult participants. 5 female. participated in the present experiment. Participants ranged in age fiom 23 to 37 years with an average age of 29 years. Al1 participants had normal or corrected-to-normal vision and were right-handed.
Stimuli The picture stimuli were 600 complex coloured photographs which were prepared from the Corel Professional Photo CD-ROM collection. Since complexity and distinctiveness of pictures has been shown to contribute to the picture superiority effect (e.g.. Anderson. 1978). pictures were selected to be both very mernorable and easily distinguishable. A variety of categories of photographs ranging from animals to people to scenery to food was used (see Figure 1 for black and white examples).
Procedure The experiment exarnined recognition memory in two different time mes: (a) within the same day. and (b) after a 24 hour hg. Pictures were presented for study on Day 1 and memory for the studied pictures was tested on both Day I and Day 2.
On Day 1. 300 pictures were randomly assigned to five study blocks. each containing 60 different stimuli. Participants were told that their memory for the pictures would be tested later in the experiment. Each study block was imrnediately followed by a test block that contained 30 of the stimuli in the preceding block dong with 30 novel stimuli. Twenty-four hours later. participants retumed to complete a second session. On this second day. 150 stimuli fiom the studied stimuli of the previous day that were not ERPs and Picture Memory 20
Figure Caption Figure 1. This figure illustrates black and white examples of the compiex. coloured stimuli used in the Experiment.
ERPs and Picture Memory 2 1
used in the recognition tests on the first day were presented dong with an equal nurnber of novel stimuli. Again. 5 blocks of test triais were presented, each containing 60 pictures (30 old and 30 new) (see Figure 2).
On both days, participants sat in a sound-attenuated room. at a distance of approximately 50 cm from a rnonitor on which al1 stimuli were presented. The average picture size was 24 cm in width by 21 cm in height and subtended an angle of approximately 26O by 23". Average overall luminance was 20 cd/m2 with a contrast range as far as 98%. Onset and offset times for picture presentation on the monitor were between 5 and 9 ms. During learning blocks, each picture was presented for 2000 ms and the screen remained blackened during an interstimulus interval of 1000 ms. This stimulus duration and stimulus off time were chosen to ensure accurate performance on the susbequent memory test (Potter & Levy. 1969; Intraub. 1980). During recognition test blocks. pictures were again presented for 2000 ms and participants were given instructions on the screen during the interstimulus interval to respond using one of four buttons on a 4-point confidence rating scale ('1' = definitely old. '2'= probably old. 'Y= probably new. 'rlt=definitely new). Responses of either 1 or 2 were judged as a correct response to old pictures and responses of 3 or 4 were judged as correct responses to new pictures. Participants were given as much time as they needrd. although they were encouraged to respond within one second. The next stimulus occurred 1000 ms derthe response.
ERP Recording ERPs were recorded during both Ieaming and test trials and on both days of the experiment. Electrophysiological data were recorded using an Electro-Cap cap with 9 mm (diameter) tin polygraphic electrodes. The thirty-two cap sites were FP1. FP2, AF3. ERPs and Picture Memory 22
Figure Caption Figure 2. This figure presents a schema of the procedure used in the Experiment.
Participants completed five sets of study-test blocks on Day 1. with 50% of test pictures derived from the previous study block. On Day?. the rernaining 50% of pictures from the study blocks that were not tested on Day 1 were presented in the tive test blocks on Day
ERPs and Picture Memory 23
AF4. Fz F3, F4, F7, F8. FC1, FC2, FC6. C3, C4. T7. T8. CP1. CP2, CP5. CP6, Pz. P3. P4. T5, T6. P03, P04. OzO 1. 02 and Iz (Amencan Electrophysiological Society. 1991). Non-cap sites were LO 1, L02, IO 1,102, F 1 1. F 12, FT9, FTIO. CB 1. CB2. TP9. TP 10, Nz and Neck. The LO IL02 and IO ln02 pairs of electrodes were positioned on the outer canthus of the eye and just inferior to the eye. respectively: the Neck electrode placement was a midline posterior site on the neck at a distance of 10% of the nasion- inion distance below the Iz site. Electrodes at AFz and Cz were used as ground and reference sites, respectively. Inter-electrode impedance mrasured at 10 Hz was kept below 3 kOhms. This was obtained by abrading the scalp sites using a blunt needle or by puncturing the skin sites with a fine needle (so as not to leave facial marks). EEG and EOG signals were amplified on a Neuroscan SynArnp at a gain of 3300 with an on-line analog filter bandpass of .O5 to 70 Hz. Data were recorded on disc at an A-D conversion rate of 250 Hz together with the coded trigger for the trials. Averaging was done offline with a sweep begiming 100 ms before stimulus onset and lasting until3000 ms afier stimulus onset. A11 data were collapsed using Brain Eiectrical Source Analysis (BESA) programs from 775 points to 180 points per sweep and recalculated to an average reference. Waveforms were measured using this average reference and plotted with an upward deflection representing positivity of the electrode relative to the reference.
EOG compensation was applied using ocular source components (Berg & Scherg.
1991 ; Lins et al. 1993a. 1993b). Using the same montage used in the experiment. a separate ocular calibration recording was made. During this calibration. participants were asked to make saccades to a stimulus that jurnped from the centre to the edge of the screen. separately in the up, dom. right and left directions. This was followed by a brief period of intermittent blinking. An ocular data set was put together by concatenating average recordings of each of the saccades and the blinks. A principal component ERPs and Picture Memory 34 analysis of this data set provided a set of componrnts that represented the variance related to the eye movements. Those two or three components which each explained more than
1% of the variance and which specifically related to the EOG waveforms were used as source components to subtract EOG contamination from the average Eus.
ERP Measurements ERPs were averaged according to the following experirnental conditions: learninp on Day 1 and new correct. old correct. new incorrect and old incorrect for both Day 1 and Day 2. Since response accuracy was high for al1 participants. there were too few incorrect responses to obtain ERPs with an adequate signal-to-noise ratio for measurement.
Five peaks were evident For al1 participants. These were labelled in tems of their polarity and peak latency after onset of the stimulus. One Iatency was selected (on the bais of the two or three electrode sites where the wave was maximally recorded) and al1 amplitudes were measured at this latency. A positive deflection. occurring maxirnally at
0 1 and 02 between 60 and 130 ms. was termed P 100. A negative deflection. occuring mavimally at PO3 and PO4 between 130 and 200 ms. was termed NljO. This negative peak rode on the upward climb of a large posterior positive wave and. therefore. the resulting amplitude measures for the NI50 were postive. As a result. peak-to-peak measures (relative to the average of the preceding and succeeding positive waves) were taken for this particular component. Two positive deflections occurred maximally at PO3 and P04. The earlier P240. occurring between 200 and 300 ms. was followed by a positive deflection occurring between 300 and 500 ms. termed P350. A later positive deflection, occumng maximally at CPl and CP2 between 450 and 900 ms, was termed the P650. For each of these components the latency was determined at the electrode sites ERPs and Picture Memory 25
where the peak was most prominent and individual subject averages and amplitudes for ai1 sites were measured at that latency. Mean amplitude during the period of 500 - 1000 ms was also measured for the P650 at CP I and CP2. Difference waveforms ('new' - 'old') were also calculated. These waveforms showed a negative deflection between 350 and 550 ms that was maximal at F3 and F4. The mean amplitude across this latency range was rneasured and termed N450. FinalIy. sustained potentials were recorded as a positive deflection in the occipital regions and negative anteriorly. Because this slow
wave activity was superimposed on earlier. more phasic components. difference waves ('Iearning' - 'new'; 'Iearning' - 'old') were calculated. These diflerence slow waves were measured as the mean amplitude during three penods of the waveform: S W 1 (500 - 1000 ms): S W2 (1000 - 1500 ms); and S W3 (1 500 - 2000 ms). A11 three slow waves were measured postenorly at 0 1 and 02 and anteriorly at FP 1 and FP3. Al1 amplitudes were measwed relative to the average voltage of a 100 ms prestimulus interval.
Statistical Analvses In order to analyse the behavioural data. the proportion of correct responses (old and new) were computed for al1 participants on both days of the experiment. A repeated
measures Day (1 vs. 2) by Condition (old vs. new) analysis of variance (ANOVAs) was performed with these proportions. These values were then used to obtain the signal detection indices of sensitivity, d' and P. Sensitivity effects across the two days of testing were assessed using paired-sarnple t-tests.
Because it was impossible to analyse data from al1 electrodes in one ANOVA, the analyses were limited for each wave to rneasurements at the maximal electrode site and the homologous site on the contralateral scalp. ERP effects for peak amplitude and slow wave measures were first compared across both days of testing using Electrode (right vs. ERPs and Picture Memory 26
lefi at maximal site) x Day (1 vs. 3) ANOVAs . The latency of the selected components
were also compared across days using paired-sample t-tests (e.g., P 100 latency for old
items on Day 1 vs. DayZ). Since none of the cornparisons across days revealed significant differences. measurernents for each individual subject were averaged across the two days.
In order to simpliQ the description of the measurements for the remainder of the
analyses. the foiiowing notations will be used: LEARNIWG wiil represent the average
waveform obtained across the five snidy block for al1 participants: NEW will represent the averaged waveform for al1 correct responses to new items (Le.. responses of 3 or 4) during test blocks; and OLD will represent the averaged waveform for ail correct
responses to old items (i-e.. responses of I or 2) during test blocks. Both OLD and NEW represent averages across both days of testing.
ERP effects for peak amplitude and slow wave measures (CP1. CP2. 500 - 100 ms only) for the collapsed data were assessed using Condition (LEARNING. OLD. NEW) x
Electrode Location (nght vs. lefi at maximal site) ANOVAs. Latency \.as assessed with one-way (conditions) ANOVAs. Within-subject factors were used for al1 ANOVAs. Al1 F-ratios were tested using degrees of freedom analysed using the Greenhouse-Geisser corrections. Individual comparisons between means were made with the Tukey's honestly significant difference. Al1 difference waves were analysed using two-tailed. one- sarnple t-tests to determine whether the mean amplitude value was significantly different from zero. Cerebral asymmetry for the N450 cornponent was tested with paired-sample t- tests. Condition (LEARNING - NEW vs. LEARN~NG- OLD) x Electrode (lefi vs right at maximal site) ANOVAs were used to analyse the difference slow waves at each intervals. Results were considered significant at E < -05. ERPs and Picrure Memory 27
Resu 1ts
Behavioural Data Table 1 presents the averaged behavioural data. The proportion of correct
responses was significantly reduced on Day 2 compared to Day 1 (F(I,9) = 72.03. e <
-001) but the difference between the proportion of correct responses to new items was not
significantly greater than that to old items (F(1.9) = 5.06. p = -05 1 and the interaction was not significant. The d' scores also revealed that performance dropped significantly on the
Day 2. t(9) = 4.33. p = 0.01. However. analysis of the response bias measure. B. did not reveal a significant difference across the two days of testing.
ERP Waveforrn Analvses
Grand mran ERPs for new correct and old correct on Day 1 and Day 2 are presented in Figure 3. Despite some visible differences across the two days (for example. the P65O. slow waves). statistical analyses of amplitude and latency information on ali components measured yielded no significant differences across days. As a result. ERP data collected at retrieval were pooled across the two days for al1 further analyses.
Pl O0 This small positive wave was recorded maximally over occipital electrodes (Figure 4). There were no significant differences in arnplitude or latency arnong the three conditions nor any significant asyrnmetry benveen the O land 02 electrode.
N 150 A negative peak occurred at approximately 150 ms afier stimulus onset (Figure 4).
Since this N 150 rode on the upward climb of a large posterior positive wave. a peak-to-
peak amplitude measure was calculated for the Nl 50. Analyses of amplitude ERPs and Picture Memory 28
Behavioural data for the two davs of the exneriment.
Mean Values (SDs in Parentheses)
Percent Percent Response Expenmentai Correct Correct Sensitivity Bias Conditions (ow (new) (d') (P)
Day 1
Day 2 ERPs and Picture Memory 29
Figure Caption
Figure 3. Cornparison of grand mean ERPs for new correct and old correct on Day 1 and
Day 2. Recordings are shown for frontal (F3. F4), centro-parietal (CPI, CP3). parieto- occipital (PO3. P04) and occipital (01,02) with the positivity at the electrode to the average reference plotted upward. The responses consist of a sequence of waves (P100-
N150-P240/P350) recorded posteriorly foliowed by a centro-parietal P650 and a slow wave (SW) recorded as a positive deflection posteriorly and negative deflection antenorly.
ERPs and Picture Memory 30
Figure Caption
Figure 4. P 100-N 1jO-P24O/P35O complex. On the lefi, grand mean ERP waveforms for the three experîmental conditions at posterior electrode sites are illustrated. Al1 of the waves in this complex are maximally recorded posteriorly. Thrre is no change across conditions other than what is caused by the supenmposed positive slow wave in the
LEARNING condition. On the right. topographical scalp maps show the distribution for each of the measured components of this complex during each of the experimental conditions.
ERPs and Picnire Memory 3 1 measures showed no effect of condition. a main efTect of electrode (F(1,9) = 6.57. c.05) and no significant interaction. The peak was more prominent at PO4 (-7.32 pV) than at PO3 (-4.97 pV). There was a significant effect of condition on the latency of the
N 150 component. Both NEW ( 1 5 1 ms) and OLD ( 150 ms) N 1 50 cornponents were significantly later than at LEARNING ( 143 ms).
P240 This large positive component, maximal at occipito-parietal sites. did not show any significant asymmetry at the PO3 and PO4 sites or any significant effect of condition (Figure 4). Howzver, an effect of condition on the latency of this cornponent was found (F(2.18) = 15-42. Q < -05). The P240 was later for LEARNING (238 rns) than for OLD (233 ms) or NEW (234 ms).
P350 A parieto-occipital positive wave followed shonly after the P240 (Figure 4). It also showed no significant asymmetry at the PO3 and PO4 sites or a significant effect of condition. Like the P240. the P3O was significantly later for LEARNING (364 ms) as compared to oLD items (349 ms) . In contrast. however. the P350 to NEW items
(361 ms) occurred at a significantly longer latency in cornparison to OLD items.
N450 Difference waves were cornposed by subtracting the waveform for OLD from NEW. This subtraction unveiled a negative peak in fiontal regions at approxirnately 450 ms after stimulus onset (Figure 5). Mean amplitude measurements taken at F3 (- 0.47 pV). F4 (-
0.92 pV) and Fz (-1 .O7 pV) were significantly different from zero (Fz: t(9) = -4.88.2 <
.O 1; F3: t(9) = -2.49, e < -05; F4: t(9) = - 4.27, e < .O 1). Although the negativity was greater at F4 compared to F3, this asymmetry did not reach significant levels. t(9) =
2.1 1, g = .06. ERPs and Picture Memory 32
Figure Caption Figure 5. New versus old. The top portion of this figure shows the grand mean ERP waveforms for each of the three experimentd conditions and the NEW - OLD difference waveforms at a selection of anterior electrode sites. The difference waves are located immediately below the particular ERP waveforms in their original form. These difference waveforms show a smail negative deflection peaking at 450 ms. The topographical scalp map at the bottom of the figure represents the distribution of the mean activity between 350 and 550 ms on the NEW - OLD difference wave.
ERPs and Picture Memory 33
P650 4 clear positive wave in the Iatency range of 500 to 900 ms was most prominent during retneval (Figure 6). Significant main effects for condition were observed for P650 amplitude (F(18,2) = 9.83. < .O 1). While amplitude measurements to oLD (4.33 pV) and NEW (3.63 pv) stimuli were not significantly different. both were significantly larger than the amplitude of the P650 at LEARNING (2.08 pv). No hemisphenc difference was obsemed for this component at the CP 1 and CP2 sites. The mean amplitude measurernents taken fiom 500 to 1 O00 ms at these two sites confirmed the significant effect of condition (F(2.18)=2.49. Q < -05) and the absence of any hemispheric difference. Due to the lack of cleariy recognizable peaks for the P650 component at LEARNMG. the analysis of latency differences was limited to OLD versus
NEW. A t-test revealed that the P650 wave occurred significantly eadier for oLD items
(67 1 ms) compared to NEW items (718 ms). t(9)= 8.43. g < -001 ).
Slow Wave Prominent slow wave activity began shortly after stimulus onset
(approximately 200 ms) and continued throughout the recording epoch (Figure 6).
Difference waves (LEARNING - OLD. LEARNING - NEW) revealed that iearning elicited posterior positive slow wave activity (associated with anterior negativity) which was most prominent at 0 1 and 02. T-tests showed that al1 mean amplitude measures of the difference waves at 0 1 and 02 were significantly different fiom zero, except those for
LEARNING - NEW at 0 1. mis was tme for the three intervals measured (i.e.. SWI: 500 - 1O00 ms, SW2: 1 O00 - 1500 ms, SW3: 1500 - 2000 ms). A summary of the results of al1 t-test is found in Table 2. The mean amplitudes of the two difference-waves were not significantly different. However. a highly significant asymmetry effect was found. In each interval. mean amplitude measures at 02were consistently larger for both types of difference waves (SW 1 : F(1,9) = 19.89, < -01 ; S W2: F( 1.9) = 12.9 1. p < -01 ; SW3 :
F(1.9) = 12.39, g < .O 1). ERPs and Picture Memory 34
Figure Caption
Fieure 6. New versus old. The top portion of this figure shows the grand mean ERP waveforms for each of the experimental conditions at central and parietal sites. The centro-parietal P650 wave is significantly iarger in the recognition condition (NEW - OLD) and peaks at a slightly longer latency for NEW compared to oLD (best seen at CPl and CP2 electrodes). The Iower part of the figure illustrates the topographical scalp maps for the the peak amplitude of the P650 for each of the conditions.
ERPs and Picture Memory 35
Figure Caption Fieure 7. Leaming versus recognition: the sustained potential. The grand mean ERP wavefoms at FP 1. FP2.01 and 02 for each experimental condition are presented on the far iefi of this figure. In the centre. the subtractions LEARNING-NEW (denoted by L-N) and
LEARN~NG-OLD(denoted by L-O) are shown for these electrode sites. The topographical scalp maps were prepared for three intervals of the recording epoch: S W 1 (500 - 1O00 rns), S W2 (1000 - 1500 ms) and S W3 ( 1500 - 2000 ms). The maps show the scalp distribution of the mean amplitude measures for each of the difference-waves (L-N. L-O).
ERPs and Picture Memory 36
Table 2.
Summary of t-tests for ost te ri or slow waves.
Conditions t-test value d f significance
SW1
LEARNING-NEW at 0 1
LEARNING-OLD at O I
LEARNING-NEW at O2
LEARNING-OLD at O2
SW2
LEARNING-NEW at O I
LE4RNING-OLD at O 1
LEARNING-NEW at O2
LEARNING-OLD at O2
SW3
LEARNING-NEW at 01 1.59
LEARNING-OLD at 0 1 2 -46
LEARNING-NEW at OS 3 -45
LEARNING-OLD at OS 3.41 ERPs and Picture Memory 37
At anterior electrode positions. significant results in the one-sample t-tests were found at the earlier intervals. During the interval of 500 to 100 ms, mean amplitude measures at
FP2 were significantly different from zero for both LEARNNG - NEW (t(9) = -2.84. < -05) and LEARNING - OLD (t(9) = -2.38. e < -05). At FPi. however. significant results were found for LEARNMG - NEW (t(9) = -7.84, p < -05) but LEARNING - OLD was not quite significant (t(9) = -2.27. p = -05). Significant differences extending in to the interval of
100 to 1500 ms for leaming - new at the FP2 site only (t(9) = -2.38. < .05).Mean amplitudes for ail other difference waves in this interval and the next (1500 to 2000 ms) were not significant. There were no significant cerebral asymmetries at FP 1 and FP2. nor were there any significant differences between LEARNING - NEW and LEARNING - OLD. at any of the three intervals. ERPs and Picture Memory 38
Discussion
Behavioural Findings Memory for pictorial information is typically quite accurate. durable and robust. demonstrating better performance than for words and showing less forgetting over time (e-g.. Shepard. 1967). In the present expenment. the recognition performance of 88% correct for the old pictures on Day 1 supports Siis picture superiority effect and concurs with the findings of other studies of similar design (e.g., Nickerson. 1965; Potter & Lew, 1969; Intraub. 1980). The delay between first and second presentation of the picnires on Day 1 in is the range of 3 to 10 minutes. Translating the item lag of Nickerson (1965) to approximate time in minutes. performance was very similar to the present results. With a lag of 120 items presented at 5 s per item (a time lag of approximately 10 minutes). Nickerson (1965) found a recognition accuracy of 87%. While pictures in the Nickerson (1 965) experiment were presented for 5 s each, Potter and Levy ( 1969) showed similar performance (i-e.. 93%) when pictures were presented for only 2 s (as in the present study). Thus. it would not be rxpected that increasing the presentation time at study in the present experiment would improve performance above the observed level, at least on Day 1.
However. the 65% accuracy on Day 2 in the current study was less than expected on the bais of past studies (e.g., Nickerson, 1968: Shepard. 1967; Standing et al.. 1970). There are two reasons for this greater decline in accuracy. First. we used a shorter presentation time than the earlier studies that showed less forgetting. The comparatively better performance of 76% found by Nickerson (1968) dera one day delay may be explained, in part. by his longer presentation tirne of 5 S. While the longer study time may not have any benefit on immediate recognition tests, it may improve accuracy at ERPs and Picture Mernory 39
longer delays. At the longer delays, participants may rely more on recollective processes which benefit from greater study time (eg. the greater elaborative processing allowed).
The second reason for the lower accuracy in the curent experiment may lie in the design of the recognition task. For exarnple. Shepard (1967) and Standing et al. (1 970) reported accuracies of 92% and 90%. respectively. after a delay of 3 days. However. both of these snidies employed a 2-AFC task in which participants are presented an old and new picture side-by-side and are asked to judge which is the old picture. The 2-AFC task allows participants to make decisions based more on comparisons than recollective processes.
ERP Findines
Potentials consistentIv evoked bv pictures: P 100-N 150-P240/P350 Complex A prominent Pl 00-NI 50-P240/P350 cornplex was observed over parieto-occipital electrode sites. The Pl00 activity was neither lateralized nor affected by any of the conditions. The N 150 wave rode on the upward climb of a large posterior positive wave. This negative peak was not affected by condition but was more prominent over the right hemisphere. A large P240 wave was followed closely by another slightly smaller amplitude positive wave. the P350. It seems likely that these waves reflect one single process, and this assumption is made in the following discussion. Both waves were larger in amplitude during leaming than during recognition. This may be due to the superimposition of these positive waves during learning by a large posterior positivity. Furthemore, this sustained positivity that begins at 200 ms probably affected the latencies of the N 150 and P240P350 waves by making the negative wave earlier and the positive wave later during learning. ERPs and Picture Memory 40
These potentials show both similarities and differences with the early potentials evoked by words. Based on PET studies suggesting a specific anatomy for feature identification, visual word foms and semantic associations (e.g.. Petersen. Fox, Posner.
Mintun & Raichle, 1989; Petersen, Fox. Snyder & Raichle. IWO). Compton.
Grossenbacher. Posner and Tucker ( 199 1) investigated the time course of word processing using ERPs and found that feature-related analysis began within the first 100
ms of word processing. ERP waveforms revealed a postenoi temporal asymrnetry at the Pl00 component that was larger over the right hemisphere than the lefi. Compton et al.
( 199 1) argued that when attending to features, participants might search a representation located in the right postenor temporal lobe. An N2OO with a left-lateralized. occipital temporal distribution discnminated words fiom non-word strings. Compton et al (199 1 ) suggested that the generator of the N200 may be the visual word fom area in the lefi ventral occipita1 lobe, as revealed by PET studies.
Compton et al. ( 199 1 ) conclude that the P !00 component indexes kature andysis in the visuai processing of words. The P 100 activity in response to pictures rnay indicate the activity associated with discriminating features of the photograph. Friedman ( 1990) reported an occipital P 100 component to line-drawn pictures. Smith and Halgren ( 1987) also report a P 105 component of approximately the sarne amplitude as the present P 100 as part of a P 1Oj/N l4OR 190 sequence of potentials to faces at occipital and posterior temporal sites. Smith and Halgren found no laterality or repetition effècts in these components. The Pl00 may indicate die initial activation of the visual cortex fiom the thalamus that is associated with feature analysis in the visual scene. ERPs and Picture Memory 4 1
The N 150 component recorded maximdly over nght-lateralized parietal-occipital sites may be the nonverbal analog to the left-lateralized N200 component reported by
Compton et al ( 199 1) and may index a nonverbal rather than verbal type of form discrimination. This activity may be associated with discrimination of coherent and separable objects and details within the visual scene. The right lateralization of the NI50 is compatible with neuropsychological evidence that the posterior areas of the right hemisphere is specialized for the integrating complex visuospatial information in general
(e.g.. Bogen & Gazzaniga, 1965).
The N 150 recorded in the present experiment rnay also be related to potentials
evoked by face stimuli. Allison et al. ( 1994) recorded activity directly from the surface of the occipitotemporal cortex of patients with intractable epilepsy. They found a large negative peak with a latency of approximately 200 ms (N200) to intact pictures of faces but not scrambled faces. cars. scrambled cars. or butterflies. This activity was generated in discrete regions of the occipital extrastriate cortex and may reflect an early stage of processing of faces or other visual stimuli The N 150 in the present expenment may also have its source in these areas.
More recently, Bentin, Allison. Puce. Perez and McCarthy (in press) conducted experiments similar to those of Allison et al. ( 1994) to determine whether face-specific ERPs could also be recorded from scalp electrodes. Bentin et al. (in press) recorded a large negative ERP with a peak latency of 172 ms (N 170) from T5 and T6 (equivalent to P7 and P8 in the present recording montage) in response to faces. Similar to Allison et al.
(1 994). this N170 was not elicited by other the other complex non-face stimuli employed in the experiment (i.e., cars. butterîlies). Bentin et al. (in press) also found that the N 170 was elicited by inverted faces and face components (Le., lips. noses and eyes, with eyes ERPs and Picture Memory 42 showing the most prominent N 170). These findings suggest that this activity is involved in the early stmctd anaiysis of features of visual stimuli leading to the categorization of a pictorial stimulus as a face. Furthemore, since distorted faces also elicited the N 170, this wave does not depend on a holistic face-processing mechanism. Thus. the N 170 does not seem to reflect face recognition but rather the detection of facial features, such as eyes. that distinguish faces from other pictorial stimuli.
The NI 50 recorded over parietal and occipital sites in the present experiment may be related to the N 170 recorded by Bentin et al (in press) in response to face stimuli.
However. the NI 50 to the picnires in the present experiment is smaller than what would be expected with face stimuli. While studies such as the one by Bentin et al. report an absence of the NI70 to non-face stimuli, the activity to the non-face stimuli at this latency and scalp distribution appears to be similar to the N 150 activity recorded in the present experiment. The activity of the N 150 may. therefore. represent a subcomponent of the activity evoked by faces. The early potentials recorded to faces may be composed of activity relating to the processing of features of complex visual stimuli in general plus the specialized processing of the socially important fèatures of face stimuli.
The P240P350 waves in the present experiment may be similar to the recognition potential (RP)related by Rudell (Rudell. 199 1 : Ruciell. 1992; RudelI. Cracco. Hassan & Eberle, 1993; Rude11 & Hua. 1996). They descnbe this as an electrical response of the brain that occurs when a subject views recognizable images such as words, pictures, or faces. A series of studies presented by Rudell (1 991) have indicated that the RP differs from other longer latency EWs such as the P3. It has a shorter latency with a maximum response occurring at about 200-250 rns whereas the latency of the P300 is typically longer than 400 ms for visual stimuli. The RP is recorded mavimally over the occipital ERPs and Picture Memory 43 cortex, unlike the typical vertex or midline parietal maximum of the P300. The RP also does not appear to be as sensitive to the probability of occurrence of stimuli as is the P300. Furthemore, Rudell et al. (1993) showed that the RP is more sensitive to where a target is presented in the visual field.
Perhaps most vital to the explanation of the RP is that. unlike the P3, a recognizable stimulus is required to produce this potential. For example. common words that were of different lengths, different print styles or surrounded by non-meaningful control stimuli reliably evoke the RP while non-meaningfùl control stimuli that matched the word stimuli in brightness, area and texture did not (Rudell. 199 1 ). Further studies. based on the notion that recognizability can be defined by a subject's individual experience, tested this hypothesis using English words. simple line-drawn pictures. Chinese ideographs and non-recognizable control stimuli (Rudell. 1992). Chinese ideographs evoked the RP for the Chinese participants but not with those unfarniliar with Chinese. These RPs were similar to those evoked by the English words for English participants. The RP to the pictures was similar in both participants groups although somewhat smaller and longer in latency. In both cases. the foreign language did not evoke the W. Thus, these results led Rudell (1992) to conclude that recognizability. and not differences in physical attributes, gives rise to the RP and that the recognition potential could be evoked with picture stimuli.
Ovenll. the very large P240P350 complex observed in the present experiment seems to be a likely candidate for Rudell's RP. The P240/P350 is maximal over occipital electrode sites and in the latency range described by Rude11 (199 1). The picture stimuli in the present experiment are, for the most part, highly meaningful. Unlike the earlier Pl00 and NI50 components that are suggested to relate to visual features and picture details. ERPs and Picture Memory 44
respectively, the P240/P350 cornplex may represent the holistic visuai processing of the photographs. In doing so. the participants may corne to recognize the photograph as a whole through the incorporation of the various features and forms/objects. Although the RP to line drawings has been shown to considerably smailer than that to words. the greater complexity of the present stimuli rnay also contribute to the large amplitude
O bserved here.
Learnine vs. Recognition: The sustained potential Given the relatively long presentation tirne of the stimuli (Le.. 2 s) in the current expenment and the complexity of the visual stimuli. two processes could occur while participants studied the picnires. First. participants could scan the picture over the entirety of the 2 s duration since the photographs contain a large arnount of information and are nch in detail. Second. encoding could be prolonged. Each piece of informaiton of detail experienced as the photograph is scanned across the 2 s duration should allow another opportunity for more elaborative encoding. Two main observations made when comparing the ERP waveforms elicited during the learning and recognition phases of the present expenment rnay be relevant to these scanning and encoding processes.
One of these observations is that the positive wave prominent at centroparietal sites in the range of 500 to 1000 ms after stimulus onset (P650)during recognition was rnuch smaller in amplitude at learning. This P650 at learning rnay be related to an effect commonly referred to as the 'subsequent memory effect' or the 'difference-due-to-memory effect' (Dm effect). Since ERPs can be recorded at both the study and retrieval phases of an experiment. many studies have addressed questions about encoding by comparing the ERP waveforms of items at first presentation based on whether they were successfully retrieved on second presentation or not. The typical finding has been that those items that ERPs and Picture Memory 45 were subsequently retrieved are associated with greater positivity (sometimes referred to as the P300) at learning compared to those that were not subsequently retrieved (e-g..
Karis et al.. 1984; Neville, Kutas, Chesney & Smith, 1 986: Paller & Kutas. 1992;
Sanquist et al., 1980). This effect is ofien explained in relation to remembered items having been more elaborately encoded dthough little is known about the nature of these processes (Rugg, 1995). The relatively high level of accuracy in the present experiment rendered the examination of the subsequent memory effect difficult since there were too few subsequently unrecognized items for averaging. Even though this analysis was not possible. given the suggested association between the greater positivity and subsequent retrievd, a large P300-like wave at learning should have been found given the high accuracy in the present experiment .
There are two reasons why a large P300-like wave was absent during the leming condition of the present experiment. First. a dichotomous task was not cmployed during leming. Perhaps the most basic characteristic of the P300 is that it is eiicited in tasks in which a subject must make some sort of discrimination between two or more stimulus categories. In an auditory oddball paradigm. for eszmple. the subject must discriminate between the standard and target tones. Thus. the somewhat ill-defined P300 observed in the Iraming phase may have resulted because the participants were not actively engaged in the task of discriminating one stimuli from another.
However, the more elaborative encoding that produces better subsequent recognition and a Dm effect may not specifically involve a P300 wave. Most experiments that explored the subsequent memory effect used a series of words (e-g.. Sanquist et al., 1980: Karis et al.. 1984) presented for a much shorter duration than the pictures in the present experiment. However, given the longer presentation time in the ERPs and Picture Memory 46
present paradigm, the more elaborative encoding that led to subsequent recognition could have been represented by a more sustained potential. Thus. the second. and perhaps more intriguing, finding is the very prominent sustained positive activity that was maximal at posterior electrode sites during leaming. This begins about 200 ms after stimulus onset and lasts throughout the recording epoch.
While it is quite probable that the prominent slow wave activity at occipital sites reflects the greater processing necessary for mernonzing picture. what actual processes
underly the activity are not clear. A recent MRi study by Stein et al. ( 1996) used a learning procedure very simila. to the procedure employed in the present experiment. In an experimental condition. participants viewed 40 complex pictures presented at a rate of 1 every 3 seconds and were told to view pictures carefuliy in order to recognize them
later. In a control condition. participants passively viewed a single picture which was presented repeatedly at the same rate as the novel pictures. The results showed that the expenmental condition was associated with increased fMN signal in the posterior quarter of the hippocampal formation and adjacent parahippocarnpal gyrus. with greater signal changes occurring in the nght hemisphere than the le fi. Additional signal intensity changes were found bilaterally in the fùsiform and lingual gyri. bilaterally. areas that are important for visual object recognition. No significanr changes were noted in the frontal lobes during picture encoding. Stein et al. ( 1996) conclude that the encoding of novel. complex pictures is caused by interactions between the posterior hippocampal regions. specialized for long term memory. and posterior ventral cortical regions. speciaiized for object recognition.
It is possible that the activity reported by Stein et al. (1996) is associated with the posterior sustained potential observed in the present experiment. This potential may ERPs and Picture Memory 47
reflect activity in both the posterior hippocampal formation and the posterior ventral corticai regions. However, Stein et al. (1 996) report only those signais that change between the experimental and control conditions. This cornparison eliminates any general activity associated with viewing complex pictures. If this is the case. the posterior sustained potential in the present experiment could include activity related to visual processing in addition to activity specifically related to encoding.
Therefore. while the results of Stein et al. (1996) might reflect the attentional and encoding resources necessary for memorizing novel pictures. the posterior sustained potential in the present experiment might reflect both the processing of complex pictures
and the processes of memorizing novel pictures. In order to delineate how much of the sustained potential reflects the latter process. a subtraction of a control condition like that
of Stein et al. ( 1996) from the presently recorded activity would need be perfomed. Finally, the encoding that takes place in this task may involve more than the simple encoding of visual detail. This type of encoding may contribute to what is often referred to as 'snapshot memory'. a type of mernory for the identity and spatial arrangement of
objects within a visual scene (Gaffan & Himison. 1989).
However. Van Petten & Senkfor (1996) suggest that retrieval also plays a role in what is 'nominally the "encoding" phase of memory paradigms' (p.493). Participants come into an experiment with pre-existing knowledge and memories. and are likely to recall some of this information during the ieaming phase. While this pre-existing knowledge is quite obvious for words and simple pictures (e.g.. the meaning of a word. the narne of a pictured object). participants may also engage in elaborative encoding that relies on retrieval of pre-existing memones. even with completely novel pictures. For example, although the subject may have never before seen the landscape presented in the ERPs and Picture Memory 48
learning phase, it rnay remind him or her about a vacation to Europe the previous summer. Thus, the activity observed in the ERP waveforms during learning (e-g.. P650. SWs) rnay partially reflect some process of retrieval that was relevant to encoding.
The sustained positive activity recorded at posterior sites during learning rnay aiso be associated with a sustained negativity recorded at anterior sites during leanùng. The difference in slow wave activity between learning and recognition conditions at the frontal electrodes rnay be partly explained as the inverted aspect of the posterior positivity. However. this explmation does not completely explain the differences at frontai sites since the inverted version of the posteriorly generated activity would be smaller and more widespread. This frontal negativity rnay therefore be partially attributed to the efforthl attention required during learning. More specifically. the areas of the prefiontai cortex rnay play an important role in initiating encoding and supervising the posterior perceptual processes.
New Versus Old: The N450 and P650 Waves
While scanning and/or encoding processes at leming should take advantage of the duration of the 2 s presentation time of the pictures and. therefore. be associated with sustained potentials, the oldhew decisions that are rnake during the recognition phase cm be made at a significantly shorter latency than 2 seconds. Differences between the ERP waveforms elicited for new and old items in the curent expenment are reflected in two potentials, the N450 and P650.
N450 The waveforms elicited by old and new pictures began to diverge about 350 ms afier stimulus onset. This difference peaked at about 450 ms and was mavimal at anterior electrode sites and slightly but not significantly right lateraiized. Two accounts ERPs and Picture Memory 49 of this difference can be put forth. One account is that this difference is a greater positivity for the old pictures during the interval of 350 to 550 ms. as part of the earlier onsei of the P650 to old pictures, than to new pictures. A second account of this difference is that it is a greater negativity for new pictures during the interval of350 to 550 ms. together with the earlier onset of the P650 to old pictures.
There has been much uncertainty regarding the processes that underly the N400 in memory paradigms. The fiequent proposition is that it may be related to the N400 component originally descnbed by Kutas and Hillyard (1980). This parietal N400 occurs when words are presented that are incongruous within the context of the sentence. From this observation, the N400 has been interpreted as reflecting the 'associative activation' evoked by an unfamiliar conjunction of a stimulus and context and. thus. is generated as the semantic attributes of the evoking stimulus are integrated with the 'cognitive context' pertaining at the tirne. Smith and Halgren ( 1989) suggested that the modulation of the N400 at retrieval is related to retrieval of associational information. That is. the repetition of an item within the same context triggers the retrieval of the episodic trace of of its first presentation (Smith and Halgren. 1989) and contextual integration is facilitated (Rugg & Doyle. 1994). thus leading to a srnaller N400 and a more postive-going ERP. Smith and Halgren (1 989) concluded that the N400 should be smaller whenever recognition judgments are based on recollection of prior encounters with the test item.
However. a very robust effect regarding this N400 component in memory paradigms is that it is rather short-lived. Rugg and Nagy (1 989) found that while the modulation of the N400 was unaffected by intervals of 6 and 19 items in a continuous recognition experiment with words. the N400 difference was absent after a 35-40 minute delay. Several other studies have also found evidence that the N400 repetition effects are ERPs and Picture Memory 50 absent over a time period greater than a few minutes (e-g.,Neville et al, 1986). Thus.
Rugg (1995) argued that the N400 effects are governed more by study-test interval than by the recollective processes responsible for discnrninating old from new items. Nonetheless, "the identity and the functional significance of the processes reflected by the modulation of N4OO in memory tasks remains unclear" (Rugg, 1995. p. 155).
While many of the proposed interpretations of the function of the N400-type component share cornmon concepts. the N400 may not represent the sarne function in al1 cases. Stuss et al. (1 983) investigated the N400 in a series of expenments involving nming of pictures of objects. They found that the amplitude of the negative peak at approximately 400 ms in latency varied with the arnount of processing required such that the more complex the stimulus. the greater the N400 amplitude. Furthemore. this NJOO showed a scalp distribution similar to the present N450 in that it was more prominent over the right hemisphere and had a fionto-central maximum. This scalp distribution is different frorn the posterior distribution that has typically been associated with the repetition effect in verbal tasks. Since it was also evoked by stimuli that had to be mentally rotated, the N400 could not be specifically semantic. Stuss et a1 ( 1983). therefore. proposed instead that the N400 may represent the initiation of processes for complex stimuli when these processes not immediately available. In the present experirnent. despite the equivalent complexity of the new and old stimuli in the recognition task. the N450 component differed between the old and new pictures. Thus. the modulation of the N450 between old and new pictures must be based on memory processes.
In our paradigrn. the N450 may be descnbed as an increased activity with novel. complex items. Perhaps a viable explanation to this view of the modulation of the N450 ERPs and Picture Memory 5 1
is that the processes that underly the N450 are related to what Tulving and his colleagues
(Tulving et al., 1994a; Tulving & Kroll, 1995: Tulving et al.. 1996) descnbe as novelty detection. That is, when participants encounter a picture that is being presented for the first time, the N450 may reflect activity that functions to detect the prescnce of the novel pictures in a strearn of pictures that also contains pictues they have previously experienced.
If the N450 observed in the difference wave is. in fact. associated with novelty detection. what function does it serve? One exphnation is that is may be associated with processes incidental to memory function. For exarnple, this activity may associated with general arousal evoked by a picture that has not been experienced previously. The findings by Rugg (with Doyle. Thomas. Perrett and Harries. reported in Rugg & Doyle. 1994: Rugg et al.. in press) that meaningless stimuli do not modulate activity in an N4OO-type component. would indicate that general arousal by any novel stimulus is not sufficient. Instead. the novel information rnust be meaninghil and relevant to the memory process to evoke this activity. Furthermore. the N450 as well as novelty activations in various PET studies (Tulving et al.. 1994a; Tulving & Kroll. 1995: Tulving et al.. 1996) indicate that novelty detection is related to activity in more discretr brain areas than would be predicted by more general increases in brain activity that accompany arousal.
The question still remains. however. what role this activity plays with regard to memory processes? Tulving and his colleagues suggest a novelty/encoding hypothesis for the novelty activations in their PET experiments (Tulving et al., 1994a; Tulving & Kroll. 1995; Tulving et al., 1996). There are two main parts to this hypothesis. First, novelty assessrnent is an early stage of memory processing and is preferentially subserved by the limbic and temporal and parietal areas of the nght hernisphere. Second, der ERPs and Picture Memory 52 significant novel happenings have been identified, higher-level. meaning-based encoding operations preferentially involving the cortical areas of the lefi frontal lobe occur.
It may be that the N450 represents the actual encoding process for the novel stimuli. It would be expected. based on the sustained potentials recorded during learning. that if the N450 was associated with encoding that a similar long-lasting activity would be observed over the 2 s presentation during recognition. However. old and new items do not appear to have any significant long-Iasting differences that would imply a scanning/encoding process similar to that at learning. This makes sense given the contnved conditions of the expenmental memory paradigm that do not require the participants to necessarily encode the new items in the test blocks. This is not to say that participants do nothing else with this new information besides deciding that they are encountenng it for the first time in the experirnent. but this is not required within the context of the task. It may be that new pictures at the test phase are not being encoding. at least not in the same mmer or to as great an extent. as those during the study phase. Thus. it seems likely that the N450 does not represent the encoding of the new pictures.
The other possibility is that the N450 represents only the first stage of the novelty/encoding hypothesis. novelty detection. This earlier stage. and perhaps more automatic process. would be better represented by the short lasting N450 component.
Although this activity is maximal at the fiontal electrodes. it is quite possible that this represents activity that is generated in the right hippocampal 1 temporal area. The anatomy of the hippocampus is such that the negative end of the dipoles potentially generating activity in this structure are oriented in such a way that they may be recorded as negative activity at frontal sites. Hence, this activity is hypothesized to be more closely related to the detection of novel information within a strearn of non-novel items ERPs and Picture Memory 53 that Tulving suggests is subserved by the right limbic. temporal and parietal areas. Two relatively new techniques, Brain Electrical Source Analysis (BESA: Scherg, 1990) and
Low Resolution Electromagnetic Tomography (LORETA: Pascual-Marqui. Michel & Lehmann, 1994) rnay confirm this hypothesis. to provide converging evidence with the
PET findings (Tulving et al.. 1994a; Tulving & Kroll. 1995: Tulving et al.. 1996). The N450 may be an automatic process that precedes conscious awareness for detecting novel information. although it may also function to initiate further processing relevant to memory (Stuss et al., 1983).
P650 The typical finding in ERP memory studies with words is that a larger late positive component, similar in latency and scalp distribution to the P650 in the present experiment. is associated with successN retrieval of old items (e-g.. Sanquist et al.. 1980:
Neville et al. 1986: Rugg & Nagy. 1989). The amplitude of correctly identified new items and incorrectly identified old and new items elicit smaller late positive components.
The finding that the amplitude of the P650 in the present experiment was not significantly larger to old pictures compared to new pictures does not concur with these previous findings of the ERP repetition effect. While this may be true for verbal stimuli. several studies with nonverbal stimuli have found results regarding this component to be
not very robust (e.g.. Friedman, 1 990). rather variable and confùsing (Noldy et al. 1 990) or have show that the old items exhibited a smaller amplitude than new items (Rugg,
Saordi & Doyle, in press: VanPetten & Senkfor, 1996). These findings. along with the present finding, weaken the proposa1 that, for pictures and other nonverbal items at Ieast. this ERP repetition effect indexes successful retrieval. These findings, in contrat to with the large number of verbal studies that have reliably shown the same effect. rnay speak to the fact that verbal and nonverbal memory may be reflected differently in the ERP waveforms. Thus, alternative explanations that do not invoke memory retrieval as a ERPs and Picture Memory 54 critical factor must therefore be considered in the explanation of the results of the present experiment. A good candidate for this discussion, and that has often been associated with the repetition effect, is the P300.
The similar amplitudes of the P650 for old and new picnires during recognition may be to other parameters affecthg the P3OO wave. First. the P300 wave occurs only when the subject is actively engaged in a task that requires discrimination of two or more stimuli, that is, the task is dichotomous in nature. In most cited exarnples of the P300. an oddball task is employed in which an improbable target must be detected among fiequent nontargets. The common finding is that the P3OO is larger when the stimulus is more improbable and had been descnbed as being inversely proportional to this probability
(Duncan-Johnson & Donchin. 1977). Old and new items in the present experiment are equiprobable and as a result, could be expected to elicit P300-like waves of similar amplitude.
Second, in dichotomous stimulus classification tasks, the 'task relevance' or 'targetness' of the stimulus influences the amplitude of the P3OO (Johnson. 1988). Participants are required to respond to both old and new items. thus both can be viewed as equal in their degree of 'targetness'. In addition. for each stimulus, old or new, participants must make a judgment that conveys the confidence wiîh which they have made their decision. Therefore, the task relevance is equivalent for both classes of stimuli (e.g., as opposed to the case wherein confidence ratings are required for the old stimulus only). Hence, while differences were expected, the absence of this difference cm be explained given the factors that affect the P300. ERPs and Picture Memory 55
Given that the P650 rnay be related to the classic P300 wave. the next question is what fiction does this P300-like wave have in the present recognition memory paradigm? Studies relating reaction time measures to the P3OO ofien conclude that the
P300 rnay index the completion of stimulus evaluation (McCarthy & Donchin. 1981 ). As
Friedman ( 1990) dso found, the P650 occurred at a shorter latency with old items than with new items in the present experiment. Although reaction time measures were not taken in the present experiment. it would be expected that response selection would be quicker with the old items.
One possibility is that P650 rnay indicate the completion of a search through memory. As suggested earlier. the N450 cornponent could initiate a search through a long-terrn memory store (Stuss et al. 1986) and the P650 rnay be associated with the completion of the search through long-term rnemory. The search rnay be faster for the old pictures. that is, the search will corne to a halt when the stored representation of the picture is found. However. the search rnay take longer for new items since no representation can be found. The detection of a novel stimulus associated with the N450 activity may predispose the search mechanism such that a routine search is undertaken that serves as a 'double-check' to the earlier decision. However. this search is aborted at some tirne when it seem reasonable that the new items will not be found in the long-term store. taking longer. perhaps than the successful retrieval of an old item. This rnay account for the latency differences at the P650. i.e.. that the P650 to old pictures occurs approximately 75 ms earlier than to new pictures. However. an alternative explanation may be that the overlap of the Iarger N450 to new pictures over the P650 wave rnay account for the earlier latency of the P650 (Stuss et al. 1986). ERPs and Picture Memory 56
There has been the suggestion that the P3OO was associated with cognitive processes that could follow such a recognition. One idea was the occurrence of the P300 indicates some type of closure. possibly perceptual or cognitive. depending on the context (e-g..Verleger, 1988) that serves to 'erase' the information that has already been processed and/or reset the set of analyzers that will be used again on the next trial (Picton. 1992). Verleger's (1988) notion of this process as 'phasic physiological deactivation' is supported by evidence that decreased reaction time occurs concomitant with the occurrence of the P300. Thus. the Po50 rnight serve to clear the date once a decision can be made regarding the present test picture so as to prepare for the next picture. This idea is plausible in the experimental memory paradigm in particular where participants are fed a constant Stream of stimuli. In order to make independent decisions for each stimulus. it would be advantageous to use such an erasing process.
Still another possibility is that the P300 rnay reflect the transfer of information to controlled processing or consciousness (Picton. Donchin. Ford. Kahneman & Norman.
1984). Responses. such a novelty detection. rnay occur automatically prior to awareness and the P300. Once information frorn processes such as novelty detection and search through long-tem mernory reaches consciousness. the behavioural response required by the task cm be made. the decision or other hypotheses may be reevaluated (given the relatively long duration of the stimulus in the present experiment). In addition. mernories may be updated if necessary, that is. information conceming the new pictures may be strengthened while not enhancing the information conceming the old pictures. However. all of these later processes, being consciously controlled responses. would occur after the P300 wave (Picton, 1992). ERPs and Picture Memory 57
Summary Three main fmdings denved from the analysis of the EEW waveforms during the learning and recognition of complex pictures. First. a prominent P 100-N 150-P240P350 complex was associated with the specidized processing of complex pictures. The P 1O0 may represent the initiai activation of visuai cortex: the right-sided Nl 50 may indicate the processing of the picture details: and the P240P350 rnay be associated with the processing of the whole picture. Second. the leaming and recognition conditions showed distinct ERP waveforms. The leaming condition elicited more positive ERPs over parieto-occipital areas (500 ms to 2000 ms) compared to the recognition condition. The centro-parietal P650 waves were more positive in the recognition condition for both the new and old pictures compared to the Iearning condition. Finally, ERPs dissociated new and old pictures in the recognition condition. New pictures elicited more negative ERPs in the N400 range over frontal electrodes and this potential may be associated with novelty detection. Old pictures elicited an earlier centroparietal P650 which may indicate the cornpletion of a memory search. ERPs and Pichire Memory
References
Allison, T., Ginter. H., McCarthy, G., Nobre. A.C.. Puce, A.. Luby. M.. &
Spencer, D.D. (1994). Face recognition in human extrastriate cortex. Journal of
Neuro~hvsioIogv.71 (2). 82 1-825.
Anderson, J.R. ( 1978). Arguments concerning representations for mental irnagery. Psvcholo~icalReview. 85(4), 249-277.
Bentin, S., Allison. T, Puce. A., Perez E.. & McCarthy. G. (in press).
Electrophysiological studies of face perception in humans. Journal of Coenitive
Neuroscience.
Bentin. S., & Peled, B.S. (1990). The contribution of stimulus encoding strategies and decision-related factors to the repetition effect for words: electrophysiological evidence. Mernorv and Coenition. 18. 359-366.
Berg, P.. & Scherg, M. (1 99 1). Dipole models of eye movements and blinks.
Electroencephalography and Clinicai Neuro~hvsiologv. 79. 36-44.
Bogen. LE.. & Gazzaniga. M.S. (1965). Cerebrai comrnissurotomy in man.
Minor hemspheric dominance for certain visuospatial fùnctions. Journal of
Neurosurp;erv. 23. 393-399.
Chao, L.L., Nielson-Bohlrnan, L., & Knight. R.T. (1 995). Auditory event-related potentials dissociate early and late memory processes. . Electroence~halogra~hyand
Chical Neuro~hvsiolorrv.96, 157-1 68. ERPs and Picture Memory
Cimbalo, R.S. (1978). Making something stand out: the isolation effect in
memory performance. In M.M. Gninneberg, P.E. Moms, & R.N. Skyes (Eds.), Practical aspects of rnemorv (pp. 2 15-223). New York: Academic Press.
Compton, P.E.. Grossenbacher. P.. Posner, M.I., & Tucker. D.M. (1991). A cognitive-anatomical approach to attention in lexical access. Journal of Cognitive
Neuroscience, 3(4), 304-3 12.
Courchesne, E.. Hillyard, S.A., & Galarnbos. R. (1975). Stimulus novelty. task retevant and the visual evoked potentid in man. Electroencephalonraphv and Clinical
Neurophvsiologv. 39, 13 1- 143.
Donchin. E.. & Fabiani. M. ( 199 1 ). The use of event-related potentials in the study of memory: is P300 a measure of event distinctiveness? in J.R. Jennings & M. G.
H. Coles (Eds.). Handbook of coenitive ~svcho~hvsiolo~v:central and automatic svstem approaches, (pp. 47 1-498.) Chichester, NY: Wiley.
Fabiani. M., Karis. D.. & Donchin. E. ( 1990). Effects of mnemonic strategy manipulation in a Von Restorff paradigrn. Electroence~halo~ra~hvand Clinical
NeurophvsioIogv. 75-22-35.
Friedman, D. (1990). Cognitive event-related potential components during continuous recognition memory for pictures. Psvcho hvsiologv. 27(3), 136- 148.
Gaffa, D.. & Harris, S. (1989). Place memory and scene memory effects of fomix transection in the monkey. Experimental Brain Research. 74. 202-2 12. ERPs and Picture Memory
Intraub. H. ( 1980). Presentation rate and the representation of bnefly glimpsed
pictures in memory. Journal of Experimental Psvcholoev: Human Learning and
Mernorv, 6(1), 1-1 2.
Johnson. R. Jr. ( 1988). The amplitude of the P3OO of the event-related potential:
review and synthesis. In P.K. Ackles. J.R. Jennings, & M.G. H. Coles (Eds.). Advances
in ~s~chophysioloev(vol.3, pp.69- 138), Greenwich. CT: JAl Press.
Johnson. R. Jr.. Pfefferbaurn. A.. & Kopell. B.S. (1985). P3OO and long-tem
memory: latency predicts recognition performance. Psvchophvsiolo~v.28. 307-3 18.
Karis, D.. Fabiani. M., & Donchin. E. (1984). P300 and memory: individual differences in the von Restorff effect. Cognitive Psvcholonv. 16, 177-f 86.
Knight. R.T. (1984). Decreased response to novel stimuli after prefrontal lesions
in man. Electroencephalo~raphyand Clinical Neurophvsioloev. 59. 9-20.
Knight. R.T. (1 996). Contribution of human hippocampal region to nowlty detection. Nature. 383.356259.
Knight. R.T. (submitted). Distributed cortical network for visual stimulus detection. Journal of Cognitive Neuroscience.
Kutas. M., & Hillyard, S.A. (1 980). Reading senseless sentences: brain potentials reflect semantic incongruity. Science. 207. 202-7.
Lins. O.G., Picton. T.W.. Berg, P., & Scherg, M. (1993a). Ocular artifacts in
EEG and event-related potentials. 1 Scalp topography. Brain Topoeraphv. 6,5 1-63. ERPs and Picture Memory
Lins, O.G., Picton. T.W.. Berg, P., & Scherg, M. (1993b). Ocular artifacts in recording EEGs and event-related potentials. II Source dipoles and source components.
Brain Topographv, 6%65-78.
Loftus. G.R. (1972). Eye fixations and memory for pictures. Cognitive
PSVC~O~O~V.3.525-55 1.
Loftus, G.R.. & Bell, S.M. (1975). Two types of information in picture memory.
Journal of Experimental Psvchologv: Human Leming and Memory. 1. 103- 1 13.
Mandler. J.M.. & Johnson. N.S. (1976). Some of the thousand words a picture is worth. Journal of Experimental Psvchologv: Hurnan Learning and Memow. 2(5). 529-
340.
McCarthy. G. & Donchin. E. (198 1). A metric for thought: a cornparison of
P3OO latency and reaction time. Science. 2 1 1. 77-79.
McIntosh. A.R. & Gonzalez-Lima. F. (1 994). Structural equation modeling and its application to network analysis in functional brain imaging. Hurnan Brain Maooine.
22-32.
Milner. B. (1968). Visual recognition afier nght temporal lobe excision in man.
Neurops~choiogv.6. 19 1-309.
Neville, H.J.. Kutas. M., Chesney, G.. & Schmidt, A.L. ( 1986). Event-related potentials during initial encoding and recognition memory of congruous and incongruous words. Journal of Memorv and Lanpluage, 25. 75-92. ERPs and Picture Mernory
Nickerson, R.S. ( 1965). Short-term memory for comlex meaningfull visual
configurations: a demonstration of capacity. Canadian Journal of Psycholow. 19(2),
155-160.
Nickerson, R.S. ( 1968). A note on long-term recognition memory for pictorial
material. Psychological Science. 1 1(2), 58.
Noldy. N.E., Stelmack. R.M.. & Campbell. K.B. (1990). Event-related potentials and recognition memory for pictures and words: the effects of intentional and incidental learning. Psvcho~hvsiolo~v.27(4). 4I 7-428.
Pascual-Marqui. R.D., Michel. C.M., & Lehmann, D.( 1994). Low resolution eiectromagnetic tomography: a new method for localizing electrical activity in the brain.
International Journal of Psvcho~hysiolog~,18, 49-65.
Petersen, S.E., Fox. P.T., Posner, M.I.. Mintun, M.. & Raichle. M.E. (1989).
Positron emission tornographic studies of the processing of single words. Journal of
Cognitive Neuroscience. 1. 153- 170.
Petersen, S.E.. Fox. P.T., Snyder. A.Z. & Raichle. M.E. ( 1990). Activation of the extrastriate and frontal cortical areas by visual words and word-Iike stimuli. Science.
-349. 1O4 1- 1044.
Picton. T.W. (1 992). The P300 wave of the human event-re1ated potential.
Journal of Clinical Neuro~hysiolony.9(4), 456-479.
Picton. T.W.. Donchin. E.. Ford, J., Kahneman, D.. & Norman. D. (1 984). The
ERP and decision and memory processes. In E. Donchin (Ed.), Cognitive
Ps~chophvsiolo~v.(pp. 19 1-200). Hillsdale, N.J: Erlbaurn. ERPs and Picture Memory
Potter, M.C., & Levy, E.I. (1969). Recognition memory for a rapid sequence of
pictutes. Journal of Experimental Psvcholoev. 8 1 ( 1), IO- 1 5.
Rudell. A.P. ( 199 1). The recognition potential contrasted with the P300.
International Journal of Neuroscience. 60. 85-1 1 1.
Rudell. A.P. (1 992). Rapid Stream stimulation and the recognition potential.
Electroencephalogra~hyand Clinical Neuro~hvsioloev, 83. 77-82.
Rudell, A.P., Cracco, R.Q.. Hassan, N.F.. & Erberle. L.P. (1 993). Recognition
potential: sensitivity to visual field stimulated. Electroencephalo~raphvand Clinical
Neurophysiolo~v. 87, 22 1-234.
Rudell. A.P.. & Hua. J. ( 1996). The recognition potential and conscious atvareness. Electroence~ha1or;raphvand Clinical Neurophvsiolog;~.98: 309-3 18.
Rugg. M.D. (1985). The effects of word repetition and semantic prirning on event-related potentials. Psvchophvsiolo~v.22. 642-647.
Rugg. M.D. (1987). Dissociation of semantic priming. word and non-word repetition by event-related potentials. Ouarterlv Journal of Experimental Psvchologv.
-39A. 223-148.
Rugg, M.D. (1 995). ERP studies of memory. In M.D. Rugg & M.G.H. Coles
(Eds.), ElectrophysioIoev of Mind. Event-related Potentials and Cognition. (pp. 1%
168). New York: Oxford.
Rugg. MD..& Doyle. M.C. (1 994). Event-related potentials and stimulus repetition in direct and indirect tests of memory. In H.J. Heinze. T. Munte & G.R.
Mangun (Eds.), Cognitive Electro~hvsiolo~~,(pp. 124- 148). Boston: Birkhauser.. ERPs and Picture Memory
Rugg, M.D., & Nagy, M.E. (1 989). Event-related potentials and recognition
memory for words. . Electroencephaloeraph~and Clinical Neurophysioloav.-. 72. 395-
406.
Rugg, M.D.. Soardi, M., & Doyle. M.C. (in press). Modulation of event-related potentials by the repetition of drawings of novel objects. Cognitive Brain Research.
Sanquist, T.F., Rohrbaugh. LW., Syndulko. K.. & Lindsley. D.B. (198%.
Electrocortical signs of levels of processing: perceptual analysis and recognition memory. Psvcho~hvsioloev.17.568-576.
Scherg, M. (1990). Fundarnentals of dlpole source analysis. In F. Grandor. M.
Hoke. & G.L. Romani (Eds.). Auditorv Evoked Maenetic Fields and Electnc Potentials.
Advances in Audioloev. Vol. 5 (pp. 40-69). Basel: Karger.
Shepard. R.N. (1967). Recognition memory for words. sentences. and pictures.
Journal of Verbal Leamine; and Verbal Behavior, 6. 156- 1 63.
Smith. ME.. & Halgren. E. (1987). Event-related potentials elicited by farniliar and unfarniliar faces. In R. Johnson. Jr.. J.W. Rohrbaugh & R. Parasuraman (Eds.).
Current Trends in Event-Related Potential Research. EEG Suml. 40 (pp 422426).
Amsterdam: Elsevier.
Smith, M.E., & Halgren. E. (1 989). Dissociation of recognition memory components following temporal lobe lesions. Journal of Ex~erimentaiPsycholoev:
Learning. Memow and Cognition. 15(1), 50-60.
Standing, L.. Conezio, J., & Haber, R.N. (1970). Perception and memory for pictures: single trial learning of 2500 visual stimuli. Psvchonomic Science. l9(2). 73-74. ERPs and Picture Memory
Stein, C.E., Corkin. S. Godez, G.R., Guimaraes. A.R.. Baker. J.R.. Jennings.
P.J.. Cm,C.A., Suigiura. R.M.. Vendantham. V.. & Rosen, B.R. ( 1996). The hippocampal formation participates in novel picture encoding: evidence fkom functional magnetic resonance imaging. Proceedings of the National Academv of Sciences. 93.
8660-8665.
Stuss. D.T., Picton, T.W., & Cem. A.M. (1986) Searching for the names of pictures: an event-related potential study. Psvcho~hvsiolow.23(2), 215-223.
Stuss. D.T..Sarazin. F.F., Leech, E.E., & Picton. T.W. (1983). Event-refated potentials during narning and mental rotation. Electroence~haioera~hvand Clinical
Neurophvsiologv. 56. 133- 146.
Tulving. E.. Kapur. S.. Markowitsch. H.J.. Craik. F.I.M.. Habib. R. & Houle. S.
( 199413). Neuroanatomical correlates of retrieval in episodic memory: Auditory sentence recognition. Proceedines of the National Academv of Sciences USA. 9 1. 20 12-20 15.
Tulving, E.. & Kroll, N. (1995). Novelty assessment in the brain and long-term rnemory encoding. Psvchonomic Bulletin & Review. 2(3). 3 87-390.
Tulving, E.. Markowitsch. H.J.. Craik. F.I.M.. Habib. R..& Houle. S. (1996).
Novelty and farniliarity activations in PET studies of memory encoding and retrieval.
Cerebral Cortex. 6.7 1-79.
Tulving, E., Markowitsch, H.J.. Kapur, S., Habib, R.. & Houle, S. (1994a).
Novelty encoding networks in the human brain: positron emission tomography data.
Neuroreport. 5, 2525-2528. ERPs and Picture Memory
Tulving, E., & Schacter. D.L. ( 1990). Pnming and hurnan mernory systems.
Science, 247.30 1-306.
Van Petten, C.. & Senkfor, A.J. (1996). blemory for words and novel visual
patterns: repetition, recognition, and encoding effects in the event-related brain potential.
Psvcho~hvsiolo~.33(5), 49 1-506.
Wallace, W.P. (1965). Review of the histoncal, empirical. and theoretical status of the von Restorff phenomenon. Psvchological Bulletin. 63.4 10-424.
Verieger. R. ( 1988). Event-related potentials and cognition: a critique of the context updating hypothesis and an alternative interpretation of P3. Behavioral and Brain
Sciences. 1 1.34347.
Yamaguchi. S., & Knight. R.T. (1 99 la). P3OO generation of novel sornatosensory stimuli. Electroencephaloeraphv and Clinical Neuro~hvsioloev. 78. 50-
55.
Yamaguchi. S.. & Knight. R.T. ( 1 99 1 b). Antenor and posterior association cortex contributions to the somatosensory P3OO. Journal of Neuroscience. 1 l(7). 3039-
2054. IMHbL LVfiLUN1 TEST TARGET (QA-3)
APPtlED IMAGE. lnc 31653 Eaçt Main Street -*- --- - Rochester. NY 14609 USA ------Phone: 716/482-0300 ---- Fax: 7161288-5989
O 1993, App(ied image. Inc.. All Rights Reserved