Steady-State Visually Evoked Potential Correlates of Object Recognition Memory

Thesis for Doctorate of Philosophy Andrew Pipingas, BAppSc(Dist) February, 2003

Brain Sciences Institute, Swinburne University of Technology

Declaration

This thesis contains no material which has been accepted for the award of any other degree at any University and to the best of my knowledge and belief contains no material previously published or written by another person or persons except where due reference is made.

Andrew Pipingas February, 2003

Acknowledgements

The author would like to acknowledge the following people whose contribution made this work possible.

My supervisor and mentor, Prof. Richard Silberstein, for his invaluable assistance, guidance and patience throughout the project and for providing me with the intellectual inspiration to undertake a PhD project in the area of brain electrophysiology.

Mr David Simpson for the design and development of the instrumentation used to conduct this work and for his assistance with many other computer-related and technical matters.

Mr Geoff Nield for his assistance with the development of a suite of software programs that were used to analyse the data.

Ms Barbara Livett for generously giving up so much time to assist with various drafts and the English expression for this thesis.

Assoc. Prof. Aina Puce and Assoc. Prof. David Crewther for their helpful comments on earlier drafts of the manuscript.

My parents for providing me with the motivation and support to pursue a higher degree and, together with my brother, for providing continual encouragement and support.

Finally, I would like to thank my wife and daughter for their encouragement, support and patience during the highs and lows associated with a PhD project.

Contents

List of figures...... v

List of equations ...... vii

List of abbreviations and units ...... viii

Abstract...... ix

Chapter 1 Introduction...... 1

Chapter 2 Neural correlates of object recognition memory...... 8

2.1 Object recognition memory: a brief overview...... 9 2.1.1 Inferotemporal cortex and primate studies of object recognition ...... 9 2.1.2 The temporal lobes and object recognition memory in human and other primates ...... 13 2.1.3 A brief summary of haemodynamic (PET and fMRI) neuroimaging correlates of episodic memory retrieval...... 18

2.2 Human electrophysiological correlates of episodic memory retrieval .....21 2.2.1 EEG and ERP functional electrophysiological techniques: a brief background...... 22 2.2.2 EEG changes associated with episodic memory retrieval...... 25 2.2.3 Event-related potential changes associated with episodic memory retrieval ...... 32 2.2.3.1 ERP correlates of episodic memory retrieval: evidence from continuous recognition paradigms ...... 32 2.2.3.2 ERP correlates of episodic memory retrieval: evidence from study-test paradigms ...... 41 2.2.3.3 Differentiation between transient item-related and sustained task- related ERP correlates of episodic memory retrieval...... 56

2.3 Summary of neural correlates of object recognition memory ...... 58

Chapter 3 Steady-State Probe Topography ...... 62

i

3.1 Steady-state evoked potentials compared with transient evoked potentials ...... 62

3.2 Steady-state evoked potentials in the study of cognitive processes...... 64

3.3 Steady-State Probe Topography (SSPT)...... 65

3.4 SSPT and recording artifacts...... 70

3.5 Overview of investigations utilizing the SSPT technique ...... 70 3.5.1 Visual vigilance...... 72 3.5.2 Planning - Wisconsin Card Sorting Task...... 73 3.5.3 Attention - Continuous Performance Task...... 74 3.5.4 Clinical application of SSPT - ADHD...... 74 3.5.5 Spatial working memory...... 75

3.6 Conclusions ...... 76

3.7 Formulation of hypotheses for the present investigation ...... 76

Chapter 4 Methods ...... 79

4.1 Introduction...... 79

4.2 Cognitive task design ...... 80

4.3 Task presentation and stimulus parameters ...... 83

4.4 Subjects ...... 84

4.5 Probe stimulus ...... 85

4.6 Recording...... 86 4.6.1 Electrodes and recording setup ...... 87 4.6.2 Recording instrumentation and phase locked data acquisition ...... 88

4.7 Analysis of behavioural data...... 90

4.8 Offline signal processing...... 90 4.8.1 Extraction of the 13Hz SSVEP from the EEG signal ...... 90 4.8.2 Automatic detection of artifact in the EEG signal ...... 92 4.8.3 Calculations for modal and transient effects...... 93 4.8.3.1 Long averaging period (sustained effects) ...... 94 4.8.3.2 Short averaging period (transient changes)...... 94

ii

4.8.4 Group averaging...... 96 4.8.4.1 Group averaging: long averaging period (sustained effects) ...... 97 4.8.4.2 Group averaging: short averaging period (transient changes) ...... 98 4.8.5 Topographic mapping of SSVEP data ...... 100 4.8.6 Statistical analysis and Significance Probability Mapping ...... 101

Chapter 5 Results...... 103

5.1 Behavioural data ...... 104 5.1.1 Individual subject...... 104 5.1.2 Group data...... 105

5.2 Electrophysiological data ...... 107 5.2.1 Sustained effects: long averaging period ...... 107 5.2.1.1 Individual subjects ...... 107 5.2.1.2 Group data...... 109 5.2.2 Transient changes: short averaging period...... 113 5.2.2.1 Changes with memory load...... 114 5.2.2.2 Targets versus non-targets...... 124

Chapter 6 Discussion...... 128

6.1 Behavioural results: the effect of increasing memory load on retrieval accuracy and response time ...... 128

6.2 Sustained SSVEP amplitude and latency changes and retrieval mode..130 6.2.1 The topography of sustained SSVEP changes ...... 131 6.2.2 Interpretation of sustained SSVEP amplitude and latency reductions..134

6.3 Transient SSVEP amplitude and latency changes: the effect of memory load on successful memory retrieval ...... 137 6.3.1 Transient parietal effects...... 139 6.3.2 Transient right frontal effects...... 143 6.3.3 Transient effects and retrieval effort...... 147 6.3.4 Interpretation of transient SSVEP amplitude and latency reductions...148

6.4 Transient SSVEP amplitude and latency changes: target versus non- target objects...... 151

6.5 Conclusions and future directions...... 154

iii

Appendix A. Task instructions ...... 157

Appendix B. Determination of optimum luminance of light-emitting diode (LED) arrays ...... 160

Appendix C. Behavioural results for practice tasks ...... 162

Appendix D. The amplitude of the SSVEP produced by turning on the probe stimulus ...... 165

Appendix E. Sustained SSVEP effects for each memory load condition relative to the baseline task: practice tasks...... 170

Appendix F. Retrieval of everyday objects...... 173

Publications by the author ...... 175

Bibliography ...... 189

iv

List of figures

Figure 3.1 The effect of various forms of artifact on an SSVEP amplitude time series ______71 Figure 4.1 Study-test experimental design ______80 Figure 4.2 Experimental task design ______81 Figure 4.3 Calculation of modulation depth______86 Figure 4.4 Sixty-four scalp recording locations ______87 Figure 4.5 Experimental recording arrangement______88 Figure 4.6 Single cycle and averaged Fourier coefficients ______95 Figure 5.1 Mean response times for target objects for each memory load for an individual subject ______105 Figure 5.2 Mean response time and mean number of errors for target objects versus memory load for 40 subjects ______106 Figure 5.3 Sustained effects: Mean SSVEP amplitude and phase topography for the baseline task and each of the memory load conditions calculated with a long (40s) averaging period for an individual subject______108 Figure 5.4 Sustained effects: Mean SSVEP amplitude and phase topography for the baseline condition and each of the memory load conditions averaged across 40 subjects ______110 Figure 5.5 Sustained effects: Topographic differences in SSVEP amplitude and latency between the average of the 3 memory load conditions and the baseline task averaged across 40 subjects ______111 Figure 5.6 Sustained effects: Topographic differences in SSVEP amplitude and latency between memory load conditions averaged across 40 subjects ______112 Figure 5.7 Normalized SSVEP amplitude time series for each of the memory load conditions calculated across 40 subjects ______116 Figure 5.8 SSVEP phase time series for each of the memory load conditions calculated across 40 subjects ______117 Figure 5.9 Normalized SSVEP amplitude time series for each of the memory load conditions calculated across 40 subjects ______118

v

Figure 5.10 SSVEP phase time series for each of the memory load conditions calculated across 40 subjects ______119 Figure 5.11 Topographic differences in SSVEP amplitude and latency between memory load conditions for target objects and Hotelling's T statistic (averaged across 40 subjects) ______122 Figure 5.12 Topographic differences in SSVEP amplitude and latency between memory load conditions for non-target objects and Hotelling's T statistic (averaged across 40 subjects) ______123 Figure 5.13 Topographic differences in SSVEP amplitude and latency between correctly identified target and non-target objects, averaged separately for each memory load condition: pooled across 40 subjects______125 Figure B.1 Determination of optimum luminance of LED arrays______161 Figure C.1 Practice tasks: Mean response times for target objects versus memory load for an individual subject ______163 Figure C.2 Practice tasks: Mean response time and errors for target objects versus memory load for 40 subjects ______164 Figure D.1 SSVEP amplitude and phase during the stimulus-off and stimulus-on intervals for an individual subject______167 Figure D.2 Mean SSVEP amplitude during the stimulus-off and stimulus-on intervals for an individual subject ______167 Figure D.3 SSVEP amplitude and phase in response to switching-on probe stimulus averaged across 40 subjects ______168 Figure D.4 Mean SSVEP amplitude topography during the stimulus-off and stimulus-on intervals averaged across 40 subjects______169 Figure D.5 SSVEP amplitude difference topography and Hotelling’s T values for stimulus-on relative to stimulus-off conditions averaged across 40 subjects______169 Figure E.1 Mean SSVEP amplitude and phase topography for the baseline task and each of the practice memory load tasks for an individual subject ______171 Figure E.2 Mean SSVEP amplitude and phase topography for the baseline task and each of the practice memory load tasks averaged across 40 subjects ______172 Figure F.1 Retrieval of everyday objects ______174

vi

List of equations

Equation 4.1 Calculation of unamplified EEG amplitude ______89 Equation 4.2 Calculation of single cycle Fourier coefficients ______91 Equation 4.3 Calculation of SSVEP amplitude and phase ______91

vii

List of abbreviations and units

BA ……………………………………………………………….….… Brodmann’s area Cd/m2 …………..………………………………………..……. candela per square metre dB ………………………………………………………………………..………. decibel DC ………………………………………………………………………... direct-coupled ECG ……………………………………………………..…………… electrocardiogram EEG ………….………………………………………...…..…….. electroencephalogram EOG ………………………………………………………………… electro-occulogram EMG …………………………………………………………………… electromyogram ERP ……………………………………………………...………. event-related potential fMRI ………………...…………………...……. functional magnetic resonance imaging Hz ………….…………...……………………………………………………..……. hertz IAF ………………………………………………………... individual’s alpha frequency LPC …………………………………………………………….. late positive component MEG ………………………………………………………….... magnetoencephalogram MΩ ……...……………………………………….…………………………… mega-ohm ms ………….………………...……………………………………………… millisecond µV …………………………………………………………...………………… microvolt ORM …………………………...…………………………… object recognition memory kΩ ……...……………………………………….……………………………… kilo-ohm PET ……………………………...…………………….... positron emission tomography s …………………………………...………………………...……………………. second SSPT ……………………………………………….……. steady-state probe topography SSVEP …...…………………………...…………. steady-state visually evoked potential VEP ……………………………………...……………………… visual evoked potential

viii

Abstract

Object recognition memory (ORM) refers to both recognition of an object and the memory of having seen it before. In humans, ORM has been investigated using functional neuroimaging and electrophysiological techniques with tests of episodic memory retrieval involving recollection of previously studied items. Processes involved in the maintenance of a mental state adopted for the performance of a retrieval task (retrieval mode) appear to involve right frontal neural regions. More transient processes occurring at the time of item recollection (retrieval success) have shown scalp activity over parietal and right frontal regions. This activity is thought to originate in the medial temporal lobes and the underlying right frontal cortex respectively. The aforementioned findings have been derived mainly from studies using verbal stimuli. It is uncertain whether the same neural regions are involved in object recollection. It is also uncertain whether sustained modal and transient item-related activity involve the same or different right frontal regions. In this study, steady-state probe topography (SSPT) was used to investigate both sustained and transient processes involved in the retrieval of abstract pictorial objects from memory. The ability to vary the evaluation period of the steady-state visually evoked potential (SSVEP) allows investigation of cognitive processes occurring over different time scales. Neural regions involved in sustained modal processes were identified by examining the SSVEP values averaged over the duration of a memory retrieval task. Sustained SSVEP effects were observed over right fronto-temporal regions. Neural regions involved in transient retrieval success processes were identified by comparing the transient SSVEP responses to tasks with different memory loads. Comparison of a higher with a lower memory load condition showed SSVEP effects over parieto-temporal and right inferior frontal regions. Larger differences between memory loads gave effects that were larger and more right lateralized. Retrieval mode and retrieval success processes showed SSVEP effects over different right frontal regions. It was also found that, in contrast to the left lateralized parietal ERP response to recollected verbal stimuli, the SSVEP effects produced with abstract pictorial shapes showed a more bilateral pattern. This was considered to reflect the relatively non-verbalizable pictorial nature of the stimuli.

ix

Chapter 1 Introduction

This study uses the technique of steady-state probe topography (SSPT) to investigate the neural correlates of object recognition memory (ORM). Object recognition memory is defined as the ability to recognize or remember previously encountered objects. Object recognition memory is only one facet of the multifaceted function we term ‘memory,’ and is usually considered to involve three main stages: encoding, storage and retrieval. For an event to be remembered it must first be encoded. This process can be thought of as the formation of new memory traces in the brain. Storage involves maintenance of these memory traces over time, and retrieval refers to the accessing of these memory traces.

It appears that memories can be encoded into different stores, characterized by the length of time that the memory traces are maintained. Some memories may be maintained only fleetingly. For example, while watching a movie, each individual still frame is remembered long enough for a succession of these to make sense (Baddeley 1999). This type of memory is sometimes referred to as immediate, or sensory, memory. Memories which are maintained for slightly longer, for example, a telephone number that is remembered only long enough to dial it, are stored in short term, or working, memory. In contrast, other memories may last several minutes, days, or even a lifetime. The store for these is referred to as long term memory.

Long term memory is generally considered to consist of two broad and largely independent types of memory referred to as explicit and implicit memory (eg. Squire and Zola-Morgan 1991; Tulving 1983). Explicit memory, also known as declarative memory, encompasses those memories that involve conscious recollection, such as, the experience of eating steak for dinner the previous night, or the fact that the Eiffel Tower is in Paris. These two examples represent a further subdivision of explicit memory into what is termed episodic and semantic memory (Tulving 1983). Episodic memory consists of context-specific memories of experiences within one’s personal past, whereas semantic memory consists of knowledge of facts that can be stated in words. Conversely, implicit memory, also known as non-declarative memory, encompasses

Introduction

those memories that do not necessarily require conscious recollection. These include skills, such as riding a bicycle, and habits. Proficiency is usually measured in terms of accuracy or speed of response.

Object recognition memory (ORM) refers to both recognition of an object and the memory of having seen it before. Object recognition memory can be part of sensory memory, working memory, and long term memory processes. However, in human studies, the longer term memory aspects of ORM have usually been studied, and tests of episodic retrieval have frequently been used. In such tests, objects are studied and then identified some time later. In a more general sense, the experiencing of an object can be thought of as an event in one’s personal past. However, items may be recognized because of their familiarity, rather than because the actual encounter is remembered (eg. Jacoby 1991; Mandler 1981). It has been suggested that when recognition is based on familiarity, implicit memory processes may be involved. It has also been suggested, however, that memory retrieval based on familiarity may also form a part of a larger explicit, or declarative, memory system (eg. Moscovitch 1992; Moscovitch 1994; Squire 1994). While the distinction between familiarity and the recollection of an experience could complicate the study of ORM, tests have been designed to influence the extent to which familiarity or recollection processes are used. With such tests it has been possible to distinguish between processes involved in familiarity-based and recollection-based recognition.

The neural regions involved in ORM have been extensively investigated in non-human primates, mainly by examining how ablating various parts of the brain affects ORM function. Such studies have shown that a series of cortical regions beginning at the primary visual cortex and ending within the inferior temporal lobe, the so-called ventral pathway, are important in object perception and recognition processes. Awareness that something has been seen previously has been shown to involve interaction of these ventral pathway regions with adjacent medial temporal lobe regions, including the hippocampus and perirhinal cortex.

Lesion studies in humans have indicated that neural regions equivalent to those in non- human primates are involved in human ORM. However, because lesions in human subjects generally result from accident, disease or surgery, they are generally larger and

2 Introduction

less precisely localized than those that can be produced experimentally in animals. Thus, confidence in conclusions drawn about functions associated with specific brain regions must be limited. Furthermore, it is generally not clear whether memory impairment stems from deficits in encoding or retrieval. Thus, differentiation of the neural correlates of encoding and retrieval on the basis of lesion data is difficult. Despite the problems inherent in such studies, findings have indicated that the same structures that are important in non-human primate recognition memory, namely, the medial temporal lobe, the hippocampus and perirhinal cortex, are also important in human recognition memory.

During the last two decades, functional neuroimaging and non-invasive electrophysiological techniques have enabled memory processes in the undamaged brains of normally functioning humans to be studied. These techniques have permitted the monitoring of brain activity at a large number of neural sites simultaneously, whilst subjects perform tasks designed to activate regions involved in ORM.

Functional neuroimaging detects changes in cerebral blood flow and can localize brain activations to within a few millimetres, but it has relatively poor temporal resolution. These techniques, used in the study of ORM, have had temporal resolutions ranging from a few seconds to a few minutes. They have been used mainly to investigate sustained, or task-related, cognitive states that are initiated by task instructions and that persist throughout the task.

On the other hand, electrophysiological techniques can monitor neural activity with a temporal resolution in the order of milliseconds. However, spatial resolution is limited to gross brain regions. Electrophysiological techniques have mainly been used to investigate more transient, or item-related, processes. These processes are initiated by the presentation of each stimulus item and may continue throughout the duration of presentation.

In both functional neuroimaging and electrophysiological studies, recognition memory processes have been investigated using tests of episodic memory retrieval. Although, these studies have mainly used word rather than object stimuli, findings have nevertheless aided the understanding of memory retrieval processes.

3 Introduction

Functional neuroimaging studies have identified seven main regions considered to be involved in episodic retrieval, namely, prefrontal, medial temporal, medial parieto- occipital, lateral parietal, anterior cingulate, occipital, and cerebellar regions (Cabeza and Nyberg 2000). Findings of medial temporal lobe activity support findings from human lesions studies. It has been shown that medial temporal lobe activity increases with increasing recognition accuracy. However, activity in the right prefrontal region appears to be the most frequently reported finding in episodic retrieval studies. While activity in this region appears to be present under a number of experimental conditions, the most consistently reported finding is that activity in this region is sustained for the duration of the retrieval task, that is, the activity is task-related. This sustained right prefrontal activity may be necessary for the maintenance of the mental set, or state that accompanies conscious recollection. This has been termed retrieval mode by Tulving (1983).

A major focus of electrophysiological research into episodic retrieval has been the difference between the brain’s electrical response to correctly recognized, previously presented, or old, words compared with the response to the presentation of words not previously presented, or new, words. This is known as the ERP old/new effect. This effect has been used to investigate processes associated with retrieval success. Most studies that used verbal materials as stimuli found the largest ERP old/new effect over left parietal regions, beginning approximately 400ms after the appearance of the stimulus and lasting throughout its duration. Transient item-related right frontal effects are found in situations when the context of item presentation has to be recalled, that is, when recognition is based on recollection rather than purely on familiarity, and are thought to be associated with both recollection and monitoring processes.

Despite the wealth of information that has been attained through use of these functional neuroimaging and electrophysiological techniques, a number of important unresolved issues remain. Firstly, it is not clear whether the right frontal activity found in functional neuroimaging and electrophysiological studies is sustained task-related activity, or more transient item-related activity. What appears to be task-related activity may, in fact, be the result of the summed intermittent activity occurring in response to individual items. Alternatively, both sustained and transient activity may be present. Furthermore,

4 Introduction

because of the limitations of the respective techniques, it is not be possible to determine whether the same or different adjacent regions are involved if both types of activity are present. Secondly, it is not clear whether non-verbal stimuli produce patterns of activity essentially the same as those observed with verbal stimuli. This is an important issue in the present study as the focus is on the retrieval of objects, rather than words, from memory. Thirdly, activity associated with retrieval success is generally investigated by comparing the activity associated with correctly retrieved old items with the activity associated with correctly identified new items. Because a comparison is made with a secondary task that entails the identification of new items, which may itself produce its own characteristic neural activity patterns, this may confound interpretation. Duzel et al. (1999) have explored the matter of whether activity associated with a word-based episodic retrieval is task and/or item-related by using a combination of functional neuroimaging and electrophysiological techniques. They found that the right prefrontal region, BA10, is active throughout the course of a retrieval task, whereas the left medial temporal lobe becomes active intermittently whenever individual words are presented and is more active for familiar than for novel words. However, this study does not address the fact that in many electrophysiological studies, transient item-related, right prefrontal effects have also been reported. Furthermore, it is not clear whether non- verbal stimuli produce the same task and item-related effects that verbal stimuli produce. The present study attempts to address the three aforementioned issues using the technique of steady-state probe topography (SSPT).

The SSPT technique was developed by Silberstein and colleagues (Silberstein et al. 1995a; Silberstein et al. 1990b). With this technique, the effects of mental activity on the steady-state evoked potential (SSVEP) generated by a rapidly repeating irrelevant, or probe, stimulus are examined at multiple scalp recording sites. A major advantage of SSPT over traditional electrophysiological techniques is that it permits temporal continuity as well as a range of time scales over which processes can be studied. This makes it a valuable tool for the investigation of both sustained task and transient item- related effects. In this study, the probe stimulus was a 13Hz visual flicker. The SSVEP produced in response to the visual flicker was recorded via 64 scalp electrodes. The spatial resolution of SSPT using 64 electrodes, although not as good as that obtained using functional neuroimaging methods, appears satisfactory for investigating neural activity within gross brain regions. Variations in the SSVEP amplitude and phase have

5 Introduction

been shown to reflect a range of cognitive processes, including working memory processes (Silberstein et al. 2001).

The main aim of the present study was to use SSPT to investigate both sustained retrieval mode activity and transient activity occurring during the actual recollection process. In this study, an episodic retrieval task was used in which previously studied, abstract, two-dimensional objects had to be identified when presented within a sequence containing a larger number of unstudied distractor objects. The memory load was varied by changing the number of objects that had to be remembered. To investigate sustained task-related processes, the average of the SSVEP values across all memory load conditions was compared with the average of the SSVEP values obtained in a separate non-episodic control task. To investigate more transient item-related processes, SSVEP amplitude and phase values obtained for different memory load conditions were compared at a point in time when subjects were considered to be engaged in the memory retrieval process. It was anticipated that increases in memory load would result in increased utilization of those neural regions necessary for retrieval success. This approach has the advantage over the traditional approach of comparing old and new items because it avoids any spurious effects that might result from the ‘new item’ comparison task. The use of abstract objects enabled exploration of the question of whether patterns of activity associated with the retrieval of object stimuli are the same as those associated with verbal stimuli.

Sustained, task-related, SSVEP effects were most prominent over right frontal regions. This is consistent with functional neuroimaging and electrophysiological studies that have associated activations in this region with the maintenance of a retrieval mode. The right frontal effects in the present study were characterized mainly by reductions in SSVEP latency. Memory-load-dependent transient SSVEP amplitude and latency reductions were found over parieto-temporal regions bilaterally and over right inferior frontal regions. In addition, increases in memory load led to larger and more right- lateralized SSVEP effects. These transient item-related effects and the ERP old/new effects are apparent during essentially the same time interval post stimulus onset. However, the parietal ERP old/new effect is usually reported as being left-lateralized, whereas a right-lateralized effect was observed in the present study. This right lateralization, however, is consistent with the abstract pictorial nature of the stimuli

6 Introduction

used. The spatio-temporal patterns of the item-related right frontal effects found in the present study are similar to those of the right frontal ERP old/new effects associated with remembering past events. While both sustained task and transient item-related effects occurred over the right frontal region, they appeared to be localized to different parts of the region. Furthermore, sustained task-related effects were characterized mainly by SSVEP latency reductions, whereas transient item-related effects were characterized by both amplitude and latency reductions that increased with increasing memory load.

This thesis is in six chapters. Chapter 1 comprises the introduction. Chapter 2 reviews the literature on the neural correlates of object recognition memory. The chapter begins with a brief overview of the way in which primate lesion studies have indicated the neural structures and pathways involved in ORM. This is followed by an overview of PET and fMRI functional neuroimaging studies of episodic memory retrieval. These overview sections are then followed by a review of studies investigating electrophysiological correlates of episodic memory retrieval. Chapter 3 comprises a description of the SSPT methodology and a formulation of the hypotheses of the present study. Chapter 4 describes in detail the experimental methods that were employed in this study. Task design, recording and analysis are explained in this chapter. Chapter 5 deals with results of the present study. Task and item-related findings are then discussed in Chapter 6.

7

Chapter 2 Neural correlates of object recognition memory

This chapter outlines findings from studies concerned with the representation of object recognition memory (ORM) in the human brain. The chapter consists of two sections. The first is an introduction to the neural regions involved in ORM (section 2.1), and the second is a comprehensive review of human electrophysiological studies that have attempted to define and localize neural substrates of ORM (section 2.2).

The introduction to ORM (section 2.1) is not intended as an extensive review of the literature. Rather, it is included as background for the reader not familiar with this area. This section begins with a brief description of the neural areas involved in visual perception and recognition, with a particular focus on the so-called ventral pathway that connects visual areas with the inferotemporal cortex (section 2.1.1). The functions of regions involved in perception and recognition have been determined largely from electrophysiological recordings and lesion studies in non-human primates. A brief discussion then follows outlining the more specific role of the medial temporal lobes in ORM (section 2.1.2). This is discussed within the context of the declarative model of memory. This model has been derived predominantly from findings of lesion studies in non-human primates, and of neuropsychological lesion studies with humans. Finally, there is a brief overview of functional neuroimaging studies, focusing on those studies that have utilized PET and fMRI techniques with tests of episodic memory retrieval to investigate neural regions involved with ORM (section 2.1.3). Studies using these imaging techniques have largely examined modal task-related activity, that is, activity that is maintained for the duration of the imaging period.

Section 2.2 contains the primary focus of this review: human electrophysiological correlates of object recognition memory. In human electrophysiological studies, visual recognition memory has been examined using tests of memory retrieval in which previously presented words or pictures are later identified. In addition to structures of the ventral pathway and the medial temporal lobe, recognition of items during a

Neural correlates of object recognition memory

retrieval task also involves other regions, such as the temporal, parietal, and frontal regions. Electrophysiological techniques that achieve a high temporal resolution have been used to examine the fast neural processes that occur during memory retrieval. A major review of electrophysiological correlates of episodic memory retrieval is presented in sections 2.2.2 and 2.2.3. These sections are concerned with EEG and ERP correlates respectively.

2.1 Object recognition memory: a brief overview

The following overview of ORM and neural regions involved in ORM processes focuses on the visual pathways, and in particular, on the role of the inferior temporal cortex in visual perception (section 2.2.1). This section is followed by a discussion of ORM within the context of a model of declarative (explicit) memory. The focus here is on the role of the medial temporal lobes in ORM (section 2.1.2). More precise localization of neural regions involved in the retrieval of items from memory has been investigated using PET and fMRI neuroimaging. A short summary of the main findings is provided in section 2.1.3.

2.1.1 Inferotemporal cortex and primate studies of object recognition

A number of stages of visual processing precede object recognition. As visual recognition memory depends on visual perception, it is important to consider the brain regions involved in the early stages of visual processing. Visual information from the retina reaches the cortex via two main pathways: one includes the lateral geniculate nucleus, which projects almost exclusively to the primary visual cortex (also known as V1, area 17 or striate cortex), and the other includes the superior colliculus and pulvinar, which projects much more extensively (eg. Kandel et al. 2000; Reid 1999). Cortical regions that receive projections from the latter pathway are not exclusive to vision and are also associated with functions such as somatosensory, auditory and motor processing. The areas receiving the densest input from the lateral geniculate nucleus and pulvinar appear to be area 17, and the extrastriate cortex (also known as visual association or areas 18 and 19). Area 17 is mainly confined to the calcarine fissure and

9 Neural correlates of object recognition memory

includes parts of the cuneus and lingual gyrus. Areas 18 and 19 are organized concentrically around area 17 and also receive direct inputs from area 17 (Kolb and Whishaw 1996 among others).

Parietal and temporal regions also receive a rich supply of connections from area 17 and appear to perform a number of important visual functions. In fact, the visual cortex projects to a large proportion of the total cortical area. Felleman and van Essen (1991) report that in the primate brain, 55% of the whole cortical surface is involved in vision, whereas only 11% is involved in somatosensory processing, and 3% in auditory processing.

Findings from a series of electrophysiological studies on non-human primates indicate that sensory information from the primary visual cortex reaches the parietal and temporal lobes via a number of cortico-cortical stages (Ungerleider and Mishkin 1982). Two relatively distinct pathways, or streams, were noted. One pathway passes dorsally into the extrastriate cortex and terminates in the posterior parietal lobule, while the other passes ventrally through the extrastriate cortex and terminates within the inferotemporal cortex. The authors proposed that the dorsal pathway is concerned with ‘where’ visual information is located, and the ventral pathway is concerned with ‘what’ the visual information is. The inferior temporal cortex represents the final cortical stage in the ‘what’ pathway. Evidence for the existence of distinct ventral and dorsal visual pathways, the so-called ‘what ‘ and where’ pathways in humans has been found using functional neuroimaging techniques (Courtney et al. 1996; Haxby et al. 1991; Kohler et al. 1995; Ungerleider and Haxby 1994).

Interconnections between primary visual areas and regions associated with these two visual pathways, or streams, have been extensively studied in the non-human primate brain. Felleman and Van Essen (1991) reported that 32 visual and visual association areas can be differentiated in the non-human primate brain. Almost half of these areas have now been mapped in the human brain (Sereno et al. 1995; Tootell et al. 1996; Tootell et al. 1997). In many respects, the organisation of visual areas in the human and non-human primate cortex appears to be similar (Tootell et al. 1996). Furthermore, as indicated earlier, PET imaging studies have revealed that the dorsal and ventral processing streams are also similar. Although there are differences between the visual

10 Neural correlates of object recognition memory

systems of humans and non-human primates, the non-human primate brain provides a good model for the investigation of the human visual system (Tootell et al. 1997).

It is the ventral pathway, the so-called ‘what’ pathway, that is of particular relevance to this study. A brief description of its interconnections will follow, with a focus on the flow of visual information through this pathway to the inferior temporal cortex.

In non-human primates, the ventral pathway includes a number of cortical regions that appear to be hierarchically organized, beginning with the primary visual cortex (V1). This area projects to all other visual areas. The second level in the hierarchy is V2 (secondary visual area in the cerebral cortex), and this also projects to all other visual areas. There are three main projections from V2: to the parietal cortex in the dorsal pathway, and to the superior temporal sulcus and inferior temporal cortex in the ventral pathway (Kandel et al. 2000).

A number of studies suggest that the last exclusively visual stage of the ventral pathway is located in the inferior temporal cortex (see Logothetis and Sheinberg 1996 for review). The inferior temporal cortex extends ventrally from just anterior to the inferior occipital sulcus to within a few millimetres posterior to the temporal pole, and from the fundus of the occipito-temporal sulcus to the fundus of the superior temporal sulcus. This region includes Brodmann’s areas 20 and 21, or area TE, named by Von Bonin and Bailey (1947). Area TE was later subdivided into two cytoarchitectonically distinct cortical regions, TEO posteriorly and TE anteriorly (Iwai and Mishkin 1969; Von Bonin and Bailey 1950). Cortical regions, roughly corresponding to TEO and TE have also been shown to be functionally specialized. Lesions within TEO result in recognition deficits for simple patterns, while lesions within TE lead to associative and visual memory deficits (Iwai 1978; Iwai 1981; Iwai 1985).

Areas TEO and TE of the visual ventral stream project to many cortical and sub-cortical regions. Of particular interest are the interconnections between the inferotemporal cortex and parts of the medial temporal cortex, in particular, the hippocampus, the amygdala, and the entorhinal and perirhinal cortices. These structures, as will be discussed later, are all implicated in various aspects of memory. Area TEO receives feedforward cortical inputs from the secondary visual areas V2, V3, and V4, including

11 Neural correlates of object recognition memory

contralateral connections from these areas via the corpus callosum. These secondary areas also receive feedback projections from area TEO. Area TEO projects in a feedforward fashion to several other cortical areas, designated TEm, TEa, and IPa, all of which also send feedback projections back to TEO (eg. Rolls 2000; Van Essen 2002). Areas designated TH and TG, and Brodmann’s area 36 also provide feedback projections to TEO. Area TE projects to regions designated TH, TF, STP, FEF and area 46 (dorsolateral prefrontal cortex). TE also projects directly to the amygdala and to the hippocampus. In addition, the hippocampus receives an indirect projection from TE via the parahippocampal gyrus (eg. Rolls 2000; Van Essen 2002). There are also indirect projections from TE to the entorhinal cortex via the perirhinal and parahippocampal cortices. Area TE is also interconnected, both directly and indirectly, with limbic structures (Desimone and Duncan 1995; Kolb and Whishaw 1996; Logothetis and Sheinberg 1996 among others). Visual association areas of TE project to two prefrontal regions, one on the dorsolateral surface and one in the orbital region (Kolb and Whishaw 1996 p. 289). Areas TEO and TE also make connections with a large number of subcortical areas (see Webster et al. 1993).

A large proportion of the inferotemporal cortex responds selectively to shapes. In fact, more than 85% of neurons in the inferotemporal cortex appear to respond to simple or complex visual patterns (Desimone et al. 1984). Tanaka (1993) reported that the inferotemporal cortex consists of ‘elaborate’ cells that respond only to composite shapes. Unlike the primary visual cortex, the inferotemporal cortex does not appear to be organised retinotopically. Instead, inferotemporal neurons are systematically organised such that neurons with similar response properties are assembled into modules extending through the thickness of the cortex (Tanaka 1993). These modules appear to be tuned to respond to similar combinations of shapes and other stimulus characteristics. These findings, among others, suggest that the general class of an object is represented by the combined activity of different modules in the inferotemporal cortex, whereas fine discriminations are represented by differences in the activity of neurons within a single module. (For a more detailed description see Fujita et al. 1992; Gawne and Richmond 1993; Tanaka 1993; Young 1993)

The inferotemporal cortex, where the final stage of the ‘what’ visual pathway is located, lies adjacent to the medial temporal cortex, which has been shown to be important in

12 Neural correlates of object recognition memory

memory. In other words, the inferior temporal cortex plays a crucial role in identifying ‘what’ an object is, whereas the medial temporal cortex plays a crucial role in ‘remembering’ whether it has been seen before. The brain regions believed to be involved in visual recognition memory, particularly the medial temporal lobe, have been determined by examining the consequences of lesions in humans and of focally produced lesions in non-human primates. This will be discussed in the next section.

2.1.2 The temporal lobes and object recognition memory in human and other primates

Although the study of the neural basis of memory began in the mid-nineteenth century with the first descriptions of memory disorders (Ebbinghaus 1964), it wasn’t until the 1980s that memory was considered to consist of distinct components that depend on different brain systems (Schacter 1987; Schacter 1992; Schacter and Crovitz 1977). A distinction has been made between a capacity for the conscious recollection of facts and events (declarative memory) and non-conscious performance of previously learned behaviours (non-declarative memory) (Squire and Zola 1996 among others). This came about because amnesic patients with bilateral medial temporal lobe damage presented with deficits in recall or recognition, but performed normally on tasks requiring a capacity for skill and habit learning and for priming (Squire and Zola 1996). These neuropsychological findings indicated that the memory processes that were intact in this form of amnesia utilized brain regions other than those that were damaged. Visual recognition memory, which is considered to be one aspect of declarative memory, has been shown to be compromised with damage to the medial temporal lobes.

Perhaps the most famous account of declarative memory loss is that of Scoville and Milner’s patient HM (Scoville and Milner 1957). This patient had both medial temporal lobes surgically removed to alleviate his chronic epilepsy. Following this procedure, and to this day, this patient has exhibited anterograde amnesia. That is, memories prior to the temporal lobe resection remain intact, whereas no subsequent long-term memories have been established. In contrast, patient HM still retains non-declarative memory function.

13 Neural correlates of object recognition memory

Petri and Mishkin (1994) described neural systems for explicit and implicit memory based on human and animal studies. For explicit memory, the regions involved include, not only the temporal lobes, but also the prefrontal cortex, thalamus, basal forebrain and neocortex. In general, experiments in monkeys and rats have indicated that the rhinal cortex is involved in object memory, the hippocampus in spatial memory, and the amygdala in emotional memory. The differing effects of brain damage on HM and other amnesic patients are considered to result from damage to different combinations of connections in the Petri and Mishkin (1994) model.

In non-human primates, bilateral lesions to specific parts of the hippocampus (CA1 and CA2) resulted in impairments in a delayed non-matching to sample task with a delay of approximately 10 minutes (Zola-Morgan et al. 1992). In this task, a presented object must be selected if it fails to match an object that had been presented previously. Furthermore, the more extensive the lesion to the hippocampus and adjacent regions, the greater the impairment.

Analysis of a number of other monkey studies also suggested that the larger the lesion within the medial temporal lobe, the greater the memory impairment (Squire and Zola 1996). While damage confined to the hippocampus resulted in significant memory impairment, lesions that included the adjacent parahippocampal and entorhinal cortices in addition to the hippocampus resulted in greater impairment, and when the damaged region extended still further into the perirhinal cortex, the severity of memory impairment was even greater.

The extent of damage is not the only factor that determines the degree of memory impairment. Various studies have indicated that certain parts of the medial temporal lobe are more important in memory than others. Firstly, it appears that the entorhinal cortex, considered to be part of the hippocampal formation, is not essential for memory (Squire and Zola 1996 among others). Most of the sensory input to the hippocampus passes through the entorhinal cortex, while the entorhinal cortex receives a large proportion of its input from the perirhinal and the parahippocampal cortices (see Squire and Zola 1996). When only the entorhinal cortex is damaged, only mild memory impairment results (Meunier et al. 1993). Furthermore, when monkeys with entorhinal lesions were re-administered delayed non-matching to sample memory tests 6-13

14 Neural correlates of object recognition memory

months later, they were found to perform normally at all delay intervals (Leonard et al. 1995). Also, localized lesions to the amygdala did not appear to affect performance in the delayed non-matching to sample task (Zola-Morgan et al. 1989).

On the other hand, it appears that the perirhinal and parahippocampal cortices, structures adjacent to the hippocampus and entorhinal cortex, are particularly important in memory. Damage to these regions results in a lasting behavioural impairment in the delayed non-matching to sample task. The degree of impairment came close to that occurring with a larger medial temporal lobe lesion that includes the perirhinal and parahippocampal cortices as well as the hippocampus and the amygdala (Squire and Zola 1996).

In fact, there are many pathways to the medial temporal lobe from other parts of the neocortex in which small specific lesions can affect performance on specific memory tasks (Squire and Zola 1996). Of particular relevance to this study is the fact that the perirhinal cortex receives greater visual input than does the parahippocampus, a finding determined using retrograde tracer techniques (Suzuki and Amaral 1994; Webster et al. 1991). It was found that lesions within the perirhinal cortex affected visual memory more so than lesions at any other single site within the medial temporal lobe (Horel et al. 1987 among others; Meunier et al. 1993). More recently, Parker and Gaffan (1998) demonstrated with monkeys that the frontal lobe must interact with the perirhinal cortex in the same hemisphere for ORM to occur. There are many pathways, both direct and indirect from the perirhinal cortex to the frontal lobes. When specific points in a number of these pathways were ablated, ORM was seriously impaired.

Area TE, in the inferior temporal lobe, is a unimodal visual area situated adjacent and lateral to the perirhinal cortex (Von Bonin and Bailey 1947), and was mentioned earlier as being important in ORM as well as in object perception. In monkeys, lesions within this area resulted in impaired visual perception as measured, for example, by pattern discrimination, and in impaired performance on the delayed non-matching to sample task. With this latter task, neurons in the inferotemporal cortex continued to discharge during the ‘memory’ period for objects that had to be remembered (Fuster and Jervey 1982 p.298). It was subsequently suggested, however, that in virtually all previous studies that described impairment resulting from lesions within TE there was also

15 Neural correlates of object recognition memory

damage to the perirhinal cortex (see Squire and Zola 1996). Squire (1996) showed that the function of TE and the perirhinal cortex can be dissociated. When monkeys were tested on various tasks, prior to and following lesioning, it was found that the perirhinal region is particularly important in declarative memory, whereas TE in the inferior temporal lobe is more important in visual perception.

The perirhinal cortex is located at the interface of TE and the medial temporal lobe. While precisely located bilateral lesions can be made in monkeys, damage to medial temporal lobes in humans is usually confined to one hemisphere and varies in size and location. Nevertheless, despite a lack of consistent human neuropsychological data, there have been some important observations confirming that regions already described as being important in ORM in non-human subjects are also important for humans. As mentioned earlier, patient HM had undergone bilateral medial temporal lobe resection, which included removal of the entire temporal lobes. Scoville and Milner’s (1957) initial interpretation was that damage to the hippocampus was responsible for his anterograde amnesia. However, subsequent experiments on monkeys and rats have indicated that the hippocampal damage would account for only the spatial impairments displayed by HM. Other medial temporal lobe regions such as the amygdala, which is responsible for emotional memory, and the perirhinal cortex, responsible for object recognition memory, also appear to be responsible for his impairment (Kolb and Whishaw 1996).

An analysis of two studies that described neuropsychological changes in patients with damage to the medial temporal lobe has indicated that the more extensive the damage, the greater the memory impairment (Squire and Zola 1996). All the patients were less severely impaired in tests of declarative memory than HM, and the damage to hippocampal and adjacent cortical regions was also less extensive than in HM. All presented with moderately severe anterograde amnesia and two with extensive retrograde amnesia also. All had damage to the CA1 area of the hippocampus. Those whose damaged areas included the entorhinal and perirhinal cortices showed the most severe memory impairment.

Buffalo et al. (1998) examined recognition memory for complex visual stimuli in two patients with extensive damage to the perirhinal cortex, and in six other amnesic

16 Neural correlates of object recognition memory

patients with damage confined to the hippocampus and other diencephalic structures. Immediate and long-term memory were tested separately, using delays from 0 to 40s in a delayed recognition memory task. They found that both patient groups exhibited intact recognition memory at delays of 0 to 2s, and both groups also displayed delay- dependent memory impairment for delays greater than 6s. Furthermore, with delays greater than 25s, the performance of the two patients with damage to the perirhinal cortex was worse than the performance of the other amnesic patients. It was thus concluded that the perirhinal cortex is not important for visual perception or immediate memory. However, like the other medial temporal lobe structures, the perirhinal cortex appears to be involved in longer-term memory processes.

It is now thought that the two temporal lobes have their own specialized functions. Milner (Milner 1958; Milner 1968; Milner 1970) found that in patients who had undergone temporal lobe removal, the specific types of memory deficit depended on the side of the lesion. Patients with damage to the left temporal lobe showed deficits in verbal memory in, for example, tests of recall of previously presented stories and pairs of words and on recognition of words or numbers (see Kolb and Whishaw 1996). Patients with right temporal lobe lesions showed deficits in non-verbal memory. These patients found it difficult to recall complex geometric figures, recognize nonsense figures, recognize tunes, and to recognize previously viewed photographs of faces (see Kolb and Whishaw 1996 p. 369). Kolb and Whishaw (1996 p.291) cautioned that although it appears that the two temporal lobes appear to have specialized functions, there is also a high degree of functional overlap. While an association of the left temporal lobe with verbal memory appears to be reasonably well accepted, the precise role of the right temporal lobe in memory is less clear.

Neuropsychological investigations with humans and other primates have thus indicated that the inferotemporal cortex and the medial temporal lobes, and in particular, the perirhinal cortex, are important in visual recognition memory. Furthermore, these areas project to other cortical and sub-cortical regions such as the frontal lobes, which need to be intact for preserved memory functioning.

17 Neural correlates of object recognition memory

In recent years, a number of functional neuroimaging techniques have been used with humans in attempts to localize more precisely the neural substrates involved in ORM. Studies using these techniques will be summarized in the following section.

2.1.3 A brief summary of haemodynamic (PET and fMRI) neuroimaging correlates of episodic memory retrieval

Neural regions involved in memory retrieval processes have been identified in humans using functional neuroimaging techniques such as positron emission tomography (PET) and functional magnetic resonance imaging (fMRI). These techniques have identified neural regions involved in memory retrieval processes through the monitoring of cerebral blood flow changes while subjects are engaged in tests of memory retrieval. While these techniques possess excellent spatial resolution, temporal resolution is relatively poor. Notwithstanding this limitation, important findings have included the identification of neural regions that are considered to be involved in the maintenance of retrieval mode, that is, those regions that remain active for the duration of a retrieval task, and that are considered to facilitate successful memory retrieval.

The premise underlying PET and fMRI neuroimaging techniques is that regional increases in blood flow or oxygenation reflect local neural activity. In the case of PET, blood-flow changes are assessed by monitoring the uptake of a short-half-life radiotracer injected into the bloodstream. Functional magnetic resonance imaging techniques assess blood flow changes through changes in blood oxygenation. Shortly after a neural region is activated, the relative concentration of oxy-haemoglobin increases and deoxyhaemoglobin decreases, producing an overall increase in the fMRI signal. For both of these techniques, image reconstruction methods are employed to localize these changes to within a few millimetres (eg. Schmitt et al. 1998).

While the spatial resolution of these techniques is excellent, the temporal resolution is relatively poor. Typically, in the case of PET, an image can only be formed reliably every 20 to 30s. For this reason, PET studies have only been used to assess task-related changes. Episodic memory retrieval studies have used blocked task designs in which two separate task blocks are presented. One task block constitutes the target task and the

18 Neural correlates of object recognition memory

other the reference task. Neural regions showing increased activity during the target task relative to the reference task are normally termed activations. These activations are considered to reflect cognitive processes that are utilized to a greater extent by the target task than by the reference task. Functional MRI techniques can, in theory, achieve a temporal resolution better than 1s. However, a haemodynamic lag (time for local uptake of blood) of around 6s cannot be measured accurately and varies depending on the neural region being examined. Consequently most fMRI-based episodic retrieval studies to date have used block designs similar to those used in PET studies. While the recently developed event-related fMRI technique can achieve a temporal resolution better than 1s (eg. see Menon and Kim 1999), the present author is not aware of any episodic memory retrieval studies in which they have been used. It must however be noted that while activations observed using a blocked design may well be indicative of modal activity, we have to be aware that summed responses to individual items may also contribute to what appears to be sustained activity.

Functional neuroimaging studies have mainly used tests of episodic memory retrieval to identify neural regions involved in ORM. Episodic memory retrieval refers to the recall of events in one’s personal past (Tulving 1993) and is generally considered to be one aspect of declarative memory. Regions involved in episodic memory retrieval include the medial temporal lobe and frontal regions.

Functional neuroimaging studies have, in the majority of cases, examined episodic memory retrieval using verbal stimuli, and in only a few instances have they used objects. A very brief summary of PET and fMRI studies into episodic memory retrieval, with both verbal and non-verbal stimuli, will follow. A more extensive discussion of this body of literature can be found in a selection of review articles (Buckner and Tulving 1995; Cabeza and Nyberg 2000; Nyberg et al. 1998b; Rugg and Wilding 2000). In a recent publication by Cabeza and Nyberg (2000) the authors reviewed 275 PET and fMRI studies, of which, a large number were concerned with episodic memory retrieval. This paper summarizes findings for all aspects of episodic memory retrieval that have been investigated using PET and fMRI techniques to the year 2000 and clarifies the consensus of opinion on a number of issues. The section in this paper on episodic memory retrieval is the main source of information for the following discussion of PET

19 Neural correlates of object recognition memory

and fMRI findings (remainder of section 2.1.3). The discussion focuses mainly on neural regions involved in retrieval success, retrieval effort, and retrieval mode.

Activations have been noted consistently in seven main regions in PET and fMRI studies dealing with episodic retrieval: prefrontal, medial-temporal, medial parieto- occipital, lateral parietal, anterior cingulate, occipital, and cerebellar regions.

Although bilateral frontal activations are sometimes observed, right prefrontal activations have been a consistent finding in PET and fMRI studies. Moreover, the majority of these studies have indicated a role for these regions in establishing and maintaining the mental set for episodic retrieval, or retrieval mode. Right prefrontal (BA10) activation reflecting retrieval mode is usually observed by contrasting an episodic retrieval task with a non-episodic retrieval task. Other studies have associated bilateral activation of prefrontal regions (BA 10, 9, 46) with processes reflecting retrieval success, and left prefrontal regions (BA 47, 10) with retrieval effort.

Although medial temporal activations are lateralized during episodic encoding, they occur bilaterally during episodic retrieval. Importantly, these medial temporal lobe activations occur bilaterally regardless of whether stimulus items are verbal or non- verbal. Medial temporal lobe activations have been associated with retrieval success, but not with retrieval mode or retrieval effort. It was therefore suggested that activation of the medial temporal lobes reflects successful memory retrieval, given also the findings of one study in which a strong relationship between medial temporal lobe activation and recognition accuracy was reported. Activation of the medial temporal lobes has also been associated with reactivation of stored memory representations and with conscious recollection.

Episodic memory retrieval tasks have also produced activation of medial parieto- occipital regions. Activation of the precuneus, located within this region, has been associated with imagery operations and retrieval success. Lateral parietal activations have been associated with the processing of spatial information during episodic retrieval, and with perceptual aspects of recognition. Activation of the occipital cortex has been associated with non-verbal retrieval involving memory-related imagery

20 Neural correlates of object recognition memory

operations, and activation of the cerebellum has been associated with self-initiated retrieval operations.

A comparison of the retrieval of object location relative to object identity yielded activation of inferior parietal regions. The opposite comparison (object identity minus object location) yielded activation of fusiform regions. This result suggests that the distinction between the dorsal and ventral pathways proposed by Ungerleider and Mishkin (1982), via which, respectively, an object’s location and identity are recognized, also applies to episodic retrieval.

2.2 Human electrophysiological correlates of episodic memory retrieval

Findings from neuropsychological and functional neuroimaging studies in humans have provided important information about brain regions involved in recognition memory (section 2.1). Furthermore, functional neuroimaging studies using PET and fMRI techniques have shown that seven main regions (as discussed in section 2.1.3) are active while subjects perform memory retrieval tasks. While these techniques can locate activated neural regions relatively accurately, information regarding the timing of these activations is limited. The majority of studies have, in fact, used PET methods, which are suitable for the investigation of sustained modal activity, that is, activations maintained for the duration of the imaging period. Such studies, while contributing to the understanding of memory processes such as retrieval mode, do not necessarily indicate which neural regions are transiently activated during successful retrieval. However, electrophysiological techniques would appear to possess the required temporal resolution required for monitoring these fast memory processes, albeit with a relatively poor spatial resolution. These techniques have been used to investigate the spatio-temporal patterns of brain electrical activity associated with the retrieval of items from memory.

While the main focus of this study is ORM, few electrophysiological studies, so far, have focused on ORM processes as such. Human electrophysiological studies have dealt more extensively with the retrieval from memory of verbal stimuli and therefore involve processes specific to humans. Retrieval of objects from memory has also been studied

21 Neural correlates of object recognition memory

using electrophysiological techniques, although the number of studies is far fewer. The retrieval of verbal and object items has mainly been investigated using tests for episodic memory retrieval. Episodic memory retrieval is defined as the explicit recollection of events in one’s personal past (eg. Tulving 1993). Studies that discuss findings in terms of episodic retrieval have not always been designed so that processes involving recollection are specifically investigated. In fact, many experimental designs may promote the tapping into an unconscious form of memory (implicit memory), in which item retrieval is based on familiarity, rather than on recollection. This does not necessarily mean, however, that familiarity processes involve different neural regions (although see Klimesch et al. 2001b, for example). The term episodic memory retrieval will therefore be used in a broad sense in the following review, indicating retrieval from some form of longer-term memory. Recognition memory may be a more useful term than episodic memory retrieval because it does not differentiate between whether items were recognized on the basis of familiarity or recollection. Recognition memory and episodic retrieval are terms that will be used interchangeably. The following review will therefore examine human electrophysiological correlates of episodic memory retrieval for verbal and non-verbal material.

Electrophysiological methodologies will be outlined in section 2.2.1. EEG correlates of episodic memory retrieval are reviewed in section 2.2.2. In section 2.2.3, ERP correlates of episodic memory retrieval are discussed. This section is sub-divided into three parts. Sections 2.2.3.1 and 2.2.3.2 contain the bulk of the ERP review and focus on transient item-related effects observed using two different experimental task designs, the continuous recognition design and the study-test design. Section 2.2.3.3 is a small section that deals with one study that used the DC-ERP technique to investigate sustained task-related effects.

2.2.1 EEG and ERP functional electrophysiological techniques: a brief background

In 1929, when Berger recorded electrical activity from the scalp of a human subject, he noted that a component of the EEG in the 8-12Hz band was present when the subject's eyes were closed but disappeared with they were open (Berger 1929). This frequency

22 Neural correlates of object recognition memory

band is now known as the alpha band. He subsequently showed various reliable correlations between the alpha power and the level of attention (Gloor 1994). Berger’s major finding was that increased task demand attenuates the alpha power. In so doing, Berger had shown that the EEG was potentially a powerful tool for studying brain function; however, it took many decades before the true value of this was realized. More recently, computers have permitted the analysis of vast quantities of data, which hitherto had not been practicable. In particular, the use of multi-channel recordings with large arrays of electrodes has resulted in greatly improved spatial resolution. These advancements in technology have therefore made the exploration of cognitive activity using EEG methods more feasible.

The alpha rhythm first described by Berger (1929) is the most prominent feature of the normal EEG. This rhythm, however, represents only a small proportion of the total EEG frequency spectrum. The EEG is usually discussed over the 1Hz to approximately 80Hz range (Regan 1989). The main frequency bands of interest have traditionally been defined as slow wave delta and theta, from 1 to 3Hz and 4 to 7Hz respectively, alpha from 8 to 12Hz, and fast beta from 12 to 30Hz. These classifications have been made to simplify comparisons of power spectra between subjects and subject groups. The boundaries of these frequency bands vary somewhat between laboratories, and bands may be further broken down into sub-bands. For example, beta can be separated into beta1 and beta2, representing the lower and upper regions of the beta band. More recently, however, because of significant inter-subject variation, frequency bands have been defined for individual subjects based on the individual’s alpha frequency (IAF). Other EEG frequency components are then ‘anchored’ to the IAF (see Klimesch et al. 1993; Klimesch et al. 1994 for more details).

Dawson (1947) first reported that distinct electrical responses produced by external stimulation could be extracted from the EEG. It was subsequently found that similar responses could also occur without an external stimulus, instead occurring with internally generated events. These event-related potentials (ERPs) are very difficult to observe in the EEG as amplitudes are typically between 1/100th to 1/10th of background (Duffy and McAnulty 1988). To achieve a satisfactory signal to noise ratio, signal averaging is usually applied to the recorded EEG. Multiple individual responses (20-2000) may be summed to obtain a waveform representative of the averaging period

23 Neural correlates of object recognition memory

(see Regan 1989, for review). This type of analysis is ideal for observing millisecond changes that occur during the processing of a stimulus, assuming that the processes being monitored do not change over the averaging period (Silberstein et al. 1990a).

ERP components that occur less than 200ms after stimulus onset are usually regarded as exogenous components generated by stimulation of the senses and related to brain activity associated with perceptual processes. For example, clicks to the auditory system, flashes to the visual system and electric shocks to peripheral nerves will produce exogenous potentials (see Regan 1989, for review). Exogenous ERPs are used clinically to test the integrity of these sensory systems. For example, visually evoked potentials to a checkerboard stimulus can be used to test optic nerve lesions associated with multiple sclerosis (eg. see Halliday et al. 1972). ERPs with longer latencies are usually generated internally by events, such as, for example, surprise, detection of an infrequent stimulus, or a remembered item. Unlike the earlier potentials, these later components are independent of the evoking stimulus and represent information processing after the initial sensory volley. These late potentials have therefore been associated with cognitive processes (Sutton et al. 1965).

ERPs, therefore, consist of a series of electrical potentials that vary in amplitude and latency, and are thought to represent the activity of brain regions involved in processing exogenous or endogenous stimuli (eg. Picton et al. 1995; Regan 1989). For example, the endogenous P3 or P300 potential (Sutton et al. 1965), which has been extensively investigated, is a target detection response, and has been linked to attentional and memory processes (Iragui et al. 1993; Pritchard 1981). This potential is termed P3 or P300 because it is positive with respect to the reference electrode (eg. Picton et al. 1995; Regan 1989) and occurs around 300ms after the evoking stimulus has been presented. Manipulation of attentional and memory parameters have been shown to alter both the amplitude and latency of the P3 potential (eg. Dujardin et al. 1993).

Electrical signals recorded at the scalp reflect the average activity of synchronously active neural populations underlying the scalp (eg. Kutas and Dale 1997). However, only EEG/ERP components generated by neural regions with the dipole orientated perpendicular to the skull can be detected at the scalp (Picton et al. 1995). Moreover, the CSF, skull and scalp act to smear the activity generated in underlying neural areas (eg.

24 Neural correlates of object recognition memory

Nunez 1981). Therefore, these techniques permit only gross localization of neural activity, which must be discussed at the level of, for example, right pre-frontal, temporo-parietal, occipital regions, etc. These gross patterns of activity have nevertheless proven useful in differentiating between spatio-temporal patterns evoked by different classes of experimental stimuli. Compared with other imaging techniques, EEG/ERP methodologies are popular because of the relatively low cost and the convenience associated with a non-invasive procedure. Thus recordings in normal and clinical subjects can be conducted comparatively easily and frequently (Johnson 1995).

A major limitation of these electrophysiological methodologies, however, is that they do not permit the kind of three-dimensional, high spatial resolution imaging that can be obtained using fMRI or PET. However, techniques exist (eg. Scherg 1990) that utilize a large number of electrodes (at least 64) to help locate underlying neural generators from scalp data. This is known as the inverse problem (eg. Nunez 1981).

2.2.2 EEG changes associated with episodic memory retrieval

In addition to a decrease in alpha amplitude with mental activity, Berger (1929) also observed a frequency increase. Martinson (1939) made similar observations. Moreover, alpha activity was found to be selectively reduced in the hemisphere that is dominant in performing a specific type of task (Adrian and Matthews 1934). More recently, these early findings have been corroborated (Donchin et al. 1977; Kinsbourne and Hiscock 1983) and extended (Osaka 1984). Osaka (1984) showed that alpha frequency increases more for difficult tasks than for easy tasks, and only in the hemisphere engaged in the task.

Pfurtscheller and his colleagues confirmed these aforementioned characteristics of alpha band power and frequency changes. In addition, they observed transient alpha attenuation (desynchronization) that appeared to reflect regional increases in cortical activity associated with the performance of cognitive or motor tasks (eg. Pfurtscheller and Aranibar 1977b; Pfurtscheller and Klimesch 1990; Pfurtscheller and Klimesch 1991).

25 Neural correlates of object recognition memory

Early studies following Berger’s observations had failed to find any correlations between alpha characteristics and memory processes, despite the many accounts in the literature of the association between alpha activity and mental activity (see Klimesch et al. 1993). In 1985, however, Saletu and Grünberger (1985) found a positive correlation between alpha frequency and memory performance, although the authors could not rule out the possible confounding influence of vigilance; that is, good performers may have been more attentive.

Klimesch et al. (1990b), obtained a similar positive relationship between mean alpha frequency and memory performance. They later addressed the possible confounding influence of attention on memory performance in a study where both memory and attentional demands were varied (Klimesch et al. 1993). A clear relationship between the individual alpha frequency (IAF) and memory performance was observed. Good performers exhibited an IAF 1.25Hz higher than poor performers during the retrieval period of a short-term memory task. Increased attentional and task demands also tended to reduce IAF although to a much lesser degree. Furthermore, poor performers showed a greater attenuation of power at this IAF than did good performers.

In a more recent study, Klimesch and colleagues (1994), found that the EEG produced during the retrieval part of a semantic memory task differed from the EEG produced during the retrieval part of an episodic task. Semantic memory was defined as ‘pure long-term memory’ by the authors, and has been described as a store of impersonal knowledge in the broadest sense (eg. knowledge about language or geography) (Doppelmayr et al. 1998). Episodic memory was defined as the intentional or incidental storage of ‘new information’ that can subsequently be retrieved. Klimesch et al. (1994) used two retrieval tasks (Kroll and Klimesch 1992) to examine firstly, semantic and secondly, episodic memory. Subjects were firstly presented with a semantic congruency task in which they had to decide whether sequentially presented ‘concept-feature’ pairs such as ‘eagle-claws’ or ‘canary-blue’ were semantically congruent or incongruent. The second task tested episodic memory. In this, subjects were presented with new concept- feature pairs interspersed with word pairs identical to those presented in the preceding semantic task, and were required to identify those pairs that had been presented in the semantic congruency task. Subjects were not informed beforehand that they would be performing this subsequent recognition task so that they would not be inclined to use

26 Neural correlates of object recognition memory

semantic encoding strategies. In this way episodic memory demands were maximized (Klimesch et al. 1994). Because both semantic and episodic retrieval processes would have occurred only after the feature word (second item in the pair) was presented, only the EEG recorded after this time point would have reflected memory retrieval processes.

Klimesch et al. (1994) predicted that the cognitive demands of the semantic memory task would be reflected in a decrease in alpha power. They also noted that a number of converging lines of research suggested that episodic retrieval would be characterized by theta band changes. Firstly, the late positive component (LPC) of event-related brain potentials has been associated with memory processes (eg. Fabiani et al. 1990 among others), and the frequency decomposition of these potentials reveals predominantly theta frequencies (Klimesch et al. 1994). Therefore, these theta band changes might reflect episodic memory processes. Secondly, the hippocampus has been shown to be involved in the generation of the P300 (eg. Halgren et al. 1980), and episodic memory is compromised with lesions to the hippocampal formation (eg. Scoville and Milner 1957). Therefore, this area may be focal in episodic memory processing. Finally, theta band activity has been shown to be the dominant frequency of the hippocampal formation (eg. Arnolds et al. 1980). This hippocampal theta activity could be transmitted to the scalp via hippocampo-cortical longitudinal pathways (Klimesch et al. 1994).

Klimesch et al. (1994) found that semantic memory processes were associated with an attenuation of upper alpha band power. This was consistent with a previous study (Klimesch et al. 1992) which showed that lower alpha band power was related to non- task related activity such as attention, whereas upper alpha band power was related to cognitive processes. In contrast, episodic memory retrieval was reflected by power changes in the theta band and not in the upper alpha band. Increased theta band power, or synchronization, was significant only in the episodic task, and only after the feature word was presented. This dissociation in EEG frequency bands could not have been due to task difficulty because alpha attenuation would then have been greater for the more difficult episodic memory task, and this was not the case. The greater alpha attenuation was exhibited during the semantic congruency task. The authors therefore speculated that different EEG frequencies reflect different cognitive processes (Klimesch et al. 1994). Klimesch (1996) suggested that short-term episodic memories are reflected by

27 Neural correlates of object recognition memory

oscillations in the anterior limbic system, and long-term semantic processes are reflected by oscillations in a posterior-thalamic system.

Klimesch et al. (1997) investigated the relationship between theta band power and episodic encoding and retrieval. Their task involved the intentional memorization of 96 words presented for 1s each, followed by the subsequent recognition of these words presented amongst 96 semantically similar distractor words (Kroll and Klimesch 1992). During the encoding phase, only words that were later correctly recognized produced a significant increase in synchronization of theta activity. Similarly, in the recognition task, only correctly identified target words produced a significant increase in theta band power. These increases in theta band power were most pronounced at frontal sites and smallest at occipital sites. Target words that were not remembered and distractor words that were correctly identified did not produce any increase in theta power. During the encoding phase, theta band synchronization appeared to be mirrored by lower alpha band desynchronization. That is, words that were later remembered produced a marked desynchronization at lower alpha band frequencies. Words that were not subsequently remembered produced a significant desynchronization in the upper alpha band. Therefore, a reduction in lower alpha band power for later remembered words was consistent with previous findings by this group, thus supporting their proposal that lower alpha band power reflects increased attention. It would be reasonable to assume that subjects had paid more attention to those words that were later remembered. The authors also argued that the upper alpha desynchronization observed with words that could not subsequently be remembered was due to a triggering of semantic associations that had produced a detrimental effect on recognition performance. Alpha band desynchronization in general was most pronounced at occipital sites and smallest at frontal sites. As already mentioned, theta band synchronization was largest frontally and smallest occipitally.

The same task design was used in a more recent study by Doppelmayr et al. (1998). The aim of this investigation, however, was to examine the relationship between EEG band power and performance in an episodic retrieval task. Band power was observed over the interval between 500 and 1000ms after the presentation of correctly identified words. Both high and low performance subject groups showed significant theta synchronization at all recording sites. Furthermore, high performance subjects showed a higher level of

28 Neural correlates of object recognition memory

theta synchronization in the right hemisphere than did poorly performing subjects. Greater right hemisphere synchronization of theta band power for high performance subjects was described by the authors as being consistent with the selective activation hypothesis (Ibatoullina et al. 1994), wherein high performance subjects show a more localized and restricted pattern of activation. The authors suggested that poorly performing subjects utilized additional resources in the left hemisphere to increase their retrieval accuracy, resulting in a more diffuse pattern of theta band synchronization. It was suggested that results for high performance subjects are consistent with Tulving’s HERA (hemispheric encoding/retrieval asymmetry) model (Tulving et al. 1994a). In the HERA model, the left hemisphere is postulated to be involved in episodic encoding, whereas the right hemisphere is involved in episodic retrieval. Most PET studies have indicated that right hemisphere activation during episodic retrieval is restricted to the pre-frontal cortex (for recent review see Nyberg et al. 1996a). However, Doppelmayr et al. (1998) observed significant theta synchronization at all right hemisphere sites. They suggested that the majority of PET studies may have failed to show activation at other right hemisphere sites because the control task used may have produced a pattern of activation similar to that of the main task at these sites. This would have reduced any differences between episodic retrieval and control task activations. In fact, a small number of PET studies have shown additional involvement of right hemisphere central and parietal sites in episodic retrieval (Nyberg et al. 1996a; Tulving et al. 1994b).

Doppelmayr and colleagues (2000) also used a similar recognition paradigm to investigate the relative contributions to theta band power of time-locked evoked and non-time locked induced theta components, and the relationship of these components to memory performance. The authors found that the early component of the post stimulus interval (<400ms) is dominated by an evoked component, whereas the late part (>400ms) is dominated by the induced theta component. Induced theta for hits was significantly larger than for correct rejections for the high performance subjects only. Theta ERPs were analysed separately in an attempt to investigate absolute band power and possible group differences in the time locked theta response. The authors found that the theta ERP was much larger for the high performance than the low performance subject groups. For the high performance group, there was also a correspondence in time between peaks of the theta ERP and the standard ERP (coincident with the P3 ERP). They suggested that for high performance subjects, theta peaks occur in a

29 Neural correlates of object recognition memory

‘preferred time window’ after the target is presented, and given that they occur maximally when the P3 can be observed, act as an orienting response that is necessary for successful episodic recognition. The authors also suggested that the induced theta component that occurs much later, and is coincident with the late positive component, is related more to the actual processes of episodic retrieval.

The relationship between induced theta band power and the P3 component was further highlighted in a study (Klimesch et al. 2000) wherein a memory–related increase in ERP positivity appeared to be functionally related to a significant increase in both theta and delta induced band power. Both timing (375-750ms) and topography (largest at left occipital sites) of these phenomena were similar. The authors suggested that hippocampal theta activity, transferred to the cortex via limbic-hippocampal-cortical re- entrant loops, generates the induced theta power and occurs simultaneously with a transient de-activation of cortical regions which generates the P3 component (see Klimesch et al. 2000, for further detail).

Studies reviewed thus far where EEG band power has been used to investigate episodic retrieval have employed verbal stimuli. Klimesch et al. (2001a) used pictures instead of words in a task design similar to that used by Klimesch et al. (1997). Once again, theta band changes associated with episodic encoding and retrieval were examined. In this study, however, changes occurring within the 250 to 750ms interval post stimulus onset were examined, rather than within the entire 1s interval of Klimesch et al. (1997). The reduced time frame was employed because this interval has been shown to be important for memory sensitive ERP components (eg. see Rugg et al. 2000, for review). Klimesch et al. (2001a) attempted to determine whether theta synchronization during retrieval is related to the access of a stored code or to more general processes. Furthermore, by using pictures and comparing findings with those of earlier studies of Klimesch and collegues (Klimesch et al. 1997; Klimesch et al. 1994), it should have been possible to determine whether theta band power during encoding and retrieval is different for pictures and words and therefore stimulus dependent. Theta synchronization was greater for retrieval than for encoding; however, there was no significant difference between hits and correct rejections. Because the degree of theta synchronization for hits and correct rejections was not significantly different, the authors suggested that theta synchronization during retrieval is related to the attempt to retrieve, rather than to the

30 Neural correlates of object recognition memory

actual access of, a stored memory trace. The authors also concluded that theta changes during encoding and retrieval do not depend on the nature of the stimulus. However, the magnitude of theta synchronization during retrieval was several times greater than that reported in previous studies utilizing verbal material. As greater theta synchronization has been linked to better performance (Doppelmayr et al. 1998), the authors argued that the greater theta synchronization for pictures was related to the fact that picture memory is superior to verbal memory.

Theta changes were also assessed in a word-based retrieval task in which the level of conscious awareness was manipulated (Klimesch et al. 2001b). Subjects made remember/know judgements by indicating whether they recollected, or remembered, the specific event of word presentation, or whether they recognized, or knew, a word based on familiarity alone without recollection of the event. Theta synchronization was observed for both ‘remember’ and ‘know’ judgements, but in different temporal windows. Theta synchronization and a more positive ERP (see section 2.2.3) was associated with ‘knowing’ in the 300 to 450ms interval post onset, and with ‘remembering’ in the 450 to 625ms interval. The authors suggested that knowing based on familiarity is associated with activity within the perirhinal cortex and medial dorsal nucleus of the thalamus, whereas remembering, which occurs later, is associated with hippocampal activity.

In summary, episodic retrieval of verbal stimuli is associated with increased synchronization in the EEG theta band, particularly at frontal sites. Moreover, theta synchronization appears to be related to task performance; high performance subjects show greater theta band synchronization in the right hemisphere than do low performance subjects. These theta band changes appear to be specific to episodic memory retrieval as they do not occur with semantic retrieval. Semantic retrieval, in contrast, is associated with upper-alpha band desynchronization. However, this theta band finding has more recently been questioned in a study indicating that theta synchronization reflects the attempt to retrieve, rather than the actual access of stored information. Notwithstanding a number of important studies that have indicated a relationship between EEG theta band power and episodic memory retrieval, the neural regions involved with generating and modulating theta changes, and the precise

31 Neural correlates of object recognition memory

relationship between theta band power and episodic memory retrieval processes remains unresolved.

2.2.3 Event-related potential changes associated with episodic memory retrieval

Compared to PET and fMRI imaging techniques, the superior temporal resolution of ERP techniques has allowed investigation of faster neural changes associated with successful memory retrieval. These transient neural changes have been investigated using a number of experimental designs and different stimulus types. Most studies that will be reviewed used scalp electrodes, although a smaller number of clinical studies have used depth electrodes located within the brain. In most cases ERPs elicited by previously studied and unstudied items (ERP old/new effect) are compared in attempts to determine the neural regions associated with successful retrieval processes. This type of approach has shown consistent parietal and right frontal ERP differences within certain time intervals following item presentation.

Sections 2.2.3.1 and 2.2.3.2 contain the bulk of the review of ERP studies and focus on transient item-related effects observed using two different experimental task designs, the continuous recognition design and the study-test design. Section 2.2.3.3 is a small section that deals with one study that used the ERP technique to investigate sustained task-related effects.

2.2.3.1 ERP correlates of episodic memory retrieval: evidence from continuous recognition paradigms

This section deals with studies utilizing the continuous recognition experimental task design to investigate ERP correlates of episodic memory retrieval. With this design, single stimulus items, such as words or pictures, are presented in a continuous sequence, and subjects are required to identify those items that are presented for the first time (new) and those that are repeated (old). Correct identification of repeated items presumably taps into some form of recognition memory. This type of experimental design has traditionally been used to explore the transfer from immediate memory, also

32 Neural correlates of object recognition memory

known as primary memory, to short or long term memory, also known as secondary memory. This is done by varying the time lag between repeated items and observing the change in various behavioural parameters, for example, the decay of recognition accuracy (Friedman 1990a). Findings from continuous recognition studies are reviewed separately from findings from study-test studies (section 2.2.3.2) as there are distinct differences in the two experimental paradigms (eg. see Friedman 1990a; Johnson 1995, for review). The main difference lies in the separation of encoding and retrieval phases in the study-test paradigm. These phases occur concurrently in the continuous recognition paradigm. This is a particularly important distinction when sustained, task- related processes (see sections 2.2.3.3 and 2.2.4) are being investigated. In the continuous recognition paradigm each new stimulus must be encoded for possible later recognition. This intermixing of retrieval and encoding processes is therefore a confounding factor when investigating sustained task effects associated solely with retrieval. Furthermore, while repetition lags of less than 1 minute are common in continuous recognition tasks, intervals between encoding and retrieval phases of approximately 5 minutes are usual in study-test tasks. It has been suggested that ERPs elicited using short repetition lags may not reflect longer-term episodic memory. Puce et al. (1991) suggest that a lag greater than 45s is necessary for transfer into long-term memory. However, definitions and models of memory are varied, and there are many other variables such as rate of presentation, familiarity, distraction, age, etc. (eg. Baddeley 1999) that may be important in considering the exact types of memory processes being accessed.

A review of studies that have used continuous recognition tasks to investigate neural correlates of episodic memory retrieval follows. A review of studies where ERPs have been elicited by verbal stimuli will firstly be presented, followed by a discussion of their possible relevance to episodic memory retrieval. Attempts have been made to distinguish between the ERPs generated in the continuous recognition paradigm and other ERPs, such as the P300, which have been associated with other cognitive processes. A small number of studies that employed both scalp and depth electrode recordings in epileptic patients, and utilized pictorial stimuli of both a verbalizable and abstract non-verbalizable nature, will then be reviewed.

33 Neural correlates of object recognition memory

Early studies utilizing the continuous recognition paradigms found differences between ERPs elicited by new and repeated verbal stimuli (eg. Johnson et al. 1985; Sanquist et al. 1980). The main finding reported in these studies is that ERPs elicited by repeated items were more positive than those elicited by new items. It was suggested that this was due to an enhancement of the P3 and related late positive components (eg. Neville et al. 1986).

Smith et al. (1986) described a negative-positive ERP complex, N400-P600, which was originally evoked during a verbal study-test task, and was also reported in a number of continuous recognition studies (eg. Nagy and Rugg 1989; Rugg and Nagy 1989; Smith and Halgren 1988). In all of these studies, compared with the presentation of new items, repeated stimuli produced an attenuated N400 component and an enhanced P600 component. These differences in ERP components generated by new and repeated items is known as the ERP old/new effect, or sometimes the ERP repetition effect. Rugg (1990) referred to the modulation of the N400 potential, occurring between 300 and 500ms, as the ‘early’ ERP old/new effect, and changes to the P600 (P3), occurring between 500 and 800ms, as the ‘late’ ERP old/new effect. Since both early and late components appeared to vary in the same direction, it was suggested that an overlapping increase in positivity or decrease in negativity mediated the old relative to the new effect (Halgren and Smith 1987).

The ERP old/new effect for words has been described as being most prominent over central and posterior electrode sites, and larger over the left than the right hemisphere (Johnson 1995). The late ERP old/new effect for verbal stimuli is more uniformly distributed from frontal to occipital regions (Johnson 1995), although in some studies larger differences have been reported over left parietal and left temporal regions (Neville et al. 1986; Rugg and Doyle 1992).

ERPs with the approximate timing and polarity of the N400 and P600 have been reported in a number of other studies not dealing with episodic retrieval. The N400, a negative centroparietal potential, occurring at approximately 400ms in response to semantic incongruities in sentences, was first reported by Kutas and Hillyard (1980). An N400 has also been recorded with other task paradigms involving, for example, picture naming and lexical decisions (eg. see for review Johnson 1995; Puce et al. 1991), while

34 Neural correlates of object recognition memory

attenuation of the N400 amplitude has been reported in unstudied, or incidental, word repetition tasks that do not involve explicit memory processes (see for example Paller and Kutas 1992; Rugg 1987). The P600 potential is reported as having a similar topography to the P300 (Sutton et al. 1965), a potential elicited, for example, in the detection of rare tones in an auditory discrimination task (eg. see Picton 1992, for review).

Friedman (1990a) argued that the scalp distributions of early and late ERPs recorded in continuous recognition paradigms are different from those of the aforementioned N400 and P300 components. Friedman (1990a) reported a frontal N400 ERP old/new effect, and argued that this pattern was quite different from the many other scalp N400 distributions that had been previously reported. These previously reported distributions were characterized mainly by parietal, frontal, or both parietal and frontal negativities. Besson et al. (1992), however, noted that the spatio-temporal pattern of the early negativity evoked in their study closely matched the N400. Halgren and Smith (1987) suggested that the N400 represents ‘associative activation’ resulting in the formation or the retrieval of a memory traces. More recently though, it was suggested that the early old/new effect is not related to longer-term episodic memory processes because this effect disappears with lags between repeated items greater than 2mins (see for example, Nagy and Rugg 1989). Johnson (1995) in fact argued that the early ERP old/new effect, as indexed by the N400, has little to do with explicit memory processes.

Friedman (1990a) also indicated that the P600 recorded using continuous recognition designs is distinct from the P300 recorded using target detection designs and suggested that these are not generated by the same processes. Traditionally the P300 has been characterized by a parietal maximum. Friedman (1990a), however, reported a frontal positive maximum which occurred significantly later than the P300. A distinction between these two positive waveforms has been further supported by many other studies (eg. Friedman 1990b; Paller and Kutas 1992; Puce et al. ; Rugg 1987; Smith 1993; Smith and Guster 1993). Smith and Guster (1993) demonstrated a distinction between the late P600 potential and a separate, temporally overlapping, P3b potential. The authors concluded that, unlike the P3b component, the P600 component was not related to the ‘targetness’ of the verbal stimuli but rather to retrieval from secondary memory. Friedman (1990b), among others, also ruled-out the possibility that modulation of the

35 Neural correlates of object recognition memory

late positive potential is due to stimulus probability effects; a well documented P300 effect (eg. see Pritchard 1981, for review). Puce et al (1991), using depth electrodes located inside the medial temporal lobes, demonstrated a double dissociation between the late positivity evoked in continuous recognition paradigms, and the P300 evoked in an oddball task. These reported functional differences between the P300 component and the late positive component have added weight to the argument that the late ERP old/new effect reflects memory retrieval processes (Johnson 1995).

Despite the majority of studies equating the late ERP old/new effect with recognition memory processes, there has been considerable debate regarding the more specific type of memory that is being accessed. Dual process models of memory (Jacoby 1991; Mandler 1981) attempt to address, for example, the frequently recounted experience that one can recognize someone’s face but not necessarily recollect that person’s name, or when or where that person was previously encountered. Dual-process models explain this phenomenon by incorporating two distinct components of memory: recognition based on familiarity and recollection. Familiarity has been described as being automatic and unconscious, whereas recollection is characterized by conscious processing requiring retrieval of context specific information. These main characteristics have led researchers to equate recognition based on familiarity and recollection with implicit and explicit memory respectively.

Rugg and his colleagues (Rugg 1990; Rugg and Doyle 1992; Young and Rugg 1992) argued that the ERP old/new effect is a familiarity effect and hence based on implicit memory. They reported the absence of a late ERP old/new effect for high frequency words but not low frequency words, and therefore concluded that repetition enhances the ease with which an item is perceived and thus increases its ‘local’ familiarity. Further support for the familiarity argument was provided by Potter et al. (1992). They administered the anticholinergic agent scopolamine because of its known detrimental effect on explicit memory processes. If the ERP old/new effect is based on explicit memory processes, then the magnitude of this effect should have be been reduced with scopolamine. However, the ERP old/new effect was unaltered despite a decrement in recognition performance. Potter et al. therefore concluded that the ERP old/new effect is not mediated by cholinergic neural activity. Instead, they concluded that the

36 Neural correlates of object recognition memory

scopolamine had an adverse effect on recollection, resulting in an increased reliance on relative familiarity as a cue for recognition judgements.

Other investigators have equated the ERP old/new effect with explicit memory processes (eg. Smith 1993; Smith and Guster 1993). Smith and Halgren (1989) first suggested this in the light of the results of a clinical study which used the study-test paradigm. Here, patients, who had undergone left temporal lobectomy, failed to show a late ERP old/new difference. This indicated that the hippocampus, a component of the left medial temporal lobe, is involved in the generation of the old/new difference. Because previous findings had implicated the hippocampal formation in explicit memory function (Squire and Zola-Morgan 1991) it was concluded that the ERP old/new effect reflects recollective processes rather than familiarity-based implicit memory processes. Johnson (1995) also argued that if the ERP old/new effect was based on aspects of implicit memory such as familiarity, the amplitude of the late component of the ERP old/new effect should steadily increase with continued repetition of items. Instead, Johnson (1995) noted that the magnitude of the ERP old/new effect does not change with repetition, a finding consistent with a basis in explicit memory processes. Smith (1993) tackled this issue more directly using a study-test experimental design in which subjects were required to indicate whether they consciously recollected the presentation of an item, or whether they simply knew that it had been presented. The P600 component was more positive going for correctly recollected items than for those simply known to have been presented. Many subsequent study-test experiments have attempted to clarify the issue of whether the ERP old/new effect is based on familiarity or recollection. A more detailed review of these studies is presented in the next section (section 2.2.3.2).

As already indicated, there are only a few studies on episodic memory retrieval that have used non-verbal stimuli. Friedman (1990a) used pictures of common objects presented in a task designed to determine whether the scalp ERP old/new effect would be the same as that described in verbal studies. They also hoped to determine whether representation of the picture would be transferred from primary to secondary memory. The authors used 3 different repetition lags of 2, 8 and 32 items. They argued that, according to the model of Waugh and Norman (1965), ERP changes should indicate a transfer from primary memory, when the lag is 2 items, to secondary memory, when the

37 Neural correlates of object recognition memory

lag is 8 or 32 items. As was the case with ERPs with verbal stimuli, an early negativity (N300) followed by a late positivity (P600) was observed, and again, repeated stimuli produced a smaller N300 and larger P600 component. Unlike verbal studies, which showed spatially diffuse ERP old/new differences between novel and repeated items, this pictorial study showed more localized differences that were largest at frontal sites for the N300, and at frontal and central sites for the P600. Friedman suggested that the N300 reflected memory retrieval processes, as this component was significantly larger when evoked by new items than by old items. He also argued that the P600 was more likely to reflect processes relating to the organization and execution of the response rather than to those involved in the recognition and/or classification of a picture. However, Friedman failed to find a difference between ERPs obtained with different lag intervals. This may have been partly because pictures require less elaboration than words during encoding, and partly because they are more differentiable and are less likely to interfere with other pictures. Friedman suggested that pictures might be transferred from primary to secondary memory more readily than words.

Findings of clinical studies using the continuous recognition task design have indicated possible neural structures involved in the generation and modulation of ERP old/new components associated with memory retrieval processes. Puce et al. (1991) recorded ERPs from epilepsy patients prior to surgery using depth electrodes located within the temporal lobes. They used a continuous recognition task with lags to repeated items greater than 45s. A relatively long lag time was used in order to engage longer-term recognition memory processes. Unlike Friedman (1990a), however, who used common objects as stimuli, Puce et al. used both abstract non-verbalizable stimuli and verbal stimuli, arguing that the ERPs evoked using abstract non-verbalizable stimuli were more likely to reflect ‘pure visuospatial recognition memory.’ The N400 was larger for novel stimuli, and the P600 larger for repeated stimuli, for both verbal and non-verbal items, findings similar to those of Smith et al. (1986). Puce et al. (1991) also found no significant difference in the ERPs evoked by verbal and non-verbal stimuli, with both stimulus types evoking larger right hemisphere ERPs. Also, polarity reversals within the mid and posterior hippocampus and hippocampal gyrus were observed, suggesting a local generator in medial temporal regions; a finding also reported by Smith et al. (1986). For 3 patients with unilateral white matter lesions, the N400 and P600 components were absent. Puce et al. (1991) discussed this finding in terms of a possible

38 Neural correlates of object recognition memory

interruption in the visual recognition pathways (Mishkin 1982) caused by the white matter lesions in these patients (see section 2.1.1).

Guillem et al. (1995b) extended previous findings by examining depth ERPs recorded at a larger number of sites, including a number in parietal, occipital and frontal regions. They used verbalizable pictorial stimuli, and examined ERPs to novel and repeated stimuli with a lag of 6 intervening items. They found that anterior temporal, prefrontal, and possibly parietal regions, were involved in generating and modulating N400 and P600 components. In addition, the authors suggested that the regions responsible for generating the resultant N400/P600 scalp ERPs were likely to interact with each other to modulate these ERPs.

Halgren et al. (1992) proposed that activity originating in anterior temporal, prefrontal and parietal regions reflects three different aspects of cognitive processes involving three different sub-systems. The first of these is involved in the generation of the scalp P3a and includes neural regions contributing to the parieto-frontal attentional system. The second sub-system is involved in the generation and modulation of the scalp N400 and includes medial temporal structures and parietal structures. The third sub-system is involved in the production of the P600 (P300 or P3b) component. This component is mediated by temporal and superior parietal structures, and reflects cognitive closure processes (see Guillem et al. 1995b, for more specific details of brain regions involved; Halgren et al. 1992).

ERP data obtained using a longer lag between repeated items was presented in a follow- up paper by Guillem et al. (1996). Once again, the ERP old/new effect was observed at various sites within the brain using depth electrodes. This study specifically examined the involvement of temporal regions in the generation and/or modulation of the N400/P600 scalp potentials. In addition, the relative contribution of temporal regions to short-term and longer-term memory processes was examined by using lags of 6 and 19 items respectively. Once again, verbalizable pictorial stimuli were used. A strong N400 old/new effect was observed in all temporal lobe structures examined, although the magnitude of the effect varied with the recording site and lag. Because the magnitude of the N400 old/new effect differed with the different lags, the authors speculated that temporal lobe structures contribute differently to short-term and long-term memory

39 Neural correlates of object recognition memory

processes. In contrast, the P600 old/new effect was not consistently observed. The authors suggested that the nature of the stimuli might have contributed to this outcome. Friedman (1990a), who also used highly verbalizable stimuli, also reported that the P600 old/new effect was less pronounced than the N400 old/new effect. Because there was a large N400 old/new difference at posterior temporal regions for a lag of 6, but not for a lag of 19, Guillem et al. suggested that the posterior temporal regions are specifically involved in short-term memory processes. The authors also found that the amygdala, hippocampus and anterior temporal regions contribute to both short-term and long-term memory processes. Moreover, the authors suggested that these regions also play an independent role in long-term memory processes in general.

In summary, the majority of verbal and pictorial studies utilizing the continuous recognition paradigm reported an increased ERP positivity for repeated items compared to new items in the 300-800ms post onset interval. The early ERP old/new effect, although consistently observed in continuous recognition tasks, decays rapidly, and therefore appears to be unrelated to memory retrieval. The late ERP old/new effect appears to be related to recognition memory; however, the precise form of memory has not been conclusively determined in continuous recognition studies. This issue has been more thoroughly investigated using the study-test experimental design and is discussed in the following section. The scalp ERP old/new effect occurs, in the majority of cases, over parietal and central regions with a left hemisphere bias for verbal stimuli, and over frontal and central regions for verbalizable pictorial stimuli. Depth electrode recordings have identified a number of brain regions involved in generating and modulating the late ERP old/new effect. These regions appear to respond differently to verbal stimuli and to both verbalizable and abstract non-verbalizable pictorial stimuli. While a number of brain regions have been implicated in the ERP old/new effect, involvement of medial temporal lobes has been reported consistently for both verbal and non-verbal stimuli. The additional involvement of parietal and prefrontal sites has also been observed with verbalizable stimuli. Due to the paucity of continuous recognition studies utilizing pictorial stimuli, it is not yet clear if the spatial distribution of the ERP old/new effect at the scalp is essentially the same for verbal and non-verbal stimuli.

40 Neural correlates of object recognition memory

2.2.3.2 ERP correlates of episodic memory retrieval: evidence from study-test paradigms

Although the continuous recognition task design has been used to study episodic memory retrieval, this design suffers from certain disadvantages. Firstly, in the continuous recognition design (section 2.2.3.1), encoding and retrieval processes are intermixed in the one task making separation of encoding and retrieval processes difficult. Furthermore, lags between repeated items are generally short, thereby reducing dependence on long-term memory. The study-test experimental design does not suffer from the aforementioned disadvantages. With this design, words or pictures are studied in an encoding task and are later retrieved in a task that is usually presented approximately five minutes later (although see Duzel et al. 1999, who used a lag of 30s). Thus, encoding and retrieval occur in separate phases, and the relatively long time lag between encoding and retrieval also increases dependence on longer-term memory processes.

A number of variants of the study-test format have been used to determine more precisely the type of memory processes associated with the ERP old/new effect. Many studies have investigated whether retrieval processes are based on recollection or on familiarity. Most studies focusing on recollection have used task designs that maximize the contribution of explicit memory processes, thereby facilitating the investigation of ERP changes associated with this aspect of memory. Under these conditions, the ERP old/new effect at parietal sites is largest and is accompanied by a right frontal effect that occurs over a similar time frame, that is, it begins approximately 500ms after stimulus appearance and lasts longer than 1s. While the parietal ERP old/new effect appears to be obligatory, and largest when retrieval is based on recollection, the presence of a right frontal effect depends on the task used and reflects monitoring processes. While most experimental designs and subsequent analyses have focused on the investigation of processes associated with retrieval success, as indexed by the ERP old/new effect, other aspects of retrieval processes such as retrieval effort and retrieval strategy have also been investigated and will be reviewed in this section. In these studies, the contribution of frontal regions is highlighted. Finally, clinical studies will be reviewed. These studies provide further insights into the origin of ERP old/new effects and the memory processes that they might reflect.

41 Neural correlates of object recognition memory

Paller and Kutas (1992) were perhaps the first to test directly the nature of the cognitive activity associated with the ERP old/new effect. They used ‘a levels-of-processing manipulation’ design, wherein subjects performed two tasks designed to give identical priming, which involves implicit memory, in the study phase but different recall associations during the subsequent test phase. Subjects had to study words in relation to their meaning in one task and letter identity in the other. In the test phase, subjects performed a word-identification task in which words were presented very briefly (33- 50ms). While behaviourally the frequency of identification of old words was the same for both study categories, there was dissociation in ERP activity. The authors argued that they had demonstrated an electrophysiological correlate of recollection because the physical characteristics of the words from the two study groups were identical, priming was the same, and the behavioural responses were not significantly different. The authors therefore hypothesized that the ERP old/new effect represents an electrophysiological correlate of conscious recollection, distinct from priming processes or other confounding influences.

Smith (1993) conducted the first study that attempted to determine whether the ERP old/new difference reflects explicit or implicit memory processes. He used an experimental paradigm developed by Gardiner (1988) designed to investigate the relationship between recognition and conscious awareness. In the test phase, in addition to making an old/new judgement for words that had been presented in the study phase, subjects also had to indicate whether they consciously remembered (R) each word, or simply knew (K) that the word was old. Gardiner (1988) had equated R responses with explicit memory and K responses with implicit memory. In Smith’s study, these corresponded to recall based on recollection and familiarity respectively. Smith found that R words elicited larger and more positive ERPs than K words. Both R and K words were, however, more positive than correctly identified new unrepeated words. Furthermore, old words that were incorrectly identified did not show the ERP changes characteristic of correctly identified old words. The authors argued on this basis that both R and K ERPs were related to explicit memory retrieval processes rather than implicit memory processes. However, Johnson (1995) later suggested that differing spatio-temporal patterns of R-K and K-new differences in Smith’s study represented distinct memory processes emanating from different brain regions. Johnson noted that

42 Neural correlates of object recognition memory

the spatio-temporal patterns of the R-K difference and the ERP old/new effects were similar, that is, the R-K difference was small and equipotential over frontal, central and parietal scalp regions, with an onset at around 550ms. The K-new difference was, however, characterized by a centro-parietal distribution, similar to that of the P300, and with an onset at around 400ms. Johnson (1995) thus concluded that the R-K difference depends on conscious explicit memory processes, whereas the K-new difference may depend on implicit memory processes.

Duzel et al. (1997) used the R/K paradigm to explore conscious awareness of memory for past events. They argued that R responses reflect ‘autonoetic’ awareness, that is, a re-living of one’s personal past (episodic memory), and K responses reflect ‘noetic’ awareness and reflect one’s interaction with the environment in the present. They argued that true and false R judgements should produce the same ERP changes, given that they both reflect the subject’s subjective awareness, independent of whether the targets were presented or not. To test this hypothesis, they used an experimental paradigm in which subjects responded to old words that were previously presented in the study phase (true targets), new words that were not presented in the study phase but were semantically related to words presented in the study phase (false targets), or words that were new and not semantically related to words presented in the study phase. Subjects had to decide which of the three categories each word belonged to. Firstly, they responded to each word in the test phase by indicating whether the word was old or new, and then, if old, whether they remembered (R) the event of the word’s presentation or simply knew (K) that the word had been in the study list. Duzel et al. argued that ERPs derived using this three-way classification of subjects’ subjective assessment should reflect three distinct states of consciousness: (i) memory of the event of the word’s presentation in the study list (remember or autonoetic awareness); (ii) memory that that word was presented, but no memory of the event (know or noetic awareness); (iii) no awareness of any kind that the item was presented (unawareness of the past).

ERPs associated with autonoetic and noetic awareness were compared with ERPs associated with correct rejection of new words. Autonoetic awareness was associated with a late positivity occurring between 600 and 1000ms post onset over left temporo- parietal and bifrontal regions. ERPs associated with noetic awareness displayed an earlier bilateral temporo-parietal positivity between 300 and 600ms, and a later bifrontal

43 Neural correlates of object recognition memory

negativity between 600 and 1000ms. Moreover, consistent with their hypothesis, the ERPs generated by true and false recognition were identical. Duzel et al. noted that the spatio-temporal ERP patterns associated with autonoetic awareness were similar to the previously reported patterns associated with the late positive component (P600). A different pattern of activity for K judgments led the authors to suggest that the late positive component is related to autonoetic conscious recollection. Furthermore, given the similarity between the ERPs elicited by true and false targets, it was concluded that these ERPs were not sensitive to whether the words were studied or not, rather, they reflected neural changes associated with the subjects’ subjective awareness.

Wilding and his colleagues (1995) used a study-test design to explore the extent to which the ERP old/new effect stems from familiarity-based retrieval and the extent to which it stems from recollection-based retrieval. During the study phase, subjects performed a lexical decision task in which they were presented with words, visually and auditorily, and had to make word/non-word judgements. Subjects were unaware there would be a subsequent test phase. During the test phase, subjects had to make a forced old/new decision, and for items judged as old, a further forced decision regarding the context in which the word was presented, namely visual or auditory. To ensure that test modality did not confound the results, test phase words were presented visually in one experiment and auditorily in a second. The authors argued that ERPs elicited by old items that had been correctly assigned contextually reflected recollection, whereas correctly judged old items with incorrect assignment of study context must have been judged old on the basis of familiarity rather than recollection.

In both experiments, that is, when the test phase words were presented both visually and auditorily, the ERPs elicited by words correctly assigned to study modality were more positive than the ERPs elicited by new unstudied words. When the test phase words were visually presented, the ERP old/new effect was absent for words incorrectly assigned to study modality, whereas, when they were presented auditorily, it was present, although in a restricted time interval (400-800ms). The authors argued that the absence of an ERP old/new effect for incorrectly assigned words with the visually presented test phase words was due to poor performance, and therefore was not used in further interpretation of the data. The ERPs elicited by studied words, whether correctly or incorrectly assigned to study context, were more positive than the ERPs elicited by

44 Neural correlates of object recognition memory

new unstudied words. Between 400 and 800ms post stimulus onset, the ERPs for the studied words, regardless of whether they were correctly or incorrectly assigned, were essentially the same. However, the correctly assigned words generated an extended positivity 800 to 1100ms post stimulus onset that was not seen with the incorrectly assigned words. This difference, 800 to 1100ms post-onset, led Wilding et al. to conclude that the late ERP old/new effect for correct contextual judgements was based on recollection rather than familiarity. However, they noted that this finding was open to interpretation because of the similarity of the ERPs evoked by correct and incorrect contextual judgements in the 400 to 800ms interval. Wilding and Rugg (1996) later proposed that the similarity of ERPs in this range was consistent with the proposal of Johnson et al. (1993), namely, that words for which study modality was incorrectly specified had engendered weak or partial recollection, insufficient for recollection of modality but sufficient for recollection of its earlier presentation.

Wilding et al. (1995) discussed their findings in relation to a model that suggests that both familiarity-based recognition and recollection may rely on declarative memory systems involving both hippocampal and frontal regions (Moscovitch 1992; Moscovitch 1994; Squire 1994). In this model, familiarity-based recognition can be achieved solely with hippocampal output. Recollection, involving retrieval of contextual information, is achieved through further integration with the prefrontal cortex. In line with this proposal, Wilding et al. (1995) suggested that because ERPs elicited by correct and incorrect contextual judgements had similar spatial distributions in the 400 to 800ms post-onset interval, they might both be generated by the hippocampal formation. Moreover, the prolongation of the ERP old/new effect for words for which study modality had been correctly specified might reflect the involvement of the frontal lobes in recollection. However, the authors cautioned that further investigation was necessary because, for the correctly assigned words, the difference between the spatial distributions of the ERP old/new effect within the two time intervals was not statistically significant.

In case fluency effects occurring when the test and study phase words were in the same modality had contributed to correct word assignation, rather than recollection per se, Wilding and Rugg (1996) performed a further study where this wouldn’t occur. In the study phase, all the words were auditorily presented, and in the test phase, all the words

45 Neural correlates of object recognition memory

were visually presented. In the study phase, subjects listened to words spoken by either a male or female voice. In the recognition phase, in addition to deciding whether the visually presented words were old or new, subjects also had to decide whether words judged old had been spoken by a male or female voice. The ERPs elicited by old words for which the study context was correctly specified should have reflected processes involving recollection rather than familiarity-based recognition.

The ERPs elicited by words correctly judged old were more positive than those elicited by correctly identified new items. This ERP old/new effect could be dissociated into two spatially and temporally distinct components, a large left parietal component between approximately 400 and 1000ms post stimulus onset, and a right frontal component beginning at about the same time and sustained for the length of the recording epoch (>1400ms). The parietal difference was likened to the widely reported old/new modulation of the P600 potential (Neville et al. 1986; Rugg 1995), while the right frontal component did not appear to have been reported previously. ERPs elicited by correctly assigned old words were more positive than ERPs for old words incorrectly assigned, although this difference was not statistically significant. The authors concluded, as did Wilding et al. (1995), that ERPs elicited by words for which the context was correctly and incorrectly specified were qualitatively the same, and were likely have been generated by the same combination of neural structures. The authors also concluded that the increased positivity over left parietal areas for old items relative to new reflects simple memory of prior occurrence, whereas the right frontal increased positivity for old items relative to new, an effect not previously reported, may reflect recovery of contextual information. Moreover, the right frontal changes could have indicated the involvement of the prefrontal cortex. This would be consistent with neuroimaging evidence of activation within the right dorsolateral prefrontal cortex during tasks requiring episodic retrieval, and the clinical observations that patients with prefrontal lesions have poor source memory (Schacter et al. 1991), that is, they have poor recollection of the details of an event while still knowing of the occurrence of the event.

In both studies by Wilding and colleagues (Wilding et al. 1995; Wilding and Rugg 1996), the authors concluded that the ERP old/new effect is more likely to reflect recollection of old items rather than familiarity with them. Moreover, given the

46 Neural correlates of object recognition memory

similarity of ERP amplitudes for correct and incorrect contextual judgments, it was suggested in both papers that recollection is a graded rather than an all-or-none process. The authors argued that their results were consistent with Rugg’s (1995) proposal that the ERP old/new effect reflects the amount or quality of information retrieved in response to the old item. To tackle this issue more directly, Wilding (2000) also used the presentation context of male versus female voice, but subjects were also required to retrieve further information about the words encountered during the study phase. During the study phase, in addition to listening to words spoken in male or female voice, subjects had to make either an ‘action’ or ‘liking’ judgement about each word. That is, they had to decide whether the word was either active or passive, or pleasant or unpleasant. During the test phase, for words judged old, subjects also had to decide whether they had been spoken in a male or female voice, and whether an action or liking judgement had been made. The magnitude of the parietal ERP old/new effect increased with the number of accurate source judgments. Wilding therefore concluded that the parietal ERP old/new effect indexes recollection in a graded fashion.

Wilding (1999) used a similar approach to explore possible variations in the spatio- temporal patterns of ERPs associated with different retrieval strategies. Here, the ERPs associated with the two retrieval tasks, namely, deciding whether the voice was male or female, and whether an action or liking judgement had been made, were compared. The task was designed to explore possible variations in ERP spatio-temporal patterns associated with retrieval strategy. Similar ERP old/new effects were seen at parietal electrodes for the two retrieval tasks. Moreover, given the similarity between the ERPs associated with correct and incorrect source judgements, the authors concluded, as previously, that the parietal ERP old/new effect indexes the amount or quality of information retrieved. The right frontal ERP old/new effect was, however, larger for the word association condition than for the voice condition. The authors argued that the parietal ERP old/new effect not only reflected successful retrieval, but also the type of source information that was retrieved from memory.

Donaldson and Rugg (1998) used an associative recognition task format to vary the likelihood that items would be recollected. This was modelled on the context-based recognition design used by Wilding and Rugg (1996), but was designed in such a way that word associations probed recollective processes. In the study phase, word pairs

47 Neural correlates of object recognition memory

were shown. In the test phase, three categories of word pairs were shown: the same word pairs shown in the study phase, re-arranged word pairs made from new combinations of studied words, and new word pairs using previously unstudied words. For each word pair presented in the test phase, subjects initially made an old/new judgment on whether the word pair contained old studied words or new unstudied words. For word pairs containing studied words, subjects made a further judgment, indicating whether the word pair was the same pairing as that presented in the study phase or whether the studied words had been re-arranged. The authors argued that same pairs should engage recollection processes to a greater extent than rearranged pairs because correct identification of same pairs would be indicative of recollection of the actual pair occurring in the study phase.

Compared with the ERPs for unstudied new pairs, the ERPs for same pairs exhibited an early left parietal and bilateral frontal positivity between 600 and 1200ms. By the end of the recording epoch (>1200ms), the left parietal positivity remained and was accompanied by a frontal positivity, now lateralized to the right. The spatio-temporal patterns of these positivities were similar to the context-based (male/female voice) ERP old/new effect reported by Wilding and Rugg (1996). Rearranged pairs showed patterns of activity that were qualitatively similar, but markedly smaller. Donaldson and Rugg argued that larger ERPs in response to same pairs compared with rearranged pairs was consistent with their proposal that correct identification of same pairs involves recollection to a greater extent than rearranged pairs. Moreover, the similarity of these ERP old/new effects to those of Wilding and Rugg (1996) suggested that associative recognition memory might involve the same recollective processes as those utilized in context-based recognition memory.

In a second experiment performed by Donaldson and Rugg (1998), designed to investigate recognition in the absence of association, subjects were only required to make old/new judgments. They were not required to judge whether the word pairs were the same as those presented in the study phase or were re-arranged. A similar pattern of results was again evident, namely, a sustained left parietal positivity followed by a prominent right frontal positivity, although the ERP old/new differences were smaller than those observed in the first experiment. The occurrence of a prominent right frontal ERP old/new effect in both experiments indicated that the additional associative

48 Neural correlates of object recognition memory

judgement made in the first experiment did not cause the right frontal effect. Instead, Donaldson and Rugg (1998) suggested that the right frontal effect may be obligatory in nature and could be related to the ‘richness’ or amount of information that is retrieved. They noted that this view is supported by similar right-frontal findings using picture stimuli that attracted high levels of recognition accuracy (Schloerscheidt and Rugg 1997).

Observations by Rugg et al. (1996) provide further evidence that the parietal ERP old/new effect in an associative recognition task reflects recollection-based retrieval. They used an associative verbal task in which word pairs were presented in the study phase, and subjects were required to incorporate each pair of words into a sentence. In the test phase, the first word of each studied pair was intermixed with an equal number of unstudied words. Subjects were required to make old/new recognition judgments, and for each word judged old, they had to recall the specific word that it had been associated with in the study phase. A strongly left-lateralized ERP old/new pattern was observed only for words judged old for which the associated word was also correctly recalled. The authors concluded that the findings support the view that the parietal ERP old/new effect reflects neural activity associated with the recollection of specific past episodes, and may therefore reflect medial temporal lobe involvement in episodic retrieval.

In the studies discussed above, context-based paradigms were primarily used to investigate whether familiarity-based and recollection-based retrieval processes involve the same or different neural systems. There are, however, other aspects of retrieval, such as strategies used and effort involved, which Rugg et al. (2000) have termed retrieval set effects. Rugg et al. studied retrieval strategy and effort using different encoding conditions to examine the effect of depth of encoding on retrieval processes. In the study phase of their task designed to promote only ‘shallow’ encoding, subjects had to decide whether the first and last letters of the study words were in alphabetical order. In the task designed to promote ‘deep’ encoding, subjects were required to incorporate each word into a meaningful sentence.

In the interval between 300 and 800ms post stimulus onset, the ERP old/new effects associated with the deep encoding task were most prominent over left parietal and right frontal regions, and were therefore described as being qualitatively the same as those

49 Neural correlates of object recognition memory

observed in context-based studies. The only ERP old/new effects generated by the shallow encoding task occurred during the 800 to 1400ms post stimulus interval and were restricted to right frontal regions. Recognition accuracy for shallowly studied words was significantly lower than for words studied in the deep encoding task.

Because other studies had reported a prominent early left parietal old/new effect associated with recollective processes, its absence with the shallowly encoded words was considered consistent with the reduced recognition accuracy for these. It was suggested, however, that the right frontal ERP old/new effect for the shallow task was inconsistent with this interpretation, given also its occurrence with context-based retrieval and recollection. Rugg et al. argued that the right frontal effect could be explained, however, by a necessity for more extensive monitoring for shallowly studied items prior to decision; a theory also supported by the longer response times associated with these items.

Retrieval set effects were investigated by comparing ERPs elicited by the new words in both the shallow encoding and deep encoding test blocks. ERPs elicited by new words presented in the shallow test block were found to be more positive than those in the deep test block. This result suggests that different retrieval sets are adopted when attempting to retrieve words that have been encoded using different linguistic attributes. It was also suggested that the prominent differences observed at left frontal sites reflected a greater retrieval effort. Rugg et al. likened this result to Ranganath and Paller’s (1999) observation of a greater positivity elicited over left frontal regions by the more difficult of two retrieval tasks.

Only few studies have used pictorial stimuli to investigate scalp-derived ERP correlates of object recognition memory. For example, Schloerscheidt and Rugg (1997) used a study-test design to compare ERPs elicited by the recognition of words and pictures of objects. In the encoding phase, subjects were asked to imagine if each represented word or picture item would, in real life, be larger than the computer monitor that they were viewing. Clinical data (Smith 1989) had suggested that damage to the left medial temporal lobe results in relatively greater impairment to verbal than to pictorial memory, and conversely, damage to the right medial temporal lobe results in relatively greater deficits in pictorial memory. Given this neuropsychological evidence, and the

50 Neural correlates of object recognition memory

possibility that the parietal ERP old/new effect reflects medial temporal lobe involvement in recognition memory (Rugg et al. 1996), Schloerscheidt and Rugg (1997) hypothesized that, in contrast to left lateralized parietal ERP old/new effect observed with word stimuli, the retrieval of pictorial stimuli would be characterized by a more bilateral distribution of the parietal old/new effect. However, both pictures and words elicited similar early ERP old/new effects that were lateralized to left parietal sites. As this result was contrary to their hypothesis, it was suggested that the parietal old/new effect may not be reflecting medial temporal lobe function. In addition, a right frontal ERP old/new effect occurred with the picture recognition but not with the word recognition task. This effect was initially left-lateralized, with an onset 100ms prior to that of the left parietal ERP old/new effect, and shifted to right frontal regions after 1s. This right frontal finding is consistent with the proposal that pictures are more richly encoded than words, and are therefore more likely to engage post-retrieval processes. An early left frontal ERP old/new effect, which preceded the left parietal effect, was previously reported by Tendolkar et al. (1997). Thus, Schloerscheidt and Rugg concluded that the left parietal ERP old/new effect is not necessarily the earliest ERP correlate of successful retrieval.

Paller and colleagues (Paller et al. 1999; Paller et al. 2000) also used a study-test format to investigate ERP correlates of successful retrieval using photographs of faces. During the encoding phase, subjects were shown photographs of faces that they were asked to memorize. Accompanying some of the photographs was a short simulated voice of the individuals shown stating who they were and what they did (‘remember faces’). Other faces were presented without voices and others with instructions to forget the faces (‘forget faces’). During the test phase, subjects were required to identify faces that were shown previously (‘remember’ and ‘forget faces’), presented in amongst photographs of new faces. The authors reported that ERPs elicited by ‘remember faces’ were significantly different from those elicited by ‘forget faces.’ In a separate behavioural study, the degree of priming for the two types of studied faces was found to have been equal, although ‘remember faces’ were much more accurately recognized. Paller and colleagues therefore concluded that differences between the ERPs elicited by the two types of studied faces represents a neural correlate of recollection.

51 Neural correlates of object recognition memory

Ranganath and Paller (2000) hypothesized that frontal effects previously associated with episodic retrieval might also reflect the recall from memory of perceptual detail of studied items. They tested this hypothesis using a study-test design with two test phase retrieval tasks: a general task in which subjects were required to make old/new recognition judgements of previously studied line drawings, and a more specific task in which an additional judgement was required, namely, whether the test phase drawings were larger or smaller than those previously studied. Frontal ERPs were more positive for the specific test than for the general test, and were largest within the 600 to 800ms interval post stimulus onset. Because this result was significant for both new and old items, the authors concluded that frontal activity not only reflects successful retrieval, but also reflects evaluative processing of specific stimulus attributes retrieved from memory. In addition, given the different requirements for the general and the more difficult specific retrieval tests, the authors proposed that both left and right prefrontal regions are engaged when retrieval demands and evaluation of perceptual detail are relatively high. A left parietal positivity was also observed for the specific relative to the general task, but only for old items. Ranganath and Paller likened this observation to the left parietal ERP old/new effect, and therefore concluded that this pattern of activity reflected reactivation of stored information.

A number of clinical studies utilizing the study-test design have provided further insight into the brain structures which could be responsible for the generation and modulation of late positive ERPs and the specific memory processes that they reflect. For example, when the response to verbal stimuli was recorded using depth electrodes sited within the temporal lobes of pre-surgery patients, N400- and P300-like components appeared to be generated and modulated by old items within medial temporal lobe structures in the left hemisphere only (Smith et al. 1986). It was suggested that the left-lateralized generation and modulation of these potentials was consistent with the involvement of the left medial temporal lobe in verbal memory. Johnson (1995) noted that the ERP components recorded in Smith et al.’s study resembled those recorded at the scalp surface, but a causal relationship was not demonstrated. A subsequent study added further weight to the view that there is a relationship between the parietal ERP old/new effect and explicit memory (Smith and Halgren 1989). Patients with intractable epilepsy who had undergone left temporal lobectomy failed to show a late ERP old/new effect. This result contrasted with findings from patients who had undergone right temporal lobectomy

52 Neural correlates of object recognition memory

and a group of control subjects. Both of these groups showed preserved ERPs. Moreover, left temporal lobectomy patients showed impaired recognition accuracy compared to the other groups. In the light of these results that indicate the importance of the hippocampal formation in the generation of the ERP old/new effect, and in conjunction with previous findings implicating the hippocampus in the formation of explicit memories, Smith and Halgren concluded that the ERP old/new effect reflects explicit recollective processes rather than implicit familiarity processes.

Guillem et al. (1995a) investigated the effects of temporal and extra-temporal (parietal and frontal) epileptogenic lesions on recognition memory-related hippocampal activity related to recognition memory. ERPs were elicited using verbalizable pictures, and were recorded using depth electrodes sited within the hippocampal formation. The authors observed ERP components with waveforms similar to those of the scalp-derived N400 and P600 potentials reported in previous studies. In addition, they recorded polarity reversals within the hippocampus for both N400 and P600 components, a finding consistent with other studies (eg. Heit et al. 1990; Puce et al. 1991; Smith et al. 1986). Polarity reversals indicate local generation rather than volume-conducted activity generated by distant sources. It was noted that the more extensive the damage, the greater the effect on the N400 and the P600 components. These components were not dramatically affected in patients with unilateral temporal lobe epilepsy. Moreover, the N400 component was preserved both ipsilaterally and contralaterally to the seizure focus. However, patients with multi-focal lesions showed greatly diminished N400 and P600 components in both left and right hippocampi. These results were compared to those reported by Puce et al. (1991), who reported absent N400/P600 components in patients with large white matter lesions. The fact that the greatly diminished N400 and P600 components recorded by Guillem et al. (1995a) were nevertheless identifiable, was attributed to less extensive lesions in their multi-focal patients. Guillem et al. concluded that N400 and P600 components are modulated by distributed, yet highly interconnected, brain regions, including hippocampal, parietal and frontal regions. This conclusion was also drawn by this group from the findings obtained using a continuous recognition memory design (Guillem et al. 1995b) (see section 2.2.3.1).

Two other clinical studies, using continuous recognition rather than study-test formats, are worthy of note as they contradict the study-test findings described above. In the first

53 Neural correlates of object recognition memory

of these studies, Rugg et al. (1991) used verbal stimuli in a continuous recognition task, and reported reduced ERP old/new effects in patients who had undergone either left or right temporal lobectomy. Furthermore, they found no relationship between the magnitude of the ERP old/new effect and scores of verbal memory performance across all subjects investigated. In contrast to Smith et al. (1986), Rugg et al. concluded that the cognitive processes reflected by their ERP data were not lateralized to one hemisphere, and moreover, because of the lack of correlation with verbal memory scores, they are not necessary for normal verbal memory functioning. Furthermore, it was suggested that while the temporal lobes appear to be involved in generating the ERPs that show the old/new differences, the primary locus of these ERPs does not appear to lie within medial temporal lobe structures (Rugg et al. 1991). This finding contradicts those of a number of studies which indicated a hippocampal origin for the ERP old/new effect (eg. Heit et al. 1990; Puce et al. 1991; Smith and Halgren 1989; Smith et al. 1986). It is possible, however, that differences in experimental conditions may account for these apparently contradictory findings (Johnson 1995). Firstly, it may be that the recognition memory processes that are employed in a continuous recognition task may differ from those employed in a study-test task. Secondly, the lag times, with 5 intervening items between repeated items, totaled 19s in Rugg et al.’s (1991) study. These were considerably shorter than the 4-5 minutes usually used in a study-test design. An insufficient delay, coupled with the much lower memory loads used by Rugg et al., could explain their failure to find a decrement in performance associated with their reduced ERP old/new effects.

In the second of these continuous recognition-based studies (described in section 2.2.3.1), intravenously administered scopolamine, known for its detrimental effects on explicit memory, was used to mimic the effects of left temporal lobectomy in normal subjects (Potter et al. 1992). Although the authors had predicted that the scopolamine would reduce ERP old/new effects, they were in fact preserved, leaving the authors to conclude that ERP old/new effects are not dependent on explicit memory processes. This conclusion is contrary to those drawn in nearly all other studies that have used a study-test task design. Johnson (1995) argued, however, that once again, factors such as low memory loads and predictable short delays between repeated items might have contributed to Potter et al.’s findings. Because increased reaction times were also

54 Neural correlates of object recognition memory

observed, Johnson also concluded that the scopolamine had affected attention as well as memory processes.

In summary, the study-test appears a better task design than the continuous recognition design for the study of neural processes involved in episodic retrieval. Additional effects more specifically associated with episodic memory retrieval have also been revealed using the study-test design. For example, context-based study-test designs have shown that compared with the presentation of new words, correctly recognized old words are associated with a prolonged scalp positivity that is most prominent over left parietal regions between 400 and 1000ms post stimulus onset, and over right frontal regions from 400ms to beyond 1400ms. The parietal ERP old/new effect has consistently been associated with episodic memory retrieval processes. The magnitude of the parietal ERP old/new effect appears to be larger when the amount or quality of information retrieved increases, or when items are better remembered or recollected. While the left parietal ERP old/new effect has always been seen in tests requiring recollection, the right frontal ERP old/new effect, thought to reflect right frontal activity, is dependent on the nature of the test and is not always observed. In tests involving contextual retrieval, the frontal effect is usually present, although because of its prolonged duration, this is thought to represent post-retrieval operations such as verification and monitoring, rather than processes specifically related to retrieval success.

Clinical studies have provided information about the origin of ERP effects based on memory retrieval. Current consensus appears to be that the scalp ERP old/new effect over parietal regions reflects the old/new effect observed with depth recordings within the medial temporal lobes. Furthermore, it appears that intact medial temporal lobes are required for the generation of the ERP old/new effect, lending additional support to the view that these ERP changes reflect explicit memory processes. Parietal and frontal regions have also been implicated in the modulation of the ERP old/new effect. Evidence for this comes from two main sources: the reported diffuse scalp ERP old/new effect over parietal and frontal areas, and depth recordings of the old/new effect within these regions. It has also been proposed that the hippocampal formation, in conjunction with parietal and frontal regions, forms a distributed, yet highly interconnected, network that modulates ERP activity based on memory retrieval. A few studies have also

55 Neural correlates of object recognition memory

suggested that both left and right frontal regions are involved in retrieval strategy and retrieval effort processes. In addition, it appears that both left and right prefrontal regions are engaged when task demands increase and when specific attributes of the stimuli are retrieved from memory. With respect to the ERP correlates of object recognition memory, there are, as yet, very few picture-based ERP studies. Furthermore, findings from these studies are confused by the use of either verbalizable or non- verbalizable pictures and a mixture of scalp-derived and intracerebral data. Picture- based scalp ERP old/new effects have been reported over frontal and left parietal regions. Picture-based intracerebral ERP old/new effects have indicated the involvement of extensive regions within the left and right medial temporal lobes and within parietal and frontal regions. The most consistent finding overall appears to be the involvement of frontal regions, and that compared with words, pictures elicit larger responses, which has been attributed mainly to the greater the richness of the material.

2.2.3.3 Differentiation between transient item-related and sustained task-related ERP correlates of episodic memory retrieval

Sections 2.2.3.1 and 2.2.3.2 were concerned with transient item-related ERPs associated, in particular, with retrieval success and post retrieval monitoring processes. These transient ERPs occur in response to the presentation of individual items. They last only while items are being processed, and reflect specific properties of these items and the operations that they engage. Sustained task-related effects, however, are largely unaffected by individual items. They are initiated by task instructions and are sustained for the duration of the task. They reflect the ‘mode’ or ‘state’ established by the task requirements. In the case of episodic memory retrieval, the term ‘retrieval mode’ (Tulving 1983) has been used to describe the task-related neural activity that is sustained throughout the retrieval task. Task-related activations are usually investigated in PET and fMRI studies, in which task blocks rather than individual items are compared (see section 2.2.4). However, a specific ERP technique known as direct- coupled recording (DC) allows monitoring of sustained task-related activity (Rockstroh et al. 1989). DC ERPs are obtained by setting a higher time constant for the high-pass filter of the recording instrumentation, such that slower changes are passed rather than filtered-out (Cabeza 1999). The low-pass filter is unchanged, however, thereby also

56 Neural correlates of object recognition memory

allowing the recording of standard transient perturbations. Thus, the DC recording technique can be used to monitor sustained task-related activity similar to that monitored using PET and fMRI methods, whilst concurrently monitoring faster, item- related changes.

Duzel and his colleagues (Duzel et al. 1999; Duzel et al. 2001) are the only group so far to utilize this technique in the study of episodic memory retrieval. Duzel et al. (1999) applied both PET (see section 2.1.3) and ERP techniques to identify brain regions that mediate task and item-related activity associated with episodic memory retrieval. All subjects were tested using both techniques, although in separate sessions. A study-test experimental design was used. During the study phase, words were encoded by making pleasant/unpleasant judgements. During the retrieval phase, both episodic retrieval and retrieval from semantic memory were examined in separate, randomly presented, blocks. For the episodic retrieval blocks, subjects were required to make old versus new judgements, and for the semantic retrieval blocks, subjects were required to make living versus non-living judgements. Prior to each retrieval block, subjects were oriented to the task requirements by instructions informing them of the type of judgements to be made.

PET data showed activation within the right prefrontal cortex (BA10) and right posterior cingulate cortex (BA23) for the episodic retrieval task relative to the semantic retrieval task. Comparison of the semantic retrieval task with the episodic retrieval task yielded left prefrontal cortex (BA45/47) and left temporal lobe (BA21) activations. Significant sustained mode-related ERPs were also observed. Compared with the semantic retrieval task, the episodic retrieval task showed a sustained positive shift over right frontopolar regions. It is not unreasonable to suppose that this is an electrophysiological reflection of the BA10 activation observed using PET. A similar correspondence with the left prefrontal cortex PET activation was not seen in the ERP data; neither the semantic nor the episodic retrieval tasks showed sustained differences in this region.

Duzel et al noted that these PET and ERP findings are consistent with the HERA (Hemispheric Encoding/Retrieval Asymmetry) model (Nyberg et al. 1996a; Nyberg et al. 1998a; Tulving et al. 1994a), in which left prefrontal regions are involved in encoding and semantic retrieval, and right prefrontal regions in episodic retrieval. They

57 Neural correlates of object recognition memory

point out that their converging PET and ERP data further substantiates the model, and furthermore note that the asymmetry of the HERA model is based on sustained task- related rather than transient item-related activity.

Duzel et al. drew further parallels between their ERP data and other PET data (Buckner et al. 1998), suggesting that the slightly sluggish time-course of the DC signal may reflect the sluggish haemodynamic response. This sluggish response was attributed to a task-related ‘neurocognitive inertia’ that reflects a strategic, intentional, orientation by the subject from the present to the past. The authors suggested that the converging PET and ERP data reflect task-related operations that, in part, characterize the episodic retrieval mode, a state in which subjects consciously think back to the encoding episode.

The PET old minus new comparison showed activations within the left medial temporal lobe. It was suggested that these indicated regions contributing to successful retrieval processes. The ERP old minus new comparison was characterized by the widely reported late positivity of the parietal ERP old/new difference, peaking at around 600ms post onset, over left parietal regions. Source analysis indicated a medial temporal lobe origin for the left parietal positivity. Duzel et al. therefore concluded that the medial temporal lobe is involved in transient item-specific episodic retrieval processes.

2.3 Summary of neural correlates of object recognition memory

Converging evidence obtained using a wide range of experimental methods indicates that an integrated network of neural regions is involved in object recognition memory (ORM). Neuropathological and neuropsychological studies in humans and non-human primates have shown that the medial temporal lobes are important in ORM. The perirhinal cortex appears to play a particularly crucial role, while neural linkages from medial temporal regions to frontal and other cortical regions are critical for normal memory function.

Object recognition memory includes recognition based both on familiarity, and on the actual recollection of the experience of seeing the object, or episodic memory. In

58 Neural correlates of object recognition memory

humans, ORM has been studied mainly by testing episodic memory using tests in which objects are studied and identified some time later.

Specific neural regions, which are dependent on the nature and context of the stimuli, are engaged for the different aspects of an episodic retrieval process. One aspect is the maintenance of the mental state adopted for the performance of a memory retrieval task, or retrieval mode. This is thought to involve sustained mental activity directing the mental focus to the past. More transient neural processes are involved during the actual moments the event of interest is remembered. Episodic retrieval also involves retrieval set processes, such as retrieval strategy and retrieval effort. Retrieval strategy refers to the way that items are recollected, and is based on the way that they were initially encoded. Retrieval effort refers to the difficulty associated with recalling specific events from memory.

Evidence from PET, fMRI, and electrophysiological DC-ERP studies suggests that a mental state or ‘mode’ of processing is established and maintained for the duration of a retrieval task. Activations within the right pre-frontal cortex and adjacent regions have consistently been reported when episodic retrieval tasks are compared with non-episodic retrieval tasks, indicating the involvement of these regions in episodic retrieval.

Electrophysiological techniques have been used to examine more transient processes involved in episodic retrieval. These have been used to identify neural regions involved at the actual time an episode is recalled. This recollection is referred to as retrieval success. Retrieval success has been extensively investigated by examining differences in the EEG or ERPs associated with the identification of previously studied old and novel new items. A significant EEG finding is that theta activity is more synchronous during the identification of old items than new items. This is thought to reflect hippocampal activity transmitted to the cortex via hippocampo-cortical feedback loops. The ERP literature has consistently reported old/new differences over left parietal regions, and also, depending on the nature of the retrieval task, over right frontal regions. Parietal old/new differences occur regardless of whether recognition is based on familiarity or on recollection. On the other hand, right frontal differences occur more specifically with recollection processes. Depth electrode findings suggest that the parietal scalp ERPs originate within the medial temporal lobes. Similarly, the scalp

59 Neural correlates of object recognition memory

ERPs over right frontal regions are thought to reflect activity in the underlying right frontal cortex. It is thought that interaction between the medial temporal lobes and frontal regions enables the recognition of the object to be linked to the context of experiencing it.

The majority of both EEG and ERP studies have utilized verbal stimuli. Most studies using pictorial stimuli have used pictures of common objects, and have reported larger ERP responses than those obtained using word stimuli. This is generally attributed to the greater richness of pictorial stimuli. This conclusion should, however, be approached with caution given the paucity of studies using pictorial stimuli.

Other aspects of episodic retrieval, such as effort and strategy, termed retrieval set effects, have also been investigated, revealing, in particular, frontal lobe involvement in these functions. With respect to strategy, transient item-related retrieval effects are influenced by the tonically maintained mental set adopted for a particular strategy. The strategy adopted for retrieval may depend on the encoding strategy that had been used. As yet, however, investigations into retrieval strategy are preliminary, and findings are inconclusive. There has been lack of agreement over the frontal lateralization of activity associated with retrieval effort. It has been suggested that distinct neural structures may not be devoted to retrieval effort processes. Instead, ‘the neural correlates of increasing effort will be manifest as increased activity of whatever brain regions are engaged by the retrieval task in question’ (Rugg and Wilding 2000, p.114).

In conclusion therefore, lesions studies in humans and non-human primates have identified the medial temporal and frontal lobes as the gross neural regions important in ORM. In normally functioning humans, ORM has been investigated using tests of episodic memory retrieval. To date, retrieval mode and retrieval success processes have, with only one exception (Duzel et al. 1999), been investigated independently using different techniques. The simultaneous investigation of these processes should help further clarify the issue of whether sustained task and transient item-related frontal activity is generated in the same neural regions. Furthermore, retrieval mode and retrieval success findings have, in the main, been derived from studies that have used verbal stimuli. Due to the paucity of studies that have used object stimuli, it is not clear whether the observed patterns of neural activity are specific for verbal stimuli, or

60 Neural correlates of object recognition memory

whether these patterns are characteristic of retrieval memory in general. The present study has attempted to add to the understanding of these issues.

61

Chapter 3 Steady-State Probe Topography

The following chapter will explain how the Steady-State Probe Topography (SSPT) technique can be used to investigate neural activity associated with cognitive processes. The main features of this technique will be explained, and its advantages and limitations in relation to other neural imaging methods such as fMRI, PET and electrophysiological techniques will be discussed. Firstly, a definition of steady-state evoked potentials and an overview of early attempts to use steady-state evoked potentials in the study of cognitive processes are presented. This overview is followed by a description of the SSPT technique, first described by Silberstein et al. (1990b). Findings from a number of studies utilizing the SSPT technique will then be summarized to provide the reader with an appreciation of the potential of this technique. Following this synopsis of the literature, hypotheses specific to this study dealing with object recognition memory will be formulated.

3.1 Steady-state evoked potentials compared with transient evoked potentials

To compare and contrast transient and steady-state evoked potentials, Regan (1977a) used an electrical engineering analogy where the brain is compared to a circuit which receives, for example, a stepped or pulsed electrical input. The system gives an initial response to the input, which dies away quickly, the so-called transient response. This transient response can be compared to the traditional ERP, which is a transient electrical potential generated by the brain in response to some stimulus (see Chapter 2). Alternatively, the same system can be driven by a long train of rapidly repeating stimuli. Once the system has settled, the observed response contains the same sequence of waveforms as the input. Steady-state potentials are unlike transient evoked potentials in that the response to one stimulus has not died away before the next stimulus is delivered.

Transient evoked potentials, where individual peaks are well defined, are usually described in the time domain. Conversely, the repetitive nature of a steady-state evoked

Steady-State Probe Topography

potential is more easily represented in the frequency domain, where it can be characterized as a number of discrete frequency ‘bands’ superimposed on the background EEG frequency spectrum. Fourier techniques are usually applied to calculate the amplitude and phase components of a steady-state evoked potential. These are usually calculated for the frequency of stimulation or for multiples of this frequency (see Regan 1977b).

In a linear system, the information contained in a transient evoked potential and a steady-state evoked potential is exactly equivalent (Regan 1977b). However, in non- linear systems, such as the human central nervous system, steady-state evoked potentials can provide information to complement information derived from transient evoked potentials. One example of the clinical utility of SSVEPs is in the diagnosis of multiple sclerosis (Regan et al. 1977). Regan et al. (1977) reported that in multiple sclerosis, potentials evoked by a medium-frequency (13-25Hz) flicker were delayed, whereas high-frequency (40-60Hz) evoked potentials were not. Furthermore, SSVEPs for patterned and unpatterned stimuli were shown to be sensitive to retrobulbar neuritis (Regan et al. 1977), whereas visually evoked potentials (VEPs) in response to a transient flash gave no reliable indication of the disorder (Halliday 1977).

Regan (1989) described a number of advantages in using steady-state evoked potentials over transient evoked potentials. One advantage is that it is easier to quantify steady- state evoked potentials as these can be defined by a small number of discrete frequency components, each consisting of two numbers, the amplitude and the phase. This reduced set of information means that recording times can be significantly shorter than those required for transient ERPs. This can be particularly advantageous in clinical situations (Regan 1976). Moreover, the quantification of transient evoked potentials can be difficult, especially when the waveform is atypical. For example, experimenters may adopt different techniques to resolve overlapping peaks and to identify baseline levels. In contrast, the quantification of steady-state evoked potentials ‘has a firm mathematical basis; the quantification can be repeated in any evoked potential laboratory, and close agreement between different experimenters is to be expected’ (Regan 1989). Regan (1989) also noted that the recording of steady-state evoked potentials is easier as mains interference is separated in the frequency domain, whereas in the time domain, it overlaps with the transient evoked potential.

63 Steady-State Probe Topography

3.2 Steady-state evoked potentials in the study of cognitive processes

In contrast to the well-documented ERP changes associated with cognition, Regan failed to show evidence for cognition-related changes in steady-state evoked potentials. Regan indicated that ‘steady-state VEPs do not seem much affected by attention, though this is a well documented effect for transient VEPs’ (Regan 1977a). Furthermore he suggested that ‘if you wish to study, eg., attention or the orienting response, then steady- state evoked potentials are most likely completely useless, so that you must use transient EPs’ (Regan 1977b).

The first study to show a link between steady-state evoked potentials and human cognitive activity was performed by Wilson and O’Donnell (1986). They found a statistically significant correlation between the behavioural scores in a Sternberg memory scanning task and the apparent latency of the SSVEP. SSVEPs generated by stimulation in the medium frequency range (15-23Hz) were correlated with the slope of the graph of response speed versus apparent latency. Those generated in the high frequency range (40-59Hz) were related to the response speed intercept for zero items. These correlations indicated that the SSVEP latency was related to the speed of cognitive processing. In a subsequent study by this group (Wilson and O'Donnell 1988), the relationship between cognitive processes and the SSVEP was investigated by examining changes in the SSVEP during the performance of a mental workload task. However, no significant correlation was found between these variables.

Steady-state evoked amplitude and phase components have been assumed to be stable over time, and have traditionally been evaluated over long time intervals (Regan 1977a). Regan (1989) explained that, in principle, a steady-state evoked potential is a repetitive evoked potential ‘whose constituent discrete frequency components remain constant in amplitude and phase over an infinitely long time period.’ In one study, SSVEP amplitude and phase components for each stimulus frequency were evaluated over a 1 minute period (Regan 1976), and in an earlier study, fluctuations in the SSVEP were evaluated over a 7 minute time period, with the filter time constant set to 7s (Regan 1977b). The author reported that over the 7 minute period, the SSVEP latency varied by 1 percent and the SSVEP amplitude varied by 7 percent. When a 1 minute evaluation period was used, much more variability was observed. This was considered the result of

64 Steady-State Probe Topography

the wider bandwidth associated with the shorter evaluation period allowing more background EEG noise through.

Galambos and Makeig (1985) suggested that the variability in steady-state evoked potentials, which Regan attributed to background EEG noise, might be related to cognitive processes. They attempted to find a relationship between ‘shifts in arousal’ and the steady-state auditory evoked potential (Galambos and Makeig 1988) but were unsuccessful. Linden et al. (1987), who examined the effect of selective attention on the auditory steady-state evoked potential, were similarly unsuccessful in demonstrating a relationship between the steady-state evoked potential and cognitive processes despite observing significant changes in the late components of the transient ERP during selective attention.

Silberstein et al. (1990b) suggested factors that may have accounted for the apparent insensitivity of steady-state evoked potentials to cognitive processes in the study of Wilson and O’Donnell (1986) and other early studies. They suggested that cognitive effects might have been reflected in the SSVEP amplitude rather than in the apparent latency of the SSVEP. Furthermore, only central, occipital and parietal sites had been investigated, limiting the possibility of observing cognition-related changes at other sites. Using a technique subsequently termed Steady-State Probe Topography (SSPT), Silberstein et al. (1990b) demonstrated, for the first time, a correlation between the amplitude of the SSVEP and visual vigilance. The SSPT technique, together with other more recent findings of SSVEP correlates of human cognitive processes, will be outlined in the next section.

3.3 Steady-State Probe Topography (SSPT)

The SSPT technique described by Silberstein et al. (1990b) incorporated three main features designed to maximize the possibility of observing cognition-related changes in the SSVEP. These were (i) the Probe-ERP approach, in which an irrelevant steady-state visual stimulus is presented in conjunction with the presentation of the cognitive task under investigation, (ii) a high density of scalp recording sites, which should reveal localized SSVEP topographic changes, and (iii) an analysis technique whereby the

65 Steady-State Probe Topography

SSVEP could be calculated in as little as 1s, allowing cognitive processes to be continuously monitored over time. Each of these three features, as explained by Silberstein et al. (1990b), will be described in turn.

‘The Probe-ERP technique was employed wherein the ERP stimulus was presented in a manner as to be distinct from and irrelevant to the cognitive task undertaken by the subjects.’ (Silberstein et al. 1990b, p. 338)

This technique was based on the ‘probe paradigm,’ first described by Galin and Ellis (1975) and reviewed comprehensively by Papanicolaou and Johnstone (1984). Galin and Ellis (1975) presented task-irrelevant visual stimuli at intervals of 3s while subjects performed block design and writing tasks. They observed a probe-ERP asymmetry that was task dependent, and was described as being similar to the task-related asymmetry of EEG alpha power distribution. Shucard et al. (1977) later used auditory evoked potentials (AEPs) as a probe response to investigate the possible lateralization of brain function during the performance of various cognitive tasks. They found a hemispheric asymmetry in the amplitude of the AEP that was related to the subject’s mode of cognitive processing.

The premise underlying the Probe-ERP technique is that increases in regional cortical activity associated with cognitive processing of the task will result in smaller potentials evoked by an irrelevant probe stimulus (Papanicolaou and Johnstone 1984; Silberstein et al. 1990b). The Probe-ERP premise is supported by evidence of an increase in regional cerebral bloodflow accompanied by a reduction in the Probe-ERP amplitude (Papanicolaou et al. 1987; Papanicolaou and Johnstone 1984). Papanicolaou and Johnstone (1984) proposed a number of models which might explain the mechanism by which the probe ERP is attenuated. They suggested that a ‘limited resource’ model best accounted for the observed changes. This model assumes that cortical regions have a limited capacity to process multiple inputs, and therefore are less responsive to the probe stimulus when simultaneously engaged in the task of interest.

More recently, however, Nield et al. (1998) questioned the ‘limited resource model’ when reductions in the 13Hz SSVEP amplitude were found to occur simultaneously

66 Steady-State Probe Topography

with increases in the evoked 40Hz SSVEP amplitude during a heightened attentional state.

Studies using the Probe-ERP technique in a variety of experimental situations have been reviewed extensively by Papanicolaou and Johnstone (1984) and by Silberstein et al. (1990b). Experimental findings include: left hemisphere attenuation of an auditory probe ERP in a covert articulation task (Papanicolaou et al. 1983), right parietal attenuation of a visual probe ERP in a visuo-spatial task requiring mental rotation of geometrical figures (Johnstone et al. 1984) and differentiation between dyslexic and control children on the basis of the amplitude of the visual probe ERP when the demands in a reading task were increased (Johnstone et al. 1984).

Silberstein and colleagues (Silberstein et al. 1995a; Silberstein et al. 1990b) employed a continuous visual flicker as their probe stimulus. The SSVEP produced by this stimulus was characterized by reductions in amplitude during a period of heightened visual vigilance (see section 3.5.1). Silberstein et al. (1995a) attributed reductions in the amplitude of the SSVEP to increased regional brain activity. They proposed that the amplitude reductions observed were akin to the phenomenon of event-related desynchronisation (ERD) described by Pfurtscheller and Aranibar (1977a) and Pfurtscheller and Klimesch (1990), in which regional increases in cortical activity are indexed by transient reductions in alpha amplitude.

More recently, however, Silberstein (1998) reported SSVEP amplitude increases occurring with certain types of cognitive activity. Transient increases in the amplitude of the 13Hz SSVEP were found during the performance of a spatial working memory task. It was proposed that with certain types of cognitive activity, specific linkages between brain regions and between cortical layers are established. In the case of a spatial working memory task, information is ‘reticulated between reciprocally related neocortical regions forming reentrant loops’ (p. 34) resulting in an increased SSVEP amplitude. Silberstein (1998) speculated that this may be the mechanism for holding information actively or ‘on-line.’

The aforementioned interpretations of SSVEP amplitude reductions and increases were reinforced in a recent study. Silberstein et al. (2001) reported that the ‘intake’ and ‘hold’

67 Steady-State Probe Topography

phases in an object working memory task were associated with SSVEP amplitude decreases and increases respectively. During the ‘intake,’ or encoding phase of the task, SSVEP amplitude reductions were observed at a number of sites, including the right occipito-parietal region. The authors noted that reductions during the intake, or perceptual, phase of the task are consistent with previous studies by this group in which amplitude reductions were observed over occipito-parietal regions during a visual vigilance task (Silberstein et al. 1990a). During the ‘hold’ phase of the task, subjects held the encoded representation of the object(s) in working memory. Increases in SSVEP amplitude associated with the ‘hold’ phase were likened to those reported in the aforementioned study (Silberstein 1998) involving spatial working memory. That is, linkages between brain regions were established, and information was reticulated around re-entrant loops.

The phase of the SSVEP for the task of interest may be in advance of, or lag behind, the phase for some other task, generally a reference or control task. Phase advances or lags are also referred to as latency reductions or increases respectively. Silberstein et al. (1996) observed that faster response times in the performance of a continuous performance task were associated with larger frontal SSVEP latency reductions. It was proposed that latency reductions are a manifestation of transient increases in neural information processing speed reflecting increased coupling strength between neural populations. More recently, Silberstein et al (2000) speculated that SSVEP latency reductions might index excitatory processes. This suggestion was based on the fact that, unlike normal controls, schizophrenic patients did not show prefrontal latency reductions during a continuous performance task.

‘Brain electrical activity was recorded from 64 scalp sites within the area defined by the International 10-20 system.’ (Silberstein et al. 1990b, p. 338)

The use of 64 scalp recording sites achieves a spatial resolution adequate for observing effects over gross brain regions. Silberstein et al. (1990b) quote a number of studies where the recommended electrode separation appears to be in the order of a few centimetres. The SSPT technique is concordant with this recommendation, using the traditional 10-20 International recording sites with additional electrodes between them (see section 4.5 for more details).

68 Steady-State Probe Topography

‘A Fourier analyser with a 10 second integration period was used to determine SSVEP magnitude.’ (Silberstein et al. 1990b, p. 338)

In this situation, variations in the SSVEP amplitude over time were shown to be related to cognitive changes, rather than to background EEG noise as suggested by Regan (1977b).

Silberstein et al. (1990b) used a temporal resolution of 10s, although in a more recent study it was suggested ‘that it is possible to estimate the amplitude and phase of the SSVEP using as little as 1 to 5s of recorded activity’ (Silberstein et al. 1995a). In theory, using a 13Hz probe stimulus, the SSVEP can be calculated using as little as 1/13s (77ms) of data; however, the level of background noise would be too high to allow any meaningful interpretation of the result. As with traditional transient ERP methods, averaging techniques can be applied to overcome this signal-to-noise problem. (Silberstein et al. 1995a). In the present study, a temporal resolution of 0.87s was used. Reliable results were obtained at this resolution by averaging data for only 5 stimulus items. Extrapolating this result to 20 or more averages could potentially yield reliable results with a greater than 4-fold improvement in temporal resolution (ie. <200ms). In a very recent study by this group, reliable results were in fact obtained using a temporal resolution of 180ms (Harris et al. 2001).

The ability to tailor the evaluation period of the SSVEP allows one to focus on the different cognitive processes that occur during a particular cognitive activity, for example, visual vigilance, whether occurring over a period of a few hundred milliseconds, seconds, minutes, or even hours. Silberstein et al. (1995a) argued that, in contrast to techniques such as positron emission tomography (PET) which have a poor temporal resolution (typically 60s), SSPT possesses the ‘temporal resolution as well as the temporal continuity’ required to monitor relatively rapid cognitive processes. Although not possessing the temporal resolution of ERP techniques, SSPT allows investigation of cognitive processes over a range of time scales. For example, a short evaluation period allows the monitoring of the more rapidly changing patterns of neural activity associated with the processing of individual stimuli. Conversely, an evaluation

69 Steady-State Probe Topography

period the length of the task being investigated allows more sustained, state related, neural activity to be evaluated.

3.4 SSPT and recording artifacts

Steady-state evoked potentials are, in general, more resilient to artifact than are transient evoked potentials (see Regan 1989). If an adequate number of stimulus cycles is averaged, non-probe related signal, such as background EEG and recording artifact, can be greatly reduced. A greater number of averages effectually reduces bandwidth, and therefore increases the signal to noise ratio. Regan (1977b) assessed the SSVEP over relatively long time periods of several minutes, and thereby achieved a high signal to noise ratio. Silberstein et al. (1990b), however, were interested in cognitive changes occurring over a much shorter time period, and therefore far fewer stimulus cycles were used when processing the SSVEP. Nevertheless, Silberstein et al. (1993) found that with an integration period of only a few seconds, artifact such as EMG, EOG, eye-blinks, and 50Hz mains interference (see Figure 3.1) had very little effect on the SSVEP. Varying amounts of known artifact were added to the EEG before the SSVEP was calculated. Where the variances of the artifact and the EEG signal were equal, giving a much poorer signal to noise ratio than normally experienced in practice, the resultant SSVEP was very similar to the SSVEP with no added artifact. The source of artifact with the greatest effect on the SSVEP, although still small, appeared to be EMG. The 50Hz mains artifact had virtually no effect. This is understandable given that mains interference is concentrated in a very narrow frequency band far removed from the 13Hz SSVEP.

3.5 Overview of investigations utilizing the SSPT technique

To date, the SSPT technique has been used to investigate various facets of cognition, such as visual vigilance, attention, learning, planning and working memory. The technique has also been employed in clinical contexts, for example, with children diagnosed with Attention Deficit Hyperactivity Disorder (ADHD). A selection of

70 Steady-State Probe Topography

Figure 3.1 The effect of various forms of artifact on an SSVEP amplitude time series Various forms of artifact were added to a 30-second epoch of EEG, and the SSVEP amplitude was calculated. The graphs in the left column show a small sample of EEG with each form of artifact added. The graphs in the right column show the comparison of the SSVEP amplitude with and without added artifact. The artifact with the largest effect on the SSVEP amplitude appeared to be EMG. This is understandable given that EMG artifact can contain frequencies within the band-pass of the Fourier analyser. (Silberstein et al. (1993); diagram supplied by Burkitt (1996))

71 Steady-State Probe Topography

studies will be briefly reviewed to help provide insight into the use of the technique and to indicate the potential of the technique in the area of object recognition memory.

3.5.1 Visual vigilance

In Silberstein et al.’s (1990b) study, where a correlation between visual vigilance and the amplitude of the SSVEP was observed, different phases of the task elicited different SSVEP response patterns. Fourteen right-handed subjects were instructed to fixate on the centre of a monitor and observe a series of shapes that would appear there. The total task was divided into three parts. Each part, lasting 3 minutes, consisted of watching a number of sequentially presented squares for 1 minute, followed by a number of sequentially presented circles for 1 minute, and then sequentially presented squares again for 1 minute. This sequence of shapes was presented twice, and subjects had simply to watch throughout. Prior to commencement of the third part, subjects were told that one of the circles had been modified and were required to identify the modification. Subjects were naïve as to the nature of the modification and where in the sequence of circles it would occur. A small monetary reward was offered for the correct identification of the modification. This reward was intended to serve two purposes: firstly, to encourage optimum performance of this part of the task, and secondly and more importantly, to increase the level of attention during this third part in relation to the second part. The first part served as a practice task to eliminate any novelty involved with the recording environment and the performance of the task.

A temporal resolution of 10s was selected for the calculation of the SSVEP amplitude and phase, as the authors were interested in monitoring slow attentional shifts associated with changes in visual vigilance during performance of the task. The SSVEP elicited during the presentation of the circles in the third part and the second part were compared.

Following the presentation of the first circle, the cue for the possible appearance of the modification, an attenuation of the SSVEP amplitude occurred at centro-parietal sites. At this point, subjects would have been anticipating the appearance of the modified circle. The identification of the modification, which was made to the last circle in the 1

72 Steady-State Probe Topography

minute period, was associated with an SSVEP amplitude attenuation over the occipito- parietal and prefrontal regions.

According to the Probe paradigm, these localized reductions in SSVEP amplitude could have reflected increases in regional cortical activity. Silberstein et al. noted that this interpretation is consistent with neuropsychological findings, cerebral blood flow and metabolism findings, findings from lesion studies, and electrophysiological findings. This demonstration of a relationship between the SSVEP amplitude and cognitive processes indicated that ‘SSPT warrants further investigation as an indicator of regional brain activity associated with cognitive processes.’

3.5.2 Planning - Wisconsin Card Sorting Task

Using a much shorter temporal resolution of only 0.77s with the SSPT technique, Silberstein et al. (1995a) observed more transient effects which occurred during the performance of the Wisconsin Card Sorting (WCS) task. The WCS task is a well- documented neuropsychological test thought to engage the prefrontal lobes (Milner 1963). Subjects are required to determine the criterion for sorting cards into four categories. Once they have determined the sort criterion, subjects perform successful sorts according to that criterion. After 6 to 10 successful sorts, the criterion is changed and the process is repeated. At the point following the change in sort criterion, subjects must dispense with the previously determined method of sorting and determine the new one, a process which is completed in a few seconds. This period of ‘planning’ the next move is known to place demands on the prefrontal cortex (Milner 1963; Milner 1964). Silberstein et al. (1995a) observed a transient SSVEP amplitude reduction and a simultaneous phase advance over prefrontal and right temporal regions at the time subjects had to determine the new criterion for sorting. These SSVEP changes were attributed to neural activation. As noted by the authors, previous techniques used to examine prefrontal involvement during performance of the WCS task had lacked the temporal resolution and sensitivity to detect cognitive processes occurring during this short time period.

73 Steady-State Probe Topography

3.5.3 Attention - Continuous Performance Task

Transient SSVEP effects have also been observed during the performance of a continuous performance task (CPT) (Silberstein et al. 1996). In this study, young adult subjects viewed a series of letters on a monitor, presented at a rate of one every 2s. Prior to the commencement of the task, subjects were instructed to identify and respond to the appearance of the letter ‘X’ only if preceded by the letter ‘A’. The time between the letter ‘A’ and ‘X’ is the point of greatest visual attention, and it was hypothesized that this interval would be associated with reductions in SSVEP amplitude at scalp sites involved in visual attention.

Again using the SSPT technique with a temporal resolution of 0.77s, SSVEP amplitude attenuation and decreased latency were observed at parietal sites during the ‘A-X’ interval. These changes were absent during an equivalent interval where an ‘X’ was preceded by a letter other than an ‘A.’ Another point of interest in the task was the correct identification of the target ‘X.’ This event was associated with transient reductions in the SSVEP amplitude and latency, interpreted as reflecting excitatory processes, over central and frontal areas. The authors noted that this interpretation is in agreement with other studies suggesting that these regions are involved with response selection and execution. It was also noted that faster response times to targets were associated with increased latency reductions frontally, suggesting that SSVEP latency reductions index the speed of information processing, and therefore the extent of coupling between cortical regions.

3.5.4 Clinical application of SSPT - ADHD

Several studies suggest that the SSPT technique may be applied to clinical situations. For example, when children diagnosed with ADHD performed the continuous performance ‘AX’ task described above, there was no attenuation in SSVEP amplitude in the interval between the ‘A’ and the ‘X’ (Farrow et al. 1996). In contrast, young control subjects showed a sustained parietal SSVEP amplitude attenuation similar to that found in the young adults. The authors suggested that the lack of SSVEP

74 Steady-State Probe Topography

attenuation in the ADHD group was due to an inability to sustain attention during the ‘A-X’ interval.

Latency effects were examined by Silberstein et al. (1998), who employed a similar task with boys diagnosed with ADHD and normal controls. In this study, only correct trials were considered. Transient latency reductions were observed over prefrontal regions in the normal controls following the presentation of the letters ‘A’ and ‘X’. However, the boys with ADHD failed to show these latency reductions. It was thus concluded that the absence of a latency reduction in ADHD subjects could have been due to inadequate excitation of prefrontal networks.

3.5.5 Spatial working memory

In contrast to the amplitude reductions associated with heightened attention described in previous studies, Silberstein et al. (1998) found amplitude increases associated with cognitive effort in a spatial working memory task. They proposed that it is the nature of the mental activity that determines whether the SSVEP amplitude increases or decreases. In this study, subjects were instructed to hold in memory the position of 3 dots located on an imaginary circle. During the 3s hold period, the 13Hz SSVEP amplitude showed a pronounced increase at frontal and parietal sites when compared with that for a control task matched for perceptual and motor aspects. The authors proposed that the increase in SSVEP amplitude in regions thought to be involved in the task reflected the reticulation of information between ‘reciprocally related neocortical regions forming re-entrant loops.’ (p. 34) This finding was consistent with a number of studies where alpha activity was enhanced during certain types of cognitive task. Ray and Cole (1985), for example, reported that during the performance of a ‘rejection’ task where subjects had to attend to mental imagery, alpha activity was significantly enhanced. A similar enhancement was also reported in a study using magnetoencephalography (MEG) (Tesche et al. 1995) where subjects had to visualize a set of imaginary movements. In another study that investigated the effect of short-term memory on the EEG (Krause et al. 1996), enhanced alpha activity was observed when

75 Steady-State Probe Topography

subjects had to determine whether a vowel had been presented previously as a target to be memorized.

3.6 Conclusions

Findings from the SSPT studies outlined above indicate that the SSPT technique possesses both the temporal resolution and the temporal continuity necessary for the investigation of cognitive processes. Moreover, the ability to vary the SSVEP evaluation period allows investigation of cognitive processes occurring over different time scales. The spatial resolution of SSPT using 64 electrodes, although not as good as that obtained using PET and fMRI methods, appears satisfactory for investigating neural activity associated with gross brain regions. Furthermore, the use of SSPT in the study of attentional and memory processes has produced results that show the value of this technique in monitoring neural activity associated with cognitive processes, and thus warrants further investigation.

3.7 Formulation of hypotheses for the present investigation

The main aim of this study is to use SSPT to investigate both sustained, task-related, and transient, item-related, neural activity associated with episodic retrieval memory. The first hypothesis is concerned with sustained task-related effects associated with the maintenance of a retrieval mode. The second hypothesis relates to transient effects associated with successful retrieval from memory.

The right prefrontal region has been implicated in the establishment and maintenance of the mental set for episodic retrieval, also referred to as retrieval mode. As explained in Chapter 2, PET and fMRI data have consistently indicated that the prefrontal cortex (BA10) and adjacent areas are involved in maintaining a retrieval mode. In a study utilizing both DC-ERP recordings and PET techniques, a sustained ERP positive shift was observed over right frontopolar regions during an episodic retrieval task, but not during a semantic retrieval task (Duzel et al. 1999). PET data from the same study revealed activation of the right anterior prefrontal cortex (BA10) for the episodic task

76 Steady-State Probe Topography

only. The authors suggested that the DC-ERP data represents the electrophysiological correlate of the PET activations, and therefore concluded that episodic retrieval mode is manifest as a tonically maintained, item-independent state. Thus, converging evidence obtained using different techniques has indicated the involvement of right frontal regions in the maintenance of a retrieval mode. One of the main aims of this study was to investigate the topography of sustained task-related SSVEP effects associated with episodic memory retrieval. This was done by comparing the average of the SSVEP amplitude and latency values obtained over the entire duration of an episodic retrieval task with those for a non-episodic retrieval baseline task.

It is hypothesized that retrieval mode processes engaged during the performance of an episodic retrieval task will be reflected in sustained SSVEP amplitude and latency effects over right frontal regions.

Transient neural activity associated with successful retrieval processes has been investigated using EEG and ERP electrophysiological techniques (see Chapter 2), albeit with a relatively low spatial resolution. ERP studies, in particular, have consistently shown that ERPs associated with the recognition of previously studied items, compared with those associated with the correct identification of unstudied items (ERP old/new effect), show a transient positivity over parietal regions between 400 and 1000ms after item presentation. This positivity is generally most prominent over left parietal regions. However, nearly all such studies have utilized verbal stimuli, and so this apparent asymmetry could be due to the nature of the stimuli. Transient ERP old/new effects have also been observed over right frontal regions between approximately 400 and 1400ms post stimulus onset. While the parietal ERP old/new effect appears to be observed consistently during successful retrieval, the right frontal ERP old/new effect appears to depend on the nature of the task and is not always observed. Right frontal ERP old/new effects have been associated with post-retrieval monitoring processes, and are mainly observed when recognition involves recollection of the study context.

The investigation of transient effects in this study will focus on the topography of SSVEP amplitude and latency changes associated with the retrieval from memory of abstract, two-dimensional objects, and how these spatio-temporal patterns vary with increases in memory load. It is anticipated that increases in memory load will result in

77 Steady-State Probe Topography

increased utilization of those neural regions involved in retrieval processes leading to recognition of previously studied objects.

It is hypothesized that increases in memory load during successful retrieval will be associated with transient graded SSVEP amplitude and latency changes over bilateral parietal and right frontal regions.

78

Chapter 4 Methods

4.1 Introduction

Neural activity associated with episodic memory retrieval can be looked at from a number of different viewpoints. For example, sustained activity associated with the maintenance of a mental state during the course of a retrieval task has been discussed in terms of retrieval mode (Tulving 1983). However, the performance of a retrieval task is also associated with rapidly changing patterns of neural activity associated with more transient processes, for example, the initial perception of a stimulus, recognition of the stimulus, deciding whether the stimulus is a target, a motor response, and also post- retrieval processes associated with monitoring and recollection. Steady-State Probe Topography (SSPT) appears to be an ideal technique to study episodic memory retrieval as the length of the evaluation period over which the SSVEP components are calculated can be chosen to suit the time-course of cognitive changes being investigated. A long evaluation period is appropriate for the investigation of modal effects, while a short evaluation period is appropriate for the investigation of transient effects that occur much more rapidly.

This chapter describes the experimental methods employed to test the hypotheses presented in Chapter 3. Section 4.2 deals with the experimental design used to investigate object recognition memory. It will include a description of the basic study- test task structure and the full battery of cognitive tasks designed to examine the effect of memory load on object recognition memory. Section 4.3 describes the procedure used to present the battery of cognitive tasks to each subject and the software used to control task presentation. Subjects who participated in the study are described in Section 4.4. The steady-state visual flicker employed as the probe stimulus is discussed in Section 4.5, and recording procedures are described in Section 4.6. Section 4.7 explains how behavioural data was analysed. Finally, section 4.8 explains the analysis of

Methods

electrophysiological data obtained using both long and short evaluation periods to investigate modal and transient neural activity respectively.

4.2 Cognitive task design

The basic structure of each task is shown in Figure 4.1. Each task consisted of a study phase and a test phase. During the study phase, subjects memorized 1, 3 or 5 sequentially presented, abstract, two-dimensional objects. Each object remained visible for a relatively extended period (3.5s) to facilitate consolidation into longer-term memory. During the test phase of the task, in the ‘task interval,’ subjects viewed sequentially a larger number of objects that included memorized objects (targets) and new objects (non-targets). Subjects were required to identify targets by pressing a button with their right hand ‘as quickly and as accurately as possible.’ The ratio of non- target to target objects was 3:1, and in each test phase, 15 non-target objects and 5 target objects were presented (see Figure 4.2). Non-target objects were designed to look similar to target objects, thereby making discrimination more difficult. Because a detailed match had to be made with objects in memory, it was expected that the use of explicit memory processes would be maximized. With more objects to remember, and a corresponding increase in the number of distractor objects with similar appearance, the task becomes more difficult and places a greater load on retrieval processes.

study phase test phase

memorize objects numbers interval task interval numbers interval (1,3, or 5) (20 trials) (20 trials) (20 trials) 40s 40s 40s

object blank object

1400ms 600ms

(1 trial)

Figure 4.1 Study-test experimental design

80 Methods

In the test phase, the task interval was both preceded and followed by a 40s ‘numbers interval.’ Subjects had been instructed to relax and press the hand-held button slowly whenever the number ‘4’ appeared in these numbers intervals that would precede and follow the task interval. The purpose of the first numbers interval was threefold: (i) to ensure that remembered objects had been transferred to longer-term memory stores, (ii) to distract subjects so that memorized items would not be rehearsed, thus limiting working-memory or short-term memory processes, and (iii) to orient subjects to a computerized task prior to performing the object retrieval operations of the task interval. The final numbers interval was included so that differences in the SSVEP due to possible differences in attention levels between the beginning and end of the task could be compared. This analysis, however, is not included in this study.

task block (test phase) 12341234123412341234*****task interval 12341234123412341234*****

*** * * bas1 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 *** * * study phase bas2 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 (memorize) ** * * * m1p * * * * * m1 * * * * * m3p * * * * * m3 * * * * * m5p * * * * * m5 * response required

Figure 4.2 Experimental task design Experimental task design for order 1 showing both study and test phases. For order 2, the m1p/m1 and m3p/m3 task block orders were reversed.

The complete experimental task design is illustrated in Figure 4.2. First, two baseline task blocks were presented to all subjects. In these blocks, the requirements of the task interval were identical to those of the numbers intervals. In other words, subjects were required to press the button when the number ‘4’ occurred for the full two minutes. However, only the middle 40s comprised the actual baseline task. During the first of

81 Methods

these baseline task blocks (bas1), the steady-state visual flicker (probe stimulus) was turned-on half way through, that is, after one minute. This was done so that any spontaneous 13Hz signal in the background EEG (first half of the task) could be compared with the driven 13Hz SSVEP response (second half of the task). Although not important to the current investigation, this was standard practice at the Brain Sciences Institute as it was a straightforward method of checking that the recording system was working correctly. The result of this analysis is provided in Appendix D. The visual flicker remained on throughout the second baseline task block (bas2). The bas2 task interval was used a reference task for the memory load tasks so that task-related effects could be investigated. The baseline tasks also served to further reduce any effects of novelty and anxiety associated with an unfamiliar environment.

Number and object stimuli were visible for 1400ms and were followed by a blank interval of 600ms, yielding a total of 2000ms per stimulus item. Each task block consisted of a 40s numbers interval, followed by a 40s task interval, followed by another 40s numbers interval, giving a total duration of 120s. In other words, the transition from numbers to task in the practice and main memory tasks occurred after 20 stimuli (40s) and from task back to numbers after another 40 stimuli (80s). The baseline interval for analysis comprised the middle 40s of the bas2 task block.

On completion of the two baseline task blocks, six task blocks of the structure illustrated in Figure 4.1 were presented. Three different memory load tasks were used in which 1, 3 or 5 objects were memorized. There were two task blocks for each memory load. The first incorporated a practice task interval (m1p, m3p or m5p, in Figure 4.2) and was included both to familiarize subjects with the memory load, and to facilitate consolidation of studied objects into a longer-term memory. The second task block incorporated the task interval (m1, m3 or m5, in Figure 4.2), the SSVEP data from which was used in the analysis. Memorized objects were identical in both practice and main memory tasks. Thus, for each memory load (m1, m3 and m5), by the time subjects performed the main memory task, each studied item had been viewed at least 3 times, that is, twice during each study phase and at least once during the practice test phase. Furthermore, the overall lag time from the first study phase to the second test phase was approximately 4 to 5 minutes. Thus, in addition to being well encoded, the likelihood of

82 Methods

using of longer-term memory processes to retrieve targets from memory was maximized.

Practice task blocks served primarily to reduce any possible novelty effects. In addition, tasks were partially counterbalanced to reduce order effects. Half of the subjects were given ‘order 1’ (shown in Figure 4.2) and the remaining subjects were given ‘order 2,’ where the m1p/m1 and m3p/m3 task orders were reversed.

As seen in Figure 4.2, targets were inserted into the same position in the sequence in the main memory task for all three memory loads. This was done so that the preceding item history would be comparable for targets and non-targets across each of the different memory load tasks. However, because targets appeared in the same position for each task interval, it was thought that subjects might have been able to predict their appearance. Nevertheless, when asked about this after completion of the recording session, all subjects indicated that the presentation of targets appeared to be random. The fact that different target locations were used in the practice tasks, acting to distract subjects from learning the sequence (see Figure 4.2) probably contributed to this. Furthermore, behavioural and electrophysiological data (see Chapter 5) are consistent with non-prediction of target position.

Task instructions given to subjects are provided in Appendix A.

4.3 Task presentation and stimulus parameters

Each of the stimuli shown in the ‘numbers’ and ‘task’ intervals were presented centrally on a computer monitor. They subtended equal horizontal and vertical angles of 1.3 degrees when viewed at a distance of 1.3 metres. The stimuli had an average luminance of approximately 15.0 Cd/m2 against the monitor background of 1.2 Cd/m2, measured using a Tektronix J16 narrow angle Digital Photometer.

All stimuli were presented on the monitor of the ‘task computer.’ The timing of stimulus presentation was controlled using an external millisecond timer that was

83 Methods

monitored through the games port of the task computer. Subjects’ responses were also monitored through the games port. The computer program that controlled stimulus presentation, timing and response was written by the author using the Microsoft Quick C Editor and was compiled using the Microsoft C compiler, version 5.1. Each stimulus item was ‘drawn’ in real-time to a virtual memory location of the graphics card in the task computer. The program then waited for the next ‘screen refresh’ pulse before displaying the stimulus (Pipingas and Maruff 1991). The games port was then simultaneously inspected for the subject’s response and the elapsed time of stimulus presentation. Once the presentation time had elapsed, a ‘blank’ screen was displayed. If the response button was pressed while the stimulus item was on the screen, the response time was stored in memory. On completion of the task, all presentation and response times were written to a file on the hard disk for off-line analysis. Presentation and response times were determined to an accuracy of 1ms.

As explained in section 4.2, the sequences of stimuli were pre-determined and were stored in ‘script files’ constructed for each task. Script files were also constructed for the presentation of target objects in the study phases. A feature of the software was that the next task was automatically loaded once the preceding task had been completed. Each task was ‘triggered’ by the EEG data acquisition computer via a separate line through the games port of the task computer.

4.4 Subjects

Fifty-one male subjects participated in the study. Their ages ranged from 17 to 48 years, with a mean of 22.50 and standard deviation of 6.28 years, and all were right handed as assessed using the Edinburgh Handedness Inventory (Oldfield 1971). At the time of testing, most subjects were completing undergraduate courses within the university and had a science or engineering background. The others were tutors and lecturers in similar disciplines. Subjects were asked to set aside approximately ninety minutes for the study, and most sessions were completed in approximately 70 minutes. A session included completion of a number of forms, preparation, recording and debriefing. All subjects gave their informed consent to participate in the study and were asked to withdraw if

84 Methods

they had a history of epilepsy. The study was approved by the Swinburne University Human Research Ethics Committee.

Prior to arrival, subjects were assigned to one of two groups, order 1, with a memory load task order of 1, 3 then 5 items, or order 2, with a memory load task order of 3, 1 then 5 items. Subjects were naïve as to the nature of the task design and the group to which they were assigned. Twenty-seven subjects completed order 1, and 24 subjects completed order 2. In a subsequent off-line analysis that was used to assess the integrity of the EEG, 11 subjects were eliminated from subsequent analysis, leaving 19 in the order 1 group and 21 in the order 2 group (total 40).

4.5 Probe stimulus

The probe stimulus used to evoke the SSVEP consisted of a 13Hz sinusoidal flicker subtending a horizontal angle of 160° and a vertical angle of 90°. This was presented to subjects using specially designed glasses. Two half-mirrored strips reflected the flicker into the eyes whilst simultaneously allowing the subject to view the task computer. The flicker was generated by a 4 x 4 array of red light emitting diodes (LEDs) housed in a small Faraday cage to prevent electrical contamination of the EEG. When viewed against the background, the maximum luminance reflected at the peak of the stimulus waveform was 3.2 Cd/m2, and the minimum was 1.2 Cd/m2. The calculated modulation depth was 45% (see Figure 4.3). The optimum luminance was contingent on both the amplitude of the SSVEP response and subject comfort. Results of a separate experiment used to determine the optimum luminance for the present study is provided in Appendix B.

85 Methods

Calculation of Modulation Depth

4

x 3 ) 2 α

2 y Luminance (Cd/m Luminance 1 β

0 0 50 100 150 200 250 Time (ms)

α − yx − 2.12.3 0.2 Modulation Depth ≡= = == 45.0 β + yx + 2.12.3 4.4

Figure 4.3 Calculation of modulation depth Modulation Depth is the ratio of the mean-to-peak to the mean amplitude of the stimulus intensity waveform. This ratio can also be expressed in terms of the measured peak and background luminances x and y.

4.6 Recording

Signals at the scalp were detected using an array of electrodes mounted in a specially designed helmet. The signals were amplified and filtered in two stages, and were digitized and stored on computer. The 13Hz probe stimulus was synchronized to data acquisition enabling comparison of the SSVEP phase with the phase of the stimulus waveform. The EEG helmet and electrodes are described in section 4.6.1. Section 4.6.2 contains a description of the recording instrumentation and the method used to generate the 13Hz probe stimulus.

86 Methods

4.6.1 Electrodes and recording setup

The electrode locations used in this study included all the scalp positions in the International 10-20 System, with additional electrodes located between these sites to give a total of 64 electrodes (see Figure 4.4). This montage gives an average separation between electrodes of 3.2cm, which is adequate for observing effects localized to gross cortical regions (Silberstein et al. 1990b). The electrodes were mounted in a helmet designed at the Swinburne Centre for Applied Neurosciences (SCAN), now the Brain Sciences Institute (Ciorciari et al. 1987). The electrodes were spring-loaded within the helmet, and retracted prior to placement on the subject’s head. Once the helmet was positioned correctly, the electrodes were gently lowered onto the scalp, and conductive gel was introduced through each to facilitate contact between the electrode tip and the scalp. The electrode tip consisted of a silver disc with a layer of silver chloride applied just prior to recording. Electrode impedances varied from a minimum of 5kΩ to a maximum of 35kΩ when measured at 40Hz and with an amplifier input impedance of 200MΩ (Ferree et al. 2001). All electrodes were referenced to a balanced non-cephalic montage (Stephenson and Gibbs 1951) with the nose serving as the ground. Silver/silver chloride surface electrodes were also used for these. The electrocardiogram (ECG) was essentially eliminated using a canceling procedure described by Stephenson and Gibbs (1951).

Fp1 1 2 3 Fp2

0 4

11 5 7 8 9 F8 F7 6 10 F3 F4 12 Fz 20 13 14 15 17 18 19 21 16 27

22 23 24 25 26 T3 C3 Cz C4 T4

28 29 30 31 32 33 34 35 36

38 42 39 40 41 P3 Pz P4 37 45 51 43 48 T5 46 47 49 50 T6

44 52 54 57 53 55 56 58

59 O1 Oz O2 63

60 61 62

Figure 4.4 Sixty-four scalp recording locations Sixty-four recording locations included all International 10-20 sites with additional electrodes located between these sites. 87 Methods

4.6.2 Recording instrumentation and phase locked data acquisition

The instrumentation used to amplify, filter, digitize and store the EEG is depicted in Figure 4.5.

The pre-amplifier stage provided initial amplification close to the subject. Each electrode was wired into one input of a differential amplifier with a fixed gain of 4000, an input impedance of 200MΩ, and a common mode rejection ratio greater than 100dB. The other input of each differential amplifier was common, and was connected to the output of the non-cephalic reference.

A second stage of instrumentation provided further amplification, filtering and sampling of the EEG. Noise was reduced using a programmable active Butterworth filter amplifier with a high frequency cut-off of 26Hz and a low frequency cut-off set to 0.5Hz. The gain could be further increased at this stage. A rotary switch was used allowing five pre-set levels of amplification to cater for inter-subject variability in the EEG amplitude. Sixty-four buffered sample-and-hold circuits were used to simultaneously sample the outputs of the filter amplifiers.

Figure 4.5 Experimental recording arrangement

88 Methods

A plug-in card manufactured by Data Translation Corporation, together with a 486, 66MHz, IBM compatible computer, was used to sample and digitize the 64 channels of EEG data. The EEG signals were digitized to an accuracy of 12 bits by an analogue to digital converter (ADC) that was incorporated into the plug-in card. The 4096 digital levels corresponded to a full-scale range of –5 to +5 volts. Equation 4.1 was used for the conversion of digital levels back to microvolts representing the EEG amplitude. Data was sampled to computer hard disk at a rate of 208Hz per channel. All instrumentation except the plug-in card and computer interface were designed and constructed at the Brain Sciences Institute (Simpson 1997).

 Amp   106   digital    Ampactual =  FSR log ueana   µVolts  FSRdigital   set GainGain var 

Ampdigital 6104.0 Ampactual = µVolts Gainvar

Equation 4.1 Calculation of unamplified EEG amplitude

Calculation of EEG amplitude where Ampactual is the EEG amplitude in microvolts, Ampdigital is the equivalent digital amplitude, FSRdigital (digital full-scale-range) equals 4096, FSRanalogue (analogue full- scale-range) equals 10, Gainset (initial amplification) equals 4000, Gainvar is the variable gain of the filter amplifiers. The factor of106 converts volts to microvolts.

Software for data acquisition was written in the DAOS1 language (Schier 1994). This software was designed to acquire a pre-determined number of samples of data equivalent to length of the task. Software was also written to set the high and low-pass levels of the programmable filter amplifier, as well as the frequency of the sinusoidal probe stimulus.

The instrumentation was designed to synchronize data acquisition with the sinusoidal probe stimulus. Data acquisition was locked to the 13Hz stimulating frequency so that acquisition always commenced on the positive zero crossing of the stimulus cycle. Sixteen data points were sampled for all stimulus cycles, and the samples were acquired at the same point in each stimulus cycle. The advantage of synchronizing data

1 Data Analysis Operating System (DAOS) by Laboratory Software Associates Pty. Ltd., Melbourne.

89 Methods

acquisition with stimulation is that the phase of the SSVEP can compared precisely with that of the stimulus waveform. Coherent demodulation of the SSVEP, utilizing Fourier techniques (see Regan 1989), can subsequently be performed off-line to calculate the SSVEP amplitude and phase for every stimulus cycle. Furthermore, the same coherent demodulation algorithm can be used for any frequency of stimulation, given that the number of samples per stimulus cycle remains the same.

4.7 Analysis of behavioural data

Response times and error scores for both practice and main memory tasks were calculated for all forty subjects. There were five targets within each ‘task interval,’ as discussed in section 4.2. Response times for correct responses only were averaged. The number of errors of omission (missed responses) and commission (responses to non- targets) were noted for each individual. Individual averaged response times and error scores were then averaged across all subjects. The Student’s t test was applied to the response time data to determine whether differences between memory loads were statistically significant. Memory loads of 3 and 1, 5 and 1, and 5 and 3 were compared for both practice and main memory tasks.

4.8 Offline signal processing

The EEG data set obtained for each individual was analysed on an IBM 486 compatible computer using a set of programs written in DAOS and in Microsoft C 5.1. The programs extracted the 13Hz SSVEP from the EEG signal, detected and replaced artifact-contaminated EEG data, averaged the SSVEP over the appropriate period of time to obtain the desired temporal resolution, and averaged the data across all subjects.

4.8.1 Extraction of the 13Hz SSVEP from the EEG signal

The SSVEP Fourier coefficients were obtained for each stimulus cycle using equation 4.2. The 16 EEG data points locked to each stimulus cycle were multiplied by the 90 Methods

corresponding 16 points in the reference waveform (generated in software) and then integrated over the 16 points to give the sine Fourier coefficient. The cosine Fourier coefficient was obtained in a similar way, however, the reference waveform was shifted by 90 degrees prior to multiplication with the EEG data points. All EEG data points for every electrode and task were analysed in a similar way and stored in a file on hard disk as a series of sine/cosine pairs. The SSVEP amplitude and phase can be calculated from these coefficients using equation 4.3 to form a new series of phase/amplitude pairs. This is essentially a conversion from a cartesian to a polar representation of the data. This last calculation was performed after the EEG and Fourier coefficients were examined and corrected for artifacts. Regan (1989) describes methods for recording SSVEPs, as well as the technique of sine/cosine multiplication for calculating the Fourier coefficients.

1 S −1  2π  an = ∑ inTf ∆+ τ cos)(  ()inT ∆+ τ  S∆τ i=0  T  1 S −1  2π  bn = ∑ inTf ∆+ τ sin)(  ()inT ∆+ τ  S∆τ i=0  T 

Equation 4.2 Calculation of single cycle Fourier coefficients

Calculation of SSVEP Fourier components, where an and bn are the cosine and sine Fourier coefficients respectively, n represents the nth stimulus cycle, S is the number of samples per stimulus cycle (16), ∆τ is the time interval between samples, T is the period of one cycle, and f(nT+i∆τ) is the EEG signal.

22 SSVEPamplitude ()+= ba nn

 bn  SSVEPphase = tana    an 

Equation 4.3 Calculation of SSVEP amplitude and phase

Calculation of SSVEP amplitude and phase where an and bn are the cosine and sine Fourier coefficients respectively. Amplitude and phase components can be calculated using either single cycle Fourier coefficients or coefficients that have been calculated by integrating across multiple cycles.

91 Methods

4.8.2 Automatic detection of artifact in the EEG signal

As already explained in section 3.4, the SSVEP is relatively insensitive to artifact because extracting narrow-band phase-locked 13Hz signal eliminates a very substantial proportion of artifact. Nevertheless artifact cannot be ignored altogether. For example, there may be excessive clipping at the input of the ADC due to intermittent contact with the scalp. Also high levels of eye and muscle movement and 50Hz mains artifact can be problematic. Muscle movement (EMG), in particular, can contain significant 13Hz activity. However all of these can be detected and minimized using a suite of programs developed at SCAN (Schier 1994).

Artifact can be detected using a two-stage process. The first stage makes use of the fact that EEG has a Gaussian amplitude distribution (McEwan and Anderson 1975). To eliminate electrodes with an unacceptable signal, amplitude histograms of EEG data for each electrode are calculated and correlated with a Gaussian function to assess the integrity of the EEG recording. This technique is very good for detecting data that exceeds the input range of the ADC. When this occurs, the power spectra show maxima at the positive and negative extremes of the amplitude histogram rather than at the mean level of the EEG2. Using this technique, electrodes with a signal with a correlation coefficient of less than 0.75 were classified as unacceptable.

The second stage makes use of the fact that the EEG from adjacent sites is highly correlated. This is because the scalp signal is spatially ‘smeared’ after it has passed through the layers of CSF, skull and scalp (Nunez 1981). This stage of detection is particularly useful for forms of artifact, such as EMG and 50Hz mains, that display a Gaussian-like amplitude distribution, and can therefore pass through the first stage of detection. For this analysis, the data was divided into a series of 16 cycle sweeps. For each sweep, the mean of the Fourier coefficients at each electrode site were correlated with the weighted mean3 of the coefficients at the four nearest neighbouring electrode sites. For each electrode, a look-up table that contains a record of that electrode’s nine

2 In an AC-coupled system (0.5Hz) the mean level of the EEG is zero volts. 3 The closer a neighbouring electrode is to the electrode being analysed, the more ‘weight’ it is given in the mean. 92 Methods

nearest neighbouring electrodes was used. Extra electrodes are necessary because an adjacent electrode may have already been flagged as unacceptable, thus requiring the next one in the list to be used. Here, an electrode was considered unacceptable if the correlation coefficient was less than 0.6. The SSVEP values for each electrode identified as unacceptable were replaced with the weighted means of the SSVEP values for that electrode’s four nearest acceptable neighbours. A subject was eliminated from further analysis if eight or more electrodes were unacceptable in any one task.

4.8.3 Calculations for modal and transient effects

Each pair of Fourier coefficients was calculated over one stimulus cycle, a time interval of 1/13s or 77ms. Such a short integration period gives a low signal to noise ratio. However, as with traditional evoked potential averaging where individual ERPs can be averaged, successive Fourier coefficients can also be averaged to give a higher signal- to-noise ratio. It must be remembered, however, that the longer the integration period, the lower the temporal resolution. Therefore, a trade-off situation exists where consideration must be given to both the signal-to-noise ratio and the temporal resolution.

In this study, Fourier coefficients were calculated using both long and short averaging periods to investigate different aspects of cognition during performance of the tasks. A long averaging period (ie. 40s) was used to investigate more sustained modal effects reflecting the mental state of subjects while performing the task. In this situation, transient SSVEP changes associated with specific events within each task cannot be observed, rather each ‘task interval’ is considered as a whole, essentially indicating the subject’s mental state, or processing ‘mode,’ during this time. A short 20-cycle (1.54s) averaging period was used to investigate the dynamics of each task; that is, the transient changes that occurred as individual stimuli were presented. With the cosine averaging window and the computer setup used, this gave an effective temporal resolution of 870ms. Although this achieves a relatively high temporal resolution, the signal-to-noise ratio is relatively poor. However, signal-to-noise ratio can be improved by the averaging of data, as is the case with traditional ERP averaging. For example, data for all target

93 Methods

objects can be averaged together, as can data for non-target objects. The methods used for both the long and short averaging periods will be described below.

4.8.3.1 Long averaging period (sustained effects)

To examine sustained or modal effects using a long averaging period of 40s, the following procedure was followed for each electrode for each subject. For each ‘task interval’ of the memory and bas2 task blocks, all Fourier coefficients were averaged. In other words, forty seconds of sine and cosine Fourier coefficients were collapsed to one pair for each task. The three resultant pairs of Fourier coefficients for each of the three memory load conditions were further averaged to give one pair of overall averaged Fourier coefficients. This overall average pair was compared with the corresponding pair of Fourier coefficients for the baseline condition to examine sustained SSVEP amplitude and latency patterns associated with retrieval mode processes. The separate individual pairs of Fourier coefficients, averaged over each of the three ‘task intervals’ (ie. m1, m3 and m5), were compared to investigate the effect of memory load on retrieval mode processes.

4.8.3.2 Short averaging period (transient changes)

To examine more transient effects occurring during the processing of an individual stimulus item, Fourier coefficients were calculated using a moving cosine (Hanning) window with a width of 20 stimulus cycles. For a stimulus frequency of 13Hz, this would, in theory, correspond to an effective temporal resolution of 769ms. However, computational limitations resulted in an effective temporal resolution of 870ms. At the start of the averaging procedure, the cosine window was positioned over the first 20 Fourier coefficients of the task block. Averaging commenced at this point, and the result was stored in a secondary file as a new Fourier pair. The window was then advanced by one pair of Fourier coefficients. Averaging was again performed, and the result was stored in the next position of the secondary file. This process was repeated until all data in the initial file had been averaged. This procedure was followed separately for each

94 Methods

electrode site, for each task, and for each subject. Figure 4.6 shows graphically the result of this procedure. It can be seen that the curves are considerably smoother for the 20-cycle averaged data than for the single cycle data. A temporal resolution of 870ms (20-cycle averaged data) was chosen to achieve a satisfactory signal-to-noise ratio in conjunction with a temporal resolution smaller that the interval between the appearance of each object (2000ms), so that SSVEP changes occurring during the course of stimulus processing could be investigated. The following section describes the procedure for averaging data across all target and non-target presentations associated with a correct response.

Recorded EEG with superimposed SSVEP 250 200 150 100

(microvolt) 50 EEG amplitude 0 53 54 55 56 57 Time (s)

Extracted single cycle SSVEP amplitude and phase 35 30 25 20 15 10

(microvolt) 5 0 SSVEP amplitude 4 3 2 1 0 -1 (radian) -2

SSVEP phase -3 -4 0 102030405060708090100110 Time (s)

SSVEP amplitude and phase 20 unit window 14 12 10 8 6 4

(microvolt) 2 0 SSVEP amplitude 4 3 2 1 0 -1 (radian) -2

SSVEP phase -3 -4 stimulus on 0 102030405060708090100110 Time (s)

Figure 4.6 Single cycle and averaged Fourier coefficients SSVEP amplitude and phase data calculated from single cycle Fourier coefficients and averaged using a 20 unit window (temporal resolution = 870ms). (subject BT; electrode 61 (Oz); first baseline task(bas1) - stim-off/stim-on)

95 Methods

4.8.3.2.1 Event averaging

Analyses of data for target and non-target objects were conducted separately. Because task presentation and data acquisition were synchronized (see section 4.3), the exact pair of Fourier coefficients corresponding to the appearance of each object could be identified. The following averaging procedure was observed for each electrode, task and subject. A five-second epoch of data, which was centred on the appearance of each correctly recognized target object, was extracted. The data for these corresponding epochs was then averaged and, as for the long averaging period analysis, sine and cosine coefficients were calculated separately. Data for non-target objects was then averaged in the same way as for target objects.

Because the five-second epochs of 20-cycle averaged data contains transient SSVEP phase changes superimposed on slow changes, when looking for transient patterns, slow changes are a confounding factor. To eliminate these slow changes, an additional step was included between the extraction of each epoch of data and the subsequent averaging of data for corresponding epochs. The data in each epoch was converted to its polar equivalent using equation 4.3. The mean phase was calculated from the series of phase coefficients. This mean phase was then subtracted from every phase coefficient in the data epoch. This process, in effect, eliminated any sustained, or DC, phase levels. The mean phase for each epoch was therefore zero radians, and phase changes were related to the transient processing of each item rather than to the sustained phase value differences that may have existed between memory loads. Amplitude and phase coefficients were converted back to sine and cosine Fourier coefficients prior to averaging the data for corresponding epochs.

4.8.4 Group averaging

For both long and short averaging period analyses, only the main memory tasks for the three memory load conditions are discussed in this section. Long averaging period analyses for the practice tasks are shown in Appendix E.

96 Methods

So far, the 13Hz SSVEP Fourier coefficients have been extracted for each task, averaged, and in the case of the short averaging period, averaged across target and non- target epochs. These analyses were carried-out separately for each task, at each electrode site, and for all subjects. Finally the data was averaged across all forty subjects for both long and short averaging period analyses.

4.8.4.1 Group averaging: long averaging period (sustained effects)

The following steps were taken to obtain group averaged data that would show sustained or mental state effects on the SSVEP. Prior to averaging across subjects, the SSVEP amplitude values for each subject had to be normalized so that those individuals showing large amplitudes did not dominate the group average. For each subject, a representative value, or normalization factor, was obtained by calculating the mean amplitude for the baseline condition across all 64 electrodes. All SSVEP amplitude values for each electrode, for all three individual memory load conditions, and for the overall average of the three memory load conditions were then divided by the unique normalization factor for that individual. The baseline task was similarly amplitude normalized.

The SSVEP phase value was also adjusted prior to averaging. This adjustment was necessary because the phase values for each subject are arbitrary, and therefore averaging across subjects could result in phase cancellation. Each individual’s phase values for each of the three memory load conditions, and for the overall average of the three memory load conditions, were calculated relative to the average phase for the baseline condition. This also allowed deviations in phase from the baseline to be examined easily. This adjustment was performed by subtracting the 64 baseline phase values (one for each electrode) from the 64 phase values for each memory load condition. In other words, the phase at every electrode site for the baseline condition took the value zero, and the phase values for the memory load conditions, that is, the three separate and the overall averaged memory load conditions, represented deviations from this zero phase. This adjustment reduces cancellation effects when averaging across subjects. It is also much easier to see whether there was a phase advance or phase

97 Methods

lag (ie. latency decrease or increase) relative to the baseline condition at each electrode site.

Once the long averaging interval data had been amplitude normalized and phase adjusted for each individual, group averaging was performed. This was done by first converting the polar data points (amplitude and phase) back to cartesian coordinates (sine and cosine coefficients) and then averaging the sine and cosine coefficients separately across all subjects. The resulting cartesian average was then converted back into polar form, allowing topographic mapping of amplitude and phase values for each task.

SSVEP amplitude and phase differences between the average of the three memory load conditions and the baseline condition were calculated using the averaged group data. SSVEP amplitude and phase differences between memory loads of 3 and 1, 5 and 1, and 5 and 3, were also calculated. These differences addressed more directly the issue of how increases in memory load affect the SSVEP response associated with retrieval mode.

4.8.4.2 Group averaging: short averaging period (transient changes)

The short averaging period analysis used the data for each individual that had been averaged using a 20-cycle moving window to give a temporal resolution of 870ms. The windowed Fourier coefficients for target and non-target stimuli were event averaged separately so that the SSVEP amplitude and phase changes engendered by the appearance of target and non-target stimuli could be compared. As already discussed, event averaging of SSVEP data involved adjustment of the phase prior to averaging across successive five-second epochs of data. Thus, phase adjustment prior to calculating the group average was not necessary as the reference phase value, in this case the mean value over the five-second epoch for targets and non-targets for each of the three memory load conditions, was already zero.

98 Methods

As already explained, for the long averaging period analysis, amplitude normalization had to be conducted prior to group averaging so that individuals showing large amplitudes did not dominate the group average. This was also required for the short averaging period analysis and was carried-out as follows. Firstly, for each electrode, the Fourier coefficients for the five-second epoch for non-targets with a memory load of 1 were averaged separately to give a single pair of coefficients for this epoch. The SSVEP amplitude at each electrode site was then calculated from these averaged coefficients and further averaged across the sixty-four sites to produce the normalization factor. A normalization factor was calculated for each subject. Each subject’s normalization factor was then divided into all SSVEP amplitude values within that subject’s five- second epochs for target and non-target averages for each electrode site for all three memory load conditions.

Once amplitude normalization had been completed, the SSVEP values were converted to cartesian form for group averaging. For each point in the five-second epoch, sine and cosine coefficients were averaged separately across all subjects, resulting in a five- second epoch of group averaged coefficients. This process was repeated for each electrode and for each experimental condition, that is, for targets and non-targets for each of the 3 memory load conditions. All pairs of Fourier coefficients were then converted back to polar form.

This group-averaged data was used to investigate the effect of memory load on transient SSVEP changes by comparing SSVEP values for the different memory loads. Differences in both amplitude and phase between memory loads were then calculated using the group averaged data. Differences in both amplitude and phase between values obtained for memory loads of 3 and 1, 5 and 1, and 5 and 3, were calculated for each electrode site. Target and non-target differences were calculated separately. This procedure showed how differences in memory load affected transient patterns in the SSVEP response during the five second epoch centred on the appearance of an object. For target analyses, topographic differences between high and low memory loads reflect the effect of increased difficulty in recognizing targets on the SSVEP response.

99 Methods

Differences between SSVEP values for target and non-target objects were also calculated. This analysis, although included for completeness, was problematic given that targets prompted a motor response and non-targets did not. Furthermore, targets occurred less frequently than non-targets, therefore SSVEP differences may have reflected probability effects.

4.8.5 Topographic mapping of SSVEP data

Two-dimensional topographic maps were constructed using the SSVEP amplitude and phase data from both the long and short averaging period analyses. Inter-electrode values were calculated using a spherical spline interpolation procedure (Cadusch et al. 1992). Maps were displayed in a 640 x 480, 256 colour mode. Different scale ranges and conventions were used depending on whether individual subject data, group data or group difference data was being mapped. Specific details for each type of data will be discussed in turn.

Amplitude and phase data for individual subjects, calculated using the long averaging period analysis, was produced mainly for illustrative purposes (see Figures 5.3 and E-i). The amplitude scale represents the ‘raw’ amplitude, in microvolts, of the SSVEP elicited in response to the 13Hz visual flicker. Warmer colours represent a larger response to the visual flicker. Topographic maps showing the SSVEP phase were produced for individual subjects using a continuous colour scale such that the same colour was used for both +π (+3.1) radian and –π (-3.1) radian. This is done because phase variations for individual subjects can be large and not necessarily centred about zero radians, resulting in phase transitions at +π and -π. In this situation, if different colours are assigned to +π and -π, sharp discontinuities in colour can occur, making interpretation of topographic variations more difficult. A transition to a more positive colour is interpreted as a phase advance, and to a more negative colour, a phase lag.

Group SSVEP amplitude and phase data was mapped somewhat differently (see Figures 5.4 and E-ii). Firstly, since amplitude data was normalized prior to group averaging (see sections 4.8.4.1 and 4.8.4.2), amplitude values are smaller. The scale range therefore

100 Methods

represents a smaller value for group data than for individual subject data. Again, warmer colours represent a larger response to the visual flicker. Variations in phase were much smaller for group data (±0.5 radian) than for individual subject data, and centred on zero radians due to the phase adjustment performed prior to group averaging (see sections 4.8.4.1 and 4.8.4.2). A continuous colour scale, like the one used for individual subject data, was therefore not required. The baseline phase was zero at all sites (yellow/green colour) as a result of the phase adjustment. Warmer colours represent a phase advance, and cooler colours a lag, relative to the mean phase for the baseline condition.

In early studies, amplitude attenuation and phase advance were interpreted as reflections of increased neural activity. Therefore, the colour convention adopted was warmer colours for amplitude attenuation and phase advance, and cooler colours for amplitude increase and phase lag. While the picture has now become more complicated (see Chapter 3) this colour convention has remained in use and is used here in the depiction of differences between task conditions. Amplitude differences were therefore calculated such that an SSVEP amplitude attenuation observed for higher relative to lower memory load conditions is represented by warmer colours. Similarly, SSVEP phase differences were calculated such that SSVEP phase advances for higher relative to lower memory load conditions is also represented by warmer colours. Phase values can also be expressed in terms of latency, where 1 radian is equivalent to a latency of 12.2ms. A phase advance corresponds to a latency decrease and a phase lag to a latency increase, therefore warmer colours represent latency decreases and cooler colours increases. Amplitude and latency group difference data are shown in Figures 5.5, 5.6, 5.11, 5.12 and 5.13.

4.8.6 Statistical analysis and Significance Probability Mapping

Because the SSVEP consists of two components, amplitude and phase, the bivariate Hotelling’s T2 test was used to determine the statistical significance of the SSVEP response to differences between task conditions. The square root of the Hotelling’s T2 parameter (Hotelling’s T) was mapped rather than the T2 parameter itself because small

101 Methods

areas could contain very large T2 values which would dominate the scale. Mapping the T values allowed for much smoother contours. This type of mapping is called significance probability mapping (SPM) (eg. Duffy et al. 1981; Hassainia et al. 1994) and more recently has been termed statistical parametric mapping (SPM) (eg. Acton and Friston 1998). On the SPM maps, iso-T contours of 2.02, 2.70, 2.97 and 3.55, corresponding to p values for a single comparison of 0.05, 0.01, 0.005 and 0.001, were used.

A p value of 0.05, or 5%, is normally considered the threshold for statistical significance. Long and short averaging period analyses however were not based on single comparisons as data was analysed from 64 recording sites. A Bonferroni correction is normally applied to p values to allow for these multiple independent measures (Abt 1983). If all electrodes were independent of each other, 0.05 would be divided by 64. However, because brain electrical activity from neighbouring scalp recording sites is highly correlated, dividing the p value by 64 is incorrect (Silberstein et al. 1995a). Rather, a value of 5 derived through spatial principal components analysis (eg. Silberstein and Cadusch 1992) represents more accurately the degree of independence for 64 separate, but correlated, recording sites. In this instance, therefore, a p value of 0.01 (1%) on the Hotelling’s T topographic map was taken as the threshold for statistical significance. It should also be noted, that where multiple time points are sampled in a temporal sequence, a p value of 0.05/(5 x no. of time point samples) should be used as the threshold for statistical significance (Silberstein et al. 1995a).

102

Chapter 5 Results

Response time and accuracy data were analysed in order to investigate whether an increase in memory load resulted in an increase in task difficulty. It was anticipated that response time would increase with memory load, and accuracy might decrease. While response time did, in fact, increase with memory load, memory load had no significant effect on accuracy.

Sustained task and transient item-related SSVEP responses produced during the performance of an episodic retrieval task were examined to investigate neural activity associated with sustained retrieval mode and transient retrieval success processes.

To investigate neural regions involved in the maintenance of a retrieval mode, the SSVEP amplitude and latency values were averaged across the entire duration of an episodic retrieval task and compared with the averaged SSVEP amplitude and latency values obtained during a non-episodic retrieval baseline task. Sustained retrieval mode processes were associated with SSVEP amplitude and latency reductions at most recording sites. While amplitude reductions were relatively uniform across all recording sites, latency reductions were most prominent at right fronto-temporal sites.

To investigate neural regions associated with successful retrieval from memory, the SSVEP amplitude and latency were calculated using a short averaging period, and values for different experimental conditions were compared at certain time points before, during, and after the appearance of the stimulus object. Transient load- dependent SSVEP amplitude and latency reductions occurred during successful retrieval at right inferior frontal, bilateral parietal and right occipito-parieto-temporal sites. Both amplitude and latency reductions at these sites were larger over the right hemisphere, particularly for the highest memory load condition.

Behavioural results are presented in section 5.1. This section is concerned primarily with the relationship between response time, errors and memory load. Results pertaining

Results

to sustained and transient electrophysiological effects are presented in sections 5.2.1 and 5.2.2 respectively.

5.1 Behavioural data

Data averaged across all forty subjects was used in the analysis of behavioural data. In addition, for illustrative purposes, the data for one individual subject was also analysed. Response times were calculated for each memory load condition, and the statistical significance of response time differences between the conditions was calculated using the Student’s t test.

5.1.1 Individual subject

Figure 5.1 shows mean response times and standard deviations for all correctly recognized target objects for an individual subject (DK), for each of the 3 memory load conditions. This subject had no errors of omission (missed responses) and no errors of commission (responses to non-targets). The mean response time to target objects was 505ms for a memory load of 1, 600ms for a memory load of 3, and 623ms for a memory load of 5. Mean response times were significantly different for memory loads of 1 and 3 (t=-6.65; p<0.02), as were those for memory loads of 1 and 5 (t=-6.76; p<0.02). Appendix C (Figure C-i) shows the equivalent chart for the practice memory tasks.

104 Results

800 700 600 500 400 300 m1 vs m3: t=-6.65; p<0.02 200 m1 vs m5: t=-6.76; p<0.02 100 m3 vs m5: NS Mean Response Time (msec) Time Response Mean 0 m1 m3 m5 Memory Load

Figure 5.1 Mean response times for target objects for each memory load for an individual subject Mean response time and standard deviation for target objects for each of the 3 memory load conditions. Data were taken from an individual subject (DK) performing task order 2 (ie. in the order m3, m1, m5). Results of unpaired t-tests for memory load comparisons are also shown. (NS = not significant)

5.1.2 Group data

From the mean response times for correctly recognized target objects for individual subjects, the group (n=40) mean response times and standard deviations for each of the 3 memory conditions were calculated. Similarly, the mean numbers of errors of omission and commission, with standard deviations, were calculated. These data are shown in Figure 5.2. Practice memory tasks were similarly analysed, and results are shown in Appendix C (Figure C.2). The group mean response time for target objects was 480ms for a memory load of 1, 590ms for a memory load of 3, and 620ms for a memory load of 5. Differences between group mean response times were statistically significant for m1 vs m3 (t=-8.08; p<0.0001), m1 vs m5 (t=-9.41; p<0.0001), and m3 vs m5 (t=-2.83; p<0.02). Group mean errors of omission and commission were generally small and were not significantly different between memory load conditions.

105 Results

800 700 600 500 400 300 m1 vs m3: t=-8.08; p<0.0001 200 m1 vs m5: t=-9.41; p<0.0001 100 m3 vs m5: t=-2.83; p<0.02 Mean Response Time (msec) Time Response Mean 0

0.8 m1 vs m3: NS m1 vs m5: NS 0.6 m3 vs m5: NS

0.4

0.2

Mean Frequency of Errors of Omission Errors of of Frequency Mean 0.0

0.8 m1 vs m3: NS m1 vs m5: NS 0.6 m3 vs m5: NS

0.4

0.2

0.0 Mean Frequency of Errors of Commission Errorsof of Frequency Mean m1 m3 m5 Memory Load

Figure 5.2 Mean response time and mean number of errors for target objects versus memory load for 40 subjects Mean response time and standard deviation for target objects for each of the 3 memory load conditions (top). Charts are shown for numbers of errors of omission (middle) and commission (bottom). Results of paired t-tests for memory load comparisons are also displayed. (NS = not significant)

106 Results

5.2 Electrophysiological data

As explained in section 4.2, during the practice baseline task block, the 13Hz probe flicker was switched-on half way through the block in order to determine whether the driven 13Hz response was significantly different from the background or spontaneous 13Hz activity. As this issue is not central to this thesis, results are not provided here, but are shown in Appendix D.

To examine the sustained modal effects associated with retrieval mode processes, an averaging period of 40s was used. This is discussed in section 5.2.1. More transient SSVEP amplitude and latency changes that occur with the rapidly changing patterns of neural activity associated with object retrieval were investigated using an averaging period of 1538ms (20 stimulus cycles). This gave an effective temporal resolution of 870ms. This analysis is explained in section 5.2.2.

5.2.1 Sustained effects: long averaging period

The following results were calculated using a long averaging period by averaging Fourier coefficients across the entire 40s ‘task interval’ of the task blocks for the baseline and each of the three memory load conditions. The resultant SSVEP amplitudes and latencies were compared to investigate neural changes associated with retrieval mode processes.

5.2.1.1 Individual subjects

Topographic maps of amplitude and phase were produced for all individual subjects. There appeared to be a large inter-subject variation in both amplitude and phase for all task conditions. However, averaging across subjects revealed statistically significant effects. Data for an individual subject is provided in Figure 5.3 for illustrative purposes (and see Appendix E, Figure E.1, for the practice task equivalent). On the topographic maps for amplitude, warmer colours represent larger SSVEP amplitudes. On the

107 Results

topographic maps for phase, warmer colours represent a phase advance relative to the 13Hz probe stimulus. Conversely, cooler colours represent smaller SSVEP amplitudes and a phase lag relative to the probe stimulus.

baseline

load 1

load 3

5.0 load 5 +3.1

0.0 υV amplitude phase -3.1 radian

Figure 5.3 Sustained effects: Mean SSVEP amplitude and phase topography for the baseline task and each of the memory load conditions calculated with a long (40s) averaging period for an individual subject Mean SSVEP amplitude and phase topography for the baseline condition and each of the memory load conditions, calculated across the entire 40s ‘task interval.’ Warmer colours represent larger SSVEP amplitudes and a phase advance relative to the 13Hz probe stimulus. Cooler colours represent smaller SSVEP amplitudes and a phase lag relative to the probe stimulus. Data for individual subject (CL).

108 Results

5.2.1.2 Group data

Amplitude and phase data for the 40 subjects were averaged as explained in section 4.8.4. The averaged results for the long averaging period analysis are shown for illustrative purposes in Figure 5.4 (see Appendix E, Figure E.2 for the equivalent practice tasks). Again, warmer colours in the topographic maps represent larger SSVEP amplitudes and a phase advance relative to the baseline phase.

To examine neural changes associated with retrieval mode processes, topographic maps depicting SSVEP amplitude and latency differences, and the associated Hotelling’s T statistical topographic maps, were calculated for the averaged results. To investigate retrieval mode processes, the overall average amplitude and latency values for the 3 memory load conditions were compared with those for the baseline condition (Figure 5.5). To examine the effect of memory load on retrieval mode processes, the SSVEP values for the separate memory load conditions were compared with each other (Figure 5.6). In contrast to the colour conventions used in Figures 5.3 and 5.4, an alternative convention has been used for the difference maps. For the amplitude difference maps, warmer colours represent a reduced SSVEP amplitude for the average of the 3 memory load conditions relative to the baseline condition (Figure 5.5), and for the higher memory load condition relative to the lower (Figure 5.6). The topographic phase maps show phase difference in terms of latency in milliseconds. Warmer colours indicate a reduced latency for the average of the 3 memory load conditions relative to the baseline condition (Figure 5.5), and for the higher memory load condition relative to the lower (Figure 5.6). The Hotelling’s T statistic indicates the consistency of SSVEP differences across the 40 subjects investigated. Warmer colours indicate a higher level of significance. Contour lines depict T values that represent p values of 0.05, 0.01, 0.005 and 0.001 for a single comparison.

109 Results

baseline

load 1

load 3

1.5 load 5 +0.5

0.0 amplitude phase -0.5 radian

Figure 5.4 Sustained effects: Mean SSVEP amplitude and phase topography for the baseline condition and each of the memory load conditions averaged across 40 subjects Mean SSVEP amplitude and phase topography for the baseline condition and each of the memory load conditions, calculated across the entire ‘task interval.’ Data taken from averaged results across 40 subjects. Warmer colours represent larger SSVEP amplitudes and a phase advance relative to the phase of the baseline condition. Cooler colours represent smaller SSVEP amplitudes and a phase lag relative to the phase of the baseline condition. The effect of the phase adjustment results in a zero phase for the baseline task (see top).

110 Results

As suggested by Silberstein et al. (1995a), a correction factor of 5 was used to allow for 64 separate, but correlated, recording sites. Therefore, p=0.01 (ie. p=0.05/5=0.01) was taken as the level for statistical significance for the difference between the average of the 3 memory load conditions and the baseline task. For the 3 memory load comparisons shown in Figure 5.6, p=0.001 (ie. p=0.05/(5x3)=0.0033) was used as a conservative value for statistical significance.

The topographic map depicting the SSVEP amplitude differences for the average of the 3 memory load conditions with respect to the average for the baseline condition (Figure 5.5) indicates that the SSVEP amplitude was diffusely attenuated (warmer colours) at all recording sites. The topographic map showing how the latency averaged over the 3 memory load conditions compared with the average latency for the baseline condition indicates an SSVEP latency reduction (warmer colours) at most recording sites, and particularly prominent reductions at right fronto-temporal sites. The Hotelling’s T topographic map indicates that SSVEP differences were statistically significant at most recording sites. Statistical significance appears greatest at right fronto-temporal sites.

Amplitude Latency Hotelling's T Difference Difference +0.3 -6.1 6.0

0.001 0.005 0.01 0.05 m(1,3,5) - base -0.3 +6.1 0.0

Figure 5.5 Sustained effects: Topographic differences in SSVEP amplitude and latency between the average of the 3 memory load conditions and the baseline task averaged across 40 subjects Topographic differences for SSVEP amplitude (normalized units) and latency (ms) between the average of the 3 memory load conditions and the average for the baseline task. Warmer colours represent reduced SSVEP amplitudes and latencies for the average of the 3 memory load conditions relative to the baseline task. Topographic maps for the Hotelling’s T statistic are shown representing the statistical strength of these differences. Four contours are shown on the scale representing T values for p=0.05, 0.01, 0.005 and 0.001 for a single comparison.

111 Results

Memory load conditions were also compared directly to investigate the effect of increased memory load, and therefore task difficulty, on retrieval mode processes. Topographic maps depicting amplitude and latency differences between the m3 and m1, the m5 and m1, and the m5 and m3 conditions are shown in Figure 5.6. It can be seen that with increases in memory load, SSVEP amplitude is attenuated, as indicated by warmer colours. The topographic map for the m5 – m1 comparison, where the memory load difference was greatest, shows an overall amplitude attenuation that is most

Amplitude Latency Hotelling's T Difference Difference +0.3 -6.1 4.0 0.001 0.005 0.01 0.05 m3-m1

-0.3 +6.1 0.0

+0.3 -6.1 4.0 0.001 0.005 0.01 0.05 m5-m1

-0.3 +6.1 0.0

+0.3 -6.1 4.0 0.001 0.005 0.01 0.05 m5-m3

-0.3 +6.1 0.0

Figure 5.6 Sustained effects: Topographic differences in SSVEP amplitude and latency between memory load conditions averaged across 40 subjects Topographic differences for SSVEP amplitude (normalized units) and latency (ms) between memory load conditions. Warmer colours represent reduced SSVEP amplitudes and latencies for high relative to low memory load conditions. Topographic maps for the Hotelling’s T statistic indicate the statistical strength of these differences. Four contours are shown on the scale representing T values of p=0.05, 0.01, 0.005 and 0.001 for a single comparison.

112 Results

prominent at occipital and right occipito-temporal sites. Topographic maps depicting latency differences between the m3 and m1, and m5 and m1 conditions indicate an increased latency (blue colour) over frontal regions for the higher memory load condition relative to the lower. This pattern is reversed over posterior regions where the higher relative to the lower memory load has produced a decreased latency (pink colour). The topographic map for the m5 – m1 comparison, where the memory load difference was greatest, shows a particularly prominent latency reduction at right occipital sites. As indicated by the Hotelling's T statistical maps, SSVEP amplitude and latency reductions at right occipital regions approached, but did not reach, statistical significance (puncorrected=0.001). Similarly, differences at left central and right temporal sites between the SSVEP response to the m5 and m3 conditions approached, but did not reach, statistical significance (puncorrected=0.001).

5.2.2 Transient changes: short averaging period

To observe transient changes in SSVEP amplitude and phase associated with the occurrence of specific events within the task, a high temporal resolution analysis was performed. The following results were calculated by firstly extracting the SSVEP Fourier coefficients from the EEG using a single cycle integration period. Signal-to- noise ratio was increased by using a 20-cycle moving cosine, or Hanning, averaging window. This gave an effective temporal resolution of 870ms. Five-second epochs of 20-cycle averaged Fourier coefficients, centred on the appearance of correctly identified target and non-target objects, were then averaged for each condition. Data for the forty individual subjects was then averaged to give a group averaged time series for amplitude and phase data for each electrode site and for each memory load. The SSVEP time averaged data for different memory loads, centred on the appearance of target objects, can be compared to highlight neural regions involved in processes associated with successful retrieval.

113 Results

5.2.2.1 Changes with memory load

The parietal ERP old/new effect has been noted most consistently in electrophysiological studies into episodic retrieval processes. It was anticipated that in this study, SSVEP load-dependent changes would also be apparent over parietal regions. Figure 5.7 shows a five-second epoch of SSVEP amplitude data, averaged over 20 cycles, for a right parieto-temporal site (electrode 42). The data for each of the 3 memory load conditions is centred on the appearance of both non-target (upper graph) and target (lower graph) objects. As can be seen in the upper graph, the SSVEP amplitude underwent transient changes associated with the appearance of non-target objects. The SSVEP amplitude was lowest just prior to the appearance of an object, and then increased, peaking 1s after the object’s appearance. The 20-cycle time-averaged data for non-target objects shows no apparent correlation between the SSVEP amplitude and memory load. When a five-second epoch of 20-cycle time-averaged SSVEP amplitude data is centred on the appearance of target objects, quite a different pattern is seen. The transient changes in amplitude appear relatively similar across all 3 memory load conditions for times up to 500ms before target presentation and beyond 1500ms after target presentation. Around the time of target presentation, however, the differences between the SSVEP amplitudes associated with each of the 3 memory load conditions markedly increase. During target presentation, at this right hemisphere parieto-temporal electrode site, an increase in memory load was associated with a decrease in SSVEP amplitude.

Figure 5.8 shows a five-second epoch of 20-cycle time-averaged SSVEP phase data recorded at the same right hemisphere parieto-temporal electrode. This is also centred on the appearance of either non-target (upper graph) or target (lower graph) objects for each of the 3 memory load conditions. For non-target objects, variations similar to the amplitude variations were evident. The SSVEP phase value was lowest, indicating a phase lag, just prior to the appearance of the non-target and peaked, indicating a phase advance, 1000ms later. The SSVEP phase data for non-target objects appeared to be similar for each of the 3 memory load conditions. The equivalent phase data for target objects (lower graph) also shows a rhythmic pattern over the 5s epoch. Before the appearance of a target object, the pattern is similar to that observed for non-target

114 Results

objects. Differences between the phase values for each of the memory loads are relatively small. However, following the appearance of a target, differences markedly increased. Here, the higher the memory load, the more advanced the phase. Phase values peaked at about 800ms after target appearance, and were approximately 0 radian for a memory load of 1, 0.1 radian for a memory load of 3, and 0.2 radian, for a memory load of 5.

Results obtained at the equivalent left hemisphere parieto-temporal site (electrode 38) are shown in Figures 5.9 and 5.10. Left parieto-temporal variations in amplitude and phase associated with the presentation of non-target objects are similar to those on the right. However, variations associated with the presentation of target objects recorded at the left hemisphere site differ from those recorded at the right, in that amplitude appears less strongly correlated with memory load, and phase appears to be uncorrelated.

115 Results

SSVEP Amplitude - Non-Target Blank 1.2 Blank Non-Target (Non-)Target (Non-)Target

1.0

0.8 m1

Normalised Amplitude m3 m5

0.6 sec post non-target 0.4 -2 -1 0 1 2 Time (seconds)

SSVEP Amplitude - Target Blank Blank

1.2 Target Non-Target Non-Target

1.0

0.8 m1

Normalised Amplitude m3 m5 Response 0.6 sec post target 0.4 -2 -1 0 1 2 Time (seconds)

Figure 5.7 Normalized SSVEP amplitude time series for each of the memory load conditions calculated across 40 subjects Normalized SSVEP amplitude time series with a temporal resolution of 870ms, across 40 subjects, for each of the memory load conditions. Time series data were taken from a right parieto-temporal site (electrode 42). Time zero represents the presentation of either a target or non-target object. For target presentations, the variation in the range of mean response times for the subjects is shown in grey, and the point selected for topographic mapping of differences (Figures 5.11 and 5.12) is shown by the vertical dotted line (between presentation and response).

116 Results

SSVEP Phase - Non-Target 0.4 Blank Blank

0.2 Non-Target (Non-)Target (Non-)Target

0.0

Phase (radians) m1 -0.2 m3 m5

-0.4 sec non-target post 0.4 -2 -1 0 1 2 Time (seconds)

SSVEP Phase - Target 0.4 Blank Blank Target Non-Target 0.2 Non-Target

0.0

Phase (radians) Phase m1 -0.2 m3 m5 Response -0.4 target sec post 0.4 -2 -1 0 1 2 Time (seconds)

Figure 5.8 SSVEP phase time series for each of the memory load conditions calculated across 40 subjects SSVEP phase time series with a temporal resolution of 870ms, across 40 subjects, for each of the memory load conditions. Time series data were taken from a right parieto-temporal site (electrode 42). Time zero represents the presentation of either a target or non-target object. For target presentations, the variation in the range of mean response times for the subjects is shown in grey, and the point selected for topographic mapping of differences (Figures 5.11 and 5.12) is shown by the vertical dotted line (between presentation and response).

117 Results

SSVEP Amplitude - Non-Target Blank 1.2 Blank Non-Target (Non-)Target (Non-)Target

1.0

0.8 m1

Normalised Amplitude Normalised m3 m5

0.6 sec0.4 post non-target -2 -1 0 1 2 Time (seconds)

SSVEP Amplitude - Target Blank Blank

1.2 Target Non-Target Non-Target

1.0

0.8 m1

Normalised Amplitude m3 m5 Response 0.6 0.4 sectarget post -2 -1 0 1 2 Time (seconds)

Figure 5.9 Normalized SSVEP amplitude time series for each of the memory load conditions calculated across 40 subjects Normalized SSVEP amplitude time series with a temporal resolution of 870ms, across 40 subjects, for each of the memory load conditions. Time series data were taken from a left parieto-temporal site (electrode 38). Time zero represents the presentation of either a target or non-target object. For target presentations, the variation in the range of mean response times for the subjects is shown in grey, and the point selected for topographic mapping of differences (Figures 5.11 and 5.12) is shown by the vertical dotted line (between presentation and response).

118 Results

SSVEP Phase - Non-Target 0.4 Blank Blank

0.2 Non-Target (Non-)Target (Non-)Target

0.0

Phase (radians) m1 -0.2 m3 m5

-0.4 sec0.4 non-target post -2 -1 0 1 2 Time (seconds)

SSVEP Phase - Target 0.4 Blank Blank Target Non-Target 0.2 Non-Target

0.0

Phase (radians) Phase m1 -0.2 m3 m5 Response -0.4 0.4 sectarget post -2-1012 Time (seconds)

Figure 5.10 SSVEP phase time series for each of the memory load conditions calculated across 40 subjects SSVEP phase time series with a temporal resolution of 870ms, across 40 subjects, for each of the memory load conditions. Time series data were taken from a left parieto-temporal site (electrode 38). Time zero represents the presentation of either a target or non-target object. For target presentations, the variation in the range of mean response times for the subjects is shown in grey, and the point selected for topographic mapping of differences (Figures 5.11 and 5.12) is shown by the vertical dotted line (between presentation and response).

119 Results

To observe the topography of the SSVEP amplitude and phase changes, differences between memory loads were calculated for each of the 64 electrode sites, and topographic maps were produced. It is possible to produce a topographic map for every time point in the time-averaged data. However, it had been anticipated that the recognition stage of object retrieval would have occurred within the interval between object presentation and response, and therefore a time point within this interval was examined. The precise location of this time point is not critical as the temporal resolution used to calculate the SSVEP Fourier coefficients was 870ms, and therefore the speed of topographic changes was of this order of magnitude. The average response time for a memory load of 1 was 480ms, and a time point approximately 400ms after either target or non-target appearance was selected and mapped. A time of 400ms after target appearance was selected because 400ms is shorter than the average response time for a memory load of 1, and therefore, SSVEP differences between memory loads were less likely to have resulted from subject response. The same time point was also used for mapping SSVEP differences for non-target objects.

Figure 5.11 shows SSVEP amplitude and phase differences between memory load conditions m3 and m1, m5 and m1, and m5 and m3 at 400ms after target appearance. Differences for non-target objects are shown in Figure 5.12. Warmer colours in the amplitude difference maps indicate SSVEP amplitude attenuation for the higher relative to the lower memory load condition. Conversely, cooler colours indicate SSVEP amplitude increases for the higher relative to the lower memory load condition. Phase differences in radians were converted to latency differences in milliseconds. Warmer colours in the latency difference maps indicate a reduced SSVEP latency for the higher relative to the lower memory load condition. Conversely, cooler colours indicate an increase in SSVEP latency for the higher relative to the lower memory load condition. The Hotelling’s T parameter for these differences was also calculated. This indicated the consistency of SSVEP differences between memory load conditions across the 40 subjects investigated. Warmer colours indicate a higher level of significance. Contour lines depict T values that represent p values of 0.05, 0.01, 0.005 and 0.001 for a single comparison. Using the correction factor of 5 (for 64 separate but correlated electrodes) suggested by Silberstein et al. (1995a), and 3 for the number of conditions, p=0.001 (ie.

120 Results

p=0.05/(5x3)=0.0033), was used as a conservative value for statistical significance (see Methods chapter, section 4.8.6).

For target objects (Figure 5.11), SSVEP differences between memory load conditions were apparent for both amplitude and latency. Again, warmer colours indicate lower amplitudes and shorter latencies, and cooler colours indicate higher amplitudes and longer latencies. The amplitude maps indicate an amplitude reduction for the higher memory load conditions relative to the lower at most electrode sites. Amplitude reductions for the m3 relative to m1 condition, and m5 relative to m1 condition, were largest over the junction of the right occipital, parietal, and temporal regions, and extended into right and left parietal, and left temporal regions. These reductions appeared to be more extensive for the m5 relative to the m1 condition than for the m3 relative to the m1 condition. Latency maps indicate a latency reduction for the higher memory load relative to the lower memory load conditions at most electrode sites. Latency reductions were largest for the m3 relative to the m1 condition over parieto- occipital regions, and for the m5 relative to m1 condition over parieto-occipital and frontal regions. For the comparison of the m3 with the m1 condition, the Hotelling’s T parameter indicates that SSVEP amplitude and latency differences are statistically significant over right occipito-temporal and bilateral parietal regions. Differences between the m5 and m1 conditions are statistically significant over additional bilateral parietal, right parieto-temporal, and right inferior frontal regions, and represent a reduced SSVEP amplitude and latency for the higher memory load condition. The m5 versus m3 comparison for target objects showed the same trend for amplitude and latency differences, that is, attenuation in amplitude and reduction in latency with increased memory load, although these differences did not reach statistical significance.

Compared with target objects, the SSVEP differences between memory load conditions for non-target objects (Figure 5.11) were generally much smaller for both amplitude and latency. There did not appear to be a correlation between amplitude or latency reductions and increases in memory load. Hotelling’s T topographic maps indicated that SSVEP differences were not statistically significant.

121 Results

Amplitude Latency Hotelling's T Difference Difference +0.4 -4.9 4.0 0.001 0.005 0.01 0.05 m3-m1

-0.4 +4.9 0.0

+0.4 -4.9 4.0 0.001 0.005 0.01 0.05 m5-m1

-0.4 +4.9 0.0

+0.4 -4.9 4.0 0.001 0.005 0.01 0.05 m5-m3

-0.4 +4.9 0.0

Figure 5.11 Topographic differences in SSVEP amplitude and latency between memory load conditions for target objects and Hotelling's T statistic (averaged across 40 subjects) Topographic differences in SSVEP amplitude (normalized units) and latency (ms) between memory load conditions at 400ms after the appearance of target objects. Warmer colours represent reduced SSVEP amplitudes and latencies for high relative to low memory load conditions. Topographic maps of the Hotelling’s T parameter are also shown indicating the statistical strength of these differences. Four contours are shown on the scale representing T values of p=0.05, 0.01, 0.005 and 0.001 for a single comparison.

122 Results

Amplitude Latency Hotelling's T Difference Difference +0.4 -4.9 4.0 0.001 0.005 0.01 0.05 m3-m1

-0.4 +4.9 0.0

+0.4 -4.9 4.0 0.001 0.005 0.01 0.05 m5-m1

-0.4 +4.9 0.0

+0.4 -4.9 4.0 0.001 0.005 0.01 0.05 m5-m3

-0.4 +4.9 0.0

Figure 5.12 Topographic differences in SSVEP amplitude and latency between memory load conditions for non-target objects and Hotelling's T statistic (averaged across 40 subjects) Topographic differences in SSVEP amplitude (normalized units) and latency (ms) between memory loads at 400ms after the appearance of non-target objects. Warmer colours represent reduced SSVEP amplitudes and latencies for high relative to low memory load conditions. Topographic maps of the Hotelling’s T parameter are also shown indicating the statistical strength of these differences. Four contours are shown on the scale representing T values of p=0.05, 0.01, 0.005 and 0.001 for a single comparison.

123 Results

5.2.2.2 Targets versus non-targets

In section 5.2.2.1, transient patterns of neural activity associated with successful retrieval processes were investigated by varying the memory load in an episodic memory retrieval task. With this approach, it was anticipated that the increased difficulty associated with the retrieval of more targets would have an effect on the SSVEP response that would help reveal the neural regions involved in successful retrieval processes.

Neural regions involved in successful retrieval were also investigated by directly comparing the SSVEP response to target and non-target objects. This type of direct comparison has been used extensively in electrophysiological studies (see Chapter 2). In this study, however, a comparison of the SSVEP response to target and non-target identification is more problematic because targets prompted a motor response, whereas non-targets did not. Therefore, preparation for the motor response could have been responsible for differences in the SSVEP patterns observed for targets and non-targets. On the other hand, it is possible for a number of reasons, that SSVEP changes prior to the motor response were not contaminated by response processes, and therefore are reflections of neural activity associated with successful recognition. These and other issues relating to the comparison of target and non-target objects will be discussed in more detail in Chapter 6.

Notwithstanding possible confounding factors, SSVEP differences between target and non-target objects were calculated. Differences were calculated separately for each of the 3 memory load conditions. Topographic maps in Figure 5.13 indicate amplitude and latency differences between the SSVEP response to correctly identified target and non- target objects at 450ms and 700ms after object appearance. The 450ms time point, which occurs between object appearance and the mean response time, was selected because it was anticipated that the recognition stage of object retrieval processes would occur within this interval. Differences calculated at the 700ms time point showed that for a memory load of 5, significant frontal differences between the SSVEP response to targets and non-targets developed after the mean motor response time.

124 Results

Amplitude Latency Hotelling's T Difference Difference 450 ms +0.4 -4.9 4.0 0.001 0.005 0.01 0.05

-0.4 +4.9 0.0 700 ms +0.4 -4.9 4.0 0.001

m1t - m1nt - m1t 0.005 0.01 0.05

-0.4 +4.9 0.0

450 ms +0.4 -4.9 4.0 0.001 0.005 0.01 0.05

-0.4 +4.9 0.0 700 ms +0.4 -4.9 4.0 0.001 0.005 m3t - m3nt - m3t 0.01 0.05

-0.4 +4.9 0.0

450 ms +0.4 -4.9 4.0 0.001 0.005 0.01 0.05

-0.4 +4.9 0.0 700 ms +0.4 -4.9 4.0 0.001 0.005 m5t - - m5nt m5t 0.01 0.05

-0.4 +4.9 0.0

Figure 5.13 Topographic differences in SSVEP amplitude and latency between correctly identified target and non-target objects, averaged separately for each memory load condition: pooled across 40 subjects Topographic differences in SSVEP amplitude (normalized units) and latency (ms) between target and non-target objects at times 450ms and 700ms following the presentation of objects. Warmer colours represent reduced SSVEP amplitudes and latencies for target relative to non-target objects. Conversely, cooler colours represent increased SSVEP amplitudes and latencies for target relative to non-target conditions. Topographic maps of the Hotelling’s T statistic are also shown representing the statistical strength of these differences. Four contours are shown on the scale representing T values of p=0.05, 0.01, 0.005 and 0.001 for a single comparison.

125 Results

Warmer colours indicate both amplitude and latency reductions for target relative to non-target objects. Conversely, cooler colours indicate both amplitude and latency increases for target relative to non-target objects. The Hotelling’s T value, calculated for both time points, reflects the consistency of SSVEP differences between target and non- target data across the 40 subjects investigated. Warmer colours indicate higher levels of statistical significance. Contour lines depict T values representing p values of 0.05, 0.01, 0.005 and 0.001 for a single comparison. Using the correction suggested by Silberstein et al. (1995a), a factor of 5 for spatial dimensionality and 6 for the number of conditions (ie. 0.05/(5x6)=0.0017), p=0.001 was used as a conservative level for statistical significance (see Methods chapter, section 4.8.6).

For a memory load of 1, the topographic maps show that differences between target and non-target amplitude and latency values were similar at both 450ms and 700ms after the appearance of the object. The cooler colours of the amplitude maps indicate that the SSVEP amplitude was greater for target objects than for non-target objects. Latency differences between the target and non-target conditions were, however, very small, as indicated by a prevalence of yellow/green colour. The Hotelling’s T parameter approaches, but does not reach, significance (puncorrected=0.001) over occipito-temporal regions.

For a memory load of 3, the pattern of amplitude and latency differences is quite different. A predominance of warmer colours on topographic maps of amplitude difference indicates an attenuation of the SSVEP amplitude for target objects relative to non-target objects at most sites. This attenuation is greatest at parieto-temporal sites bilaterally. The predominance of warmer colours on the difference maps for latency indicates that latencies were generally reduced for targets relative to non-targets. The most prominent latency reductions are apparent over left parietal regions, particularly for the earlier 450ms time point. Hotelling’s T values are highest over parietal regions, once again approaching, but not reaching, significance (puncorrected=0.001).

Amplitude and latency differences for a memory load of 5 were somewhat similar to those for a memory load of 3. Again, there were SSVEP amplitude and latency reductions, indicated by warmer colours, for target relative to non-target objects. The

126 Results

amplitude reductions, however, appeared to be more frontal and were more marked at the later (700ms) time point. Hotelling’s T scores over pre-frontal regions indicate that SSVEP amplitude and latency differences in these regions are statistically significant (p<0.001) at the 700ms time point.

127

Chapter 6 Discussion

The discussion of findings for the present study is divided into five main sections. Section 6.1 contains a brief account of behavioural results. Behavioural data were analysed in order to determine whether increases in memory load led to corresponding increases in task difficulty. Response time data was also required so that time points indicative of the mental processes under investigation could be selected for data analysis. Section 6.2 focuses on sustained SSVEP effects over frontal and right hemisphere regions when memory load conditions are compared with the baseline condition. These results are discussed in relation to PET, fMRI and DC-ERP findings, namely, that right frontal regions have been associated with the establishment and maintenance of a retrieval mode. Section 6.3 is concerned with transient effects, in particular, load-dependent SSVEP changes associated with successful memory retrieval. The observed SSVEP effects over parietal and right frontal regions are discussed primarily in relation to findings from studies into the ERP old/new effect. Section 6.4 also deals with transient SSVEP changes; however here, the direct comparison of the response to target and non-target objects is examined. Findings of frontal effects will again be discussed in relation to the ERP old/new findings that were extensively reviewed in Chapter 2. Finally, in section 6.5, major conclusions relating to this study will be presented.

6.1 Behavioural results: the effect of increasing memory load on retrieval accuracy and response time

In the present study, subjects were required to memorize 1, 3 or 5, two-dimensional abstract objects during the study phase for the corresponding memory load condition. These objects had to be retrieved from memory in the subsequent retrieval phase. Recognition accuracy was high for all three memory load conditions, as evidenced by the very low error scores across the 40 subjects. In addition, performance appeared to improve with practice (compare Figure 5.2 with Figure C.2).

Discussion

The group mean response times increased with increases in memory load. The mean response times for 1, 3 and 5 memory load conditions were approximately 480, 590 and 620ms respectively. Furthermore, each of the 40 subjects showed an increased response time with increased memory load. There was considerable variability in response times because some subjects responded slowly to all tasks, while others responded more quickly overall. However, Student’s t tests showed that the differences in response times associated with the different memory load conditions were highly significant.

In a post-recording interview, all subjects reported that increases in memory load led to noticeable increases in task difficulty. That is, the task with 3 objects to be recognized was more difficult than the task with 1 object, and the task with 5 objects was more difficult than the task with 3. In the light of the aforementioned behavioural findings and these personal accounts, it seems reasonable to assume that an increase in memory load resulted in increased retrieval effort. Retrieval effort is a term that has been used to refer to ‘the mobilization of processing resources in service of a retrieval attempt’ (Rugg and Wilding 2000, p. 110). While task difficulty is a more general term used to describe the overall complexity of the task, retrieval effort involves transient processes that are specifically engaged whilst determining whether an object is new or old.

The study-test task was specifically designed so that subjects would experience increases in difficulty with increases in memory load. This was especially important since one of the main aims of the study was to identify neural regions associated with successful retrieval by increasing the demand on these regions. In other words, to remember whether a presented item was studied or not should require more effort with higher memory loads. As explained in the Methods section (section 4.2), tasks were also designed to facilitate the encoding of objects into a more permanent store by incorporating both multiple presentations of the studied items and relatively long presentation times. Therefore, a high retrieval accuracy for all three memory load conditions was not unexpected. Increased response times with increased load were also not unexpected in the light of subjects’ personal accounts and the fact that higher memory load tasks required the scanning of more items in memory. Sternberg (1969) first demonstrated that response time increases linearly with the number of items stored in short-term memory. It also appears that memory search processes are reflected in the

129 Discussion

EEG. For example, Stuss et al. (1986) showed that both the N400 amplitude and the response time increase as the number of pictures to be named increases, and Noldy- Cullum and Stelmack (1987) suggested that the ERP old/new effect reflects an ‘exhaustive search’ of all items in the memory set to determine whether a presented item is old or new.

In the present experimental design, as greater numbers of target objects had to be remembered, they also had to be differentiated from a greater number of distractor objects of similar appearance. That is, subjects had to make a judgement based on perceptual detail. In a similar way, Ranganath and Paller (2000) required their subjects to make specific perceptual judgements when they had to recall whether line drawings of objects were the same size as those studied. The authors argued that such memory monitoring processes are an essential component of episodic memory, and furthermore, may also be important in conscious recollection processes. It is suggested that, in the present study, increases in response time with increasing memory load were due, in part, to a reliance on perceptual detail for the identification of targets and non-targets.

In summary, behavioural results, particularly the response time data, served as confirmation that increases in memory load led to increases in task difficulty. Furthermore, given also the personal accounts that difficulty increased with memory load, it seems reasonable to assume that retrieval effort also increased with memory load.

6.2 Sustained SSVEP amplitude and latency changes and retrieval mode

Retrieval mode is a term specifically used in relation to episodic retrieval, and has been defined by Tulving (1983) as a tonically maintained mental state that is necessary for the successful retrieval of items from episodic memory. Studies using PET, fMRI and DC-ERP techniques to investigate retrieval mode have compared patterns of activity during the performance of episodic and non-episodic retrieval tasks. In the present study, to identify neural regions involved in sustained retrieval mode processes, the amplitude and latency of the SSVEP were averaged over a 40s period, during which,

130 Discussion

subjects were required to recognize previously studied items. The sustained SSVEP effects engendered during the episodic retrieval tasks were compared with those engendered during a low vigilance baseline task. In addition, this study attempts to determine effects of task difficulty on retrieval mode processes by incorporating retrieval tasks with different memory loads.

In section 6.2.1, the topography of sustained SSVEP amplitude and latency values will be discussed, particularly in relation to the first hypothesis of this study (see section 3.7), namely, that sustained SSVEP amplitude and latency changes reflecting retrieval mode will be most prominent over right frontal regions. While the topography of SSVEP effects is central to this study, the direction of SSVEP effects, that is, whether the SSVEP amplitude and SSVEP latency increase or decrease relative to a baseline task, is also an important issue. Thus, the interpretation of SSVEP amplitude and latency changes will be discussed in section 6.2.2.

6.2.1 The topography of sustained SSVEP changes

The first hypothesis of this study states that sustained SSVEP changes, reflecting retrieval mode processes, will be most prominent over right frontal regions. When the sustained SSVEP response to the episodic memory retrieval condition was compared with the sustained response to the baseline condition, the most prominent and most statistically significant SSVEP amplitude and latency reductions were observed over right frontal regions. While sustained SSVEP amplitude reductions were fairly uniform over most regions, sustained SSVEP latency reductions were clearly asymmetric and largest over right fronto-temporal regions. The topography of these reductions is consistent with findings obtained using other functional imaging techniques where episodic and non-episodic memory tasks have been compared. As indicated in Chapter 2, a majority of PET and fMRI studies have reported prominent activations within right prefrontal regions (Cabeza and Nyberg 2000). Moreover, most of these studies reported greater right frontal than left frontal activation. This is in accordance with the HERA model (Nyberg et al. 1996a; Tulving et al. 1994a) in which episodic memory encoding and semantic retrieval is lateralized to left prefrontal regions, whereas episodic memory

131 Discussion

retrieval is lateralized to right prefrontal regions. In addition, Duzel et al. (1999), using PET methods, showed that the right prefrontal cortex, BA10, was activated for an episodic retrieval task relative to a semantic retrieval task (see Chapter 2). In the same study, using DC-ERP techniques, sustained positive-going ERP activity was observed over right frontal regions. Using source localization techniques, the authors found that this activity was generated in the right frontopolar region, that is, in a region that overlaps with BA10. In the present study, while the precise source of SSVEP amplitude and latency reductions was not investigated, the observed patterns are suggestive of underlying right fronto-temporal activity. The use of source localization techniques, such as those used by Duzel et al. (1999), could help identify the neural origin(s) of these sustained SSVEP changes.

While PET and fMRI studies have consistently yielded evidence of right prefrontal activations with retrieval mode processes, other neural regions have also been implicated. Activations within medial temporal, medial parieto-occipital, lateral parietal, anterior cingulate, occipital, and cerebellar regions have also been reported in PET and fMRI studies (Cabeza and Nyberg 2000). In the present study, when the SSVEP values were averaged over the entire ‘task interval,’ the episodic retrieval condition relative to the baseline condition showed significant SSVEP amplitude and latency reductions over frontal and right hemisphere regions. The involvement of right parieto-temporal regions, in particular, may reflect spatial and perceptual processes involved in recognizing target objects presented amongst a larger number of distractor objects. Moreover, it is suggested that the bias towards right hemisphere sites in this study is consistent with the relatively abstract pictorial nature of the stimulus objects used. While the largest sustained SSVEP amplitude and latency reductions occurred over right frontal regions, left frontal SSVEP amplitude and latency reductions were also highly statistically significant. As indicated in the electrophysiological literature, the involvement of left frontal sites has been associated with task difficulty and effort in performing an episodic retrieval task (Rugg et al. 2000). Nolde et al. (1998) suggested that while the right prefrontal cortex is activated while subjects remember events, left prefrontal regions are activated when more complex and reflective retrieval is required.

The investigation of neural regions involved in retrieval mode processes, as discussed

132 Discussion

above, was conducted by comparing the overall average of the SSVEP amplitude and latency values across all three memory load conditions with the average SSVEP amplitude and latency values for the baseline condition. A secondary analysis, also utilizing SSVEP amplitude and latency values averaged over the 40s ‘task interval,’ was performed to investigate the effect of task difficulty on sustained retrieval mode processes. This was done by directly comparing the SSVEP amplitude and latency data for the three individual memory load conditions, that is, for m3 with m1, m5 with m1, and m5 with m3. SSVEP amplitude and latency differences for these comparisons were statistically significant at right occipital sites for all comparisons, and at left central and right temporal sites for the comparison of the m5 with the m3 condition. The largest differences occurred between the m5 and m1 conditions. They were located over right occipital regions, and were characterized by SSVEP amplitude and latency reductions for the m5 relative to the m1 condition. In contrast to the differences between the sustained SSVEP responses associated with the episodic memory and baseline conditions, sustained right frontal SSVEP differences between memory load conditions were not statistically significant. Given the suggested role of right frontal regions in sustained retrieval mode processes, the absence of load-dependent differences over right frontal regions suggests that retrieval mode processes are not dependent on memory load, and therefore on task difficulty. Load-dependent amplitude and latency reductions over occipital regions are consistent with previous findings indicating such reductions are associated with visual vigilance (Silberstein et al. 1990b), heightened visual attention (Silberstein et al. 1996) and object encoding (Silberstein et al. 2001). In the present study, it seems reasonable to suppose that a more difficult task would have promoted higher levels of attention. It is therefore suggested that the sustained load- dependent SSVEP amplitude and latency reductions over occipital regions reflect heightened visual attention and are not unique to retrieval mode processes.

The effect of memory load on retrieval processes can also be discussed in terms of retrieval effort. As discussed in section 6.1, all subjects reported that the larger the memory load, the more difficult was the retrieval task. Subjects also stated that they felt that it took longer to respond to targets as the memory load increased, and such personal accounts were supported by the response time data. It therefore seems reasonable to suppose that retrieval effort also increased with increasing memory load. Rugg and

133 Discussion

Wilding (2000) suggested that retrieval effort is an item-specific process, defined as ‘the mobilization of processing resources in service of a retrieval attempt’ (p. 110). However, PET findings have suggested that retrieval effort is manifest as a sustained task-related effect. A number of PET studies have reported frontal activations, particularly on the left side, for more effortful relative to less effortful tasks (eg. Cabeza and Nyberg 2000). Left frontal activation has also been described as being more common in older adults than young adults, and has therefore been interpreted as a compensatory effect (Cabeza et al. 1997a). These findings, however, do not rule-out the possibility that left frontal activations reflect the summed activity of transient item- related effects. In the present study, frontal differences in the SSVEP response to different memory load conditions, reflecting differences between more effortful and less effortful tasks, were not statistically significant. It is therefore suggested that retrieval effort is not manifest as a sustained task-related process. It is possible, however, that task-related SSVEP amplitude and latency differences between different memory loads, and therefore different retrieval effort requirements, may have been too small to detect using the SSPT technique. This is in contrast to the larger, statistically significant, differences that were observed for the overall average of these memory load conditions relative to the baseline condition. However, the explanation that the task-related SSVEP differences were too small seems unlikely in light of the significant load-dependent transient differences that were found (see section 6.3). These transient SSVEP amplitude and latency differences were calculated using a much smaller number of Fourier coefficients, therefore giving a much smaller signal to noise ratio, yet differences between memory loads were significant.

6.2.2 Interpretation of sustained SSVEP amplitude and latency reductions

Thus far, SSVEP effects have been discussed in terms of topography, or scalp location. Another important aspect of the data is the nature of the effects, that is, whether a comparison between particular conditions produces amplitude and latency increases or reductions. In this study, to examine sustained retrieval mode processes, the SSVEP response to the retrieval condition was compared to the response to the baseline condition. This comparison yielded amplitude and latency reductions for the retrieval

134 Discussion

condition relative to the baseline condition. To examine the effect of memory load on retrieval processes, the SSVEP response to a higher memory load was compared directly with the response to a lower memory load. Again, amplitude and latency reductions were obtained. The interpretation of these SSVEP amplitude and latency reductions will be discussed below.

As already mentioned, an attenuation of the 13Hz SSVEP amplitude has been associated with a state of heightened attention in a number of experimental situations. These include the heightened attention phases during the performance of a visual vigiliance task (Silberstein et al. 1990b) and a continuous performance task (Silberstein et al. 1996), and the ‘intake’ period of an object working memory task (Silberstein et al. 2001). Pipingas et al. (2000) showed graded SSVEP amplitude changes during a visual attention task. Subjects who responded faster to the appearance of target letters in a continuous performance task also showed a greater attenuation in SSVEP amplitude prior to target presentation than did subjects who responded more slowly. The authors equated a greater attenuation of the SSVEP amplitude with a heightened attention prior to target presentation that resulted in faster responses. Silberstein et al. (1995b) has suggested that the attenuation of the 13 Hz SSVEP amplitude associated with increased regional brain activity may be related to the alpha event-related desynchronization (ERD) that has been shown to index regional cortical activity in various cognitive and motor tasks (Pfurtscheller and Aranibar 1977b; Pfurtscheller and Klimesch 1990).

More recently, however, SSVEP amplitude increases have been observed during periods of mental effort occurring at certain phases of a cognitive task. For example, an amplitude increase was observed during the ‘hold’ period of a spatial working memory task (Silberstein 1998). A similar amplitude increase was also observed during the ‘hold’ period of an object working memory task (Silberstein et al. 2001). However, during the ‘intake’ or encoding period of the same task, an amplitude decrease was observed and was attributed to perceptual processes.

In the light of these apparently inconsistent effects of mental effort on the SSVEP amplitude, it was posited that the type of SSVEP amplitude response depends on the exact nature of the cognitive activity being undertaken (Silberstein et al. 2001). There

135 Discussion

appear to be similarities between the amplitude of the SSVEP response and the amplitude of the alpha response noted by Ray and Cole (1985), namely, that attending to visual targets engendered a decrease in alpha amplitude, whereas attending to mental imagery engendered an increase.

In the present study, the SSVEP amplitude shows a sustained reduction for the overall average of the three memory load conditions relative to the average of the baseline condition. This amplitude reduction, in conjunction with a pattern of more localized latency changes, was considered a reflection of sustained retrieval mode processes. Because the amplitude reductions were diffuse and non-localized compared with the latency reductions, these may not have reflected retrieval mode processes. Instead, the amplitude reductions may have reflected non-specific aspects of the task, such as attention or those associated with task difficulty. When the SSVEP responses to the various individual memory load conditions were compared directly to examine the effect of task difficulty on retrieval mode processes, the largest differences occurred at right occipital sites and were characterized by SSVEP amplitude and latency reductions for a higher relative to a lower memory load. Thus the observed occipital amplitude reductions may well be attentional effects. This would be consistent with the previous SSVEP findings showing amplitude reductions associated with perceptual attention.

The observed latency effects may specifically reflect retrieval mode processes, given the correspondence of the location of SSVEP latency reductions over right frontal regions to findings from neuroimaging and electrophysiological studies. Silberstein et al. (1996) observed that faster response times in a continuous performance task were associated with greater transient decreases in prefrontal SSVEP latency, and suggested that latency reflects speed of information processing. The transient latency reductions were considered to reflect transient increases in neural information processing speed resulting from increased coupling strength between neural populations. In the present study, sustained latency reductions occurred over right frontal regions, and in-line with the proposal of Silberstein et al (1996), it is suggested that these are a reflection of increased coupling between those regions required in maintaining a retrieval mode.

In summary, the sustained SSVEP effects for the retrieval task relative to the baseline

136 Discussion

task were characterized by diffuse non-localized amplitude reductions, and latency reductions that were more prominent and were localized over right fronto-temporal regions. While the source of these reductions was not investigated in the present study, they occurred in the vicinity of those underlying cortical regions believed to be involved in retrieval mode processes that have been more precisely localized in PET, fMRI, and DC-ERP studies using source localization techniques. It is therefore proposed that SSVEP amplitude and latency reductions at right fronto-temporal regions reflect sustained retrieval mode processes. Furthermore, increases in task difficulty did not appear to significantly alter this pattern of latency reduction. It is therefore also proposed that retrieval mode processes are independent of task difficulty or retrieval effort.

6.3 Transient SSVEP amplitude and latency changes: the effect of memory load on successful memory retrieval

In the previous section, sustained SSVEP amplitude and latency effects were discussed in relation to the maintenance of a retrieval mode. The sustained SSVEP effects were obtained using a long (40s) averaging period so that neural activity maintained for the duration of the task could be investigated. This approach essentially averages SSVEP amplitude and latency across all time points during the task, and includes all target and non-target presentations. However, more transient effects that reflect more rapid neural processes, or that may differ for target and non-target objects, are masked using this approach. In a separate analysis, SSVEP amplitude and latency data were calculated using a short averaging period so that more transient patterns of neural activity could be examined. This section discusses the transient SSVEP amplitude and latency changes that occurred at a point in time when subjects were successfully retrieving target objects from memory.

One of the main aims of this study was to investigate the SSVEP amplitude and latency changes associated with the retrieval from memory of abstract, two-dimensional, objects. Retrieval processes were examined by varying the memory load so that retrieval became more difficult with increasing memory load. It was anticipated that

137 Discussion

increases in memory load would be associated with increased utilization of those neural regions involved in successful retrieval processes. Electrophysiological findings led to the second hypothesis: that successful retrieval would be associated with transient SSVEP amplitude and latency changes over parietal and right frontal regions. Furthermore, because abstract pictorial stimuli were used, it was further hypothesized that the SSVEP changes would show a pattern with a more bilateral distribution than that of the widely reported left parietal ERP old/new effect observed with verbal stimuli.

Transient load-dependent reductions in both SSVEP amplitude and latency that followed the appearance of target objects were observed. More specifically, transient amplitude reductions for the m3 relative to m1 comparison were most prominent at occipital and bilateral parieto-temporal sites. For the m5 relative to the m1 comparison, amplitude reductions were larger and more extensive, particularly at right hemisphere sites. Latency reductions were more diffuse than amplitude reductions, although they also appeared to be larger and more right lateralized as the memory load increased. These SSVEP differences were statistically significant for the m3 relative to the m1 comparison at parietal and right occipital sites. The SSVEP differences for the m5 relative to the m1 comparison were statistically significant mainly over right hemisphere regions, including occipital, parietal, parieto-temporal and right inferior frontal sites. SSVEP differences for the m5 relative to the m3 comparison, while also characterized by SSVEP amplitude and latency reductions were not statistically significant. It is noteworthy that, compared with the load-dependent amplitude and latency differences obtained for target objects, load-dependent differences for non-target objects were much smaller and diffuse, and were not statistically significant at any electrode site or for any time point.

While the topographies of transient SSVEP amplitude and latency differences were calculated for the time point 400ms after the appearance of a target, when subjects were likely to be engaged in memory retrieval, transient SSVEP amplitude and latency reductions were evident throughout the entire 1400ms period that the target object was visible. These reductions were statistically significant in the interval from approximately 200 to 900ms after target appearance, and were located over regions similar to those at 400ms.

138 Discussion

The second hypothesis predicted that transient SSVEP amplitude and latency load- dependent changes would be observed at parietal and right frontal regions, regions consistently noted in the electrophysiological literature as being involved in successful retrieval processes. Furthermore, as the stimulus shapes were not easy to verbalize, it was hypothesized that parietal changes would be bilateral rather than exclusively left- sided. It appears that these hypotheses were supported. Relative to lower memory loads, higher memory loads produced transient SSVEP amplitude and latency reductions over bilateral parieto-temporal and right inferior frontal regions. A detailed discussion of the transient SSVEP response over parietal and right frontal regions will follow in sections 6.3.1 and 6.3.2 respectively. The neural regions considered responsible for the observed effects will be discussed mainly in relation to previous studies that have associated ERP old/new differences with either familiarity or recollection-based memory processes. Several studies have indicated that frontal changes are related to aspects of episodic retrieval such as retrieval strategy and retrieval effort. Transient frontal SSVEP effects will also be discussed in relation to retrieval effort in section 6.3.3. Finally, in section 6.3.4, the interpretation of transient SSVEP amplitude and latency reductions is discussed.

6.3.1 Transient parietal effects

The parietal ERP old/new effect, where previously studied old items give rise to more positive ERPs than novel new items, has frequently been reported in episodic memory retrieval studies (see Chapter 2). In the present study, transient load-dependent effects were most prominent over bilateral parietal and adjacent scalp regions, as predicted in the second hypothesis of this study. Unlike the parietal ERP old/new effect, however, the SSVEP differences were load-dependent, and therefore more specifically associated with those regions more extensively utilized when subjects are making a greater effort to retrieve studied objects from memory.

Clinical evidence, reviewed in section 2.1.2, has indicated involvement of the medial temporal lobes in retrieval processes. It has been suggested that the parietal ERP

139 Discussion

old/new effect, which has been used to investigate neural activity associated with successful retrieval, is a reflection of medial temporal lobe activity (Rugg et al. 1996). Evidence supporting this proposal comes from patient studies where ERP old/new differences, similar to those recorded over scalp areas, were obtained using depth electrodes within the medial temporal lobes (eg. Guillem et al. 1995a; Guillem et al. 1995b; Guillem et al. 1996; Heit et al. 1990; Puce et al. 1991; Smith et al. 1986), and where left temporal lobectomy patients failed to show a late ERP old/new effect (eg. Smith and Halgren 1989). Guillem and colleagues (Guillem et al. 1995a; Guillem et al. 1995b) recorded directly from a number of intracerebral areas and concluded that the ERP old/new effect is modulated by distributed, yet highly interconnected, brain regions, including hippocampal, parietal and frontal regions. While volume conduction of parietal and frontal cortical potentials to the scalp appears plausible, transmission from deeper medial temporal lobe structures seems less likely. Klimesch et al. (1996), however, suggested that theta activity, which appears to index episodic retrieval processes, might be transmitted to the cortex via hippocampo-cortical feedback loops.

Findings from the present study do not provide further insights into the sources of the ERP old/new effect. However, spatio-temporal similarities between transient load- dependent SSVEP effects and ERP old/new effects suggest common origins. In future studies, the use of source derivation techniques similar to those used by Duzel et al. (1999) should provide valuable information regarding the sources of transient SSVEP amplitude and latency changes. Furthermore, the use of electrophysiological techniques with their high temporal resolution, in combination with PET and fMRI techniques with their high spatial resolution, should also help answer this question.

Atlhough they lack the temporal resolution of electrophysiological techniques, PET and fMRI functional neuroimaging techniques, with their superior spatial resolution, have helped us identify neural regions involved in retrieval processes. Because these techniques do not permit a high temporal resolution, studies using a block design have been popular. For example, within a study, different blocks may contain different proportions of studied to unstudied items (Cabeza and Nyberg 2000). In this way regions showing greater activation with a greater proportion of studied items can be identified. Such studies have indicated that the medial temporal lobes are involved in

140 Discussion

successful memory retrieval. Nyberg et al. (1996b) showed that medial temporal lobe activation was correlated with the number of correctly retrieved old words (Nyberg et al. 1996b).

Activation of medial temporal lobes during successful retrieval has generally been found to occur bilaterally, regardless of the nature of the test items used (Cabeza and Nyberg 2000). In the present study, significant transient load-dependent SSVEP effects were found bilaterally at parietal and adjacent scalp sites. However, to suggest that these bilateral SSVEP effects are in some way related to bilateral activity within the medial temporal lobes would be highly speculative. The increased right lateralization of SSVEP effects that occurred with increasing memory load is, however, discussed below.

A number of additional cortical regions have been associated with retrieval processes, in particular, the medial parieto-occipital region, the occipital region, and lateral parietal regions (Cabeza and Nyberg 2000). However, activation of these areas has been attributed more specifically to processes associated with discerning the perceptual and spatial detail of items rather than with processes specifically related to retrieval success. These regions of cortex also lie below recording sites that showed significant load- dependent SSVEP amplitude and latency effects and may also be involved in discriminating between the perceptual or spatial details that determine whether items are targets or non-targets. The spatial resolution of the SSPT technique used in the present study did not allow a more precise localization of the specific neural regions involved. Source localization and multimodal imaging approaches may help to address this problem in future studies.

In addition to predicting that transient load-dependent SSVEP changes would occur over parietal and right frontal regions, the second hypothesis also refers to the lateralization of transient parietal effects. Because of the non-verbal nature of the stimuli used, a relatively bilateral pattern of parietal effects was predicted rather than the widely reported left-lateralized pattern obtained using verbal stimuli. For the m3 relative to the m1 comparison, SSVEP amplitude and latency reductions over parietal regions were indeed located bilaterally. For the m5 relative to the m1 comparison, both amplitude and latency reductions were larger and right lateralized. Two possible

141 Discussion

explanations are suggested for this increasing right lateralization of amplitude and latency reductions with increasing memory load. Firstly, with increasing memory load, regions specifically involved in the retrieval of abstract objects become more apparent, and these regions are located in the right hemisphere. Secondly, the right hemisphere is engaged in a more general way when the task becomes more difficult. While both of these explanations appear plausible, the fact that load-dependent effects were not observed for non-target objects makes the second explanation less likely. However, the identification of non-targets may have occurred more quickly and less laboriously than the identification of target objects, thereby reducing differences between memory load conditions. This, however, could not be determined because subjects were only required to respond to target objects.

Johnson (1995) suggested that the absence of ERP old/new effects in some studies resulted from the use of low memory loads or insufficient delay between first and second presentations. He argued that only with sufficient memory load do retrieval related differences become apparent. Similarly, the present findings indicate that when sufficient memory load is applied, the right hemisphere is shown to be involved in the retrieval of abstract objects from memory.

Schloerscheidt and Rugg (1997) hypothesized that the retrieval of objects from memory would be associated with a relatively bilateral pattern for parietal ERP old/new differences, while retrieval of words would show more prominent effects over left parietal regions. This hypothesis was based on observations of verbal deficits in individuals with damage to left medial temporal lobe structures, and deficits in retrieval from pictorial memory in individuals with damage to right medial temporal lobe regions. However, Schloerscheidt and Rugg (1997) observed a similar left-lateralized parietal ERP old/new effect for both objects and words. They thus concluded that the medial temporal lobes are not involved in the generation of the ERP old/new effect. This conclusion appears at odds with a large body of literature suggesting that medial temporal lobe structures are indeed associated with explicit memory processes and are involved in the generation and modulation of the ERP old/new effect.

Schloerscheidt and Rugg may have obtained left parietal lateralization of ERP old/new

142 Discussion

effects with pictorial stimuli because the stimuli used were pictures of common objects which may have been encoded and subsequently retrieved using verbal processes. The abstract stimuli used in the present study were not as easily verbalized, and this could explain the observed right lateralization of effects. An alternative explanation for the left-sided ERP old/new effects observed by Schloerscheidt and Rugg, in line with the proposal of Johnson (1995), is that the retrieval task contained insufficient memory load.

It is also possible that right hemisphere load-dependent effects may reflect processes involved more specifically in distinguishing target objects from similar-looking non- target objects, rather than reflecting processes related to episodic memory retrieval per se. This view is consistent with observations of activations within right lateral parietal regions during the processing of spatial information in an episodic retrieval task. (Moscovitch et al. 1995). Lateral parietal activations have also been associated with perceptual aspects of recognition memory processes (Cabeza et al. 1997b), and occipital activations have been reported for non-verbal retrieval and memory-related imagery operations (Cabeza and Nyberg 2000). Although such processes may be engaged while processing target stimuli in the present study, it is expected that they would also be engaged during the processing of non-target stimuli to enable a decision to be made as to whether a test stimulus is a target or a non-target. Given that SSVEP amplitude and latency differences between memory loads were not statistically significant following the presentation of non-target stimuli, it is reasonable to suppose that target SSVEP differences arise mainly from retrieval effort, and to a lesser extent from other processes that are necessary for performance of the task.

6.3.2 Transient right frontal effects

In addition to occurring over parietal scalp regions, the ERP old/new effect has also been observed over right frontal regions. Moreover, the investigation of the combination of parietal and frontal ERP old/new effects may provide information about the type of recognition occurring. That is, it may indicate whether recognition is based on recollection, reflecting episodic memory retrieval processes, or is based on familiarity,

143 Discussion

possibly involving implicit memory retrieval processes.

As discussed in Chapter 2, right frontal ERP old/new effects have been observed when the context of studied items is remembered (eg. Wilding and Rugg 1996), when studied associations are correctly retrieved (eg. Donaldson and Rugg 1998), or when specific perceptual or spatial details of studied pictures are correctly recalled (eg. Ranganath and Paller 2000). Wilding and Rugg (1996) required subjects to remember whether old words had been spoken by a male or female voice. Correct contextual judgements produced left parietal and right frontal ERP old/new effects. These effects were prolonged, particularly at frontal regions, lasting well beyond the motor response. Given that these effects were associated with retrieval of the specific study context, the authors argued that they index recollection-based recognition involving episodic memory processes. They also argued that the prolonged frontal effects were associated with the monitoring processes that occur during recollection-based retrieval.

ERP old/new effects for incorrect contextual judgements were observed over the same regions, but these were much smaller and less prolonged. These differences were attributed to familiarity-based retrieval since the specific study context was not correctly identified. However, because the ERP old/new effects for both conditions were qualitatively similar, the authors proposed that their observations were consistent with a model in which both recollection- and familiarity-based retrieval utilize the same explicit memory system (Moscovitch 1992; Moscovitch 1994; Squire 1994). In this model, familiarity-based recognition can be achieved solely with hippocampal output. Retrieval of contextual information, or recollection, is achieved through further integration with the prefrontal cortex.

Other researchers, however, claim that familiarity-based recognition utilizes the implicit memory system (Squire 1992; Tulving and Schacter 1990). This view has recently been supported by a study in which familiarity-based processes appeared to involve the perirhinal cortex and the medial dorsal nucleus of the thalamus, and not the hippocampus (Appleton and Brown 1999).

The location and timing of transient load-dependent effects in the present study are

144 Discussion

consistent with the findings of Wilding and Rugg (1996). Statistically significant transient SSVEP amplitude and latency reductions over right inferior frontal regions were observed with both the m3 relative to m1, and the m5 relative to m1, comparisons. As already mentioned in section 6.3.1, retrieving targets from memory produced prolonged parieto-temporal amplitude and latency reductions for higher relative to lower memory load conditions. Amplitude and latency reductions for higher relative to lower memory load conditions over right frontal regions were also prolonged, lasting more than one second. They were statistically significant in the interval between 400 and 800ms after the appearance of the target. Right frontal effects described by Wilding and Rugg (1996) occurred over a large area. In the present study, SSVEP frontal effects were localized to right inferior frontal sites. The less diffuse nature of the effect in the present study can possibly be explained by the fact that responses to targets under different memory load conditions were compared. Wilding and Rugg (1996), however, compared ERPs for targets relative to non-targets. Present findings may, in fact, have reflected activity in underlying neural regions involved when retrieval becomes more demanding.

As already explained, right frontal effects have been observed in studies that used the ERP old/new effect to examine episodic memory retrieval. In the present study, transient load-dependent effects were also observed over right frontal regions and may also reflect episodic memory retrieval processes. Certain aspects of the observed SSVEP amplitude and latency reductions support this proposal. Firstly, both the location and the timing of the parieto-temporal and right frontal SSVEP amplitude and latency reductions were similar to the spatio-temporal effects described in studies designed to investigate context-based retrieval, and in studies where specific detail relating to the studied events was retrieved. Secondly, transient frontal and parieto- temporal SSVEP reductions increased with increases in memory load. These greater reductions, in conjunction with increased response times and perceived effort, indicate that that retrieval becomes more demanding as memory load increases. If recognition resulted purely from familiarity, involving more automatic and less effortful processes, load-dependent amplitude and latency reductions would not be expected. Furthermore, while the actual cognitive operations used to retrieve objects from memory have not been fully determined, they are likely to involve memory scanning and/or some form of

145 Discussion

perceptual or spatial discrimination in order to determine whether an object is a target or a distractor. It seems plausible that such processes would also be affected in some way by increases in memory load. Familiarity-based retrieval is faster (Yonelinas and Jacoby 1995) and is therefore unlikely to involve these more elaborate processes. The similarity in the appearance of target and non-target objects in the present study meant that subjects had to retrieve specific features of the stimuli, thus increasing the likelihood of context-based episodic memory retrieval rather than familiarity-based retrieval occurring. Thirdly, because encoding was intentional rather than incidental, and was also reinforced in a practice task, targets would have been relatively well encoded, and therefore subjects would have been confident that their judgements were correct. All of these factors would have increased the likelihood that recognition was based on explicit or declarative memory rather than implicit or familiarity-based memory. However, the use of familiarity-based processes cannot be totally discounted.

As was discussed earlier, separate neural systems may be involved in familiarity-based and recollection-based recognition (Appleton and Brown 1999). That is, familiarity- based recognition may involve perirhinal regions, whereas recollection-based recognition involves medial temporal regions. Given the spatial resolution of the SSPT technique used, the neural regions responsible for the observed scalp SSVEP effects cannot be localized more precisely. Source localization techniques or multimodal imaging could prove useful in localizing more precisely the neural regions utilized, therefore possibly indicating whether familiarity-based or recollection-based recognition is being accessed.

While the time-course of the observed SSVEP load-dependent changes is not dissimilar to findings from context-based memory retrieval studies, it must be remembered that SSVEP amplitude and latency values were derived from Fourier coefficients averaged over a moving window of 20 cycles, giving an effective temporal resolution of 870ms. As a result, faster neural processes may have been reflected in the analysed data by slower and more prolonged SSVEP amplitude and latency changes. It is therefore possible that the conclusion drawn, namely, that recognition was more likely to have been based on recollection rather than familiarity, is incorrect. However, the fact that transient amplitude and latency reductions were maximum 200 to 400ms after the motor

146 Discussion

response, rather than prior to the response, indicates that these reductions reflected slower processes more consistent with recollection-based memory retrieval.

6.3.3 Transient effects and retrieval effort

Another aspect of retrieval processing that has been discussed in the electrophysiological literature is the effort required to retrieve items from memory (eg. Rugg and Wilding 2000). Retrieval effort has been investigated by comparing the ERP response to retrieval tasks with varying levels of difficulty. For such comparisons, significant differences between ERP components over frontal regions are commonly reported (eg. Ranganath and Paller 1999; Ranganath and Paller 2000; Rugg et al. 2000). For example, a greater positivity was elicited over left frontal regions by the more difficult of two retrieval tasks (Ranganath and Paller 1999). In contrast, right frontal ERP changes were associated with the retrieval of words that had been encoded shallowly rather than with words that had been encoded more deeply (Rugg et al. 2000). The lack of agreement in the ERP literature over frontal lateralization of transient retrieval effort effects also exists in the PET and fMRI literature (Rugg and Wilding 2000).

In section 6.1 it was argued on the basis of response time data and subjects’ personal accounts that increased memory load results in both increased retrieval difficulty and increased retrieval effort. During the successful retrieval of target objects, for higher relative to lower memory load conditions, significant transient amplitude and latency reductions over bilateral parieto-temporal and occipital regions, and right inferior frontal regions were observed. These reductions may therefore reflect the increased difficulty in retrieving objects when the memory load is increased. As discussed in section 6.3.1, the source of the right hemisphere load-dependent SSVEP amplitude and latency reductions may be either neural regions involved in the retrieval of abstract objects, or neural regions recruited when the task becomes more difficult. Rugg and Wilding (2000) suggest that these two possibilities could be reflections of the same neural processes. That is, retrieval effort may not have a distinct neural signature; instead, ‘the neural correlates of increasing effort will be manifest as increased activity

147 Discussion

of whatever brain regions are engaged by the retrieval task in question’ (Rugg and Wilding 2000, p.114). This concept is of particular relevance to this study as it underpins the use of different memory loads to identify the neural regions involved in successful memory retrieval. The present author suggests that the employment of tasks with different memory loads warrants further investigation as this may help to identify more precisely neural regions associated with successful retrieval processes than do the more traditional methods of comparing the responses to old and new items.

In order to achieve a clearer picture of the relationship between memory load and the SSVEP response, a more recent study performed by the present author incorporated five different memory load levels. While the experimental conditions differed considerably from those of the present study, preliminary results indicate a correlation between the SSVEP response and memory load (see Appendix F for poster presentation).

6.3.4 Interpretation of transient SSVEP amplitude and latency reductions

Thus far, transient SSVEP effects have been discussed in terms of scalp location, but the nature of these effects, that is, whether they involved amplitude or latency, and whether these were increased or reduced is also of significance. In section 6.2.2 it was explained that amplitude reductions are thought to reflect increased neural activity, and latency reductions are thought to reflect increased coupling between neural regions leading to increased processing speed. It was suggested in section 6.2.2 that amplitude and latency reductions reflect sustained retrieval mode processes, and given the topography of latency reductions in particular, it was suggested that latency reductions might index the coupling of neural regions during the maintenance of a retrieval mode. Differences that exist in the interpretation of sustained and transient SSVEP amplitude and latency reductions lie primarily in the cognitive processes that they may be indexing. Transient SSVEP amplitude and latency reductions occur much faster than the observed sustained reductions, and are more likely to reflect neural activity associated with more specific memory retrieval processes. Another difference in the interpretation of sustained and transient reductions lies in the fact that the sustained reductions were obtained for a memory retrieval task relative to a low demand control task, whereas the transient

148 Discussion

changes were load-dependent and appear to be graded. The brief discussion below extends the previous interpretation of sustained effects to include an interpretation of faster load-dependent amplitude and latency reductions.

As indictated in Chapter 3, SSVEP amplitude reductions demonstrated during a visual vigilance task (Silberstein et al. 1990b) and during the performance of the Wisconsin Card Sorting Task (Silberstein et al. 1995b) were likened to the phenomenon of event related desynchronisation (ERD) (eg. Pfurtscheller and Aranibar 1977a; Pfurtscheller and Klimesch 1990) where transient alpha amplitude reductions were considered to index regional increases in cortical activity associated with the performance of cognitive and motor tasks. Furthermore, the extent of alpha attenuation has been associated more specifically with the difficulty and/or relevance of a task (eg. Klimesch et al. 1990a; Klimesch et al. 1993). Similarly, in relation to the present findings, it is suggested that transient SSVEP amplitude reductions index the extent of utilization of those neural regions associated with successful retrieval processes. Because SSVEP amplitude reductions were observed for higher relative to lower memory load conditions, it is more specifically suggested that SSVEP amplitude reductions index increased difficulty or effort associated with performance of a higher memory load condition.

Comparisons between the m3 relative to the m1 condition, the m5 relative to the m1 condition, and the m5 relative to the m3 condition, all show SSVEP amplitude and latency reductions over parietal and right frontal regions during the time that subjects were considered to be actually retrieving target objects from memory. The amplitude and latency reductions for the higher relative to the lower memory load conditions were statistically significant for the m5 relative to the m1, and the m3 relative to the m1 comparisons, but did not reach significance for the m5 relative to the m3 comparison. The Hotelling’s T score is a bivariate statistic that combines each pair of amplitude and phase values into a single vector, thereby clouding the independent contribution of each of these components. Separate statistical analyses of amplitude and latency are planned for future studies.

Klimesch and colleagues (eg. Doppelmayr et al. 1998; Klimesch et al. 1997; Klimesch et al. 1994) demonstrated that episodic retrieval processes are associated with specific

149 Discussion

EEG theta band changes. Transient theta increases, or theta synchronization, during a time interval when subjects are involved in successfully recognizing previously studied items, was reported in a number of studies by this group. It has been suggested that these theta increases reflect hippocampal activity that is transferred to the scalp via longitudinal hippocampal-cortical pathways (Klimesch et al. 1994). Similarly, it appears that load-dependent 13Hz SSVEP amplitude reductions reflect certain memory retrieval processes. An important distinction exists between the theta synchronization approach described in the literature and the SSPT technique used in the present study. Theta activity is a component of the brain’s intrinsic EEG, and has been observed to vary in response to mental activity. In contrast, the 13Hz SSVEP is generated by an externally produced visual flicker, and is therefore a driven response. This driven response appears to be modified by neural processes associated with mental activity. It has been suggested that SSVEP amplitude changes may in fact reflect neural processes associated with the alpha band resonant system (Silberstein et al. 2001). In the light of the reported relationship between theta activity and episodic memory processes, an extension to the present study would be to investigate SSVEP amplitude and latency variations using a probe visual flicker with a frequency in the theta range.

As mentioned above, the effects of retrieving objects from memory on the SSVEP amplitude were load-dependent, with increases in memory load resulting in transient reductions in amplitude. Similarly, the effects on latency were also load dependent. Increases in memory load resulted in transient decreases in latency over parietal and right frontal regions. In section 6.2.2, latency reductions were discussed in relation to retrieval mode processes. It was considered that latency reductions over right frontal regions reflect processes involved in the maintenance of an episodic retrieval mode. Silberstein et al. (1996) have suggested that latency reductions are a manifestation of transient increases in coupling strength between neural populations leading to increases in information processing speed. In line with this proposal, it is speculated that transient load-dependent latency reductions over parieto-temporal and right inferior frontal regions reflect increased coupling strength between neural regions involved in successful retrieval processes. It is further speculated that as memory load is increased, connections between these regions are strengthened to enable the performance of a more demanding retrieval task.

150 Discussion

In summary, transient load-dependent SSVEP amplitude and latency reductions occurred while subjects were considered to be engaged in the process of retrieving previously studied abstract objects from memory. These reductions occurred over parietal and right frontal regions as predicted by the second hypothesis of this study. While bilateral parietal effects were predicted, the increasing right lateralization of amplitude and latency reductions with increasing memory load is consistent with the fact that the abstract objects used were difficult to encode verbally. The experimental task was specifically designed to enhance the likelihood of stimuli being stored in a longer-term form of memory. This, in conjunction with the observation that the parietal and right inferior frontal latency reductions were prolonged, lasting until well after the motor response, suggests that the observed reductions were more likely to have been associated with episodic memory retrieval processes rather than with retrieval processes based on familiarity. An association between load-dependent SSVEP amplitude and latency reductions and memory retrieval processes is reinforced by the fact that there were no statistically significant findings for non-target objects.

6.4 Transient SSVEP amplitude and latency changes: target versus non-target objects

Although problematic, for reasons that will be discussed, differences between target and non-target objects are presented here for completeness, given that this comparison most closely parallels the old versus new comparison of the ERP old/new effect discussed in Chapter 2.

Transient SSVEP amplitude and latency differences between target and non-target objects were calculated separately for each memory load condition. Only differences for a memory load of 5 were statistically significant. Significant differences occurred at prefrontal and right inferior frontal sites at around 700ms after the appearance of an object, and were characterized by SSVEP amplitude and latency reductions for targets relative to non-targets. It is suggested that amplitude reductions reflect increased utilization of underlying neural regions for target objects, and latency reductions

151 Discussion

represent more efficient coupling between neural regions, enabling faster information processing (see sections 6.2.2 and 6.3.4). Although response times to non-targets were not recorded, subjects indicated that non-target objects were more rapidly and easily identified than target objects. It is considered that memory scanning or some form of perceptual or spatial discrimination was used to identify targets. Thus, SSVEP amplitude and latency reductions, for target compared with non-target objects, are consistent with more effortful and extended processing for these items. Johnson (1995) suggested that the magnitude of the memory load is important in determining whether ERP old/new effects are observed. The results of the present study are consistent with this viewpoint since significant differences were only observed for a memory load of 5.

Very few studies have examined ERP old/new differences using pictorial stimuli. Those that have can not be directly compared because their experimental conditions differ in, for example, experimental design (continuous recognition versus study-test) and/or type of stimulus (verbalizable versus non-verbalizable). In contrast to ERP old/new effects observed with verbal studies, parietal differences are not always observed with pictorial studies (eg. Friedman 1990a). It appears that the only common finding with pictorial studies is that the ERP old/new effects involve frontal regions. In one study, ERP old/new differences were found to be much larger for picture than for word stimuli (Schloerscheidt and Rugg 1997). Moreover, while both pictures and words showed early ERP old/new effects at left parietal sites, only pictures elicited significant right frontal effects. The authors concluded that these right frontal differences are consistent with the suggestion that pictures are more richly encoded than words, and are therefore more likely to engage post-retrieval processes. It is difficult to determine whether the SSVEP amplitude and latency reductions observed in the present study support the interpretation of ERP old/new findings because of the paucity of studies and the variety of experimental conditions used. The timing and topography of frontal SSVEP reductions, however, are comparable to those of frontal ERP old/new effects.

The interpretation of SSVEP effects for targets relative to non-targets should be approached with caution because a number of variables were not well controlled. Firstly, targets required a motor response whereas non-targets did not. The observed frontal differences may therefore be attributable to the motor response, although this

152 Discussion

seems unlikely as significant differences were largest well after the motor response. Furthermore, differences were statistically significant only for a memory load of five; they were not significant for loads of three and one. If differences resulted from the motor response, then similar differences for all three memory load conditions might be expected. Secondly, targets appeared with a probability of 0.25 while non-targets appeared more frequently with a probability of 0.75. It could therefore be argued that target/non-target differences reflect probability effects. If this were the case, however, a diffuse P300-like response would be expected over parietal regions at around 300ms after object appearance (eg. Picton 1992). In the present study, however, differences were more frontal and occurred much later. Thirdly, there was a mismatch between the number of targets and non-targets presented; far fewer targets were averaged resulting in a lower signal to noise ratio. This may have reduced observed differences, but the effect would have been the same for all memory load conditions. However, significant differences occurred only for a memory load of five. It therefore appears unlikely that the aforementioned factors contributed significantly to the transient amplitude and latency differences observed. Consequently, it is tentatively suggested that the observed transient SSVEP amplitude and latency differences reflect processes associated with retrieval success in the same way that ERP old/new effects reflect these processes. Furthermore, as the frontal SSVEP amplitude and latency reductions for targets occurred after the motor response, it is further speculated that these reductions more specifically reflect monitoring processes associated with episodic memory retrieval.

Despite the assumption that differences between the SSVEP response to targets and non-targets were probably unaffected by the aforementioned confounding factors, the conclusions drawn should be re-examined when such factors have been eliminated. To address all three issues, the experiments could be repeated with an increased number of stimulus items, equal numbers of targets and non-targets, and responses to both target and non-target objects. Problems with interpretation due to the motor response could be reduced in two ways. Firstly, a larger number of stimulus items would permit a higher temporal resolution. If effects associated with memory retrieval and the motor response occurred at different times, a higher temporal resolution would help to differentiate these processes. Secondly, a motor response to both target and non-target objects should lead to its effects subtracting out when comparing one condition with the other.

153 Discussion

Probability effects should be eliminated simply by the use of equal numbers of target and non-target stimuli. For a given temporal resolution, increased numbers of target and non-target stimuli would also improve the signal to noise ratio, increasing the likelihood of significant effects being observed. Additional experiments employing both verbal and pictorial material could help determine whether frontal memory retrieval effects depend on the nature of the stimulus used. This could be extended to investigate other types of visual stimuli such as numbers or colours, or stimuli encoded and tested in different sensory modalities.

6.5 Conclusions and future directions

The SSPT technique was used to investigate neural correlates of object recognition memory. Both sustained and transient SSVEP effects were analysed to examine retrieval mode and retrieval success processes respectively during the performance of an episodic memory retrieval task.

The first hypothesis of this study predicted that retrieval mode processes would be reflected in sustained SSVEP effects over right frontal regions. Sustained amplitude and latency reductions were found for the memory retrieval condition relative to the baseline condition, and these sustained reductions were most prominent and statistically significant over right frontal regions. While the amplitude reductions were diffuse, latency reductions were most prominent over right fronto-temporal regions. It is suggested that sustained SSVEP latency reductions over right fronto-temporal regions reflect the maintenance of a retrieval mode during the performance of the task. While this finding is consistent with the first hypothesis, and therefore with the current literature, further investigation is required to confirm whether the source of these effects is the same as those reported in fMRI, PET and DC-ERP studies. A secondary finding of the present study is that sustained SSVEP amplitude and latency values do not change significantly when the memory load is increased. It is suggested, therefore, that retrieval mode processes are independent of the level of effort applied when performing a memory retrieval task.

154 Discussion

The second hypothesis of this study predicted that successful retrieval of abstract objects would be associated with transient load-dependent SSVEP changes over bilateral parietal and right frontal regions. Load-dependent SSVEP amplitude and latency reductions were found over these regions during a time interval when it was considered that subjects were involved in retrieving these objects from memory. With a small increment in memory load, load-dependent differences were observed bilaterally over parietal regions. This finding is therefore consistent with the second hypothesis. However, a larger increment in memory load produced larger and more right lateralized effects. It is suggested that this pattern of right lateralized changes is consistent with the use of stimuli that are not easily verbalized, and was probably not observed in previous picture-based studies because of the use of lower memory loads in these studies. This explanation is in line with Johnson’s (1995) proposal that episodic retrieval effects become apparent when the memory loads applied, and the delays between studied and tested items, are sufficient.

It is proposed that these transient load-dependent effects reflect episodic retrieval processes rather than processes based on familiarity. This conclusion is drawn from a number of observations. Firstly, the spatio-temporal patterns of both the parietal and right frontal effects are similar to the spatio-temporal patterns of the ERP old/new effect observed in context-based retrieval studies that have specifically investigated episodic memory retrieval processes. Secondly, changes were load-dependent, meaning that retrieval was effortful and not automatic. Thirdly, the experimental design was structured to promote recollection of the perceptual details of items to be retrieved.

It is also proposed that the method of varying the memory load to investigate successful retrieval processes may give more valid indications of retrieval processes than other methods which have compared activity in response to old and new items, as used in studies into the ERP old/new effect. The advantage of the method used in the present study is that activity in response to retrieved, or old, items is not contaminated by activity generated by a comparison task requiring, for example, the identification of new items. While it is possible that the load-dependent transient effects of the present study may reflect processes specific to task effort, rather than to memory retrieval, it is suggested that this scenario is unlikely in view of the spatio-temporal similarities

155 Discussion

between transient load-dependent SSVEP effects and the ERP old/new effect. Rugg and Wilding (2000, p. 114) have, in fact, suggested that retrieval effort doesn’t have a unique neural signature. They suggest, instead, that activity associated with retrieval effort may be observed in whatever neural regions are involved with retrieval success. The present results support Rugg and Wilding’s suggestion, given the similarity between the spatio-temporal patterns of load-dependent SSVEP effects and ERP old/new effects. It is therefore suggested that the method of varying the memory load to investigate retrieval processes warrants further investigation.

There are a number of important issues that remain unresolved and further questions that arise from the present findings. Firstly, what are the neural regions that generate these sustained and transient load-dependent effects, and are these effects generated by the same neural regions that generate the ERP old/new effects? Secondly, while the present findings indicate that right frontal regions may be involved in both sustained and transient retrieval processes, it is not clear whether the same or different underlying neural regions are involved. Thirdly, are the findings of sustained and transient effects stimulus dependent? That is, would the use of stimuli such as words, numbers, or even sounds result in the same patterns of activity observed in the present study with abstract objects?

The combination of SSPT, which allows investigation of cognitive processes that occur over different time scales, with the method of varying memory load to help identify neural regions involved in retrieval processes, appears to be a potentially useful approach for the investigation of these unresolved issues. Furthermore, a combination of SSPT with source localization techniques and multi-modal imaging approaches may also prove valuable in identifying the source of the sustained and transient effects observed in the present study.

156

Appendix A. Task instructions

Appendix A. Task instructions

TASK INSTRUCTIONS

Baseline

In the centre of the screen in front of you, a series of blue numbers, 1, 2, 3 and 4 will be presented. The numbers will be shown to you over and over again for about 2 minutes. Your task during the presentation of these numbers is very simple; just R-E-L-A-X, watch the numbers and press the hand-held button S-L-O-W-L-Y whenever you see the number 4 appear .

Memory

In the centre of the screen in front of you, a series of numbers and shapes will be presented one at a time. Your task is to respond to certain shapes by pressing the hand- held button. The shapes that you must respond to will be shown to you before the task begins.

The task will begin with the presentation of a series of BLUE numbers, 1, 2, 3 and 4. The numbers will be shown to you over and over again for about 40 seconds. Your task during the BLUE numbers is very simple; just R-E-L-A-X, watch the numbers and press the button S-L-O-W-L-Y whenever you see the number 4 appear.

Following the BLUE numbers, a series of RED shapes will appear. Your task during this segment is to C-O-N-C-E-N-T-R-A-T-E and press the button as Q-U-I-C-K-L-Y and A-C-C-U-R-A-T-E-L-Y as you can whenever you see a TARGET shape. Remember the target shapes will be shown to you at the beginning of each task. This section of the task will last for about 40 seconds.

After the RED shapes segment, once again, a series of BLUE numbers will be shown to you. As before, just RELAX and press the button SLOWLY whenever you see the

158 Appendix A. Task instructions

number 4 appear. This section will also last for about 40 seconds.

Each task, consisting of BLUE numbers, RED shapes, BLUE numbers will take about 2 minutes to complete.

Before each task the INSTRUCTIONS will be repeated to you. If the instructions are not clear please do not hesitate to ask.

159

Appendix B. Determination of optimum luminance of light-emitting diode (LED) arrays

Appendix B. Determination of optimum luminance of LED arrays

The maximum luminance of the light emitting diode (LED) arrays that produce the probe stimulus was determined in a separate experiment. The voltage driving the LED arrays was varied in a pseudo-random fashion so that the intensity of the flicker ranged from very dim to very bright. A fixation point on the task computer was viewed through the flicker, while the EEG was recorded for 50s at the occipital electrode, Oz. Two subjects were used in this experiment. After every 50s recording period, each subject gave a subjective comfort rating for the flicker: 10 represented ‘very comfortable’ and 1 represented ‘very uncomfortable.’ For each 50s period, the average SSVEP amplitude across subjects was determined off-line and plotted against voltage. As can be seen in Figure B.1), initially the SSVEP amplitude increased almost linearly and then reached a plateau, further increases in voltage having little effect on the SSVEP amplitude. It is important to use a luminance in the plateau region so that the SSVEP amplitude is not affected by small changes in modulation depth. Regan (1972) suggested that the modulation depth should exceed 30%. A voltage of 0.4 volts was chosen as it gives an SSVEP amplitude in the plateau region, a modulation depth of 45%, and was not uncomfortable for either subject.

Determination of optimum luminance of LED arrays

100

10

5 Comfort (0-10) Comfort

10 Comfort SSVEP amplitude Modulation depth = 45% depth Modulation Average SSVEP amplitude (Normalized units) (Normalized SSVEP amplitude Average 0 0.0 0.2 0.4 0.6 0.8 1.0 1.2

Peak-to-peak voltage into LED goggle driver (volt)

Figure B.1 Determination of optimum luminance of LED arrays The peak LED-driver input voltage was determined by assessing the combination of both the SSVEP amplitude response at Oz and the subject comfort rating. A peak-to-peak voltage of 0.4 was selected giving a Modulation Depth of 45% (see also Figure 4.3).

161

Appendix C. Behavioural results for practice tasks

Appendix C: Behavioural results for practice tasks

800 700 600 500 400 300 m1 vs m3: NS 200 m1 vs m5: NS 100 m3 vs m5: NS Mean Response Time (msec) Time Response Mean 0 m1 m3 m5 Memory Load

Figure C.1 Practice tasks: Mean response times for target objects versus memory load for an individual subject Mean response time and standard deviation for target objects for each of the 3 practice task intervals. Data were taken from an individual subject (DK) performing task order 2 (ie. in the order m3, m1, m5). Results of unpaired t-tests for memory load comparisons are also shown. (NS = not significant)

163 Appendix C: Behavioural results for practice tasks

800 700 600 500 400 300 m1 vs m3: t=-3.84; p<0.001 200 m1 vs m5: t=-6.27; p<0.0001 100 m3 vs m5: NS Mean Response Time (msec) Time Response Mean 0

1.0 m1 vs m3: NS 0.8 m1 vs m5: NS m3 vs m5: NS 0.6

0.4

0.2

Mean Frequency of Errors of Omission of Frequency Mean 0.0

1.0 m1 vs m3: NS 0.8 m1 vs m5: t=-2.1; p<0.005 m3 vs m5: NS 0.6

0.4

0.2

0.0 Mean Frequency of Errors of Commission m1 m3 m5 Memory Load

Figure C.2 Practice tasks: Mean response time and errors for target objects versus memory load for 40 subjects Mean response time and standard deviation for target objects for each of the 3 practice task intervals. Similar graphs are shown for errors of omission (missed targets) and errors of commission (responses to non-targets). Results of paired t-tests for memory load comparisons are also displayed. (NS = not significant)

164

Appendix D. The amplitude of the SSVEP produced by turning on the probe stimulus

Appendix D: The amplitude of the SSVEP produced by turning on the probe stimulus

The 13Hz probe flicker was switched-on half way through the baseline practice task to determine whether the driven 13Hz response (stimulus-on) was significantly different from the background (spontaneous) 13Hz activity contained in the EEG. The SSVEP amplitude and phase were calculated for each subject with both a high temporal resolution (870ms) and a low temporal resolution (60s) averaging period. The high temporal resolution analysis was performed so that the transition from stimulus-on to stimulus-off could be visualized for each recording site. The low temporal resolution analysis was performed by averaging all Fourier coefficients during the 60s stimulus-off period and all Fourier coefficients during the 60s stimulus-on period. Hotelling’s T values were used to determine the consistency of the difference between the SSVEP response at 13Hz (stimulus-on) and the spontaneous 13Hz activity (stimulus-off) across all 40 subjects.

The SSVEP amplitude and phase time series for the high temporal resolution averaging period, calculated for an individual subject (BO) are shown in Figure D.1). Following the switching on of the probe stimulus sixty seconds after commencement of the task, the SSVEP amplitude increased and the phase appeared to stabilize. The appearance of the SSVEP remained similar throughout the 60s stimulus on period. The topographic maps in Figure D.2) show the mean SSVEP amplitude for the stimulus-on and stimulus- off conditions, calculated over the whole 60s interval for each. The increase in SSVEP amplitude is most apparent at occipito-parietal sites, although there were also smaller increases at other recording sites.

166 Appendix D: The amplitude of the SSVEP produced by turning on the probe stimulus

6 electrode - Oz 5 4 3 2

(microvolt) 1 0 SSVEP amplitude 4 3 2 1 0 -1 (radian) -2

SSVEP phase -3 -4 on stimulus 0 102030405060708090100110 Time (sec)

Figure D.1 SSVEP amplitude and phase during the stimulus-off and stimulus-on intervals for an individual subject Steady-state visually evoked amplitude and phase changes occurring with the switching-on of the probe stimulus, recorded from the occipital scalp location Oz. Temporal resolution of 870ms. Data taken from individual subject (BO).

stimulus off stimulus on

3.0

0.0 υV

Figure D.2 Mean SSVEP amplitude during the stimulus-off and stimulus-on intervals for an individual subject Mean SSVEP amplitude calculated across the entire stimulus-off (left) and stimulus-on (right) intervals (60s of data) Data taken from individual subject (BO).

167 Appendix D: The amplitude of the SSVEP produced by turning on the probe stimulus

Figure D.3) shows the group averaged time series data at occipital site Oz, calculated with a high temporal resolution (870ms) averaging period. As with the single subject, the SSVEP amplitude increased and the phase stabilized when the probe stimulus was switched-on. The amplitude, averaged over 60s across 40 subjects, for both the stimulus-off and the stimulus-on conditions is shown in Figure D.4). Compared to the stimulus-off condition, the stimulus-on condition shows increased SSVEP amplitude at all sites. The SSVEP amplitude for the stimulus-on condition relative to the stimulus-off condition is shown in the difference map in Figure D.5). The Hotelling’s T topographic map, also shown in Figure D.5), indicates that differences between stimulus conditions were statistically significant across the 40 subjects at all recording sites. The T-values mapped were statistically significant (p<0.001) for a single comparison at all sites. When corrected for multiple comparisons (multiple recording sites), as suggested by Silberstein et al. (1995a), differences at all sites remained statistically significant (p<0.005).

1.0 electrode - Oz 0.8 0.6 0.4 0.2 (normalized) 0.0 SSVEP amplitude 4 3 2 1 0 -1 (radian) -2

SSVEP phase -3 stimulus on stimulus -4 0 102030405060708090100110 Time (sec)

Figure D.3 SSVEP amplitude and phase in response to switching-on probe stimulus averaged across 40 subjects Steady-state visually evoked amplitude and phase changes related to the onset of the probe stimulus, recorded from occipital scalp location Oz, with a temporal resolution of 870ms. Data averaged across the 40 subjects.

168 Appendix D: The amplitude of the SSVEP produced by turning on the probe stimulus

stimulus off stimulus on

0.8

0.0

Figure D.4 Mean SSVEP amplitude topography during the stimulus-off and stimulus-on intervals averaged across 40 subjects Mean SSVEP amplitude calculated across the entire probe stimulus-off interval (60s off data) and probe stimulus-on interval across the 40 subjects.

stimulus on - stimulus off Hotelling’s T

+1.0 12.0

0.001 0.005 0.01 0.05

-1.0 0.0

Figure D.5 SSVEP amplitude difference topography and Hotelling’s T values for stimulus-on relative to stimulus-off conditions averaged across 40 subjects Topography of SSVEP amplitude differences between the mean amplitude during stimulus-on and the mean amplitude during the stimulus-off intervals. The Hotelling’s T values indicate the statistical strength of this difference. N.B. No contours are shown on the Hotelling’s T topographic map as p values for differences at all electrode locations are less than p=0.001

169

Appendix E. Sustained SSVEP effects for each memory load condition relative to the baseline task: practice tasks

Appendix E: Sustained SSVEP effects for each memory load condition relative to the baseline task: practice tasks

baseline

load 1 (practice)

load 3 (practice)

5.0 load 5 +3.1 (practice)

0.0 υV amplitude phase -3.1 radian

Figure E.1 Mean SSVEP amplitude and phase topography for the baseline task and each of the practice memory load tasks for an individual subject Mean SSVEP amplitude and phase topography for each of the practice memory load tasks, calculated across the entire 40s ‘task interval.’ The mean SSVEP amplitude and phase for the baseline task was calculated over the equivalent 40s period. Data taken from individual subject (CL).

171 Appendix E: Sustained SSVEP effects for each memory load condition relative to the baseline task: practice tasks

baseline

load 1 (practice)

load 3 (practice)

1.5 load 5 +0.5 (practice)

0.0 amplitude phase -0.5 radian

Figure E.2 Mean SSVEP amplitude and phase topography for the baseline task and each of the practice memory load tasks averaged across 40 subjects Mean SSVEP amplitude and phase topography for each of the practice memory load tasks calculated across the entire 40s ‘task interval.’ The mean SSVEP amplitude and phase for the baseline task was calculated over the equivalent 40s period across 40 subjects.

172

Appendix F. Retrieval of everyday objects

Appendix F: Retrieval of everyday objects

Introduction Method: data acquisition Episodic memory retrieval was associated with sustained SSVEP amplitude reductions relative to the control task over parieto-temporal, parietal, temporal and frontal regions. The largest reductions were at left parieto-temporal regions (Fig. 4A). Episodic memory retrieval refers to the process of accessing Brain electrical activity was recorded from 64 scalp sites that personally experienced past events (1). were referenced to the nose, with the chin serving as ground. The consistency of these differences across subjects was Brain electrical activity was amplified and bandpass filtered examined using the Student’s t score. The highest scores were Recent electrophysiological and neuroimaging studies suggest (3dB down at 0.1 Hz and 80 Hz) prior to digitization to 16 bit observed at left parieto-temporal sites (Fig. 4B). that episodic retrieval is associated with both state(tonic) and accuracy at a sampling rate of 500 Hz (Fig. 2). transient neural activity (2-4). A continuous 13Hz irrelevant sinusoidal flicker, subtending Tulving (1) suggested that for successful retrieval, an angles of 165º horizontally and 95º vertically was individual must be in a ‘tonically’ maintained cognitive state, superimposed on the task display (Fig. 2). termed retrieval mode. Retrieval Mode and SSVEP amplitude Silberstein et al. (5) demonstrated attenuation of the A. Difference B. Paired t -test amplitude of the steady-state visually evoked potential (SSVEP) during a visual vigilance task. This attenuation is considered to be indicative of increased regional cortical activity and appears akin to the regional reductions in alpha activity associated with cognitive tasks.

In this study we investigate how the amplitude of a SSVEP is affected firstly by the retrieval mode and secondly by the memory load during the retrieval period. R

p=0.05 0.01

-0.5 +0.5 -2.8 +2.8

Fig. 4. A. Topographic map of the SSVEP amplitude of the combined retrieval tasks relative to the control task. Warmer Method: subjects and colours represent a relative SSVEP amplitude attenuation during the retrieval task. B. Topography of Student’s t score. cognitive task Fig. 2. Recording set-up with 13 Hz irrelevant flicker

A negative correlation between SSVEP amplitude and Twenty-eight right-handed male subjects aged from 18 to 30 memory load was found at parieto-temporal sites, particularly years (mean=22.5) participated in the study. in the right hemisphere, indicating that with increasing memory load the SSVEP amplitude at these sites was reduced All subjects performed 5 retrieval tasks and 1 control task (see (Fig. 5). Fig. 1). The order of task presentation was counterbalanced For each of the 64 electrodes, the SSVEP amplitude of the across subjects. Each of the 5 retrieval tasks commenced with response to the irrelevant flicker was extracted from the an encoding interval during which a picture of 1 everyday background EEG using Fourier techniques (5). The average object or sets of 2,3,4 or 5 everyday objects were presented SSVEP amplitude was calculated for the control task and for sequentially. Each object was displayed five times for 2 Effect of memory load on SSVEP amplitude the retrieval periods of each of the 5 retrieval tasks. The seconds. Subjects were instructed to pay careful attention as resulting 6 sets of 64 SSVEP amplitudes were subsequently Pearson’s r recognition of these images would be tested in the ensuing averaged across all subjects. retrieval interval. To investigate the affect of retrieval mode on the SSVEP In the retrieval interval of each task, new and previously 1.15 amplitude, the data from the 5 retrieval tasks were averaged e

d

u

displayed (old) objects were presented for 2 seconds. The t rp=-0.967; =0.007 i 1.10 and compared with the control task data. l probability of an old object being presented was 0.25. Subjects p

m

a 1.05

were required to respond using a forced choice button press, P

To investigate the effect of memory load on the SSVEP E 1.00 yes or no, corresponding to old/new objects respectively. For V amplitude, the 5 separate sets of retrieval data were compared. S

S the control task subjects were merely required to press 0.95 alternately yes and no buttons in response to pictures of 1 2345 Memory load everyday objects similar to those in the retrieval task. p=0.01 0.05

-1.0 +1.0 Fig. 5. Topographic distribution of Pearson’s correlation coefficients. Cooler colours represent negative correlations between SSVEP amplitude during retrieval and memory load.

Increases in memory load were associated with an increase in the mean response time (r=0.92; p<0.01, 1-tailed) (Fig. 3B). However, no relation between memory load and accuracy, determined by the number of correctly identified old objects was apparent (r=-0.109; NS, 1-tailed) (Fig. 3A).

Episodic memory retrieval is associated with sustained SSVEP amplitude reductions, predominantly in left parieto- temporal regions. Increasing memory load is associated with Accuracy and Response Time reductions in the SSVEP amplitude in right parieto-temporal versus Memory Load regions. yes no yes no yes no yes no A B

)

.

9898 c 700700

e )

s .

) 680680 c

m 9696 e

%

(

s ) ( 660660

m %

e (

y ( 640640

c

94 e

m y 94

i

c a

m i

r

T a

t 620620 r

u

u e

e

c

c 92

92 s

s 600600 c

c n

n

A o

A p o 580580 90 s

no noyes no no yes no no 90 p e

s

R 560560 e 1. Tulving (1983). Elements of Episodic Memory. 8888 R 540540 123451 2 3 45 123451 2 345 2. Donaldson., Petersen, S. E., Ollinger, J. M., & Buckner, R. L. (2001). MemoryMemor Loady Load MemoryMemor Loady Load Neuroimage, 13(1), 129-142. Fig. 1. Cognitive task design. Example images shown for a 3. Duzel, E., Cabeza, R., Picton, T. W., Yonelinas, A. P., Scheich, H., Heinze, H. J., & Tulving, E. (1999). Proc Natl Acad Sci U S A, 96(4), memory load of 2 and for the control task. Fig. 3. Behavioural data: Number of correctly identified old 1794-9. objects(accuracy) (A) and time to respond to these objects (B) 4. Rugg, M. D., & Wilding, E. L. (2000). Trends Cogn Sci, 4(3), 108-115. yes 5. Silberstein, R., Ciorciari, J., & Pipingas, A. (1995). versus memory load. Electroencephalography and Clinical Neurophysiology, 96, 24-35.

Figure F.1 Retrieval of everyday objects Poster presented at the Seventh Annual Meeting of the Orgainsation for Human Brain Mapping in Brighton, UK, 10-14 June 2001

174

Publications by the author

Journal Articles

Farrow, M., Silberstein, R.B., Levy, F., Pipingas, A., Wood, K., Hay, D.A. and Jarman, F.C. (1996). “Prefrontal and parietal deficits in ADHD suggested by brain electrical activity mapping during children's performance of the AX CPT.” Educational and Developmental Psychologist (The Australian), 13, (1), 59-68. Harris, P. G., Silberstein, R. B., Nield, G. and Pipingas, A. (2001). “Frontal lobe contributions to perception of rhythmic group structure.” Annals of The New York Academy of Sciences: The Biological Foundations of Music, 930, 414-417. Patterson, J., Owen, C.M., Silberstein, R.B., Simpson, D.G., Pipingas, A. and Nield, G. (1998). “Steady state visual evoked potential (SSVEP) changes in response to olfactory stimulation.” Annals of the New York Academy of Sciences, 855, 625- 7. Silberstein, R. B., Ciorciari, J. and Pipingas, A. (1993). “Rapid changes in steady-state visually evoked potential topography associated with the Wisconsin card sort.” Biological Psychology, 37, 43-71. Silberstein, R. B., Ciorciari, J., and Pipingas, A. (1995). ‘Steady-state visually evoked potential topography during the Wisconsin card sorting test.’ Electroencephalography and Clinical Neurophysiology, 96, (1), 24-35. Silberstein, R.B., Farrow, M., Levy, F., Pipingas, A., Hay, D.A. and Jarman, F.C. (1998). “Functional brain electrical activity mapping in boys with attention- deficit/hyperactivity disorder.” Archives of General Psychiatry, 55, (12), 1105- 1112. Silberstein, R. B., Harris, P. G., Nield, G. E. and Pipingas, A. (2000). “Frontal steady- state potential changes predict long-term recognition memory performance.” International Journal of Psychophysiology, 39, (1), 79-85. Silberstein, R. B., Line, P., Pipingas, A., Copolov, D., and Harris, P. (2000). “Steady- state visually evoked potential topography during the continuous performance

175

task in normal controls and schizophrenia.” Clinical Neurophysiology, 111, (5), 850-7. Silberstein, R. B., Nunez, P. L., Pipingas, A., Harris, P. and Danieli, F. (2001). “Steady state visually evoked potential (SSVEP) topography in a graded working memory task.” International Journal of Psychophysiology, 42, 219-232. Silberstein, R.B., Schier, M.A., Pipingas, A., Ciorciari, J., Wood, S.R. and Simpson, D.G. (1990). “Steady-state visually evoked potential topography associated with a visual vigilance task.” Brain Topography, 3, (2), 337-347. Van Rooy, C., Stough, C., Pipingas, A., Hocking, C. and Silberstein, R. B. (2001). “Spatial working memory and intelligence: Biological correlates.” Intelligence, 29, (4), 275-292. Wallace, J. G., Silberstein, R. B., Bluff, K. and Pipingas, A. (1994). “Semantic transparency, brain monitoring and evaluation of hybrid cognitive architectures.” Connection Science, 6, (1), 43-58. Wheaton, K. J., Pipingas, A., Silberstein, R. B. and Puce, A. (2001). “Human neural responses elicited to observing the actions of others.” Visual Neuroscience, 18, (3), 401-406.

Book Chapters

Harris, P. G., Silberstein, R. B., Pipingas, A. and Pressing, J. (1999). “Perceptual grouping of pitch sequences in the steady-state visually evoked potential (SSVEP) responses of musically trained subjects.” Music, Mind and Science, S. W. Yi, ed., Seoul National University Press, Seoul, Korea, 144-165.

Conference Papers

Buchan, R.J., Nagata, K., Silberstein, R.B., Nield, G., Shinohara, T., Sato, M., Pipingas, A., Simpson, D. and Hirata, Y. (1998). “Steady state probe topography and PET during a japanese continuous performance task.” Brain Topography Today - Proceedings of the III Pan-Pacific Conference on Brain Topography (BTOPPS III), 183-188. Paper presented at the III Pan-Pacific Conference on Brain

176

Topography (BTOPPS III) in Tokyo Bay, Japan, 1-4 April, 1997. Harris, P. G., Silberstein, R. B., Pipingas, A. and Pressing, J. (1998). “Steady-state visually evoked potential (SSVEP) responses to changes of note duration in pitch sequences.” Proceedings of the Fifth International Conference on Music Perception and Cognition, 115-121. Paper presented at the Fifth International Conference on Music Perception and Cognition in Seoul, Korea, 26-30 August, 1998. Nield, G. E., Silberstein, R. B., Pipingas, A., Simpson, D. G. and Burkitt, G. (1998). “Effects of Visual Vigilance Task on Gamma and Alpha Frequence Range Steady-State Potential (SSVEP) Topography.” Brain Topography Today - Proceedings of the III Pan-Pacific Conference on Brain Topography (BTOPPS III), 189-194. Paper presented at the III Pan-Pacific Conference on Brain Topography (BTOPPS III) in Tokyo Bay, Japan, 1-4 April, 1997. Pipingas, A. and Silberstein, R. B. (1995). “SSVEP changes with memory load in a visual recognition task.” Recent Advances in Event-Related Brain Potential Research. Proceedings of the 11th International Conference on Event-Related Potentials (EPIC)., Paper presented at the 11th International Conference on Event-Related Potentials (EPIC). in Okinawa, Japan, 25-30 June 1995. Senova, M., Nagata, K., Buchan, R., Silberstein, R.B., Matsuoka, S., Nield, G., Pipingas, A., Simpson, D.G. and Yaguchi, K. (1998). “Effects of smoking on brain dynamics during the continuous performance task: A pilot study.” Brain Topography Today: Proceedings of the III Pan-Pacific Conference on Brain Topography (BTOPPS III), 822-4. Paper presented at the III Pan-Pacific Conference on Brain Topography (BTOPPS III) in Tokyo Bay, Japan, 1-4 April 1997. Silberstein, R. B., Cadusch, P. J., Nield, G., Pipingas, A. and Simpson, D. G. (1996). “Dynamic changes in the topography of the Steady State Visually Evoked Potential associated with cognition.” Recent Advances in Event-Related Brain Potential Research. Proceedings of the 11th International Conference on Event- Related Potentials (EPIC), Paper presented at the Proceedings of the 11th International Conference on Event-Related Potentials (EPIC) in Okinawa, Japan, 25-30 June, 1995. Silberstein, R. B., Ciorciari, J., Schier, M. A., Pipingas, A. and Wood, S. (1991). “The

177

steady-state visually evoked potential topography and vigilance.” Brain impairment: Advances in applied research, 267-274. Silberstein, R. B., Pipingas, A., Ciorciari, J., Schier, M. A. and Wood, S. R. (1991). “Dynamic changes in brain evoked potential laterality in a visual vigilance task.” Brain Impairment: Advances in applied research., 267-274.

Conference Abstracts

Aranda, G., Silberstein, R. B., Nield, G. and Pipingas, A. (2000). “ Steady state visually evoked potential topography of unfamiliar facial processing.” International Journal of Psychophysiology, 35, (1), 31. Paper presented at the 10th World Congress of the International Organization of Psychophysiology in Sydney Australia, 8-13 February 2000. Balog, O., Silberstein, R. B. and Pipingas, A. (1994). “Steady-state visually evoked potential topography in visual and auditory variants of the continuous performance task (CPT).” Fourth Australasian Psychophysiology Conference Abstracts, Paper presented at the Fourth Australasian Psychophysiology Conference in Melbourne, Australia., December 1994. Buchan, R., Nagata, K., Silberstein, R.B., Nield, G., Shinohara, T., Satoh, M., Pipingas, A., Simpson, D.G. and Hirata, Y. (1998). “Simultaneous steady state probe topography (SSPT) and positron emission tomography (PET) during a Japanese visual vigilance task.” Proceedings of the 3rd Australian Symposium on Functional Brain Mapping, 25. Paper presented at the 3rd Australian Symposium on Functional Brain Mapping in Howard Florey Institute of Experimental Physiology and Medicine, University of Melbourne, Melbourne, Australia, 16-17 April 1998. Buchan, R., Nagata, K., Silberstein, R.B., Nield, G., Shinohara, T., Satoh, M., Pipingas, A., Simpson, D.G. and Hirata, Y. (1997). “Simultaneous Steady State Probe Topography (SSPT) and PET Cerebral Blood Flow Measurements During a Japanese Visual Vigilance Task.” Proceedings of the Seventh Australasian Psychophysiology Conference, Paper presented at the Seventh Australasian Psychophysiology Conference in Swinburne University of Technology, Melbourne, Australia, 6-8 December, 1997.

178

Carter, J.D., Farrow, M., Silberstein, R.B., Tucker, A., Stough, C. and Pipingas, A. (2000). “Functional brain mapping in ADHD and control children during performance of the Stop-Signal Task.” Program of the 10th Annual Conference of the Australasian Society for Psychophysiology (ASP), 18. Paper presented at the 10th Annual Conference of the Australasian Society for Psychophysiology (ASP) in Adelaide, Australia, 3-5 December 2000. Carter, J. D., Farrow, M., Silberstein, R. B., Tucker, A., Stough, C. and Pipingas, A. (2000). “The stop-signal task: a comparison of auditory and visual stop-signals set proportional to mean reaction time.” International Journal of Psychophysiology, 35, 42. Poster presented at the 10th World Congress of the International Organization of Psychophysiology in Sydney, Australia, 8-13 February 2000. Chua, P., Egan, G. F., Morris, P. L. P., Saling, M., Pipingas, A., Berlangieri, S. U., Fitt, G., Schweitzer, I. and Burrows, G. D. (1995). “A Positron Emission Tomography (PET) activation study of an orthographic lexical retrieval task.” The Australian Society for Psychiatric Research Annual Scientific Meeting Abstract, Paper presented at the The Australian Society for Psychiatric Research Annual Scientific Meeting November, 1995. Farrow, M., Silberstein, R.B., Pipingas, A., Hay, D.A., Levy, F. and Jarman, F.C. (1997). “Brain electrical activity mapping during the continuous performance task in attention deficit hyperactivity disorder.” Proceedings of the Seventh Australasian Psychophysiology Conference, Paper presented at the Seventh Australasian Psychophysiology Conference in Melbourne, Australia, 6-8 December, 1997. Farrow, M., Silberstein, R. B., Pipingas, A., Hay, D. A., Levy, F. and Jarman, F. C. (1997). “Brain electrical activity mapping in attention deficit hyperactivity disorder.” Brain Sciences Institute Symposium Program and Abstracts, Paper presented at the Brain Sciences Institute Symposium in Melbourne, Australia, 4- 5 December, 1997. Farrow, M., Silberstein, R. B., Pipingas, A., Levy, F., Jarman, F. C. and Hay, D. A. (1998). “Functional brain electrical activity mapping in attention deficit hyperactivity disorder.” The Faculty of Child and Adolescent Psychiatry, The Royal Australian and New Zealand College of Psychiatrists Abstracts, 11th

179

Annual Conference, Paper presented at the The Faculty of Child and Adolescent Psychiatry, The Royal Australian and New Zealand College of Psychiatrists, 11th Annual Conference in Sydney, Australia, 24 October 1998. Farrow, M. A., Silberstein, R. B., Sergejew, A. A., Hay, D. A., Levy, F., Pipingas, A. and Jarman, F. C. (1996). “High temporal resolution functional brain electrical activity mapping in attention deficit hyperactivity disorder.” European Neuropsychopharmacology, 6, (Suppl. 3), 203. Paper presented at the XXth Collegium Internationale Neuro-psychopharmacologicum Congress in Melbourne, Australia, 23-27 June 1996. Farrow, M. A., Silberstein, R. B., Sergejew, A. A., Hay, D. A., Pipingas, A., Wood, K., Levy, F. and Jarman, F. C. (1995). “Dynamics of brain electrical activity topography in normal and ADHD children.” The Australasian Society for Psychiatric Research - Annual Scientific Meeting Abstracts, 62. Paper presented at the The Australasian Society for Psychiatric Research - Annual Scientific Meeting in Melbourne, Australia, 30 November - 1 December 1995. Harris, P. G., Silberstein, R. B. and Pipingas, A. (1998). “Steady-state visually evoked potential (SSVEP) responses correlate with musically trained subjects' working memory task performance.” Australian Journal of Psychology, 50 (supplement), 90. Paper presented at the 33rd Annual Conference of the Australian Psychological Society in Melbourne, Australia, 30 September - 4 October 1998. Harris, P. G., Silberstein, R. B., Pipingas, A. and Pressing, J. (1997). “Steady-state visually evoked potential responses to changes in the note duration in pitch sequence.” Proceedings of the Seventh Australasian Psychophysiology Conference, Paper presented at the Seventh Australasian Psychophysiology Conference in Swinburne University of Technology, Melbourne, Australia, 6-8 December, 1997. Line, P., Silberstein, R. B. and Pipingas, A. (1992). “Steady-state visually evoked potential topography and mental rotation.” 2nd Australasian Psychophysiology Conference, Paper presented at the 2nd Australasian Psychophysiology Conference in Nelson Bay, Australia, 1992. Owen, C., Patterson, J., Silberstein, R. B., Simpson, D. G., Nield, G. E. and Pipingas, A. (1997). “Respiratory Monitoring and Stimulus Delivery Apparatus for use with Brain Electrical Activity Recording.” Chemical Senses. Proceeding of the

180

International Symposium on Olfaction and Taste., 22, 765. Paper presented at the International Symposium on Olfaction and Taste. in San Diego, California, July, 1997. Pantelis, C., Egan, G., Maruff, P., Pipingas, A., O'Keefe, G., Velakoulis, D., Collinson, S. and Chua, P. (1995). “Practice dependent alterations in activation of the anterior cingulate cortex during the Stroop task: a positron emission tomography study.” Functional Brain Mapping Symposium Abstracts, Paper presented at the Functional Brain Mapping Symposium in Melbourne, Australia, November, 1995. Pantelis, C., Egan, G. F., Maruff, P., Velakoulis, D., Pipingas, A., Tharan, A. S., Tochon-Danguy, H. J. and Stuart, G. (1998). “Functional neuroanatomy of stroop performance in schizophrenia: Inhibition and facilitation with practice.” Proceedings of the 3rd Australian Symposium on Functional Brain Mapping, 21. Poster presented at the 3rd Australian Symposium on Functional Brain Mapping in Howard Florey Institute of Experimental Physiology and Medicine, University of Melbourne, Melbourne, Australia, 16-17 April 1998. Pantelis, C., Egan, G. F., Maruff, P., Velakoulis, D., Pipingas, A., Tochon-Danguy, H. J. and McKay, W. J. (1997). “Functional neuroanatomy of attentional abnormalities in chronic schizophrenia.” Second Functional Brain Mapping Symposium Abstracts., Paper presented at the Second Functional Brain Mapping Symposium in Newcastle, Australia, 2 February, 1997. Pipingas, A. (1991). “Topography variations in the steady-state visually evoked potential during a recognition memory task.” International OSET Congress Abstracts, Paper presented at the International OSET in Melbourne, Australia, 1991. Pipingas, A. and Maruff, P. T. (1991). “Sources of timing errors in IBM PC controlled experiments.” International OSET Congress Abstracts, Paper presented at the International OSET Congress in Melbourne, Australia, 1991. Pipingas, A., Silberstein, R., Van Rooy, C. and Aranda, G. (2001). “State dependent changes in the steady-state visually evoked potential amplitude associated with retrieval of everyday objects.” Neuroimage, 13, (6 (part 2)), S726. Poster presented at the Seventh Annual Meeting of the Orgainsation for Human Brain Mapping in Brighton, UK, 10-14 June 2001.

181

Pipingas, A. and Silberstein, R. B. (1993). “SSVEP in a memory scanning task.” Australasian conference on physical science and engineering in medicine and the biomedical engineering conference abstracts, Paper presented at the Australasian conference on physical science and engineering in medicine and the biomedical engineering conference. in Melbourne, Australia, 1993. Pipingas, A. amd Silberstein, R. B. (1994). “Steady-state probe topography in a memory scanning task.” Pan Pacific Conference on Brain Electric Topography Abstracts, Paper presented at the Pan Pacific Conference on Brain Electric Topography in Sydney, Australia, 1994. Pipingas, A., Silberstein, R. B., Maruff, P., Pantelis, C., Egan, G. F. and Velakoulis, D. (1997). “Topographic variations in the steady state visually evoked potential during the Stroop task.” Second Functional Brain Mapping Symposium Abstracts., Paper presented at the Second Functional Brain Mapping Symposium in Newcastle, Australia, 2 February, 1997. Pipingas, A., Silberstein, R. B. and Nield, G. E. (2000). “Correlation between pre- target 13Hz SSVEP amplitude and response time in a visual vigilance task.” Brain Topography , 12, (4), 314. Poster presented at the 10th World Congress of the International Society for Brain Electromagnetic Topography 1999 (ISBET99) in Adelaide, Australia, 9-13 October, 1999. Pipingas, A., Silberstein, R. B. and Nield, G. E. (1998). “Correlation between response time and the 13Hz SSVEP amplitude preceding target presentation in a visual vigilance task.” 8th Australasian Psychophysiology Conference Abstracts, 40. Paper presented at the The 8th Australasian Psychophysiology Conference and Annual Meeting of the Australasian Society for Psychophysiology in University of Queensland, Brisbane, Australia, 4-6 December 1998. Schier, M. A., Silberstein, R. B., Cadusch, P. J., Pipingas, A. and Wood, S. R. (1990). “Spatial deconvolution of the steady state visually evoked potential topography.” Fourth International Evoked Potential Symposium Abstracts., 161. Paper presented at the Fourth International Evoked Potential Symposium in Toronto, Canada, 1990. Schier, M. A., Silberstein, R. B., Pipingas, A. and Ciorciari, J. (1993). “Steady-state visually evoked potentials during a continuous performance task.” Brain Topography, 5, 447.

182

Silberstein, R. B., Aranda, G., Pipingas, A. and Nield, G. (1998). “Changes in steady state visually evoked potential associated with human face perception.” Australasian Society for Psychiatric Research Program and Abstracts, P12. Poster presented at the Australasian Society for Psychiatric Research in University of Queensland, Brisbane, Australia, 1998. Silberstein, R. B., Aranda, G., Pipingas, A., Nield, G. and Simpson, D. G. (1997). “Dynamic Changes in the Steady-State Visually Evoked Potential Topography Associated with Human Face Perception.” Neuroimage, S98. Poster presented at the Third International Conference on Functional Mapping of the Human Brain. in Copenhagen, Denmark, 19-23 May, 1997. Silberstein, R. B., Burkitt, G. R., Line, P., Nield, G. E. and Pipingas, A. (1995). “Steady-state visually evoked potential topography during the continuous performance task (CPT): performance effects.” Fifth Australasian Psychophysiology Conference Abstracts, Paper presented at the Fifth Australasian Psychophysiology Conference in University of Wollongong, Australia, 1995. Silberstein, R. B., Ciorciari, J., Pipingas, A., Schier, M. A. and Ma, S. (1993). “Effects of the Wisconsin Card Sort Test on the topography of the steady-state visually evoked potential.” Brain Topography, 5, 447. Silberstein, R. B. et al. (1998). Effects of Stimulant Medication on the Steady State Visually Evoked Potential Latency Topography in Attention Deficit Hyperactivity Disorder (ADHD). 3rd Australian Symposium on Functional Brain Mapping Abstracts, 11. Paper presented at the 3rd Australian Symposium on Functional Brain Mapping in University of Melbourne, Melbourne, Australia, 16-17 April , 1998. Silberstein, R. B., Farrow, M. A., Levy, F., Pipingas, A., Jarman, F., and Hay, D. A. (1997). “Steady state visually evoked potential latency topography in attention deficit hyperactivity disorder.” Second Functional Brain Mapping Symposium Abstracts., Paper presented at the Second Functional Brain Mapping Symposium in Newcastle, Australia, 2 February, 1997. Silberstein, R. B., Harris, P. G., Nield, G. E. and Pipingas, A. (2000). “Prefrontal steady-state visually evoked potential (SSVEP) latency changes predict recognition memory performance after 7 days.” International Journal of

183

Psychophysiology, 35, (1), 57. Paper presented at the 10th World Congress of the International Organization of Psychophysiology (IOP) in Sydney, Australia, 8-13 February 2000. Silberstein, R. B., Line, P., Nield, G. and Pipingas, A. (1994). “Steady State visually evoked potential topography changes during the continuous performance task.” Fourth Australasian Psychophysiology Conference Abstracts, Paper presented at the Fourth Australasian Psychophysiology Conference in Melbourne, Australia., December 1994. Silberstein, R. B., Nield, G., Pipingas, A. and Simpson, D. (1998). “Dynamic changes in gamma and alpha frequency range steady state visually evoked potential (SSVEP) in a visual vigilance task.” The XIIth International Conference on Event-Related Potentials of the Brain (epic xii) Abstract Book, S01-02. Paper presented at the The XIIth International Conference on Event-Related Potentials of the Brain (epic xii) in Cambridge, Massachusetts, USA, 19-23 July, 1998. Silberstein, R. B., Nield, G. E., Pipingas, A. and Simpson, D. G. (1997). “Changes in the Alpha and Gamma Frequency Range Steady-State Visually Evoked Potential (SSVEP) Topography during a Visual Vigilance Task.” Proceedings of the Seventh Australasian Psychophysiology Conference, Paper presented at the Seventh Australasian Psychophysiology Conference in Melbourne, Australia, 6- 8 December, 1997. Silberstein, R. B., Pipingas, A., Ciorciari, J., Schier, M. and Ma, S. (1990). “Steady- state visually evoked scalp topography in a visual vigilance task: effects of eye movements.” Second International Congress on Brain Electromagnetic Topography Abstracts, Paper presented at the Second International Congress on Brain Electromagnetic Topography in Toronto, Canada, 1990. Silberstein, R. B., Pipingas, A., Copolov, D., Line, P. and Harris, P. (2000). “Steady state visually evoked potential topography during the continuous performance task in normal controls and schizophrenia.” International Journal of Psychophysiology, 35, (1), 21. Paper presented at the 10th World Congress of the International Organization of Psychophysiology (IOP) in Sydney, Australia, 8-13 February 2000. Silberstein, R. B., Pipingas, A., Harris, P. G., Nield, G., Saling, M., and O'Sullivan, B. O. (1999). “Steady state visually evoked potential topography in a spatial

184

working memory task.” Society for Neuroscience Abstracts, 1, (25), 1142. Poster presented at the Society for Neuroscience in Miami Beach, Florida, 23-28 October, 1999. Silberstein, R. B., Pipingas, A., Saling, M. and O'Sullivan, B. (1998). “Steady state visually evoked potential topography in a spatial working memory task.” The 8th Australasian Psychophysiology Conference Abstracts, 43. Paper presented at the 8th Australasian Psychophysiology Conference and Annual Meeting of the Australasian Society for Psychophysiology in University of Queensland, Brisbane, Australia, 4-6 December, 1998. Silberstein, R. B., Robb, D., Stanley, R., Burrows, G. and Pipingas, A. (1998). “Word emotional valence influences steady state visually evoked potential (SSVEP) amplitude and latency topography.” Brain Topography, 11, 71-2. Paper presented at the The 9th World Congress of the International Society for Brain Electromagnetic Topography 6-9 October, 1998. Silberstein, R. B., Schier, M. A., Pipingas, A. and Ciorciari, J. (1991). “Topography of phase and amplitude variations in the steady state visually evoked potential during a vigilance task.” Abstracts of the Second International Congress on Brain Electromagnetic Topography, Paper presented at the Second International Congress on Brain Electromagnetic Topography in Toronto, Canada, 1991. Silberstein, R. B., Schier, M. A., Pipingas, A., Ciorciari, J. and Simpson, D. G. (1990). “Steady state probe topography, a new technique for measuring brain electrical activity.” Proceedings of the 1990 Australian Society for Psychiatric Research Scientific Meeting, Paper presented at the Australian Society for Psychiatric Research Scientific Meeting in Melbourne, Australia, 1991. Silberstein, R. B., Schier, M. A., Pipingas, A., Ciorciari, J., Wood, S. R. and Cadusch, P. J. (1990). “Topography of the steady state visually evoked potential associated with visual vigilance.” Fourth International Evoked Potential Symposium Abstracts., 59. Paper presented at the Fourth International Evoked Potential Symposium in Toronto, Canada, 1990. Silberstein, R. B., Stough, C., Pipingas, A., Dennison, S. and Celi, E. (1996). “The relationship between steady-state evoked potentials (SSVEPs) and cognitive processes.” Australian Journal of Psychology, 48, 141. Paper presented at the 31st Annual Conference of the Australian Psychological Society September

185

1996. Silberstein, R. B., Stough, C., Pipingas, A., Line, P., Celi, E. and Dennison, S. (1996). “Steady state visually evoked potential topography in a mental rotation task: Performance effects.” Paper presented at the Australian Psychophysiology Conference December 1996. Silberstein, R. B., Stough, C. K., Pipingas, A., Line, P., Celi, E. and Dennison, S. (1997). “Dynamics of Steady-State Visually Evoked Potential Topography Latency in a Mental Rotation Task: Performance Effects.” Brain Topography, 10, (1), 50. Paper presented at the 8th World Congress of the International Society for Brain Electromagnetic Topography (ISBET) The Key Foundation Symposium in Zurich, Switzerland, 6-8 March, 1997. Silberstein, R. B., Wallace, I. G., Pipingas, A. and Bluff, K. (1993). “Steady-state visually evoked potential topography in a counting task.” Third Australasian Psychophysiology Conference Abstracts, Paper presented at the Third Australasian Psychophysiology Conference Silberstein, R. B., Wallace, J. G., Pipingas, A. and Bluff, K. (1992). “Steady-State visually evoked topography and task automatization.” The Third International Congress on Brain Electromagnetic Topography Abstracts, Poster presented at the Third International Congress on Brain Electromagnetic Topography in Amsterdam, Netherlands, 9-12 June, 1992. Stough, C., Silberstein, R. B. S., Pipingas, A., Dennison, S., Celi, E. and Gillespie, N. (1997). “The relationship between SSVEP and psychometric intelligence.” Program and Abstracts of the Biennial Conference of the International Society for the Study of Individual Differences , Poster presented at the Biennial Conference of the International Society for the Study of Individual Differences in Aarhus, Denmark, July 1997. Stough, C. K., Silberstein, R. B., Celi, E., Pipingas, A. and Dennison, S. (1997). “The relationship between steady-state visual evoked potentials (SSVEPs) and intelligence.” Fifth European Congress of Psychology Abstracts, 32. Paper presented at the Fifth European Congress of Psychology in Dublin, Ireland, 6-11 July, 1997. Thompson, J., Tzambazis, K., Stough, C., Pipingas, A. and Silberstein, R. B. (2000). “Changes in prefrontal steady state visually evoked potential in a visual

186

vigilance task: effects of nicotine.” International Journal of Psychophysiology, 35, (1), 33-34. Paper presented at the 10th World Congress of the International Organization of Psychophysiology (IOP) in Sydney, Australia, 8-13 February 2000. Thompson, J. C., Tzambazis, K., Stough, C., Pipingas, A., Silberstein, R. B. and Nathan, P. J. (1999). “Changes to prefrontal electrophysiological activity during enhanced attention following nicotine.” European Neuropsychopharmacology, 9, (Suppl. 5), S356 (P.6.029). Poster presented at the European College of Neuropsychopharmacology (ECNP) in London, UK, 21-25 Septemer, 1999. Thompson, J. C., Tzambazis, K., Stough, C., Pipingas, A., Silberstein, R. B. and Nathan, P. J. (1998). “Changes to prefrontal electrophysiological activity during attention following nicotine.” The Australasian Society of Psychiatric Research (ASPR) Conference Abstracts, Paper presented at the Australasian Society of Psychiatric Research (ASPR) Conference in Brisbane, 1998. Van Rooy, C., Stough, C. K., Pipingas, A. and Silberstein, R. B. (1997). “Spatial Working Memory and Intelligence.” Proceedings of the Seventh Australasian Psychophysiology Conference, Paper presented at the Seventh Australasian Psychophysiology Conference in Melbourne, Australia, 6-8 December, 1997. Wallace, J. G., Silberstein, R. B., Bluff, K. and Pipingas, A. (1993). “Learning in a hybrid cognitive architecture: a brain monitoring study.” World Conference on Artificial Intelligence in Education, Paper presented at the World Conference on Artificial Intelligence in Education in Edinburgh, Scotland., 1993. Wallace, J. G., Silberstein, R. B., Bluff, K. and Pipingas, A. (1992). “Semantic tranparency, brain monitoring and the definition of hybrid systems.” International Conference on Artificial Neural Networks, Paper presented at the International Conference on Artificial Neural Networks 1992. Wallace, J. G., Silberstein, R. B., Bluff, K. and Pipingas, A. (1992). “Semantic transparency, brain monitoring and the integration of neural and symbolic processes.” Tenth National Conference on Artificial Intelligence, Paper presented at the Tenth National Conference on Artificial Intelligence in San Hose., July, 1992. Wheaton, K. J., Pipingas, A., Silberstein, R. B. and Puce, A. (2000). “ERPs elicited to

187

observing the actions of others.” International Journal of Psychophysiology, 35, (1), 59-60. Paper presented at the 10th World Congress of the International Organization of Psychophysiology (IOP) in Sydney, Australia, 8-13 February 2000.

188

Bibliography

Abt, K. (1983). “Significance testing of many variables - problems and solutions.” Neuropsychobiology, 9, 47-51. Acton, P. D., and Friston, K. J. (1998). “Statistical parametric mapping in functional neuroimaging: beyond PET and fMRI activation studies.” European Journal of Nuclear Medicine, 25(7), 663-667. Adrian, E. D., and Matthews, B. H. C. (1934). “The berger thythm: Potential changes from the occipital lobes in man.” Brain, 57, 355-385. Appleton, J. P., and Brown, M. W. (1999). “Episodic memory, amnesia and the hippocampal anterior thalamic axis.” Behavioural Brain Sciences, 22, 452-489. Arnolds, D. E., Lopes da Silva, F. H., Aitink, J. W., Kamp, A., and Boeijinga, P. (1980). “The spectral properties of hippocampal EEG related to behaviour in man.” Electroencephalography and Clinical Neurophysioliology, 50(3-4), 324-8. Baddeley, A. D. (1999). Essentials of human memory, Psychology Press, U.K. Berger, H. (1929). “Uber das elektrenkephalogramm des menschen.” Arch Psychiat Nervenkr 1929;87:527-570. English translation by Gloor P. Hans Berger on the electroencephalogram of man. Electroencephalography and Clinical Neurophysiology 1969;(Suppl. 28):37-73. Besson, M., Kutas, M., and Van Petten, C. (1992). “An event-related potential (ERP) analysis of semantic congruity and repetition effects in sentences.” Journal of Cognitive Neuroscience, 4, 132-149. Buckner, R. L., Koutstaal, W., Schacter, D. L., Dale, A. M., Rotte, M., and Rosen, B. R. (1998). “Functional-anatomic study of episodic retrieval. II. Selective averaging of event-related fMRI trials to test the retrieval success hypothesis.” Neuroimage, 7(3), 163-175. Buckner, R. L., and Tulving, E. (1995). “Neuroimaging studies of memory: theory and recent PET results.” Handbook of Neuropsychology, J. C. Baron, ed., Elsevier, New York, 439-466. Buffalo, E. A., Reber, P. J., and Squire, L. R. (1998). “The human perirhinal cortex and recognition memory.” Hippocampus, 8(4), 330-339.

189

Burkitt, G. (1996). “Steady-state visual evoked responses and travelling waves,” PhD thesis, Swinburne University of Technology, Melbourne. Cabeza, R. (1999). “Functional neuroimaging of episodic memory retrieval.” Memory, consciousness and the brain: the Tallinn conference, E. Tulving, ed., Psychology Press, Philadelphia, 76-90. Cabeza, R., Grady, C. L., Nyberg, L., McIntosh, A. R., Tulving, E., Kapur, S., Jennings, J. M., Houle, S., and Craik, F. I. (1997a). “Age-related differences in neural activity during memory encoding and retrieval: a positron emission tomography study.” Journal of Neuroscience, 17(1), 391-400. Cabeza, R., Kapur, S., Craik , F., and McIntosh, A. (1997b). “Functional neuroanatomy of recall and recognition: A PET study of episodic memory.” Journal of Cognitive Neuroscience, 9(2), 254-265. Cabeza, R., and Nyberg, L. (2000). “Imaging cognition II: An empirical review of 275 PET and fMRI studies.” Journal of Cognitive Neuroscience, 12(1), 1-47. Cadusch, P. J., Breckon, W., and Silberstein, R. B. (1992). “Spherical splines and the interpolation, deblurring and transformation of topographic EEG data.” Brain Topography, 5. Ciorciari, J., Silberstein, R. B., Simpson, D. G., and Schier, M. S. “The multichannel electrode helmet.” Conference on Engineering and Physical Sciences in Medicine, Melbourne, Australia. Courtney, S. M., Ungerleider, L. G., Keil, K., and Haxby, J. V. (1996). “Object and spatial visual working memory activate separate neural systems in human cortex.” Cerebral Cortex, 6(1), 39-49. Dawson, G. D. (1947). “Cerebral responses to electrical stimulation of peripheral nerve in man.” Journal of Neurology, Neurosurgery, and Psychiatry, 10, 134-140. Desimone, R., Albright, T. D., Gross, C. G., and Bruce, C. (1984). “Stimulus-selective properties of inferior temporal neurons in the macaque.” Journal of Neuroscience, 4(8), 2051-2062. Desimone, R., and Duncan, J. (1995). “Neural mechanisms of selective visual attention.” Annual Review Neuroscience, 18, 193-222. Donaldson, D. I., and Rugg, M. D. (1998). “Recognition memory for new associations: electrophysiological evidence for the role of recollection.” Neuropsychologia, 36(5), 377-395.

190

Donchin, E., Kutas, M., and McCarthy, G. (1977). “Electrocortical indices of hemispheric utilization.” Lateralization in the nervous system, S. Harnad, R. W. Doty, L. Godstein, J. Jaynes, and G. Krauthamer, eds., Academic Press, New York, 339-384. Doppelmayr, M., Klimesch, W., Schwaiger, J., Auinger, P., and Winkler, T. (1998). “Theta synchronization in the human EEG and episodic retrieval.” Neuroscience Letters, 257(1), 41-44. Doppelmayr, M., Klimesch, W., Schwaiger, J., Stadler, W., and Rohm, D. (2000). “The time locked theta response reflects interindividual differences in human memory performance.” Neuroscience Letters, 278(3), 141-144. Duffy, F. H., Bartels, P. H., and Burchfiel, J. L. (1981). “Significance probability mapping: An aid in the topographic analysis of brain electrical activity.” Electroencephalography and Clinical Neurophysiology, 51, 455-462. Duffy, F. H., and McAnulty, G. (1988). “Electrophysiological studies.” Geriatric neuropsychology, M. S. Albert and M. B. Moss, eds., The Guildford Press, New York, 262-289. Dujardin, K., Derambure, P., Bourriez, J. L., Jacquesson, J. M., and Guieu, J. M. (1993). “P300 component of the event-related potentials (ERP) during an attention task: effects of age, stimulus modality and event probability.” International Journal of Psychophysiology, 14, 255-267. Duzel, E., Cabeza, R., Picton, T. W., Yonelinas, A. P., Scheich, H., Heinze, H. J., and Tulving, E. (1999). “Task-related and item-related brain processes of memory retrieval.” Proceedings National Academy Sciences U S A, 96(4), 1794-1749. Duzel, E., Picton, T. W., Cabeza, R., Yonelinas, A. P., Scheich, H., Heinze, H. J., and Tulving, E. (2001). “Comparative electrophysiological and hemodynamic measures of neural activation during memory-retrieval.” Human Brain Mapping, 13(2), 104-123. Duzel, E., Yonelinas, A. P., Mangun, G. R., Heinze, H. J., and Tulving, E. (1997). “Event-related brain potential correlates of two states of conscious awareness in memory.” Proceedings National Academy Sciences U S A, 94(11), 5973-5978. Ebbinghaus, H. (1964). Memory, Reprinted by Dover (originally published in 1885), New York. Fabiani, M., Karis, D., and Donchin, E. (1990). “Effects of mnemonic strategy

191

manipulation in a Von Restorff paradigm.” Electroencephalography and Clinical Neurophysioliogy, 75(2), 22-35. Farrow, M., Silberstein, R. B., Levy, F., Pipingas, A., Wood, K., Hay, D. A., and Jarman, F. J. (1996). “Prefrontal and parietal deficits in ADHD suggested by brain electrical activity mapping during children's performance of the.” Educational and Developmental Psychologist (The Australian), 13(1), 59-68. Felleman, D. J., and Van Essen, D. C. (1991). “Distributed hierarchical processing in the primate cerebral cortex.” Cerebral Cortex, 1(1), 1-47. Ferree, T. C., Luu, P., Russell, G. S., and Tucker, D. M. (2001). “Scalp electrode impedance, infection risk, and EEG data quality.” Clinical Neurophysiology, 112(3), 536-544. Friedman, D. (1990a). “Cognitive event-related potential components during continuous recognition memory for pictures.” Psychophysiology, 27(2), 136-148. Friedman, D. (1990b). “ERPs during continuous recognition memory for words.” Biological Psychology, 30(1), 61-87. Fujita, I., Tanaka, K., Ito, M., and Cheng, K. (1992). “Columns for visual features of objects in monkey inferotemporal cortex [see comments].” Nature, 360(6402), 343-346. Fuster, J. M., and Jervey, J. P. (1982). “Neuronal firing in the inferotemporal cortex of the monkey in a visual memory task.” Journal of Neuroscience, 2(3), 361-375. Galambos, R., and Makeig, S. “Studies on the auditory high-rates responses: Methods and major findings.” Abstract in Proceedings of International conference on dynamics of sensory and cognitive processing in the brain. Galambos, R., and Makeig, S. (1988). “Dynamic changes in steady-state responses.” Springer Series in Brain Dynamics 1, E. Basar, ed., Springer-Verlag, Berlin, 103-122. Galin, D., and Ellis, R. (1975). “Asymmetry in evoked potentials as an index of lateralized cognitive processes: Relation to EEG alpha asymmetry.” Neuropsychologia, 13, 45-50. Gardiner, J. M. (1988). “Functional aspects of recollective experience.” Memory Cognition, 16, 309-313. Gawne, T. J., and Richmond, B. J. (1993). “How independent are the messages carried by adjacent inferior temporal cortical neurons?” Journal of Neuroscience, 13(7),

192

2758-2771. Gloor, P. (1994). “Berger lecture. Is Berger's dream coming true?” Electroencephalography and Clinical Neurophysiology, 90, 253-266. Guillem, F., N'Kaoua, B., Rougier, A., and Claverie, B. (1995a). “Effects of temporal versus temporal plus extra-temporal lobe epilepsies on hippocampal ERPs: physiopathological implications for recognition memory studies in humans.” Brain Research Cognitive Brain Research, 2(3), 147-153. Guillem, F., N'Kaoua, B., Rougier, A., and Claverie, B. (1995b). “Intracranial topography of event-related potentials (N400/P600) elicited during a continuous recognition memory task.” Psychophysiology, 32(4), 382-392. Guillem, F., N'Kaoua, B., Rougier, A., and Claverie, B. (1996). “Differential involvement of the human temporal lobe structures in short- and long-term memory processes assessed by intracranial ERPs.” Psychophysiology, 33(6), 720-730. Halgren, E., Clarke, J. M., and Herve, N. (1992). “Roles des enregistrements profonds dans la localisation, l'agencement sequential, et la caracterisation des etapes de traitement de l'information dans le cerveau human.” Psychologie Francaise, 37, 149-166. Halgren, E., and Smith, M. E. (1987). “Cognitive evoked potentials as modulatory processes in human memory formation and retrieval.” Human Neurobiology, 6(2), 129-139. Halgren, E., Squires, N. K., Wilson, C. L., Rohrbaugh, J. W., Babb, T. L., and Crandall, P. H. (1980). “Endogenous potentials generated in the human hippocampal formation and amygdala by infrequent events.” Science, 210(4471), 803-5. Halliday. (1977). “Clinical applications of transient, flash visually evoked potentials.” Visual evoked potentials in man: New developments, J. E. Desmedt, ed., Clarendon Press, Oxford. Halliday, A. M., McDonald, W. I., and Mushin, J. (1972). “Delayed visual evoked response in optic neuritis.” Lancet, 1, 982-985. Harris, P. G., Silberstein, R. B., Nield, G. E., and Pipingas, A. (2001). “Frontal lobe contributions to perception of rhythmic group structure. An EEG investigation.” Annals of the New York Academy of Sciences, 930, 414-417. Hassainia, F., Petit, D., and Montplaisir, J. (1994). “Significance probability mapping:

193

the final touch in t-statistic mapping.” Brain Topography, 7(1), 3-8. Haxby, J. V., Grady, C. L., Horwitz, B., Ungerleider, L. G., Mishkin, M., Carson, R. E., Herscovitch, P., Schapiro, M. B., and Rapoport, S. I. (1991). “Dissociation of object and spatial visual processing pathways in human extrastriate cortex.” Proceedings National Academy Sciences U S A, 88(5), 1621-1625. Heit, G., Smith, M. E., and Halgren, E. (1990). “Neuronal activity in the human medial temporal lobe during recognition memory.” Brain, 113, 1093-1112. Horel, J. A., Pytko-Joiner, D. E., Voytko, M. L., and Salsbury, K. (1987). “The performance of visual tasks while segments of the inferotemporal cortex are suppressed by cold.” Behavioural Brain Research, 23(1), 29-42. Ibatoullina, A. A., Vardaris, R. M., and Thompson, L. (1994). “Genetic and environmental influences on the coherence of background and orienting response EEG in children.” Intelligence, 19, 65-78. Iragui, V. J., Kutas, M., Mitchiner, M. R., and Hillyard, S. A. (1993). “Effects of aging on event-related brain potentials and reaction times in an auditory oddball task.” Psychophysiology, 30, 10-22. Iwai, E. (1978). “The visual learning area in the inferotemporal cortex of monkeys.” Integrative control functions of the brain, M. Ito, ed., Kodansha, Tokyo, 419- 427. Iwai, E. (1981). “Visual mechanisms in the temporal and prestriate association cortices of the monkey.” Advances Physiological Sciences, 17, 279-286. Iwai, E. (1985). “Neuropsychological basis of pattern vision in macaque monkeys.” Vision Research, 25(3), 425-439. Iwai, E., and Mishkin, M. (1969). “Further evidence on the locus of the visual area in the temporal lobe of the monkey.” Exp Neurol, 25(4), 585-94. Jacoby, L. L. (1991). “A process dissociation framework: separating automatic from intentional uses of memory.” Journal Memory Language, 30, 513-541. Johnson, M. K., Hashtroudi, S., and Lindsay, D. S. (1993). “Source monitoring [Review].” Psychology Bulletin, 114, 3-28. Johnson, R. (1995). “Event-related potential insights into the neurobiology of memory systems.” Handbook of Neuropsychology, J. C. Baron, ed., Elsevier, New York, 135-163. Johnson, R., Jr., Pfefferbaum, A., and Kopell, B. S. (1985). “P300 and long-term

194

memory: latency predicts recognition performance.” Psychophysiology, 22(5), 497-507. Johnstone, J., Galin, D., Fein, C., Yingling, C., Herron, J., and Marcus, M. (1984). “Regional brain activity in dyslexic and control children during reading tasks: visual probe event-related potentials.” Brain and Language, 21, 233-254. Kandel, E. R., Schwartz, J. H., and Jessell, T. M. (2000). Principles of neural science, McGraw-Hill, New York. Kinsbourne, M., and Hiscock, M. (1983). “The normal and deviant development of functional lateralization of the brain.” Carmichael's manual of child psychology, P. H. Mussen, ed., John Wiley, New York, 157-208. Klimesch, W. (1996). “Memory processes, brain oscillations and EEG synchronization.” International Journal Psychophysiology, 24(1-2), 61-100. Klimesch, W., Doppelmayr, M., Schimke, H., and Ripper, B. (1997). “Theta synchronization and alpha desynchronization in a memory task.” Psychophysiology, 34(2), 169-176. Klimesch, W., Doppelmayr, M., Schwaiger, J., Winkler, T., and Gruber, W. (2000). “Theta oscillations and the ERP old/new effect: independent phenomena?” Clinical Neurophysiology, 111(5), 781-793. Klimesch, W., Doppelmayr, M., Stadler, W., Pollhuber, D., Sauseng, P., and Rohm, D. (2001a). “Episodic retrieval is reflected by a process specific increase in human electroencephalographic theta activity.” Neuroscience Letters, 302(1), 49-52. Klimesch, W., Doppelmayr, M., Yonelinas, A., Kroll, N. E., Lazzara, M., Rohm, D., and Gruber, W. (2001b). “Theta synchronization during episodic retrieval: neural correlates of conscious awareness.” Brain Research Cognitive Brain Research, 12(1), 33-38. Klimesch, W., Pfurtscheller, G., Mohl, W., and Schimke, H. (1990a). “Event-related desynchronization, ERD-mapping and hemispheric differences for words and numbers.” International Journal Psychophysiology, 8(3), 297-308. Klimesch, W., Pfurtscheller, G., and Schimke, H. (1992). “Pre- and post stimulus processes in category judgement tasks as measured by event related desynchronization (ERD).” Journal Psychophysiology, 6, 188-203. Klimesch, W., Schimke, H., Ladurner, G., and Pfurtscheller, G. (1990b). “Alpha frequency and memory performance.” Journal of Psychophysiology, 4, 381-390.

195

Klimesch, W., Schimke, H., and Pfurtscheller, G. (1993). “Alpha frequency, cognitive load and memory performance.” Brain Topography, 5(3), 241-251. Klimesch, W., Schimke, H., and Schwaiger, J. (1994). “Episodic and semantic memory: an analysis in the EEG theta and alpha band.” Electroencephalography Clinical Neurophysiology, 91(6), 428-441. Kohler, S., Kapur, S., Moscovitch, M., Winocur, G., and Houle, S. (1995). “Dissociation of pathways for object and spatial vision: a PET study in humans.” Neuroreport, 6(14), 1865-1868. Kolb, B., and Whishaw, I. (1996). “The occipital lobes.” Fundamentals of Neuropsychology, W.H. Freeman and Co., New York, 243-264. Krause, C. M., Lang, A. H., Laine, M., Kuusisto, M., and Porn, B. (1996). “Event- related desynchronization and synchronization during an auditory memory task.” Electroencephalography and Clinical Neurophysiology, 98, 319-326. Kroll, N. E. A., and Klimesch, W. (1992). “Semantic memory. Complexity and connectivity?” Mem.Cogn., 20, 192-210. Kutas, M., and Dale, A. (1997). “Electrical and magnetic readings of mental functions.” Cognitive Neuroscience, M. D. Rugg, ed., Psychology Press, London, 169-242. Kutas, M., and Hillyard, S. A. (1980). “Reading senseless sentences: brain potentials reflect semantic incongruity.” Science, 207(4427), 203-205. Leonard, B. W., Amaral, D. G., Squire, L. R., and Zola-Morgan, S. (1995). “Transient memory impairment in monkeys with bilateral lesions of the entorhinal cortex.” Journal of Neuroscience, 15(8), 5637-5659. Linden, R. D., Picton, T. W., Hamel, G., and Campbell, K. B. (1987). “Human auditory evoked potentials during selective attention.” Electroencephalography and Clinical Neurophysiology, 66, 145-159. Logothetis, N. K., and Sheinberg, D. L. (1996). “Visual Object Recognition.” Annual Review Neuroscience, 19, 577-621. Mandler, G. (1981). “Recognizing: the judgment of prior occurrence.” Psychology Review, 87, 252-271. Martinson. (1939). “A study of brain potentials during mental blocking.” Journal of Experimental Psychology, 21(1), 101-105. McEwan, J. A., and Anderson, G. B. (1975). “Modeling the stationarity and gaussianity of spontaneous electroencephalographic activity.” IEEE Transactions and

196

Biomedical Engineering, 22(5), 361-369. Menon, R. S., and Kim, S. G. (1999). “Spatial and temporal limits in cognitive neuroimaging with fMRI.” Trends in Cognitive Sciences, 3, 207-215. Meunier, M., Bachevalier, J., Mishkin, M., and Murray, E. A. (1993). “Effects on visual recognition of combined and separate ablations of the entorhinal and perirhinal cortex in rhesus monkeys.” Journal of Neuroscience, 13(12), 5418-5432. Milner, B. (1958). “Psychological defects produced by temporal lobe excision.” Recent publications of the association for research in nervous and mental disease, 38, 244-257. Milner, B. (1963). “Effects of different brain lesions on card sorting.” Archives of Neurology, 9, 90-100. Milner, B. (1964). “Some effects of frontal lobectomy in man.” The frontal granular cortex and behaviour, J. M. Warren and K. Akert, eds., McGraw-Hill, New York, 313-334. Milner, B. (1968). “Visual recognition and recall after right temporal lobe excision in man.” Neuropsychologia, 6, 191-209. Milner, B. (1970). “Memory and the medial temporal regions of the brain.” Biological Bases of Memory, K. H. Pribham and D. E. Broadbent, eds., Academic Press, New York. Mishkin, M. (1982). “A memory system in the monkey.” Philos Trans R Soc Lond B Biol Sci, 298(1089), 83-95. Moscovitch, C., Kapur, S., Kohler, S., and Houle, S. (1995). “Distinct neural correlates of visual long-term memory for spatial location and object identity: a positron emission tomography study in humans.” Proceedings National Academy Sciences U S A, 92(9), 3721-3725. Moscovitch, M. (1992). “Memory and working-with-memory: a component process model based on modules and central systems.” Journal Cognitive Neuroscience, 4, 257-267. Moscovitch, M. (1994). “Models of consciousness and memory.” The cognitive neurosciences, M. S. Gazzaniga, ed., MIT Press, Cambridge, Mass. Nagy, M. E., and Rugg, M. D. (1989). “Modulation of event-related potentials by word repetition: the effects of inter-item lag.” Psychophysiology, 26(4), 431-436. Neville, H. J., Kutas, M., Chesney, G., and Schmidt, A. L. (1986). “Event-related brain

197

potentials during initial encoding and recognition memory of congruous and incongruous words.” Journal of Memory and Language, 25, 75-92. Nield, G. E., Silberstein, R. B., Pipingas, A., Simpson, D. G., and Burkitt, G. R. (1998). “The effects of a visual vigilance task on gamma and alpha frequency range steady-state visually evoked potential (SSVEP) topography.” Brain Topography Today, Y. Koga and K. Nagata, eds., Elsevier Science, Amsterdam, 189-194. Nolde, S. F., Johnson, M. K., and Raye, C. L. (1998). “The role of prefrontal cortex during tests of episodic memory.” Trends in cognitive sciences, 2(10), 399-406. Noldy-Cullum, N. E., and Stelmack, R. M. (1987). “Recognition memory for pictures and words: The effect of incidental and intentional learning on N400.” Current research in event-related potentials. Electroencephalography and Clinical Neurophysiology, Supplement 40, R. Johnson, J. W. Rohrbaugh, and R. Parasuraman, eds., Elsevier, Amsterdam, 350-354. Nunez, P. L. (1981). Electric fileds of the brain: the neurophysics of EEG, Oxford University Press, New York. Nyberg, L., Cabeza, R., and Tulving, E. (1996a). “PET studies of encoding and retrieval: The HERA model.” Psychonomic Bulletin & Review, 3(2), 135-148. Nyberg, L., Cabeza, R., and Tulving, E. (1998a). “Asymmetric frontal activation during episodic memory: what kind of specificity?” Trends in Cognitive Sciences, 2(11), 419-420. Nyberg, L., McIntosh, A. R., Houle, S., Nilsson, L. G., and Tulving, E. (1996b). “Activation of medial temporal structures during episodic memory retrieval [see comments].” Nature, 380(6576), 715-717. Nyberg, L., McIntosh, A. R., and Tulving, E. (1998b). “Functional brain imaging of episodic and semantic memory with positron emission tomography.” J Mol Med, 76(1), 48-53. Oldfield, R. C. (1971). “The assessment and analysis of handedness: The Edinburgh Inventory.” Neuropsychologia, 9, 97-113. Osaka, M. (1984). “Peak alpha frequency of EEG during a mental task: task difficulty and hemispheric differences.” Psychophysiology, 21(1), 101-105. Paller, K. A., Bozic, V. S., Ranganath, C., Grabowecky, M., and Yamada, S. (1999). “Brain waves following remembered faces index conscious recollection.” Brain Research Cognitive Brain Research, 7(4), 519-531.

198

Paller, K. A., Gonsalves, B., Grabowecky, M., Bozic, V. S., and Yamada, S. (2000). “Electrophysiological correlates of recollecting faces of known and unknown individuals.” Neuroimage, 11(2), 98-110. Paller, K. A., and Kutas, M. (1992). “Brain potentials during retrieval provide neurophysiological support for the distinction between consious recollection and priming.” Journal Cognitive Neuroscience, 4, 375-391. Papanicolaou, A. C., Deutsch, G., Bourbon, W. T., Will, K. W., Loring, D. W., and Eisenberg, H. M. (1987). “Convergent evoked potential and cerebral blood flow evidence of task-specific hemispheric differences.” Electroencephalography and Clinical Neurophysiology, 66, 515-520. Papanicolaou, A. C., Eisenberg, H. M., and Levy, R. S. (1983). “Evoked potential correlates of left hemisphere dominance in covert articulation.” International Journal of Neuroscience, 20, 289-294. Papanicolaou, A. C., and Johnstone, J. (1984). “Probe evoked potentials: theory, method, and applications.” International Journal of Neuroscience, 24, 107-131. Parker, A., and Gaffan, D. (1998). “Interaction of frontal and perirhinal cortices in visual object recognition memory in monkeys [In Process Citation].” European Journal of Neuroscience, 10(10), 3044-3057. Petri, H. L., and Mishkin, M. (1994). “Behaviorism, cognitivism and the neuropsychology of memory.” American Scientist, 82, 30-37. Pfurtscheller, G., and Aranibar, A. (1977a). “Event-related cortical desynchronization detected by power measurements of scalp EEG.” Electroencephalography and Clinical Neurophysiology, 42, 138-146. Pfurtscheller, G., and Aranibar, A. (1977b). “Event-related cortical desynchronization detected by power measurements of scalp EEG.” Electroencephalography and Clinical Neurophysiology, 42(6), 817-826. Pfurtscheller, G., and Klimesch, W. (1990). “Topographic display and interpretation of event-related desynchronization during a visual-verbal task.” Brain Topography, 3, 85-93. Pfurtscheller, G., and Klimesch, W. (1991). “Event-related desynchronization during motor behaviour and visual information procesing.” Event-related brain research, C. H. M. Brunia, G. Mulder, and M. N. Verbaten, eds., Elsevier, Amsterdam, 58-65.

199

Picton, T. W. (1992). “The P300 wave of the human event-related potential.” Journal of Clinical Neurophysiology, 9(4), 456-479. Picton, T. W., Lins, O. G., and Scherg, M. (1995). “The recording and analysis of event- related potentials.” Handbook of neuropsychology, F. Boller and J. Grafman, eds., Elsevier Science B.V., Amsterdam, 3-73. Pipingas, A., and Maruff, P. T. “Sources of timing errors in IBM PC controlled experiments.” International OSET Congress, Melbourne, Australia. Pipingas, A., Silberstein, R. B., and Nield, G. E. (2000). “Correlation between pre-target SSVEP amplitude and response time in a visual vigilance task.” Brain Topography, 12(4), 314; Poster presented at the 10th World Congress of the International Society for Brain Electromagnetic Topography (ISBET), in Adelaide, Australia, 9-13 October, 1999. Potter, D. D., Pickles, C. D., Roberts, R. C., and Rugg, M. D. (1992). “The effects of Scopolamine in event-related potentials in a continuous recognition memory task.” Psychophysiology, 29(1), 29-37. Pritchard, W. S. (1981). “Psychophysiology of P300.” Psychol. Bull., 90, 506-540. Puce, A., Andrewes, D. G., Berkovic, S. F., and Bladin, P. F. (1991). “Visual recognition memory. Neurophysiological evidence for the role of temporal white matter in man.” Brain, 114(Pt 4), 1647-1666. Ranganath, C., and Paller, K. A. (1999). “Frontal brain activity during episodic and semantic retrieval: insights from event-related potentials.” J Cogn Neurosci, 11(6), 598-609. Ranganath, C., and Paller, K. A. (2000). “Neural correlates of memory retrieval and evaluation.” Brain Research Cognitive Brain Research, 9(2), 209-222. Ray, W. J., and Cole, H. W. (1985). “EEG alpha activity reflects attentional demands, and beta activity reflects emotional and cognitive processes.” Science, 228(4700), 750-2. Regan, D. (1972). Evoked potentials in psychology, sensory physiology and clinical medicine, Chapman and Hall, London. Regan, D. (1976). “Latencies of evoked potentials to flicker and to pattern speedily estimated by simultaneous stimulation method.” Electroencephalography and Clinical Neurophysiology, 40, 654-660. Regan, D. (1977a). “Fourier analysis of evoked potentials; some methods based on

200

Fourier analysis.” Visual evoked potentials in man: New developments, J. E. Desmedt, ed., Clarendon Press, Oxford, 110-117. Regan, D. (1977b). “Steady-state evoked potentials.” J. Opt. Soc. Am., 67(11), 1475- 1488. Regan, D. (1989). Human Electrophysiology, Elsevier, New York. Regan, D., Milner, B. A., and Heron, J. R. (1977). “Slowing of visual signals in multiple sclerosis, measured psychophysically and by steady-state evoked potentials.” Visual evoked potentials in man: New developments, J. E. Desmedt, ed., Clarendon Press, Oxford, 461-469. Reid, R. (1999). “Vision.” Fundamental Neuroscience, F. Bloom, S. Landis, J. Roberts, and L. Squire, eds., Academic Press, San Diego, 821-851. Rockstroh, B., Elbert, T., Canavan, A., Lutzenberger, W., and Birbaumer, N. (1989). Slow cortical potentials and behaviour, Urban and Schwarzenberg, Baltimore M.D. Rolls, E. T. (2000). “Functions of the primate temporal lobe cortical visual areas in invariant visual object and face recognition.” Neuron, 27(2), 205-218. Rugg, M. D. (1987). “Dissociation of semantic priming, word and non-word repetition effects by event-related potentials.” Q. Journal Experimental Psychology, 39, 123-148. Rugg, M. D. (1990). “Event-related brain potentials dissociate repetition effects of high and low frequency words.” Memory and Cognition, 18, 367-379. Rugg, M. D. (1995). “Cognitive event-related potentials: intracranial and lesion studies.” Handbook of neuropsychology, F. Boller and J. Grafman, eds., Elsevier Science B.V., Amsterdam, 165-185. Rugg, M. D., Allan, K., and Birch, C. S. (2000). “Electrophysiological evidence for the modulation of retrieval orientation by depth of study processing.” Journal Cognitive Neuroscience, 12(4), 664-678. Rugg, M. D., Cox, C. J., Doyle, M. C., and Wells, T. (1995). “Event-related potentials and the recollection of low and high frequency words.” Neuropsychologia, 33(4), 471-484. Rugg, M. D., and Doyle, M. C. (1992). “Event-related potentials and recognition memory for low- and high-frequency words.” Journal Cognitive Neuroscience, 4, 69-79.

201

Rugg, M. D., and Nagy, M. E. (1989). “Event-related potentials and recognition memory for words.” Electroencephalography and Clinical Neurophysiology, 72(5), 395-406. Rugg, M. D., Roberts, R. C., Potter, D. D., Pickles, C. D., and Nagy, M. E. (1991). “Event-related potentials related to recognition memory. Effects of unilateral temporal lobectomy and temporal lobe epilepsy.” Brain, 114(Pt 5), 2313-2332. Rugg, M. D., Schloerscheidt, A. M., Doyle, M. C., Cox, C. J., and Patching, G. R. (1996). “Event-related potentials and the recollection of associative information.” Brain Res Cogn Brain Res, 4(4), 297-304. Rugg, M. D., and Wilding, E. L. (2000). “Retrieval processing and episodic memory.” Trends in Cognitive Sciences, 4(3), 108-115. Saletu, B., and Grunberger, J. (1985). “Memory dysfunction and vigilance: neurophysiological and psychopharmacological aspects.” Annals New York Academy Sciences, 444, 406-427. Sanquist, T. F., Rohrbaugh, J. W., Syndulko, K., and Lindsley, D. B. (1980). “Electrocortical signs of levels of processing: perceptual analysis and recognition memory.” Psychophysiology, 17(6), 568-576. Schacter, D. L. (1987). “Memory, amnesia, and frontal lobe dysfunction.” Psychobiology, 25, 21-36. Schacter, D. L. (1992). “Implicit knowledge: New perspectives on unconscious processes.” Proceedings National Academy Sciences U S A, 89(23), 11113- 11117. Schacter, D. L., and Crovitz, H. F. (1977). “Memory function after closed head injury: A review of quantitative research.” Cortex, 13, 150-176. Schacter, D. L., Kaszniak, A. W., Kihlstrom, J. F., and Valdiserri, M. (1991). “The relation between source memory and aging.” Psychology Aging, 6(4), 559-568. Scherg, M. (1990). “Fundamentals of dipole source potential analysis.” Auditory evoked magnetic fields and potentials. Anvances in audiology, F. Grandori and G. L. Romani, eds., Basel: Karger, 40-69. Schier, M. A. (1994). “Human steady-state visually evoked potential topography and attention,” PhD, University of Melbourne, Melbourne. Schloerscheidt, A. M., and Rugg, M. D. (1997). “Recognition memory for words and pictures: an event-related potential study.” Neuroreport, 8(15), 3281-3285.

202

Schmitt, F., Stehling, M. K., and Turner, R. (1998). “Echo-planar imaging. Theory, technique and application.” , Springer-Verlag, Berlin, 662. Scoville, W. B., and Milner, B. (1957). “Loss of recent memory after bilateral hippocampal lesions.” Journal of Neurology, Neurosurgery and Psychiatry, 20, 11-21. Sereno, M. I., Dale, A. M., Reppas, J. B., Kwong, K. K., Belliveau, J. W., Brady, T. J., Rosen, B. R., and Tootell, R. B. H. (1995). “Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging.” Science, 268, 889- 893. Shucard, D. W., Shucard, J. L., and Thomas, D. G. (1977). “Auditory evoked potentials as probes of hemispheric differences in cognitive processing.” Science, 197, 1295-1298. Silberstein, R. B. (1998). “Neocortical resonances and holding memory.” Brain Topography Today, Y. Koga, K. Nagata, and K. Hirata, eds., Elsevier Science, Amsterdam, 34-38. Silberstein, R. B., Burkitt, G. R., and Wood, S. R. (1993). “Artifact sensitivity of Fourier-based analysis of steady-state visually evoked potentials.” Biological Psychology, 37, 43-71. Silberstein, R. B., and Cadusch, P. J. (1992). “Measurement processes and spatial principal components analysis.” Brain Topography, 4, 267-276. Silberstein, R. B., Cadusch, P. J., Nield, G. E., Pipingas, A., and Simpson, D. G. (1996). “Steady-state visually evoked potential topography dynamics and cognition.” Recent advances in event-related brain potential research, C. Ogura, Y. Koga, and M. Shimokochi, eds., Elsevier, Amsterdam, 379-385. Silberstein, R. B., Ciorciari, J., and Pipingas, A. (1995a). “Steady-state visually evoked potential topography during the Wisconsin card sorting test.” Electroencephalography and Clinical Neurophysiology, 96, 24-35. Silberstein, R. B., Ciorciari, J., and Pipingas, A. (1995b). “Steady-state visually evoked potential topography during the Wisconsin card sorting test.” Electroencephalography and Clinical Neurophysiology, 96(1), 24-35. Silberstein, R. B., Farrow, M., Levy, F., Pipingas, A., Hay, D. A., and Jarman, F. C. (1998). “Functional brain electrical activity mapping in boys with attention- deficit/hyperactivity disorder.” Archives of General Psychiatry, 55(12), 1105-

203

1112. Silberstein, R. B., Line, P., Pipingas, A., Copolov, D., and Harris, P. (2000). “Steady- state visually evoked potential topography during the continuous performance task in normal controls and schizophrenia.” Clinical Neurophysiology, 111(5), 850-857. Silberstein, R. B., Nunez, P. L., Pipingas, A., Harris, P., and Danieli, F. (2001). “Steady state visually evoked potential (SSVEP) topography in a graded working memory task.” International Journal of Psychophysiology, 42(2), 125-138. Silberstein, R. B., Schier, M. A., Pipingas, A., Ciorciari, J., Wood, S. R., and Cadusch, P. J. (1990a). “Topography of the steady state visually evoked potential associated with visual vigilance.” Fourth International Evoked Potential Symposium. Book of abstracts. Toronto, 1990. p. 59. Silberstein, R. B., Schier, M. A., Pipingas, A., Ciorciari, J., Wood, S. R., and Simpson, D. G. (1990b). “Steady-state visually evoked potential topography associated with a visual vigilance task.” Brain Topography, 3(2), 337-347. Simpson, D. G. (1997). “Instrumentation for high spatial resolution steady-state visually evoked potentials,” Masters thesis, Swinburne University of Technology, Melbourne. Smith, M. E. (1993). “Neurophysiological manifestations of recollective experience during recognition memory judgements.” Journal Cognitive Neuroscience, 5, 1- 13. Smith, M. E., and Guster, K. (1993). “Decomposition of recognition memory event- related potentials yields target, repetition, and retrieval effects.” Electroencephalography and Clinical Neurophysiology, 86(5), 335-343. Smith, M. E., and Halgren, E. (1988). “Attenuation of a sustained visual processing negativity after lesions that include the inferotemporal cortex.” Electroencephalography and Clinical Neurophysiology, 70(4), 366-369. Smith, M. E., and Halgren, E. (1989). “Dissociation of recognition memory components following temporal lobe lesions.” Journal of Experimental Psychology: Learning Memory and Cognition, 15(1), 50-60. Smith, M. E., Stapleton, J. M., and Halgren, E. (1986). “Human medial temporal lobe potentials evoked in memory and language tasks.” Electroencephalography and Clinical Neurophysiology, 63(2), 145-159.

204

Smith, M. L. (1989). “Memory disorderss associated with temporal-lobe lesions.” Handbook of neuropsychology, F. Boller and J. Grafman, eds., Elsevier, Amsterdam, 91-106. Squire, L. R. (1992). “Declarative and non-declarative memory: multiple brain systems supporting learning and memory.” Journal of Cognitive Neuroscience, 2, 232- 243. Squire, L. R. (1994). “Declarative and nondeclarative memory: multiple brain systems supporting learning and memory.” Memory systems, D. L. Schacter and E. Tulving, eds., MIT Press, Cambridge (MA), 203-231. Squire, L. R., and Zola, S. M. (1996). “Structure and function of declarative and nondeclarative memory systems.” Proceedings of the National Academy of Sciences U S A, 93(24), 13515-13522. Squire, L. R., and Zola-Morgan, S. (1991). “The medial temporal lobe memory system.” Science, 253, 1380-1386. Stephenson, W. A., and Gibbs, M. D. (1951). “A balanced non-cephalic reference electrode.” Electroencephalography and Clinical Neurophysiology, 3, 237-240. Sternberg, S. (1969). “The Discovery of processing stages: extensions of Donder's Method.” Attention and Performance II (Acta Psychol., 30), W. G. Koster, ed., North-Holland Publ., Amsterdam, 276-315. Stuss, D. T., Picton, T. W., and Cerri, A. M. (1986). “Searching for the names of pictures: An event-related potential study.” Psychophysiology, 23, 215-223. Sutton, S., Braren, M., Zubin, J., and John, E. R. (1965). “Evoked potential correlates of stimulus uncertainty.” Science, 150, 1187-1188. Suzuki, W. A., and Amaral, D. G. (1994). “Perirhinal and parahippocampal cortices of the macaque monkey: cortical afferents.” Journal of Comparative Neurology, 350(4), 497-533. Tanaka, K. (1993). “Neuronal mechanisms of object recognition.” Science, 262(5134), 685-688. Tendolkar, I., Doyle, M. C., and Rugg, M. D. (1997). “An event-related potential study of retroactive interference in memory.” Neuroreport, 8(2), 501-506. Tesche, C. D., Uusitalo, M. A., Ilmoniemi, R. J., and Kajola, M. (1995). “Characterizing the local oscillatory content of spontaneous cortical activity during mental imagery.” Cog Brain Res, 2, 243-249.

205

Tootell, R. B., Dale, A. M., Sereno, M. I., and Malach, R. (1996). “New images from human visual cortex.” Trends in Neurosciences, 19(11), 481-489. Tootell, R. B., Mendola, J. D., Hadjikhani, N. K., Ledden, P. J., Liu, A. K., Reppas, J. B., Sereno, M. I., and Dale, A. M. (1997). “Functional analysis of V3A and related areas in human visual cortex.” Journal of Neuroscience, 17(18), 7060- 7078. Tulving, E. (1983). Elements of episodic memory, Oxford University Press, New York. Tulving, E. (1993). “What is episodic memory?” Current Directions in Psychological Sciences, 2, 67-70. Tulving, E., Kapur, S., Craik, F. I. M., Moscovitch, M., and Houle, S. (1994a). “Hemispheric encoding/retrieval asymmetry in episodic memory: positron emission tomography findings.” Proceedings of the National Academy of Sciences U S A, 91(6), 2016-2020. Tulving, E., Kapur, S., Markowitsch, H. J., Craik, F. I., Habib, R., and Houle, S. (1994b). “Neuroanatomical correlates of retrieval in episodic memory: auditory sentence recognition.” Proceedings of the National Academy of Sciences U S A, 91(6), 2012-2015. Tulving, E., and Schacter, D. L. (1990). “Priming and human memory systems.” Science, 247, 301-306. Ungerleider, L. G., and Haxby, J. V. (1994). “'What' and 'where' in the human brain.” Current Opinion in Neurobiology, 4(2), 157-165. Ungerleider, L. G., and Mishkin, M. (1982). “Two cortical visual systems.” Analysis of visual behavior, D. J. Ingle, M. A. Goodale, and R. J. W. Mansfield, eds., MIT Press, Cambridge, Mass. Van Essen, D. C. (2002). “Organization of visual areas in macaque and human cerebral cortex.” Visual Neurosciences, L. M. Chalupa and J. S. Werner, eds., MIT Press, Cambridge MA. Von Bonin, G., and Bailey, P. (1947). The Neocortex of Macaca Mulatta, Univ. Ill. Press, IL. Von Bonin, G., and Bailey, P. (1950). The Neocortex of the Chimpanzee, Univ. Ill. Press, IL. Waugh, N. C., and Norman, D. A. (1965). “Primary memory.” Psychological Review, 72, 89-104.

206

Webster, M. J., Bachevalier, J., and Ungerleider, L. G. (1993). “Subcortical connections of inferior temporal areas TE and TEO in macaque monkeys.” Journal of Comparative Neurology, 335(1), 73-91. Webster, M. J., Ungerleider, L. G., and Bachevalier, J. (1991). “Connections of inferior temporal areas TE and TEO with medial temporal- lobe structures in infant and adult monkeys.” Journal of Neuroscience, 11(4), 1095-1116. Wilding, E. L. (1999). “Separating retrieval strategies from retrieval success: an event- related potential study of source memory.” Neuropsychologia, 37(4), 441-454. Wilding, E. L. (2000). “In what way does the parietal ERP old/new effect index recollection?” International Journal of Psychophysiology, 35(1), 81-87. Wilding, E. L., Doyle, M. C., and Rugg, M. D. (1995). “Recognition memory with and without retrieval of context: an event-related potential study.” Neuropsychologia, 33(6), 743-767. Wilding, E. L., and Rugg, M. D. (1996). “An event-related potential study of recognition memory with and without retrieval of source [published erratum appears in Brain 1996 Aug;119(Pt 4):1416].” Brain, 119(Pt 3), 889-905. Wilson, G. F., and O'Donnell, R. D. (1986). “Steady state evoked responses: correlations with human cognition.” Psychophysiology, 23(1), 57-61. Wilson, G. F., and O'Donnell, R. D. (1988). “Measurement of operator workload with the neuropsychological workload test battery.” Human mental workload, P. A. Hancock and N. Meshkati, eds., Elsevier Science Publishers B. V., North Holland, 63-100. Yonelinas, A. P., and Jacoby, L. L. (1995). “The relation between remembering and knowing as the basis for recognition: effects of size congruency.” Journal of Experimental Memory and Language, 34, 622-643. Young, M. P. (1993). “Visual cortex: modules for pattern recognition.” Current Biology, 3, 44-46. Young, M. P., and Rugg, M. D. (1992). “Word frequency and multiple repetition as determinants of the modulation of event-related potentials in a semantic classification task.” Psychophysiology, 29, 664-676. Zola-Morgan, S., Squire, L. R., and Amaral, D. G. (1989). “Lesions of the amygdala that spare adjacent cortical regions do not impair memory or exacerbate the impairment following lesions of the hippocampal formation.” Journal of

207

Neuroscience, 9(6), 1922-1936. Zola-Morgan, S., Squire, L. R., Rempel, N. L., Clower, R. P., and Amaral, D. G. (1992). “Enduring memory impairment in monkeys after ischemic damage to the hippocampus.” Journal of Neuroscience, 12(7), 2582-2596.

208