Event-related functional magnetic resonance imaging: modelling, inference and optimization

Oliver Josephs1* and Richard N. A. Henson1,2 1Wellcome Department of Cognitive Neurology, Institute of Neurology, 12 Queen Square, London WC1N 3BG, UK 2Institute of Cognitive Neuroscience, University College London, 10 Queen Square, London WC1N 3BG, UK

Event-related functional magnetic resonance imaging is a recent and popular technique for detecting haemodynamic responses to brief stimuli or events. However, the design of event-related experiments requires careful consideration of numerous issues of measurement, modelling and inference. Here we review these issues, with particular emphasis on the use of basis functions within a general linear model- ling frameworkto model and makeinferences about the haemodynamic response. With these models in mind, we then consider how the properties of functional magnetic resonance imaging data determine the optimal experimental design for a speci¢c hypothesis, in terms of stimulus ordering and interstimulus interval. Finally, we illustrate various event-related models with examples from recent studies. Keywords: single trial; event-related fMRI; statistics; echo-planar MRI; BOLD contrast

1. INTRODUCTION 2. THE IMPORTANCE OF BEING EVENT RELATED The term event-related functional magnetic resonance Previous methods of functional neuroimaging, such as imaging (efMRI) can be de¢ned as the use of functional positron emission tomography (PET), have limited magnetic resonance imaging (fMRI) to characterize and temporal resolution, which precludes the imaging of all detect transient haemodynamic responses to brief stimuli but prolonged states of brain activity. Such state-based or tasks. Event-related, or trial-based, measurement is designs were initially adopted for fMRI studies. However, already standard in the ¢eld of electrophysiology, namely it did not take long before the greater sensitivity and stimulus-locked, event-related potentials (ERPs). In temporal resolution of fMRI, particularly at higher contrast, efMRI is a relatively new technique. fMRI magnetic ¢eld strengths, were used to allow the more measurements of the haemodynamic response are as £exible, event-related approach. recent as those of DeYoe et al. (1992), and statistical methods for detecting event-related responses are as (a) Trial-based designs recent as those of Boynton et al. (1996). The method has The cardinal advantage of efMRI over state-related since proved popular: more than 91 published papers fMRI is that it allows trial-based rather than block-based have used the technique. experiments. Trial-based experiments o¡er several advan- Here we review the technique. The stress is on the tages. statistical modelling techniques used to detect reliable event-related responses. It is hoped that this will comple- (i) The order of trials can be randomized, meaning that ment other review articles by Rosen et al.(1998)and the response to a trial is neither confounded by a D'Esposito et al. (1999), which concentrate on the nature subject's cognitive set nor systematically in£uenced of the event-related response and give examples of appli- by previous trials (Johnson et al. (1997), for example, cations in cognitive neuroscience. However, we shall showed that ERPs di¡ered depending on whether recapitulate the advantages of the event-related approach stimuli were randomized or blocked). over previous methods (½ 2), and brie£y outline the basics (ii) Trials can be individually categorized or parame- of haemodynamic responses and their measurement (½ 3). trized post hoc according to a subject's performance, We shall then discuss the reliable detection of event- as indexed by accuracy or reaction time for example related fMRI responses, which involves creating models (see, for example, Wagner et al.1998). (½ 4) and using the models to make inferences (½ 5). We (iii) Some experiments involve events that cannot be shall then introduce two areas of current research: the blocked, such as `oddball' paradigms, in which the applications of random e¡ect models in single subject event of interest is a stimulus that violates the efMRI and the optimization of event-related experi- prevailing context (B. A. Strange, R. N. A. Henson, mental designs (½ 6). Finally, we shall present several K. J. Friston and R. J. Dolan, unpublished data). recent neuroscienti¢c applications that illustrate the (iv) Some events can occur unpredictably, and can be modelling techniques (½ 7). indicated only by the subject (such as spontaneous transitions in the perception of ambiguous ¢gures *Author for correspondence (o.josephs@¢l.ion.ucl.ac.uk). (see, for example, Kleinschmidt et al.1998).

Phil. Trans. R. Soc. Lond. B(1999)354, 1215^1228 1215 & 1999 The Royal Society 1216 O. Josephs and R. N. A. Henson Event-related functional magnetic resonance imaging

(v) Trial-based fMRI results are more directly compar- 1998), visual cortex (see, for example, Boynton et al. 1996) able with other trial-based neuroscienti¢c methods and auditory cortex (see, for example, Josephs et al.1997) (for example, ERPs and reaction-time measure- suggests that the basic form of the BOLD response is ments). quite typical across the brain, one might expect variability arising from di¡erences in the blood supply to (b) Block-based designs di¡erent regions and the extent to which di¡erent Not only do event-related techniques make trial-based components of the venous compartment contribute to the experiments feasible but, more generally, they provide MRI signal. However, there are surprisingly few quanti- improved methods for analysing other forms of fMRI tative studies of this spatial variability. Schacter et al. experiment. For example, from an event-related perspec- (1997) found that responses in anterior prefrontal cortex tive, a state can be modelled, to ¢rst order, as a contin- were best ¢tted by gamma functions that peaked 4 s later uous train of events, each representing one trial within than the functions that best ¢tted the responses in the the block. If the trials are relatively sparse within the visual cortex. Although the vasculature of the two regions block, the event-related approach can account for £uctua- does not di¡er markedly, it remains uncertain whether tion of the response within the block. This means that the this relatively delayed prefrontal BOLD response re£ects experimental variance is likely to be better modelled than haemodynamic or neuronal e¡ects. Clari¢cation of this with a simple state-based approach (Price 1999). Another issue requires the mapping of venous £ow in di¡erent example is when a state has variance associated with the brain regions (see, for example, Lee et al.1995). event of entering the state, an early e¡ect (Friston et al. 1995a). This can simply be treated by modelling an (iii) Individual additional event representing the beginning of the block By far the largest source of variability observed by in addition to the state-related variance. Aguirre et al. (1998) was across subjects, presumably re£ecting considerable individual di¡erences in physiology. Physiological manipulations have been shown to o¡set the 3. MEASURING EVENT-RELATED RESPONSES BOLD response (Cor¢eld et al.1998),althoughinthiscase In this section, we brie£y examine the nature of the they did not modulate the magnitude, relative to the base- event-related haemodynamic response and its measure- line, of responses evoked visually. ment using echo-planar magnetic resonance imaging Thus preliminary studies suggest that the BOLD (EPI). The fMRI signal is caused by the (sluggish) response shows reasonable reproducibility across sessions haemodynamic response after neuronal activation. The and brain regions within a subject, but signi¢cant varia- biophysical mechanisms are not fully understood, but the bility across subjects. Further workis clearly needed: for signal change is clearly a function of blood £ow, volume example, to quantify the extent to which this variability and oxygenation state (Howseman & Bowtell, this issue). a¡ects the appropriateness of statistical models that assume This blood oxygenation level dependent (BOLD) contrast a single canonical form for the event-related response. Ã is usually measured with T2 -weighted gradient-recalled Models that use more general basis sets (} 4(c)) are better EPI. able to accommodate variability in the BOLD response.

(a) Variability in the response (b) Linearity The question arises of whether the BOLD response is a The assumption that the magnitude of the BOLD reliable and consistent marker of neuronal activity. response is linearly related to the magnitude of under- lying neuronal activity is useful in simplifying the model- (i) Temporal ling and detection of event-related responses. Various Aguirre et al. (1998) examined the variability of the studies (Boynton et al. 1996; Dale & Buckner 1997; BOLD response in the central sulcus during a simple Pollmann et al. 1998) have concluded that a reasonable reaction-time taskperformed across scanning sessions degree of linearity is to be expected, although this is within a day, between days and between subjects. Varia- clearly not always true (Vasquez & Noll 1998), particu- bility across sessions within a day was signi¢cant for only larly for short interstimulus intervals (Friston et al.1998b). one subject. Signi¢cant variability across days was Workis needed to correlate these e¡ects with electro- observed in three out of four subjects (although, because physiological investigations to discover which nonlinear scanning across days entailed repositioning subjects in the e¡ects occur at the neuronal level and which occur at the scanner, it was not possible to attribute this variability to haemodynamic level. Some manifestations of nonlinearity temporal £uctuations in BOLD response itself or to might include the following. di¡erences in the scanning environment). Reproducibility of state-related responses across multiple sessions in (i) Saturation several regions of interest has been demonstrated (McGo- Whereas the PET response seems to increase linearly nigle 1999), although other brain regions showed signi¢- with stimulus intensity, the fMRI response seems to cant session^state interactions. These studies highlight saturate at high levels (Howseman & Bowtell, this issue). the riskof false-positive results when generalizing from single sessions. (ii) Habituation The response to a trial might depend on the history of (ii) Spatial previous trials, over and above the linear superposition of Although a super¢cial comparison of responses previous responses. ERPs to rapid tone presentations, for measured in motor cortex (see, for example, Aguirre et al. example, are attenuated at short interstimulus intervals,

Phil. Trans. R. Soc. Lond. B(1999) Event-related functional magnetic resonance imaging O. Josephs and R. N. A. Henson 1217 presumably owing to habituation of the cells in the audi- intervals, time-courses can be plotted over peristimulus tory cortex. This form of habituation is also likely to be scans (see, for example, Cohen et al.1997).However,a re£ected by the haemodynamic responses measured in more powerful technique is to model the predicted time- efMRI. course of the fMRI signal (Friston et al.1995a; Josephs et al. 1997). Below, we describe the generation of a model (iii) Slew limits from component models of the stimulus, neuronal activity Slew limits are e¡ectively the maximum rate at which and haemodynamic response. the response can change. It might be that the haemo- dynamic system can reach a ¢nal level that depends line- (a) The general linear model arly on the applied stimulus, but can only reach that level We formulate models within a general linear model at a certain rate. (GLM) framework. By following this approach we can From the perspective of modelling, however, we can vary the complexity of the model to include higher-order remain agonistic on the question of linearity. We shall e¡ects such as di¡erential activation, interactions with demonstrate in ½ 4 that, even if the linear assumptions are state or time, and even (linear approximations to) violated, we simply expand our models to include nonlinear nonlinear e¡ects (½ 5(f )). In addition, we can account for terms and proceed to make statistical inferences on the the temporal covariance structure of fMRI data. basis of the linear or nonlinear component of the response. In a GLM, we assume that the observed data can be expressed as a linear combination of explanatory vari- (c) Whole-brain EPI measurement ables plus an error term. In matrix notation, fMRI data are usually acquired by using EPI, which is fast relative to other MRI imaging procedures: a single y ˆ X ‡ e, (1) slice can be measured in less than 100 ms. For single-slice where y is a vector of the measured results (a time-series studies this temporal resolution is su¤cient to sample sampled every scan), X is the design matrix, in which event-related haemodynamic responses, the spectral each column represents one explanatory time-course power density of which falls rapidly above 0.1Hz (see (covariate), is a vector of weights or parameters for ¢gure 1a). However, a multi-slice scan of the whole brain each covariate, and e is a residual error term with zero at reasonable spatial resolution, which is necessary for mean and covariance V. A least-squares solution, 0 for many experiments, entails a volume repeat time (TR) of in such a model is several seconds, depending on the number of slices. Thus the measurement of a haemodynamic response can often 0 ˆ pinv(XTinv(V)X)XTinv(V)y, undersample the experimental variance. One method to overcome this problem is to stagger stimulus times which, for independent, identically distributed residuals, relative to scan acquisition times, realizing a higher reduces to e¡ective sampling rate (Josephs et al.1997). 0 ˆ pinv(XTX)XTy. (d) Noise sources Thus an important assumption in the analysis of time- Event-related responses are not recorded in isolation, series is that the residuals are white, that is they depend but rather in the context of various non-deterministic only on the number of degrees of freedom (in fact, the noise confounding e¡ects. These include instrument noise, is often coloured owing to serial autocorrelations in the pulsatile physiological noise due to cardiac and respira- time-series, which requires corrections to the degrees of tory rhythms, endogenous haemodynamic £uctuations freedom; see Worsley & Friston (1995) for a complete and residual e¡ects of subject movement (motion e¡ects description). Under the null hypothesis, the model is also can persist even after image realignment, because of the expected to model variance depending only on the number non-rigid body nature of the multi-slice imaging). These of degrees of freedom (covariates) in the model (½ 5(d)). noise sources dominate at low frequencies, having a 1/f form (Aguirre et al. 1997; Zarahn et al.1997b). The noise (b) Stimulus model can be removed by high-pass ¢ltering during preproces- Weassume that the stimulus or taskcan be characterized sing or modelling (½ 5(b)) (Holmes et al.1997). by a multivariate function of time, s(t), that has one dimen- sion per experimental factor 1, :::, n. Values of s(t) ˆ 0 4. MODELLING EVENT-RELATED RESPONSES denote the baseline (no stimulus being applied).The occur- rence of a brief stimulus or taskis modelled as a Kroneker In ½ 3 we summarized the nature of the haemodynamic delta function. This model can also account for state response to brief stimuli and the means by which we can changes (e.g. a blockof stimulation) with appropriate `top- measure it with MRI. In this section we explain how, hat' functions. The resolved magnitude of s(t) can be set armed with this knowledge, we can predict the forms of arbitrarily to 1, to act as an indicator representing the variance that we expect to measure in an event-related presence of a categorical factor, or to a value representing experiment. the instantaneous magnitude of a parametric factor. Several early event-related studies did not model the BOLD response explicitly, but implicitly allowed for (c) Neuronal activation model haemodynamic lag by equating the event-related response The neuronal activation to stimulation, u(t), can be to the signal measured in scans acquired 4^6 s after the expressed as a function of s(t): stimulus (McCarthy et al. 1996). By synchronizing stimuli with scan onsets and employing long interstimulus u(t) ˆ fs(t)g‡es(t),

Phil. Trans. R. Soc. Lond. B(1999) 1218 O. Josephs and R. N. A. Henson Event-related functional magnetic resonance imaging

Figure 1. (a) EHRF in time (s) and frequency (Hz) domains after temporal smoothing with a FWHM Gaussian kernel of 4 s and¢lteringwithanidealhigh-passcut-o¡of1/60s.(b) Sequence from a simulation of the predicted signal energy per scan (bottom plot) after convolving a di¡erential stimulus function (top plot) with a canonical HRF at a resolution of 0.1 s (second plot), sampling every TR of 1 s (third plot), smoothing with a FWHM Gaussian kernel of 4 s (fourth plot), and high-pass ¢ltering to 1/60 s (¢fth plot). The smoothing and ¢ltering stages are separated (rather than being convolved with the EHRF in (a)) to illustrate their e¡ects separately.

Phil. Trans. R. Soc. Lond. B(1999) Event-related functional magnetic resonance imaging O. Josephs and R. N. A. Henson 1219

where es(t) represents spontaneous neuronal activity. In assumptions, such as linearity at the neuronal or haemo- general, the function  is nonlinear but we can expand it dynamic levels, as illustrated by the examples in ½ 4(f ). as a power series, (e) Measurement model T T u(t) ˆ s b ‡ s Bs ‡ ...‡ es(t), (2) The high-resolution discrete-time model in equation where b is a vector of n linear coe¤cients and the sym- (3) is sampled every TR, such that the predicted MRI signal is y(mT ), where T is the scan repetition time and metrical nÂn matrix B contains quadratic coe¤cients. R R The diagonal components of B represent nonlinear m is the scan number. The model (and data) can also be in£uences of a given factor on neuronal activity (e.g. smoothed temporally to swamp intrinsic autocorrelation saturation of receptors at high levels of stimulation). The in the data with a known autocorrelation (Worsley & o¡-diagonal terms encode interactions between factors Friston 1995).

(e.g. B12 ˆ B21 represents an interaction between factor 1 and factor 2). (f) Examples We now consider two special cases of the above model and demonstrate the GLM form for the measured time- (d) Haemodynamic response model We know that the haemodynamic response caused by course that ensues. neuronal activation lasts for a ¢nite duration after the event. This is equivalent to observing that a given (i) Univariate indicator stimulus function measurement is in£uenced only by neuronal activation Here we assume that only one stimulus type is that occurred within an equal, ¢nite period before the presented (n ˆ1), its intensity is constant, and neuronally generated noise is negligible (e (t) ˆ 0).With these simpli¢- measurement. This restricted temporal in£uence on a s scan can be expressed by a ¢nite memory model. With the cations there are no interactions between event types and use of a discrete-time formulation (for notational simpli- no nonlinearity in the neuronal activity, because only two city), the ¢nite memory model can be expressed as values of neuronal activity are generated. Thus the neuronal activation model is y(t) ˆ fu(t)g‡e (t), m u(t) ˆ bs(t) where the T-dimensional vector u(t) ˆ‰u(t),u(t À 1) ... and the recent history of the neuronal activation is uft À (T À 1)gŠ represents the neuronal activation at time t to a memory depth of T,andem(t) represents measure- u(t) ˆ bs(t), ment noise at time t. In general,  is a nonlinear function but it can be expanded as a Volterra series (Friston et al. where s(t) is formed by analogy with u(t) above. This can 1998b), be substituted into the haemodynamic response model, T 2 T T T y(t) ˆ bs h ‡ b s Hs ‡ ...‡ em(t), y(t) ˆ u h ‡ u Hu ‡ ...‡ em(t), (3) and substituting for h and H yields where h is the linear convolution term and H is a TÂT matrix of second-order coe¤cients. The nonlinearity T T T 2 T y(t) ˆ a1bs g1 ‡ a2bs g2 ‡ ...‡ apbs gp ‡ A1b s G1s coded by H includes the saturation of the BOLD response ‡ A b2sTG s ‡ ...‡ A b2sTG s ‡ e (t). to neuronal activation. 2 2 q q m We assume that h can be expressed as a linear combi- This is a GLM for y with (p+q) unknown parameters nation of i ˆ 1, ...p temporal basis functions, gi(t). In comprising neuronal and haemodynamic variables of the discrete-time matrix form, known stimulus function (event train) and known temporal basis functions (compare equation (1)). h ˆ a1g1 ‡ a2g2 ‡ ...‡ apgp. In a similar manner, we assume that H can be expressed (ii) Linear haemodynamic response as the linear combination of j ˆ1...q second-order basis We now consider the case for n stimulus factors and for functions, Gj(t): which the haemodynamic response is linear (H ˆ 0). For simplicity we again neglect the neuronal noise term, es(t). H ˆ A1G1 ‡ A2G2 ‡ ...‡ AqGq. In this case, u(t) is a vector containing Telements, each of which is the sum of n+1 n(n+1) terms: In the general case, we cannot separately estimate the 2 neuronal and haemodynamic parameters implicit in u(t) ˆ STb ‡ STBs, equation (3), given only the stimulus model as input and the data as output. For example, we cannot distinguish where S is an NÂT matrix, and thus between saturation at the neuronal level (the on-diagonal y(t)ˆ a b sTg ‡ a b sTg ‡ ...‡ a b sTg ‡ ... terms of B in equation (2)) and saturation at the haemo- 1 1 1 1 1 2 2 1 2 1 1 2 T T T dynamic level (the terms of H in equation (3)). Nonethe- ‡ apbnsn gp ‡ ...‡ a1B11s1(t)s1 g1 ‡ a1B21s2(t)s1 g1 less, other parameters (such as the o¡-diagonal terms of T T ‡ ...‡a2B11s1(t)s1 g ‡ ...‡apBnnsn(t)sn g ‡em(t). B that capture interactions between types of event) can 2 p be estimated, at least to a proportional level, which The resulting GLM has pfn+ 1 n(n+1)gˆpn 1 (n+3) 2 2 su¤ces for inferences by means of a variance ratio (F-test; unknown parameters. Thus in a two-factor experiment ½ 5(c)). Moreover, this general frameworkpermits the modelled by three linear basis functions, the model would derivation of simpler models by following additional contain 15 covariates: six to model the two main e¡ects,

Phil. Trans. R. Soc. Lond. B(1999) 1220 O. Josephs and R. N. A. Henson Event-related functional magnetic resonance imaging six to model nonlinearity in the factors, and three to su¤ce because the event-related response is contaminated model the interaction. by at least two sources of error. First, instrument noise is typically quite high (although much lower than in PET). (g) Haemodynamic response function Ratios of 2 or 3 for signal to (r.m.s.) noise are typical, We now turn to examine the choice of basis functions, and so any activation will only be detected with a low gi(t). These can be either direct expansions of time or signi¢cance. Higher ¢eld systems, with higher sample expansions of a prior estimate of a typical haemodynamic magnetization, might allow the detection of the event- response function (HRF). An example of the former related response to single trials (Richter et al.1997). approach is the use of Fourier series components to esti- Second, even though the sensitivity to detect the haemo- mate the full e¡ective sampling-rate-limited component dynamic response is high, the underlying brain response of the response (Josephs et al. 1997). For the latter itself often has a random component. One reason is that approach, we typically employ a multivariate Taylor the brain region in question might have functions unre- expansion of a gamma-function approximation to a cano- lated to the taskunder investigation. Furthermore, nical HRF (Friston et al.1998a). The higher-order basis subjects might vary their strategy from trial to trial, functions in this expansion include the partial derivatives yielding a random, experimentally induced variance of the HRF with respect to time and dispersion. The component. For this reason one wishes to detect a typical Fourier basis set has the advantage of generality and rather than a single response. The question therefore insensitivity to slice timing (½ 4(h)), whereas the partial becomes `Which voxels consistently show an event-related derivatives of a canonical HRF have the advantage that response?' However, we begin by discussing the concepts the parameter for each covariate is interpretable in terms of confounds, parameter estimation, and contrasts. of response magnitude, latency or duration (½ 5(f )(iv) and ½ 7(f )). The canonical form and its derivatives can be (a) Confounds tested separately by means of univariate tests, whereas We routinely model several confounding e¡ects when the Fourier components are best tested by reduced F-tests analysing fMRI data. These e¡ects include the mean over (½ 5(d)). time, all low frequencies up to some cut-o¡ point (typi- cally 60 s), to remove low-frequency noise (½ 3(d)), and a (h) Slice-timing issues global mean e¡ect over space (i.e. all voxels). The impor- A single model is often assumed for all voxels (i.e. scan- tant point with respect to efMRI is that the experimental based or volume-based design matrices). At present, variance is predominantly of a relatively high frequency however, whole-brain EPI acquisitions (½ 3(c)) require a compared with a state-related design. This allows us to time comparable to the haemodynamic rise time, apply a higher cut-o¡ to the high-pass ¢lter, removing meaning that di¡erent slices are acquired at di¡erent more of the low-frequency e¡ects that would otherwise times relative to the response. This can be partly reduce sensitivity. corrected for by temporal interpolation. However, interpolation is not ideal, particularly if signi¢cant (b) Estimation of parameters and residual variance experimental power exists at frequencies above the Our statistical inference uses an analysis of covariance Nyquist sampling limit (2TR)À1. Alternatively, slice- procedure. We need to discover the parameter estimates timing di¡erences can be accommodated by a suitable for the linear components of the full model and, in choice of basis functions, such as the phase-invariant addition, the residual variance accounting for (i) the Fourier set, or the inclusion of temporal derivatives confounds only (the reduced model (Buechel et al.1996)), (Henson et al. 1999). However, the most general solution and (ii) both the e¡ects of interest and the confounds. is to construct separate models for each slice that take The purpose of the reduced model is to estimate the resi- into account their acquisition time relative to the stimulus dual variance and degrees of freedom when accounting (i.e. slice-based design matrices). only for the confounding e¡ects, which can be used in an F-test of signi¢cance (½ 5(d)). 5. INFERENCE IN efMRI (c) Contrasts Having constructed a model for the event-related In many cases we wish to test speci¢c, a priori hypotheses variance, we now discuss the estimation of the parameters about di¡erences between two or more types of event. required to ¢t the model to the data and methods for Such contrasts (Friston et al.1995b) are linear combina- performing statistical inference with the use of the para- tions of event-related e¡ects, speci¢ed as vectors of weights meter and residual variance estimates. The questions that for each event type. The signi¢cance of these contrasts can can be addressed mirror the models that have been used. be indexed by t-tests (½ 5(d)). We can incorporate these We consider only inferences at the level of individual contrasts into the model by multiplying the design matrix voxels (for issues regarding whole-brain inferences, see X by a matrix C containing each contrast. This rotation Friston et al.(1995b)). Inference involves the use of resi- produces a new model, X', in which each column is a dual variance estimates to determine the distribution of linear combination of columns from X, with weightings the parameter estimates for e¡ects of interest under the according to the appropriate row (contrast) from C. null hypothesis, and hence the probability of falsely rejecting the null hypothesis (the signi¢cance), assuming (d) Fixed-e¡ects inference the null hypothesis to be true. Event-related inference can be performed within the We could, in principle, test for activation by following a GLM by two types of signi¢cance test: Student's t-test single event-related trial. However, this is unlikely to and Fisher's F-test. In both cases we use the results of

Phil. Trans. R. Soc. Lond. B(1999) Event-related functional magnetic resonance imaging O. Josephs and R. N. A. Henson 1221

¢tting the full GLM. For the F-test we additionally ¢t the (f) Inference about multiple types of event reduced model (½ 5(c)). Here we are interested in the di¡erential e¡ects of two The t-test is used when we wish to determine the or more event types. We construct an NÂM column signi¢cance of a particular contrast. By comparing the design matrix of N event types and M basis functions parameter estimate against a t-distribution, we deter- that spans the space of experimental variance. We can mine a signi¢cance level for the contrast. For the F-test now rotate the model (½ 5(c)) to form a design matrix we do not refer to the parameter estimates but rather the comprising linear combinations of the original covariates. ratio of error variances obtained from the full and the Under certain useful rotations, this rotated model is reduced models. Under the null hypothesis, this ratio is equivalent to the original model but the ¢tted parameter expected to follow an F-distribution with D7N and estimates can be more directly interpretable. Note that if D7(N7M) degrees of freedom, where D is the number events of each type are close together in time and not of degrees of freedom in the data, N is the degrees of completely randomized in order (such that the covariates freedom in the complete model and M is the degrees of for the di¡erent event types are correlated), then some of freedom in the confounds. For independent residual the experimental variance due to one event type might be error variance, the degrees of freedom in the data equal inappropriately modelled by the other event types, the number of scans (for temporally smoothed data, the resulting in a loss of sensitivity to detect responses to each e¡ective degrees of freedom are less than the number of eventtype(atypeIIerror). scans (Worsely & Friston 1995)). For non-collinear covariates, the degrees of freedom are the number of (i) Di¡erential responses covariates in the model. An example rotation for two trial types is to form The advantages of the t-test are that multiple tests can the sum and di¡erence between the two responses to be performed after ¢tting only one model and that the (note that the sum of two event-type covariates, A+B, direction of e¡ects can be tested (e.g. an activation or is orthogonal to the di¡erence, A7B). These partitions deactivation). The F-test is required when we wish to test of the model can be interpreted as accounting for the for the combined signi¢cance of a multiple-component mean (common) activation and the di¡erential model (e.g. with multiple basis functions). With a single activation. component, the F-value obtained in an F-test equals the square of the t-value from an equivalent t-test. (ii) Factorial responses One can extend the idea of modelling commonalties (e) Inference about a single type of event and di¡erences to more general factorial experiments. For The question asked here is `Did the voxel respond to example, the model for a two-factor, two-level experi- the stimulus?' This is the simplest hypothesis, in which ment comprises four partitions, one for each event type. we attempt to disprove the null hypothesis that no event- This model can be rotated to represent, for example, the related activation occurred. Under the null hypothesis the common e¡ect of a trial, the two main e¡ects and an variance in the subspace of the model due to the event interaction term. would be expected to depend only on the degrees of freedom in the model relative to the degrees of freedom in (iii) Parametric responses the data. The ratio of these modelled-to-residual The levels of a given factor can be continuous rather variances allows an F-test of the signi¢cance of the activa- than discrete. In this case the model for the response tion. In the context of confounding e¡ects, the F-ratio magnitude (s(t) in ½ 4(b)) depends on the level of the can be formed between the extra variance accounted for factor. Alternatively, the response magnitude might be by the e¡ects of interest relative to the residuals from the determined post hoc by behavioural measures (e.g. full model (Buechel et al.1996). galvanic skin response) that are associated with each In some designs, the response of interest is that to a trial. single event in the context of background events of no interest. An example of this design is an `oddball' para- (iv) Response parameters digm, in which the event of interest is a stimulus that The components of some basis sets can have di¡erent deviates from surrounding context events. For example, interpretations, and it can therefore be useful to test low-pitched tones might be presented rapidly with occa- them separately. The use of a canonical HRF and its sional high-pitched tones. If the context events are derivatives with respect to time and dispersion, for presented rapidly enough (relative to the time constants example, allows separate t-tests of di¡erences in magni- of the HRF), they can be treated as a raised baseline, for tude, latency and duration of event-related responses which we assume that a tonic haemodynamic state is (Friston et al.1998a). Latency di¡erences can then be reached with respect to the common e¡ect (of tones). If compared with psychophysical di¡erences, such as reac- the interval between e¡ects of interest (high tones) is tion-time di¡erences. Note, however, that the dispersion su¤ciently long, we can detect a signi¢cant event-related derivative is not orthogonal to the canonical form, and e¡ect superimposed on this baseline (assuming linearity). although the temporal derivative is orthogonal to the If context events are not presented rapidly enough for a canonical form, this orthogonality will not remain if the tonic state to be reached, they may need to be modelled model and data are undersampled. Thus, strictly inde- explicitly as confounds (otherwise the assumption that pendent tests of latency or duration di¡erences can residual noise is white might be compromised, and type I require explicit orthogonalization of the sampled deriva- errors can arise if the associated variance is misattributed tive and dispersion covariates with respect to the to the e¡ect of interest). sampled canonical form.

Phil. Trans. R. Soc. Lond. B(1999) 1222 O. Josephs and R. N. A. Henson Event-related functional magnetic resonance imaging

(g) Non-stationary reponses no reason why such corrections could not be applied The event-related response, or derived e¡ects, can across trials when signi¢cant autocorrelation does exist). adapt to (interact with) other factors. Whereas the non- One problem with separate partitions of the design linearities described in ½ 3(b) can be interpreted as inter- matrix for each trial is that several degrees of freedom actions between (typically) adjacent event-related might be required to obtain an accurate estimation of responses, non-stationary responses can be regarded as response parameters. This implies several scans per event, the interaction of other parameters. For example, the with a TR short enough to sample the haemodynamic response can depend on the task(e.g. the context of a response su¤ciently, and relatively long interstimulus trial), the subject (e.g. the level of arousal), or simply the intervals. As shown below, however, long interstimulus time during the experiment. Such interactions can be intervals are non-optimal for most designs. modelled as the product of the parameter and the event- related e¡ect. A particularly interesting case arises when the non-stationarity of a response can be engineered to be 6. OPTIMIZATION OF EVENT-RELATED orthogonal to stationary components. For example, the EXPERIMENTAL DESIGNS variance in a trial of varying duration can be modelled in In this section we shall discuss the optimization of terms of constant components due to the beginning and event-related experimental designs, which is an area of end of the trial and a varying component depending on much current interest. Optimization involves maximizing the duration of the trial. the sensitivity (signal:noise ratio) for particular contrasts (hypotheses) as a function of the stimulus ordering and (h) Random-e¡ect inference stimulus onset asynchrony (SOA). For simplicity we So far we have assumed that event-related responses assume the noise to be invariant across changes in are ¢xed e¡ects; that is, neglecting possible non- ordering or SOA, and concentrate on maximizing the stationarity and nonlinearity, the neuronal response from signal. We begin by introducing several concepts that are trial to trial can be considered constant. In many cases, useful in characterizing the space of experimental however, the actual event-related responses are in fact designs. random e¡ects, i.e. they are themselves drawn from a distribution of possible values. The appropriate null (a) Some de¢nitions hypothesis is no longer that there is no activation in the (i) Stimulus transition matrices experiment but rather that there is no consistent activa- An event-related experiment is fully determined by the tion in the population of trials from which the present SOA and a transition matrix. For N di¡erent event types, experiment sampled. In this section we consider how the N m ÂN transition matrix, T, details the probability of including, or failing to include, a random-response a given event type (in the columns of T) conditional on component a¡ects the statistical inference drawn from the previous m event types (the rows of T). Table 1 shows experiments. Much of this discussion re£ects the issues a ¢rst-order (m ˆ1) transition matrix for two randomly raised by Petersson et al. (this issue) on multiple-subject ordered event types, A and B (1), a ¢rst-order transition inferences (see also Holmes & Friston 1998) (Strange et al. matrix for two ordered (alternating) event types (2), and 1999). a second-order (m ˆ 2) transition matrix for two We consider a hypothetical di¡erential event-related permuted event types (3). Note that the permuted experiment in which the neuronal response varies sequence is pseudo-randomized in the sense that it is randomly from trial to trial. We assume that there is in randomized up to ¢rst-order contingencies. fact no systematic di¡erential activation between trials Deterministic designs are those in which the elements

(i.e. the responses are drawn from the same population of the transition matrix (the probabilities, pe, of each distribution with zero mean but non-zero standard devia- event type e) are either 0 or 1. Stochastic designs (Heid et tion), and that the instrument noise is white. al. 1997) are those in which pe51, and a special case of A ¢xed-e¡ect model for this experiment is the convolu- stochastic designs is fully randomized designs in which tion of a delta-function sequence by an HRF. Fitting this pe ˆ1/N for all event types e ˆ1, :::, N. In most situa- model to the data can result in a small but non-zero para- tions, a fully randomized design is desirable (for psycho- meter estimate. Crucially, however, the residual variance logical reasons, for example). Some designs, however, is non-white (confounding variability between responses have an inherent non-random ordering (such as alter- with variability between scans), violating the assumption nating transitions between bistable perceptions; see, for required to use the residual variance in setting the appro- example, Kleinschmidt et al. 1998). As demonstrated priate signi¢cance thresholds. A random-e¡ect model below, di¡erent event-type orderings are most sensitive treats each trial separately, in its own partition of the for a given contrast in di¡erent ranges of SOA. design matrix, meaning that residuals are more likely to Events can be varied in time as well as order. Such be white. To test for the di¡erential e¡ect, the parameter designs can be realized by transition matrices in which estimates from each trial are taken to a second level of the rows sum to p ˆ Spe51. In the fully randomized inference, in which their distributions are compared design of Dale & Buckner (1997), for example, an addi- directly. These two stages e¡ectively separate inter-scan tional null trial is introduced (when no event occurs), and inter-response variability. such that pe ˆ1/(N+1). The introduction of null trials is An additional advantage of the random-e¡ect model is equivalent to the stochastic selection of SOAs for the that, under the null hypothesis, the parameter estimates trials of interest (see ½ 6(c)). Stochastic designs can also be are likely to be less correlated and hence not require stationary, where pe is constant over time, or dynamic, corrections for autocorrelation (½ 5(d)) (although there is where pe changes over time (K. J. Friston, E. Zarahn,

Phil. Trans. R. Soc. Lond. B(1999) Event-related functional magnetic resonance imaging O. Josephs and R. N. A. Henson 1223

Table 1. Examples of (1) random, (2) alternating and (3) contrast. The IEI is the time between repetitions of permuted (pseudo-random) sequences and their transition event types with the same contrast weight. The IEI for a matrices simple di¡erence between two alternating event types (a [171] contrast), for example, is twice the SOA, as is sequence the IEI for the interaction between four alternating type A B example event types in a 2Â2 factorial design (a [171711] contrast), whereas the IEI for the main e¡ect (a [11] or 1A 0.50.5 [1111] contrast, respectively) is equal to the SOA. As B 0.5 0.5 ABBABBBAAABA . . . shown below, the optimal SOA for a given contrast in a 2A 01 deterministic design is that with an experimental B 1 0 ABABABABABAB . . . frequency, 1/IEI, that is closest to the peakof the EHRF 3AA01 power spectrum. AB 0.5 0.5 ABBABAABBABA . . . BA 0.5 0.5 (iv) Estimated measurable power BB 1 0 The signal magnitude associated with a particular contrast and a particular experimental design can be indexed by the estimated measurable power (EMP). The O. Josephs, R. N. A. Henson and A. Dale, unpublished EMP can be simulated by taking the linear neuronal data). An extreme example of a dynamic stochastic design model (½ 4(c)), multiplying by the contrast of interest, is the blocked-trial activation^rest design, where pe convolving with the EHRF (again assuming linearity of switches from 0 to 1 in each activation block. We concen- the haemodynamic response; ½ 4(f )(ii)), sampling every trate on deterministic designs below, to illustrate impor- scan (TR) and calculating the total energy (sum of tant concepts in the detection of event-related responses, squared signal across scans). The EMP is the total and consider stochastic designs later. energy divided by the number of scans (to normalize for di¡erent experimental durations). Assuming that the (ii) E¡ective HRF noise level is ¢xed (i.e. not in£uenced by the parameters Although the HRF contains low-frequency compo- being varied), then the experimental design with the nents, these components are likely to be contaminated highest EMP is likely to be the most sensitive for that by noise due to subject motion and other long-term drift contrast. EMP is therefore proportional to the e¤ciency e¡ects (½ 3(d)). Its high-frequency components are also of a design for a particular contrast, which is propor- redundant when data are smoothed to accommodate tional to the variance of a column in a design matrix temporal autocorrelation in the GLM (½ 4(e)). It is (K. J. Friston, E. Zarahn, O. Josephs, R. N. A. Henson therefore sensible to introduce the concept of the e¡ec- and A. Dale, unpublished data). For multiple contrasts, tive HRF (EHRF). The EHRF is simply the HRF or contrasts involving multiple basis functions, the e¤- ¢ltered by the band-pass ¢lter comprising the high-pass ciency of a design can be characterized by the trace of ¢lter and the temporal smoothing required for statistical the covariance of the contrasted design matrix. The inference. By considering this e¡ective response, as EMPs for single contrasts in some example designs are opposed to the actual response, we can visualize the discussed below. measurable component of the haemodynamic response to neuronal activity. (b) Simulated optimization Figure 1a shows the EHRF with the use of a canonical Figure 2 shows EMP as a function of SOA for the main HRF (½ 4(f )), a high-pass ¢lter up to 60 s, and Gaussian and di¡erential e¡ects in various orderings of two event smoothing with a full width at half maximum (FWHM) types. These simulations involved 10 000 events (to ensure kernel of 4 s. The long side-lobes of the EHRF in the time stable estimates for randomized designs), the EHRF domain arise from our use of an ideal high-pass ¢lter. shown in ¢gure 1a, and continuous sampling with a TR of Since we are not interested in the time-course of the 1s. We emphasize that resulting EMP is not necessarily haemodynamic responses itself, but rather the ability to linearly related to sensitivity (for example the resulting t- detect responses that re£ect our experimental manipula- value)öwe assume only that EMP is a monotonic index tion, these long time-scale artefacts do not usually of sensitivityöand that for SOAs of less than 2 s (Friston concern us. However, this does provide a mechanism et al.1998b), nonlinear (saturation) terms will prevent the whereby, if the model for the event-related response is EMP from growing without limit. incomplete, power evoked by one event can `leak' into a Figure 2a shows sensitivity for the random, alter- region of time that is modelled by another event (an issue nating and permuted designs in table 1. Maximal sensi- that needs further exploration). The e¡ect of temporal tivity to a di¡erential e¡ect (a [171] contrast) arises smoothing can be seen in the attenuation of the high- with randomized designs and short SOAs. Maximal frequency components in the frequency domain. The opti- sensitivity to a main e¡ect (a [11] contrast) is of course mization of an experimental design can then be viewed as independent of stimulus ordering, and occurs with an the maximization of experimental power under the SOA of ca. 18 s (i.e. with a dominant experimental frequency pro¢le of the EHRF. frequency close to that of the EHRF). Thus we immedi- ately see that whether one's hypothesis concerns a (iii) Inter-e¡ect interval di¡erential e¡ect or a main e¡ect determines whether For deterministic designs, it is the inter-e¡ect interval one should use a short or a long SOA respectively (in (IEI) that determines optimal sensitivity for a particular deterministic designs).

Phil. Trans. R. Soc. Lond. B(1999) 1224 O. Josephs and R. N. A. Henson Event-related functional magnetic resonance imaging

(a)(b)(c) randomized ME randomized ME randomized ME 5 randomized DE 5 randomized DE 5 blocked1 DE permuted DE null event ME blocked2 DE alternating DE null event DE blocked4 DE

4 4 4

3 3 3 log(EMP)

2 2 2

1 1 1

0301020 01003020 30 10 20 SOA (s) SOA (s) SOA (s)

Figure 2. Logarithm of the scaled EMP against SOA for the main e¡ect (main e¡ect, ME, solid lines) and di¡erential e¡ect (di¡erential e¡ect, DE, broken lines) for two event types in (a) randomized, alternating and permuted designs (table 1), (b) randomized designs with and without null events (} 6(b)), and (c) randomized designs and blocked designs of one (alternating), two or four events.

At short SOAs, the main e¡ect of two event types (the sequence might not, of course, be appropriate for psycho- constant component) is e¡ectively a raised baseline that is logical considerations, so a permuted design, which has a removed by the high-pass ¢lter. For alternating designs, similar sensitivity pro¢le but is randomized to ¢rst order, the high-frequency components of the di¡erential e¡ect at might be a good compromise. short SOAs are lost by the temporal smoothing inherent Figure 2b shows the sensitivity for a design suggested in the HRF and smoothing kernel. For randomized by Dale & Buckner (1997). This fully randomized design designs, the di¡erential e¡ect is actually enhanced at involves a third null event, in which no stimulus occurs short SOAs through the summation of overlapping HRFs (i.e. the two event types are e¡ectively randomized in (see, for example, Burock et al. 1998). In fact, the time as well as order). If null events are treated as a third maximum energy for such rapid, randomized designs event type, main and di¡erential e¡ects can be simulated actually occurs in the scans after runs of the same event by [11 0] and [1710] contrasts, respectively. This design type (see ¢gure 1b and ½ 6(d)). a¡ords improved sensitivity to the main e¡ect even at At intermediate SOAs between 6 and 25 s, the optimal short SOAs. Thus, although this design is non-optimal for design for a di¡erential e¡ect is an alternating or di¡erential contrasts, relative to a randomized design permuted design. With such designs, the optimal SOA is with only two event types, it is a good design for ca. 9 s, giving an IEI (twice the SOA; ½ 6(a)(iii)) that is retaining sensitivity to both main and di¡erential e¡ects close to the dominant components of the EHRF. More at short SOAs. generally, the optimal SOA for a given contrast in an Figure 2c shows the sensitivity of short, blocked alternating design is that with an IEI close to 18 s (with designs, with alternating blocks of one, two or four events the present parametrization of high-pass and low-pass of each type. These blocked designs are more sensitive ¢ltering). Thus the optimal SOA for a pairwise di¡erence than the randomized design at short SOAs, particularly between two of four alternating event types (a simple as blocklength increases (at least until the frequency of e¡ect, or a [1710 0] contrast) would be ca. 4.5 s. The blockalternation approaches the high-pass cut-o¡ decreased sensitivity of randomized designs in these inter- (Hutton et al. 1998)). Again, this is because, under the mediate SOA regimes arises because low-frequency linearity assumption, responses to events of the same type components resulting from long runs of an event type are are summed, concentrating power at the blockalternation attenuated by the high-pass ¢lter. An alternating frequency.

Phil. Trans. R. Soc. Lond. B(1999) Event-related functional magnetic resonance imaging O. Josephs and R. N. A. Henson 1225

(c) Analytical optimization experimental and control trials of a visuospatial working Many of the simulated results in the previous section memory taskpresented every 30 s. Paired t-tests on indi- can be con¢rmed by analytical methods. Friston et al. vidual HRFs revealed signi¢cant activations in sensori- (K. J. Friston, E. Zarahn, O. Josephs, R. N. A. Henson motor and prefrontal regions associated with temporally and A. Dale, unpublished data), for example, character- distinct perceptual, mnemonic and response components ized the sensitivity to responses of a single event type as a of each experimental trial. Dale & Buckner (1997) used function of the minimal SOA, SOAmin,andtheprob- gamma functions (parameterized by Boynton et al. ability, p, of an event occurrence every SOAmin. Under (1996)) to model the response to 1s bursts of one, two or the linearity assumption, optimal sensitivity occurs in a three £ickering visual chessboards. Signi¢cant activation stationary stochastic design with p ˆ 0.5 and minimal was observed in the primary visual cortex, with responses SOAmin. Thus, contrary to the deterministic cases consid- within bursts summing linearly even for SOAs as short as ered above, long SOAs are not necessary to measure 2 s. These examples illustrate the range of di¡erent model- main e¡ects if stochastic designs are employed: the ling techniques that have been used to detect signi¢cant random `jitter' in time introduces lower-frequency compo- event-related responses. nents that survive smoothing, much like the Dale^ Buckner null events in ¢gure 2b. Future investigation of (b) Raised baseline event-related optimization will need to parameterize and The two principal disadvantages of multiple-trial quantify the in£uence of nonlinear components of experiments with a single event type relate to the predict- neuronal and haemodynamic responses on sensitivity, ability of events and the lackof any high-level control. particularly for short SOAs and for variable durations of The subject knows in advance what type of event they are events. about to experience and a voxel might be activated not because of the e¡ect of interest (e.g. a word being (d) Interpretation in rapid, randomized designs presented) but because the event is more salient, or atten- Figure 1b shows a section of the simulated signal in a tion-grabbing, than the inter-trial baseline. One might di¡erential contrast with a randomized design and an even detect event-related deactivations in brain regions SOA of 2 s. The bottom plot, showing the instantaneous that re£ect, for example, a growing anticipation of an power at each scan point, illustrates that, at short SOAs, event. A raised-baseline technique might su¤ce in some runs of events of the same type generate large amounts of cases (½ 5(e)). This form of experiment has been used energy. Conversely, periods of alternating event types (R. Vandenberghe, O. Josephs, L. K. Tyler, C. J. Price, produce minimal energy. This observation has implica- R. Turner, R. S. J. Frackowiak and K. J. Friston, unpub- tions for the interpretation of any signi¢cant di¡erential lished data) when the event of interest was temporally e¡ect: any such e¡ect is driven mainly by a di¡erence extended, comprising a short sequence of seven words between responses to event type A and event type B in (lasting 3.85 s) within a background sequence of conso- the context of runs of A or B. In other words, we are left nant letter strings presented every 550 ms. The prolonged with the same type of context-dependent interpretation HRF was modelled with a Fourier basis set (½ 4(g)). As that is the lacuna of blocked designs. For example, if a expected, responses associated with word-processing were negative priming e¡ect existed for the immediate repeti- identi¢ed in the left supramarginal, superior temporal tion of event type A but not B (such that the response to and Sylvian ¢ssure regions (although these responses did A diminished with immediate repetition), the ¢nding of a not seem to di¡erentiate between meaningful sentences signi¢cant di¡erence between event types A and B might and random word sequences). apply only to primed events, and there might not in fact A similar design has been used (B. A. Strange, be any di¡erence between unprimed event types A and B. R. N. A. Henson, K. J. Friston and R. J. Dolan, unpub- This is one reason why the greater EMP for short SOAs lished data) in which sequences of semantically related might be sacri¢ced in favour of the clearer interpretation neutral-context words were presented at a rate of one a¡orded by longer SOAs in which haemodynamic overlap word every 3 s (to allow subjects to make a judgement is reduced. about each word). Three `oddball' words were randomly inserted into these sequences: a perceptual `oddball' (a word presented in a di¡erent font), a semantic `oddball' (a 7. EXAMPLE EVENT-RELATED EXPERIMENTAL semantically unrelated word) and an emotional `oddball' DESIGNS (a word with aversive emotional connotations). The Now that we have covered measurement, modelling, slower presentation rate in this case meant that a tonic inference and optimization, we can discuss how these tech- baseline might not have been reached, so context words niques have been used to address neuroscienti¢c questions. were explicitly modelled as confounds (½ 5(e)). Enhanced responses relative to context words were identi¢ed in the (a) Single event type fusiform cortex for perceptual `oddballs', in the left Josephs et al. (1997) used a Fourier basis set of 16 prefrontal cortex for semantic `oddballs', and in the amyg- components up to 0.5 Hz (½ 4(g)) to model the response dala for emotional `oddballs'. to words heard every 33 s. An F-test revealed signi¢cant activation in auditory and periauditory regions. Plots of (c) Simple di¡erential response the event-related response peaked at 5^8 s after the Most event-related experiments use two or more event stimulus, followed by an undershoot 10 and 25 s after the types and are primarily interested in the di¡erential stimulus. Zarahn et al.(1997a) used a set of time-shifted, e¡ect between event types (½ 5(f )(i)). A popular empirically derived HRFs to model responses during design for di¡erential e¡ects is the rapid, randomized

Phil. Trans. R. Soc. Lond. B(1999) 1226 O. Josephs and R. N. A. Henson Event-related functional magnetic resonance imaging presentation with null events developed by Dale & T. Shallice and R. J. Dolan, unpublished data) that exam- Buckner (1997). Buckner et al. (1998), for example, used ined the interaction between repetition priming within an this design to examine object priming by comparing experimental context (as in Buckner et al. (1998), discussed responses to novel object pictures with responses to above) and whether or not the stimulus was familiar previously studied pictures. Several brain regions, before the experiment. In one experiment, the interaction including the extrastriate visual, inferior temporal and was between ¢rst and second presentations of famous and left dorsal prefrontal cortices, showed reduced responses non-famous (unfamiliar) faces. A region in the right to familiarized compared with novel pictures. One advan- lateral fusiform cortex showed a decreased response to the tage of this randomization scheme is that plots of the repetition of famous faces, comparable to the results of di¡erential response between each event type and ¢xation Buckner et al. (1998), but an increased response to the (e.g. novel pictures compared with null events) can be repetition of non-famous faces. Furthermore, the same generated simply by selective averaging of the data region showed the same interaction in a second experi- (although statistical tests of these di¡erences remain best ment in which the stimuli were simple line-drawings that achieved with temporal basis functions). were either familiar (symbols) or unfamiliar (rearranged Wagner et al. (1998) reported another use of this design symbols). One possibility is that the enhanced response to to examine di¡erential responses to words presented the repetition of unfamiliar stimuli in this region re£ects visually as a function of whether or not the words were the formation of new stimulus representations. subsequently remembered. Enhanced responses in the left posterior prefrontal, parahippocampal and fusiform (e) Parametric responses cortices predicted which words were later recognized. In the study of repetition priming described above This experiment illustrates one of the important advan- (R. N. A. Henson, T. Shallice and R. J. Dolan, unpub- tages of event-related designs (½ 2(a)), namely the ability lished data), the ¢rst and second presentations of familiar to classify event types post hoc as a function of the and unfamiliar stimuli were randomly intermixed subject's behaviour. However, one associated problem is to throughout the scanning period. This entailed a range of ensure that the order of the subjectively classi¢ed event lags (number of intervening stimuli) between the ¢rst types remains random (for example, the well-established and second occurrence of a speci¢c stimulus. The para- tendency for stimuli towards the start and end of study metric e¡ect of this repetition lag was examined by multi- lists to be better remembered might mean that remem- plying the event-related covariate for the second bered events tend to co-occur early and late in the scan- occurrence of stimuli by its mean-corrected lag ning period). (½ 5(f )(iii)). Although the main e¡ect of lag is confounded Another example of a subjective classi¢cation of events by time (because larger lags are necessarily associated was reported by Henson et al.(1999a), in which subjects with stimuli occurring towards the start and the end of indicated the nature of their conscious experience when the scanning session), the interaction between lag and trying to remember words studied previously. Words that stimulus familiarity is not (illustrating the value of produced a clear recollection of their study episode were factorial designs; ½ 5(f )(ii)). The same region that showed given a `remember' (R) judgement, whereas words that an interaction between face repetition and face famil- produced a feeling of familiarity in the absence of recoll- iarity also showed a modulation by lag, such that the ection were given a `know' (K) judgement. R judgements response to the second presentation of famous faces were associated with enhanced responses in left anterior increased with lag, whereas the response to the second prefrontal and superior parietal cortices, whereas K presentation of a non-famous face decreased with lag. judgements were associated with enhanced responses in This interaction suggests that both the repetition inhibi- right dorsal prefrontal and bilateral anterior cingulate tion for famous faces and the repetition facilitation for cortices. This study also e¡ected a univariate random- non-famous faces were short-lived. e¡ects inference across subjects (½ 5(h)) by reducing event-related responses to a single parameter (the height (f) Response parameterization of a canonical HRF). A ¢nal example of a di¡erential The use of multiple basis functions can permit tests of e¡ect that can be indexed only by subjectively de¢ned di¡erent aspects of the haemodynamic response. The use events is illustrated by Kleinschmidt et al.(1998),who of temporally shifted gamma functions, for example, scanned subjects while they viewed perceptually bistable allowed Schacter et al. (1997) to di¡erentiate between ¢gures. By comparing button presses made by subjects early- and late-onset HRFs, with the best-¢tting function when a percept was stable with button presses made when for the anterior prefrontal cortex being delayed relative to the percept spontaneously reversed, these authors identi- that for the visual cortex during a visual recognition task ¢ed increased responses associated with reversals in the (although this delay cannot be unequivocally attributed ventral occipital and intraparietal cortex, and decreased to haemodynamic or neural di¡erences; ½ 3(a)(ii)). Less responses in the primary visual cortex and the pulvinar, equivocal latency di¡erences in neuronal activity can be even though the visual stimulus remained constant inferred from di¡erences between ¢tted responses for throughout. Event-related experiments such as these are event types within the same brain region. Friston et al. therefore beginning to identify the neural correlates of (1998a), for example, illustrated how the use of a cano- di¡erent types of conscious experience. nical HRF and its ¢rst-order derivative with respect to time allows one to test for di¡erences in response latency. (d) Factorial responses The use of the derivative of a canonical HRF with Fewer studies have used factorial, event-related designs respect to its dispersion similarly allows one to test for (½ 5(f )(ii)). One example is a study (R. N. A. Henson, di¡erences in response duration.

Phil. Trans. R. Soc. Lond. B(1999) Event-related functional magnetic resonance imaging O. Josephs and R. N. A. Henson 1227

The temporal derivative of a canonical HRF has been CS+ responses are measured in the context of an extinc- used (Henson et al. 1999c) to test for latency di¡erences in tion schedule, a confound explicitly acknowledged by a lexical decision task. Regions that exhibited enhanced such studies (for example, Morris et al. 1998). Event- responses to words relative to non-words (greater height related designs with a partial reinforcement schedule, parameter estimates), including the left angular and left like that of Buechel et al. (1998), permit the separation of inferior temporal cortex, also exhibited shorter latencies intermixed trials of CS+ with the US, and CS+ without (greater derivative parameter estimates). This could the US. re£ect nonlinearities in the underlying haemodynamic response (Vasquez & Noll 1998), such as the non- (h) Responses as random e¡ects independence of response magnitude and response onset, The modelling of event-related responses as random or it could re£ect a true di¡erence in the onset of neural e¡ects is a relatively recent suggestion and is an impor- activity after words and non-words. tant consideration if the variability of e¡ect size (e.g. Another advantage of using the derivative to test for response magnitude) is large relative to the interscan latency di¡erences is that it provides an approximation to variability.This type of analysis has recently been applied the size of the latency di¡erence. By comparison with the (Josephs & Friston 1999) to epileptic spikes (triggered ¢rst-order Taylor expansion of a haemodynamic function, between scans in a `burst-mode' procedure (Josephs et al. f(t), 1999)), which is one situation in which a large inter-

0 response variability would seem likely. Several brain f (t ‡ dt)  f (t) ‡ f (t)dt, regions showed a signi¢cant di¡erence in the mean para- where f ' is the derivative of f with respect to time, the meter estimate for spike events and the mean parameter parameter estimates b and b obtained from ¢tting a estimate for control events. We are currently investigating 1 2 the relative size of inter-response and interscan varia- canonical basis function g1(t) and its temporal derivative g (t) (½ 5(f )(iv)), bility, to address design issues concerning the appropriate 2 number of events and scans for e¡ective statistical power. f (t)  b1g1(t) ‡ b2g2(t), allow an approximation of dt as 8. CONCLUSIONS dt  b /b , Having summarized the important advantages of 2 1 event-related designs and the measurement of event-

(assuming the canonical HRF ¢ts well, i.e. b140). With related haemodynamic responses with the use of echo- the use of this approximation, values of dt have been planar fMRI, we have discussed current issues in the found (Henson et al.1999c) that di¡ered by ca.0.3^3sin modelling, inference and optimization of event-related brain regions that were di¡erentially responsive to words designs. Future development in the analysis of efMRI and non-words. Latencies of this size are likely to re£ect would seem to require further investigation of (i) the haemodynamic rather than neural di¡erences (given that reproducibility of the haemodynamic response, especially reaction times were less than 1s for both word and non- over di¡erent brain regions and di¡erent subjects (½ 3(a)); word decisions). This use of multiple basis functions is an (ii) the degree of nonlinearity in the measured response, alternative to the nonlinear ¢tting of parametrized func- which has particular relevance to the appropriate model- tions, such as the delay and dispersion of a Gaussian ling of experimental variance (½ 4) and the optimization HRF (Krueggel & Von Cramon 1998). of event-related designs with short SOAs (½ 6(c)); and (iii) the appropriateness of modelling event-related (g) Non-stationary responses responses as random, rather than ¢xed, e¡ects (½ 5(h)), An example of the analysis of non-stationary responses which has implications for the statistical power of experi- is provided by Buechel et al. (1998), who sought to charac- mental designs. terize changes in event-related responses during aversive conditioning of a neutral face (CS+) by a loud noise This workwas supported by the Wellcome Trust. We thankKarl (US). By using a partial reinforcement schedule, the main Friston for useful comments on an earlier draft. events of interest were presentations of the CS+ in the absence of the US (unpaired CS+ events). Covariates sensitive to the learning of the CS^US contingency were REFERENCES created by convolving the unpaired CS+ event trains Aguirre, G. K., Zarahn, E. & D'Esposito, M. 1997 Empirical with a canonical HRF and multiplying by an exponen- analyses of BOLD fMRI statistics. II. Spatially smoothed data tially decreasing function of time. The only region collected under null-hypothesis and experimental conditions. showing such a response^time interaction was the left NeuroImage 5,199^212. amygdala, suggesting that this structure shows adaptation Aguirre, G. K., Zarahn, E. & D'Esposito, M. 1998 The varia- to the CS+ during conditioning. bility of human, BOLD hemodynamic responses. NeuroImage This study also illustrates another advantage of event- 8, 360^369. Boynton, G. M., Engel, S. A., Glover, G. H. & Heeger, D. J. related designs: in previous conditioning studies, blocks 1996 Linear systems analysis of functional magnetic reso- of CS+ stimuli have been compared with blocks of CS7 nance imaging data in humanV1. J. Neurosci. 16, 4207^4221. stimuli (stimuli that are never followed by the US). To Buckner, R. L., Goodman, J., Burock, M., Rotte, M., Koustaal, avoid the confound of the US itself, however, these W., Schacter, D., Rosen, B. & Dale, A. M. 1998 Functional^ studies have needed to present CS+ stimuli alone during anatomic correlates of object priming in humans revealed by the scanning window. Unfortunately, this means that rapid presentation event-related fMRI. Neuron 20, 285^296.

Phil. Trans. R. Soc. Lond. B(1999) 1228 O. Josephs and R. N. A. Henson Event-related functional magnetic resonance imaging

Buechel, C., Wise, R. J. S., Mummery, C. J., Poline, J.-B. & Josephs, O. & Friston, K. 1999 Event-related fMRI: which error Friston, K. J. 1996 Nonlinear regression in parametric activa- variance measure for statistical inference? NeuroImage 9,186. tion studies. Ne u roImage 4,60^66. Josephs, O., Turner, R. & Friston, K. 1997 Event-related fMRI. Buechel, C., Morris, J., Dolan, R. J. & Friston, K. J. 1998 Brain Hum. Brain Mapp. 5, 243^248. systems mediating aversive conditioning: an event-related Josephs, O., Lemieux, L., Krakow, K. & Friston, K. 1999 Burst fMRI study. Neuron 20, 947^957. mode event-related fMRI. Ne u roImage 9,219. Burock, M. A., Buckner, R. L., Woldor¡, M. G., Rosen, B. R. & Kleinschmidt, A., Buechel, C., Zeki, S. & Frackowiak, R. S. J. Dale, A. M. 1998 Randomised event-related experimental 1998 Human brain activity during spontaneously reversing designs allow for extremely rapid presentation rates using perception of ambiguous ¢gures. Proc. R. Soc. Lond. B 265, functional MRI. NeuroReport 9, 3735^3739. 2427^2433. Cohen, J. D., Perlstein, W. M., Braver, T. S., Nystrom, L. E., Kruggel, F. & von Cramon, D. Y. 1998 Modeling the hemo- Noll, D. C., Jonides, J. & Smith, E. E. 1997 Temporal dynamic response in single trial fMRI experiments. Technical dynamics of brain activation during a working memory task. report 12/98. Leipzig: Max-Planck-Institute of Cognitive Nature 386,604^607. Neuroscience. Cor¢eld, D. R., Murphy, K., Josephs, O., Adams, L. & Turner, Lee, A. T., Glover, G. H. & Meyer, C. H. 1995 Discrimination R. 1998 Modulation by hypercapnia of activation-related of large venous vessels in time-course spiral blood- BOLD signal. NeuroImage 7, 259. oxygenation-level-dependent magnetic-resonance functional Dale, A. M. & Buckner, R. L. 1997 Selective averaging of neuroimaging. Magn. Reson. Imag. Med. 33, 745^754. rapidly presented individual trials using fMRI. Hum. Brain McCarthy, G., Puce, A., Constable, R. T., Krystal, J. H., Gore, Mapp. 5,329^340. J. C. & Goldman-Rakic, P. 1996 Activation of human D'Esposito, M. D., Zarahn, E. & Aguirre, G. K. 1999 Event- prefrontal cortex during spatial and nonspatial working related functional MRI: implications for cognitive psychology. memory tasks measured by functional MRI. Cerebr. Cortex 6, Psychol. Bull. (In the press.) 600^611. DeYoe, E. A., Neitz, J., Bandettini, P. A., Wong, E. C. & Hyde, McGonigle, D. J., Howseman, A. M., Athwal, B. S., Friston, J. S. 1992 Time-course of event-related MR signal enhance- K.J.,Frackowiak,R.S.J.&Holmes,A.P.1999An ment in visual and motor cortex. In Proceedings, International examination of inter-session di¡erences in fMRI. Ne u roImage Society of Magnetic Resonance in Medicine, Inc. 11th Annual Meeting, 9,38. Berlin, p. 1824. Morris, J. S., Ohman, A. & Dolan, R. J. 1998 Conscious and Friston, K. J., Frith, C. D., Turner, R. & Frackowiak, R. S. J. unconscious emotional learning in the human amygdala. 1995a Characterizing evoked hemodynamics with fMRI. Nature 393, 467^470. NeuroImage 2, 157^165. Pollmann, S., Wiggins, C. J., Norris, D. G., Von Cramon, D. Y. Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. B., Frith, & Schubert, T. 1998 Use of short intertrial intervals in single- C. D. & Frackowiak, R. S. J. 1995b Statistical parametric trial experiments: a 3T fMRI study. NeuroImage 8, 327^339. maps in functional imaging; a general linear approach. Hum. Price, C. J., Veltman, D. J., Ashburner, J., Josephs, O. & Friston, Brain Mapp. 2, 189^210. K. J. 1999 The critical relationship between the timing of Friston, K. J., Fletcher, P., Josephs, O., Holmes, A., Rugg, M. D. stimulus presentation and data acquisition in blocked designs & Turner, R. 1998a Event-related fMRI: characterising with fMRI. NeuroImage.(Inthepress.) di¡erential responses. Ne uroImage 7, 30^40. Richter, W., Andersen, P., Georgopoulos, A. P. & Kim, S. G. Friston, K. J., Josephs, O., Rees, G. & Turner, R. 1998b 1997 Time-resolved fMRI measurement of brain activity Nonlinear event-related responses in fMRI. Magn. Reson. Med. during a delayed motor task. In Proceedings, International Society 39, 41^52. of Magnetic Resonance in Medicine, Inc. 5th Scienti¢c Meeting, Heid, O., Gonner, F. & Schroth, G. 1997 Stochastic functional Vancouver,p.11. MRI. NeuroImage 5, 476. Rosen, B. R., Buckner, R. L. & Dale, A. M. 1998 Event-related Henson, R. N. A., Rugg, M. D., Shallice, T., Josephs, O. & functional MRI: past, present and future. Proc. Natl Acad. Sci. Dolan, R. J.1999a Recollection and familiarity in recognition USA 95,773^780. memory: an event-related fMRI study. J. Neurosci. 19, Schacter, D. L., Buckner, R. L., Koustaal, W., Dale, A. M. & 3962^3972. Rosen, B. R. 1997 Late onset of anterior prefrontal activity Henson, R. N. A., Shallice, T., Price, C., Dolan, R. J., Friston, during true and false recognition: an event-related study. K. & Turner, R. 1999b Lexical decision: di¡erences in magni- NeuroImage 6, 259^269. tude and onset as indexed by event-related fMRI. NeuroImage Strange, B. A., Portas, C. M., Dolan, R. J., Holmes, A. P. & 9, 1044. Friston, K. J. 1999b Random e¡ects analyses for event-related Henson, R. N. A., Buechel, C., Josephs, O. & Friston, K. 1999c fMRI. NeuroImage 9,36. The slice-timing problem in event-related fMRI. NeuroImage Vasquez, A. L. & Noll, D. C. 1998 Nonlinear aspects of the 9,125. BOLD response in functional MRI. NeuroImage 7,108^118. Holmes, A. P. & Friston, K. J. 1998 Generalisability, random Wagner, A. D., Schacter, D. L., Rotte, M., Koutstaal, W., Maril, e¡ects and population inference. NeuroImage 7,754. A., Dale, A. M., Rosen, B. R. & Buckner, R. L. 1998 Building Holmes, A. P., Josephs, O., Buechel, C. & Friston, K. J. 1997 memories: remembering and forgetting of verbal experience as Statistical modelling of low frequency confounds in fMRI. predicted by brain activity. Science 281, 1188^1191. NeuroImage 5,480. Worsley, K. J. & Friston, K. J. 1995 Analysis of fMRI time- Hutton, C., Friston, K. & Turner, R. 1998 The e¡ect of inter- series revisitedöagain. NeuroImage 2,173^181. stimulus interval on signal response in fMRI. NeuroImage 7, Zarahn, E., Aguirre, G. K. & D'Esposito, M. 1997a A trial- 591. based experimental design for fMRI. Ne uroImage 6, 122^138. Johnson, M. K., Nolde, S. F., Mather, M., Kounios, J., Zarahn, E., Aguirre, G. K. & D'Esposito, M. 1997b Empirical Schacter, D. L. & Curran, T. 1997 The similarity of brain analyses of BOLD fMRI statistics. I. Spatially unsmoothed activity associated with true and false recognition depends on data collected under null-hypothesis conditions. NeuroImage 6, test format. Psychol. Sci. 8, 250^257. 179 ^ 197.

Phil. Trans. R. Soc. Lond. B(1999)