Electroencephalography of the Trail Making Test

Zhongmin Lin

A thesis submitted in conformity with the requirements for the degree of Master of Science Department of Medical Biophysics University of Toronto

Electroencephalography of the Trail Making

Zhongmin Lin

Master of Science

Department of Medical Biophysics University of Toronto

2021 Abstract

The Trail Making Test (TMT) is a neuropsychological test consisting of Parts A and B, which involve linking numbers and numbers alternating with letters, respectively. The temporal dynamics of TMT-related behavioural and neural changes remain to be characterized. Sixteen healthy young adults underwent TMT performance with electroencephalography (EEG) recording using tablet technology to capture enhanced performance metrics such as the speed of linking movements, and linking and non-linking periods - the time spent executing and preparing movements, respectively. Seconds per link (SPL) was also employed to evaluate TMT performance. A strong effect of TMT Part A and B was found for SPL values, whereas a strong effect of linking and non-linking periods was found in multiple EEG frequency bands, as well as effects consistent with previous studies of TMT-related brain activity performed by functional magnetic resonance imaging.

Acknowledgments

I would like to indicate my wholehearted appreciation to all the people who have helped me to conduct this research. First, I would like to thank the members of my supervisory committee, Dr. Bradley McIntosh, Dr. Tom Schweizer, and Dr. Fa-Hsuan Lin, for their constructive criticisms, advice, and guidance. I want to especially express my sincere gratitude to my supervisor Dr. Simon Graham for his research guidance and advice over the entirety of this thesis, for unparalleled support, kindness, and encouragement, and for this unique and wonderful experience. In addition, I must thank Dr. Dan Nemrodov for his guidance, as well as Dr. Nathan Churchill for his crucial and timely data analysis support.

Last but not least, I am grateful to my family and my girlfriend for their continual support and encouragement, and patience. A special thanks to Fred, Benson, Nicole, Anton, and Lucy for being the best lab mates in the world.

iii

Table of Contents

Acknowledgments...... iii

Table of Contents ...... iv

List of Tables ...... vi

List of Figures ...... vii

List of Abbreviations ...... ix

Chapter 1 ...... 1

1 Introduction ...... 1

1.1 Electroencephalography (EEG) ...... 1

1.1.1 EEG Oscillations ...... 3

1.1.2 Event-related potentials (ERPs) ...... 7

1.1.3 Time-Frequency Analysis ...... 9

1.1.4 Task Partial Least Squares (PLS) ...... 12

1.2 Trail Making Test (TMT) ...... 15

1.2.1 Behavioural studies ...... 18

1.2.2 Lesion studies...... 20

1.2.3 Neuroimaging studies ...... 23

1.3 Tablet Technology ...... 28

1.4 Application of Tablet Technology in TMT ...... 33

1.5 Hypotheses and Thesis Outline ...... 35

Chapter 2 ...... 37

2 EEG Brain Activity during the Trail Making Test...... 37

2.1 Introduction ...... 37

2.2 Methods...... 40

2.2.1 Participants ...... 40

2.2.2 TMT Design ...... 40 iv

2.2.3 Tablet Technology ...... 42

2.2.4 EEG Recording ...... 43

2.2.5 Tablet Data Analysis ...... 44

2.2.6 EEG Data Analysis ...... 46

2.3 Results ...... 49

2.3.1 TMT Performance ...... 49

2.3.2 EEG Time-Frequency Power ...... 52

2.4 Discussion ...... 56

2.4.1 Behaviour ...... 56

2.4.2 Brain Activity...... 60

Chapter 3 ...... 68

3 Conclusion ...... 68

3.1 Summary ...... 68

3.2 Limitations ...... 69

3.3 Future Directions ...... 70

3.3.1 Simultaneous EEG-fMRI of TMT Performance ...... 71

3.3.2 TMT Performance in Aging and Patient Populations ...... 75

3.4 Improvements to the Tablet Technology ...... 76

3.5 Final Remarks ...... 77

Bibliography ...... 78

List of Tables

Table 2.1 Behavioural metrics for participants (n = 16) performing the tablet TMT*...... 50

List of Figures

Figure 1.1 Time-locked, non-time-locked and phase-locked properties of EEG signals...... 8

Figure 1.2 Example of paper-based Trail Making Test (TMT). Left = TMT-A, Right = TMT-B...... 16

Figure 1.3 Tablet prototype 2 (A) with a stylus (B) and a video camera (C) to enable visual feedback of stimulus response and hand position (D)...... 30

Figure 2.1 Task design for the EEG experiment. See text for details. Fix = fixation, TMT = Trail Making Test...... 40

Figure 2.2 The experimental setup inside (left) and outside (right) the acoustically shielded room. (A) Touch-sensitive tablet, (B) Stylus, (C) Video camera, (D) Visual feedback of stimulus response and hand position, (E) EEG cap and electrodes, (F) Stimulus/response computer, (G) EEG recording computer...... 42

Figure 2.3 Speed time course of the stylus during TMT-A performance, for a representative participant. Linking periods are characterized by rapid acceleration to peak stylus speed, followed by similar deceleration, whereas non-linking periods are characterized by much lower stylus speed. The horizontal line indicates the threshold separating linking and non-linking periods...... 45

Figure 2.4 Task Partial Least Squares (PLS) algorithm for the average EEG time-frequency power...... 48

Figure 2.5 Box and whisker plots of (a) SPL, (b) average link speed, (c) linking period, and (d) non-linking period in TMT-A and TMT-B. Each value was averaged across seven trials. The box represents the interquartile range (IQR), and the top and bottom boundaries of the box represent the third quartile (Q3) and the first quartile (Q1), respectively. The horizontal line in the middle of the box represents the median. The maximum whisker length is 1.5 times IQR, which corresponds to approximately ± 2.7 standard deviation and 99.3% coverage if the data were normally distributed. The top and bottom whiskers extend to the most extreme non-outlier data values within the maximum whisker length above Q3 and below Q1, respectively. The crosshairs

vii

represent outliers, which are data values beyond the maximum whisker length. Circles representing data points of the same participant are connected using a straight line. SPL = seconds per link, * = p < 0.05, ** = p < 0.01, *** = p < 0.001, n.s = p > 0.05...... 51

Figure 2.6 Overall distributions of (a) SPL, (b) average link speed, (c) linking period, and (d) non-linking period across participants for TMT-A and TMT-B. Both TMT-A and TMT-B consist of individual behavioural measures from the seven trials excluding the first trials. The 95% confidence intervals (CIs) of the overall distributions were obtained by a bootstrapping procedure using the histogram values for the individual participants...... 52

Figure 2.7 Omnibus task PLS analysis of EEG power for TMT performance. (a) BSRs of EEG scalp electrodes in the delta, theta, alpha and beta bands (first latent variable). Significant BSRs are shown according to the colour scale given, after correction for multiple comparisons using the false discovery rate (FDR) at q = 0.05 and an additional threshold of |BSR| > 2 to remove results that were unstable during the resampling procedure. The spatial pattern for the gamma band is not shown due to lack of statistical significance. (b) Mean loadings of task condition weights. Error bars indicate standard deviations. (c) Mean loadings of task contrast weights. Error bars indicate standard deviations. L = left, R = right, BSR = bootstrap ratio, Link A = linking period of TMT-A, Link B = linking period of TMT-B, Nonlink A = non-linking period of TMT-A, Nonlink B = non-linking period of TMT-B...... 53

Figure 2.8 The BSRs of the first latent variable in the Link (A+B) vs. Nonlink (A+B) task PLS subtest. All electrode BSRs were FDR-corrected and thresholded as described in the text...... 54

Figure 2.9 The BSRs of the first latent variable in the Link B vs. Link A task PLS subtest. All electrode BSRs were FDR-corrected and thresholded as described in the text. The electrode spatial patterns in alpha, beta, and gamma bands were not significant...... 55

viii

List of Abbreviations

Abbreviations Term

AD Alzheimer’s Disease

ANOVA Analysis of Variance

BOLD Blood Oxygenation Level Dependent

BSR Bootstrap ratio

CI Confidence interval

D-KEFS Delis-Kaplan Executive Function System

DMN Default Mode Network

DTI Diffusion Tensor Imaging

ECG Electrocardiography

EEG Electroencephalography

EMG Electromyography

EOG Electrooculography

EP Evoked potentials

EPI Echo Planar Imaging

EPSP Excitatory postsynaptic potential

ERP Event-related potential

FDR False discovery rate fMRI Functional Magnetic Resonance Imaging

IFG Inferior frontal gyrus

IPSP Inhibitory postsynaptic potential

IPL Inferior parietal lobule

IQ Intelligence quotient

IQR Interquartile range

LED Light emitting diodes

LFP Local field potentials

LV Latent variables

MANOVA Multivariate analysis of variance

MCI Mild Cognitive Impairment

MDD Major Depressive Disorder

MiFG Middle frontal gyrus

MiTG Middle temporal gyrus

MRI Magnetic Resonance Imaging

NPT Neuropsychological test

OCS Oxford Cognitive Screen

PL Polyester laminate

PLS Partial least squares

RF Radiofrequency

REM Rapid eye movement

ROI Region of interest

SMA Supplementary motor area

SNR Signal to noise

SPL Second per link

STFT Short-time Fourier transform

SVD Singular value decomposition

TBI Traumatic brain injury

TMT Trail Making Test

TR Repetition time

USB Universal serial bus

VFHP Visual feedback of hand position

VLSM Voxel-wise lesion symptom mapping

Chapter 1 1 Introduction

This thesis examines the temporal dynamics of mental processes involved during the Trail Making Test (TMT) using a digitizing tablet device and electroencephalography (EEG) recording. The subsequent structure of Chapter 1 provides the necessary introductory material for the reader to understand the background motivation and rationale for the project. The current status of the research field and the major gaps in knowledge are addressed. Sufficient review is provided for the reader to understand the hypotheses that will be tested in thesis.

1.1 Electroencephalography (EEG)

In 1875, Richard Caton, a physician from Liverpool, found an electrical phenomenon on the exposed brain of rabbits and monkeys1. Based on Dr. Caton’s work, Hans Berger recorded the first human electroencephalography (EEG) signal in 1924. Since then, EEG and other technological developments have advanced our knowledge about human brain function. Today, modern EEG electrode design and data acquisition systems permit signals to be recorded simultaneously from billions of neurons spanning the whole brain. Continuing advancements in computing power have also enabled complex analyses of EEG signals that were difficult or impossible to implement previously.

Neurons in the brain are electrically charged due to the unequal distribution of ions inside and outside their cell membranes, and constantly undergo ion exchange with the extracellular environment via transmembrane ion pumps. Interconnected via gap junctions and synapses, the neurons enable effective transmission of electrical signals throughout the central nervous system. Action potentials and local field potentials are two types of such neural activity, each with different spatial and temporal properties2. Action potentials are generated by the rapid depolarization of the neuronal membrane when the membrane exceeds a certain voltage threshold, leading to a change of about 110 mV in the membrane potential within 1 ms3. The rapid depolarization produces an impulse or spike that propagates with minimal amplitude decay along the axon. Action potentials form the basis of fast neural communication that mediates mental processes such as precise sensorimotor control in humans. This type of neural activity is most evident in intracellular recordings and exhibits at frequencies above 300 Hz in extracellular

1 2 recordings4. However, action potentials have a very brief duration (1-2 ms) and have negligible contribution to scalp EEG signals due to poor temporal constructive summation and highly localized amplitudes that strongly attenuate outside a 50 µm radius4,5. Unlike action potentials, local field potentials (LFPs) exhibit lower frequency content (<250 Hz) and smaller changes in the membrane potential in response to synaptic activation. There are the two types of LFPs received by neurons as synaptic inputs: excitatory postsynaptic potentials (EPSPs), which act to increase the membrane potential of the postsynaptic neuron toward the action potential threshold, and inhibitory postsynaptic potentials (IPSPs), which act to further decrease the membrane potential. When the summation of these LFPs exceeds the threshold of membrane potential in the postsynaptic neuron, the neuron becomes active and the voltage-gated ion channels in the membrane rapidly open, causing the membrane potential to depolarize rapidly. Then, an action potential propagates down to the synaptic connections with subsequent neurons.

The LFPs generated through synaptic connections in the grey matter also exhibit more temporal and spatial summation in the postsynaptic neuron compared to the action potentials. In the synaptic cleft, an active presynaptic neuron releases vesicles containing neurotransmitters, which bind to neurotransmitter-gated ion channels on the membrane of the postsynaptic neuron, triggering ion exchange between the postsynaptic neuron and extracellular space. The resulting change in membrane potential creates electromagnetic fields surrounding the postsynaptic neuron. However, the electromagnetic field generated by a single postsynaptic neuron (which receives inputs from multiple synapses) is far too small to be detected by electrodes located remotely on the scalp. For scalp electrodes to receive a detectable EEG signal, tens of thousands of neurons with similar spatial orientation must be activated synchronously to produce a powerful electromagnetic field.

Pertinent to this requirement, pyramidal neurons are found in brain regions such as the cerebral cortex and the hippocampus. Cortical pyramidal neurons have a conic shaped cell body, large apical dendrites and are aligned perpendicular to the cortical surface. Given the proximity of cortex to the scalp surface, as well as the high connectivity and uniform spatial orientation among cortical neurons, neuroscientists believe that the cerebral cortex is the primary structure to generate scalp EEG signals6,7. When populations of pyramidal neurons become synchronously activated by LFPs or other mechanisms, the weak electromagnetic fields created by each active neuron sum and become strong enough to be detected on the scalp with the presence of barriers

3 of brain tissue, fluid, skull, and skin. Subsequently, the coherent, time-varying, and relatively powerful electromagnetic fields can be recorded by EEG scalp electrodes2.

The EEG signal is thus a measure of cortical electrical activity at a mesoscopic- to macroscopic- level. The temporal dynamics of cortical synaptic action occur on the millisecond time scale and can be captured by EEG (i.e. with excellent temporal resolution). However, the electromagnetic fields created by synchronous synaptic response decrease with distance, which makes detecting neural activities from deeper subcortical structures than the cerebral cortex difficult8, limiting the sensitivity and spatial resolution of EEG. With cerebrospinal fluid, the skull, and the scalp in between EEG electrodes and current sources inside the brain, it is difficult to discern these sources spatially. Due to the low conductivity, the skull may also distort the electrical potential distribution detected on the scalp, compared to the actual distribution in the brain. Other than the inherent constraints of non-invasiveness, EEG hardware also has an impact on the spatial resolution. In low-noise measurement environment, traditional EEG systems with low channel count (20 or lower) demonstrate poorer spatial resolution than dense electrode systems (64 or higher)9.

There are several common approaches to examine and interpret EEG signals, for example: 1) by analyzing the oscillatory signal content in characteristic frequency bands; 2) by generating event- related potentials (ERPs) or evoked potentials (EPs) in response to brief, precisely timed stimuli; and 3) by using time-frequency decomposition. The EEG oscillatory activity was discovered in the first report of human EEG recordings10. Later, different oscillations were grouped into frequency bands based on their localization and function. With appropriate experiment design, ERPs or EPs can be obtained by averaging multiple recordings of a transient response to the same event or stimuli. From these data, the phase- and time-locked spatiotemporal dynamics of the scalp potentials can be extracted around the stimulus or event onset. As a supplement or alternative, the time-frequency decomposition method may reveal spectral changes that are neither phase-locked nor time-locked. Each of these three approaches is described in more detail immediately below.

1.1.1 EEG Oscillations

The brain oscillations recorded by EEG are produced by synchronous neural activity, reflecting fluctuations in the excitability of neuron ensembles. These oscillations are ubiquitous

4 neurophysiological phenomena which support various aspects of brain function at synaptic, cellular, and system levels11. Typically, EEG oscillations are characterized by frequency, power, and phase. Frequency describes the rapidity of the oscillation in cycles per second (Hz); power describes to the time-averaged amount of energy in a frequency band using the squared amplitude of the oscillation (휇푉2); and phase (radians or degrees) indicates the temporal offset of the oscillatory wave relative to the idealized start of the wave cycle (time zero). The spectral content of oscillations recorded in the EEG signal is generally divided into five frequency bands (delta, theta, alpha, beta, gamma) ranging approximately from 0.3 to 100 Hz, based on neurobiological mechanisms of oscillation such as synaptic decay and signal transmission dynamics12–16. In the EEG literature, each frequency band has been associated with different brain states, such as those providing cognitive and sensorimotor functions12,15,16. Frequency bands in the sub-delta and supra-gamma ranges (up to 600 Hz) have also been identified, which have yet to demonstrate a clear link to cognitive processes17. Notably, neural oscillations at these frequency bands are not completely independent. They can occur simultaneously and interact with one another, leading to mixtures of brain states4. In addition, there are neither precise boundaries to define the frequency bands nor consistent grouping standards in the field of EEG research. Inter-individual variability in the exact frequency of oscillatory activity has been shown to relate to certain brain functions such as attention18, placing some limits on how EEG results are interpreted. With these provisos, the five major frequency bands are briefly described below.

Delta: Delta waves are high amplitude neural oscillations with a frequency range between 0.3-4 Hz. In EEG recordings of healthy adults, they are found in the midline cingulate structures and during stage 3 and 4 of non-rapid eye movement (REM) sleep, which characterizes deep sleep19. Delta waves are the predominant EEG oscillations in infants, who spend considerable time sleeping, but it is also normal to observe delta waves in infants during wakefulness7. The appearance of delta waves in adults indicates a pathological state, however. For example, sleep deprivation leads to increased delta wave amplitude. Simultaneous EEG and functional magnetic resonance imaging (fMRI) research has shown that significant increases in blood oxygen level dependent (BOLD) response associated with delta and slow waves are found in brain areas such as the precuneus, posterior cingulate, inferior frontal, and medial prefrontal cortex20. These areas partially overlap with the default mode network (DMN)21, a network of regions that are engaged during wakeful rest and self-reflections, as well as abstractions of the outer world without

5 external input22,23. Therefore, delta waves may partially be a concomitant phenomenon of DMN activity.

Theta: Theta waves are neural oscillations with a frequency range between 4-8 Hz. These oscillations are also more predominant during infancy and childhood, and their appearance in adults indicates drowsiness and sleep, especially deep sleep stages. Theta waves are mainly found in the hippocampus24, sensory cortex25, prefrontal cortex26, and dorsal anterior cingulate cortex27. Hippocampal theta waves are associated with the state of drowsiness and sleep, whereas cingulate theta waves are associated with working memory performance28,29. Theta waves are thought to support multiple functions including memory30,31, synaptic plasticity32, top-down control33, and long-range synchronization (between distant brain regions)33. Using simultaneous EEG-fMRI experiments in awake adults, positive correlations between theta waves and BOLD responses were found in the insular cortex, hippocampus and the cingulate cortex34. However, other studies either found no such correlation35, or only negative correlations in areas that partially overlapped with the DMN36, suggesting the decreased alertness37.

Alpha: First discovered by Hans Berger along with beta waves in the 1920s, alpha waves are neural oscillations with a frequency range between 8-13 Hz10. Regardless of the long history of investigation, the neuronal generators of alpha waves and their functional significance remain to be clearly understood12,38. Alpha waves are most evident in the occipital cortex during wakeful rest, suggesting that the alpha wave is an “idling rhythm” associated with sensory input suppression39. This theory is further supported by the observation of reduced alpha waves when the eyes are open (versus eyes closed) and during mental activity (versus rest). However, the increase of alpha waves in response to cognitive and memory performance also challenges the idling rhythm theory28,40–42. Alpha waves have been found in the thalamus43, hippocampus44, reticular formation44, and sensory cortex10. Alpha waves found in motor-related areas, such as motor cortex45 and supplementary motor area46, are specifically designated as mu rhythms, whereas those observed in the primary auditory cortex are designed as tau rhythms47,48. Alpha waves located in different brain regions have also been reported to exhibit distinct functional roles in inhibition38, attention49, consciousness50, top-down control33, and long-range synchronization51. Known sources of alpha rhythm are the thalamic pacemaker cells52 and various cortical areas with different alpha frequency characteristics2,53. Through neural networks, the cortical regions interact not only with each other52, but also with thalamic structures54,

6 providing a potential source for cortical alpha waves4. However, alpha waves in the occipital cortex (marginal gyrus) have demonstrated larger coherence with other cortical alpha sources than with thalamic sources (lateral geniculate nucleus) in dogs, suggesting that intra-cortical interactions play a crucial role in propagating alpha waves throughout the cerebral cortex2,55,56. Alpha waves also demonstrate different spatial distributions during wakefulness and deep sleep57. Simultaneous EEG-fMRI studies consistently have found that alpha power is positively correlated with BOLD signal in the thalamus, whereas negative correlations are observed in occipital, parietal, and frontal regions37,53,58–60 that overlap with the DMN21. Such results support that the thalamus is a key alpha wave generator.

Beta: With lower amplitudes than alpha waves, beta waves (13-30 Hz) are considered the arousal response of the cortex, that suppresses slower waves in the state of elevated alertness12,61. Beta waves are found throughout the cerebral cortex and in subcortical structures such as the subthalamic nucleus62, basal ganglia62, and olfactory bulb63. Originally suggested by Hans Berger to indicate mental activity64, beta waves have also shown associations with brain functions such as sensory gating65, attention66,67, motor control68, and long-range synchronization69. Cortical beta waves have distinct functional significance depending on the signal topology. Beta waves originating from the prefrontal cortex are associated with working memory encoding, retention, retrieval, and reallocation70–72. Beta waves from the primary motor cortex during motor activity have been shown to associate the sensory input with the motor command47,73,74, and networks within the basal ganglia are responsible for generating transient beta bursts75–77. Furthermore, simultaneous EEG-fMRI studies have shown positive correlation between beta power and BOLD signal in the posterior cingulate, precuneus, and prefrontal cortex35, which overlaps with the DMN21. This suggests that beta waves play an important role in the resting state. Notably, the cortical beta waves associated with the resting state likely have distinct sources in comparison to motor-related beta waves78.

Gamma: Gamma waves (30-100 Hz) are the most extensively studied neural oscillations with respect to sensory and cognitive brain function. A dominating gamma rhythm indicates neural network engagement, and high neuronal excitability (i.e. increased sensitivity to synaptic inputs) during complex cognitive processes. Primarily using animal studies, gamma waves have been found to be distributed across various brain structures, including the retina79 and olfactory bulb80, playing a role in perception81, attention82, memory83, consciousness84, and synaptic plasticity85.

Despite the popularity of studying gamma waves consequent to their prominent role in the theory of feature binding (i.e. gamma waves are thought to link stimulus features to form a unified, conscious percept)81, there has been an ongoing debate over their functional significance. Some research supports the feature binding mechanisms of gamma waves and their aggregation of brain functions through long-range synchronization86,87. Alternatively, the findings that gamma waves are intrinsic network properties during sleep88,89, as well as artifacts of stimulus evoked saccades90, and facial muscle artifacts during cognitive tasks91,92 challenge the feature binding theory. The latter reports emphasize in particular that the contribution of non-neuronal electrical activity to the EEG signal needs to be recognized, and carefully suppressed using topographic analyses and source estimates to differentiate scalp signal artifacts from generators in the brain4,93. Although the cortical generators of gamma waves still remain to be confirmed in humans, animal studies have revealed gamma rhythm generators in the somatosensory cortex94 and in the primary visual cortex95.

1.1.2 Event-related potentials (ERPs)

Event-related potentials (ERPs) are relatively weak electrical fluctuations in the scalp EEG signal evoked by certain stimuli or events96. When generated solely by a sensory stimulus, they are designated as an evoked potential (EP). Over many decades, ERPs have been used extensively to study the neuropsychological correlates and the timing of various behavioural and mental processes. Like other EEG scalp potentials, ERPs also reflect the synchronized synaptic response produced by LFPs. Because a single ERP has a small amplitude (~1 µV), whereas the variability of the background EEG signal is about 10 µV in healthy adults, the signal-to-noise ratio (SNR) of each trial in an experiment is extremely low. In an ERP experiment, therefore, a stimulus is typically repeated up to hundreds of times to generate many ERP trials. These ERP trials are subsequently averaged to increase the SNR and to allow for visualizing the ERP of interest with the minimized background noise and preserved time-locked changes in the scalp potential in response to sensory input, motor output, or cognitive activity. The time-locked changes refer to signal amplitude effects with highly consistent latency across many trials, whereas phase-locked changes refer to effects with consistent phase synchrony across trials, such that oscillations of EEG subcomponents add constructively (Figure 1.1). Both time-locked and phase-locked changes are only discernable with careful data acquisition and averaging that is

8 time-synchronized to a trigger that is positioned precisely, appropriately and consistently within each experimental trial (e.g. 500 ms before sensory stimulus onset).

Figure 1.1 Time-locked, non-time-locked and phase-locked properties of EEG signals.

Typically, the nomenclature of ERPs is based on the individual peaks or components in the waveforms, and is characterized by features such as polarity and latency. For example, N200 refers to a negative wave deflection that occurs 200 ms after the stimulus presentation. Alternatively, researchers also name ERP components based on the ordinal position after stimulus presentation, with separate counting for positive and negative peaks. For example, P1, N1, and P2 denote the first positive component after the stimulus, first negative component, and second positive component, respectively97. Generally, components within 100 ms after stimulus

9 onset indicate early sensory perception, those occurring between 100 - 200 ms reflect a mixture of late sensory and early cognitive processes, and those after 250 ms may be associated with higher-order cognitive processes or behavioural responses, depending on the experimental design.

As a time-domain EEG measurement, ERPs are advantageous for their simplicity, high temporal precision, and extensive literature for comparison and data quality checks17. The computation of ERPs requires minimal parameters and data processing, allowing fast and easy visualization of main effects, and very precise estimation of scalp potentials at each time point, permitting more accurate latency estimates of ERP components than is achievable by other EEG processing methods (such as time-frequency analysis, summarized below). However, ERPs have several disadvantages such as the need for large number of trials, which leads to longer experiments and more artifacts98. To meaningfully interpret ERP findings, at least 40 trials are required in a typical ERP study involving 20 participants99. To collect such large number of trials, long experiments are needed, leading to fatigue, decreased attention and concentration in participants. These declines impact the cognitive performance of participants and may produce more EEG artifacts such as oculomotor and muscle artifacts.

1.1.3 Time-Frequency Analysis

Besides time-locked and phase-locked signals visible in ERPs, EEG recordings also contain other dynamic signals that be may relevant to certain behaviours and mental activities. In the context of this discussion, temporal resolution, precision, and accuracy are important parameters that should be defined and distinguished. Temporal resolution is determined by the number of data samples per unit time; temporal precision refers to the certainty of the measurement at each time point; and temporal accuracy describes the consistency between the timing of the EEG signal and the underlying biophysical events.

In addition, the oscillatory activity in different frequency bands may also have implications for brain states and behaviours. The Fourier transform is a well-known method to obtain a frequency-domain representation of a time-dependent signal. However, Fourier transformation is only valid under the assumptions that the time-dependent data are stationary stochastic signals and that the activity in different frequencies is constant throughout the whole signal. Unfortunately, both of these assumptions are violated for EEG signals. To overcome these

10 limitations, time-frequency analysis is employed to decompose the spectral content of EEG signals while preserving the temporal dynamics.

In this thesis, Morlet wavelets are used for time-frequency decomposition. Named after a French geophysicist Jean Morlet, the approach involves convolving the EEG signal with complex Morlet wavelets (kernels). A Morlet wavelet is created by multiplying a sine wave of a certain frequency by a Gaussian taper window, such that the sine wave gradually decreases to zero at both ends, providing a bandpass filter in convolution. Due to their shape, Morlet wavelets are relatively free from edge artifacts and focus on the frequency characteristics around the center time point.

In Morlet wavelet convolution, there is a trade-off between temporal and frequency precision: a temporally precise Gaussian window produces a Morlet wavelet with low frequency precision (i.e. wide frequency spectrum) and vice versa. However, depending on the experimental design and hypotheses, the wavelet can usually be adjusted to yield acceptable trade-offs. This is achieved by manipulating the number of cycles in the Gaussian window17. Furthermore, the use of a complex Morlet wavelet containing real and imaginary parts enables extraction of the bandpass-filtered signal at peak frequency, as well as time-frequency power and phase information. Thus, the complex Morlet wavelet is well suited to extract spectral information that varies with time, as observed in EEG signals.

Alternatively, time-frequency power and phase information can be obtained using the filter- Hilbert method, which involves passing the EEG signal through a bandpass filter (with a certain frequency range) and computing the Hilbert transform on the filtered signal, thus determining how the bandpass frequencies are amplitude-modulated. With a long history in sound wave analysis100, short-time Fourier transform (STFT) is also applicable to extract time-frequency power from the EEG signal. To preserve temporal dynamics while extracting frequency information using the Fourier transform, the EEG signal is segmented into short signal segments by sliding a time window along the time-course a few milliseconds at a time. In each signal segment, the time-frequency power is obtained by combining the frequency power spectrums at all frequencies of interest. These three approaches differ in the number of analysis parameters, computation time, temporal resolution, and frequency spectral shape. The complex Morlet wavelet convolution requires less parameters and shorter computation time than the filter-Hilbert

11 method and the STFT, and is the approach adopted in this thesis. It should be noted, however, that the filter-Hilbert method has more fine control over the spectral envelope of specified frequencies than the wavelet convolution method. Specifically, the filter-Hilbert method allows flexible adjustments of the frequency characteristics of the filter such as range and boundaries, whereas the frequency spectrum of a complex Morlet wavelet is always a Gaussian (although in principal different families of wavelet could also be applied). Typically, the time-frequency power produced by STFT has a lower temporal resolution than the other two approaches, because the time window moves a few milliseconds at a time rather than time point by time point. Increasing the temporal resolution by reducing the step of the time window will lead to a steep increase in computation time. Despite their differences in implementation, the three methods produce highly consistent results if appropriately applied101.

Although not the perfect analysis in all circumstances, conceptualizing EEG signals as multidimensional data with frequency as a prominent dimension allows joint analyses with behavioural performance, experimental conditions, patient groups and other neurophysiological processes. The time-frequency analysis has two advantages: result interpretability and task- related dynamics. With extensive literature, neural oscillations and their underlying neurophysiological mechanisms interpreted from time-frequency results may bridge findings across species and disciplines. Because the EEG signal dynamics are well preserved in time- frequency results, regardless of whether the activity is time- and phase-locked or not, there are more opportunities to extract and analyze task-related cognitive processes from the highly complex EEG signal. However, time-frequency analysis also has limitations. As mentioned briefly above, the use of temporal filtering can lead to decreased temporal precision. Lower EEG frequency bands, such as delta and theta waves, are generally affected more heavily by temporal smoothing and filtering, and thus suffer more loss of precision than the high frequency bands such as beta and gamma waves17. Regardless, the temporal precision of EEG time-frequency analysis is always better than that of functional magnetic resonance imaging (fMRI) and poorer than that of ERPs. Compared to the generation of ERPs, time-frequency analysis is mathematically complex to implement, making it more susceptible to human errors, suboptimal analysis choices, and inappropriate result interpretations. In recent years, however, the time- frequency analysis has been increasingly adopted in the EEG literature, given the utility of the method.

1.1.4 Task Partial Least Squares (PLS)

In general, brain signal can be highly noisy, but multivariate techniques can exploit the fact that brain regions are highly correlated, in order to improve model power and identify distributed networks. A key element of this thesis involves partitioning multiple effects of interest from noise, to improve understanding of the brain activity associated with TMT performance. It is evident from the review given above that effects in the EEG data are likely to be multivariate in nature, given the dependence of the EEG signals on time, frequency, space (electrode) and task- related behaviour. Furthermore, patterns (or features) of covariance are expected in the EEG signals. For example, substantial neural activity at the cortex is likely to be identified across multiple electrode locations. The question thus becomes how to identify covariances that represent TMT-related features of interest, from those that are of no interest. This is difficult to answer using univariate analyses such as such as t-test and Analysis of Variance (ANOVA), which would require difficult assumptions about what aspects of the EEG data can be considered statistically independent variables (and subsequent corrections for multiple statistical comparisons). Another alternative is to consider common multivariate tests, such as the Multivariate Analysis of Variance (MANOVA) – although the MANOVA is restricted by assumptions such as multivariate normality, homogeneity of covariance matrices, and collinearity, and can be hard to interpret in multi-factor analysis and within-participant design102,103. A much better choice in the present context is to use a data-driven multivariate test is to fully capture the effects of interest. Partial Least Squares (PLS) provides a more flexible multivariate model that can be used to capture these effects of interest. This technique is based on eigen-decompositions of correlation and covariance matrices, and is therefore well-suited to high-correlated dependent variables, such as EEG time-frequency power and fMRI signals, as well as to data with more dependent variables (e.g. electrode location, time, frequency) than observations (e.g. participants)104. In addition, PLS combined with non-parametric resampling approaches such as permutation tests and bootstrap estimates can evaluate effects while making minimal assumptions about data distributions.

The name “PLS” stems from computing the least-squares fit of part of a covariance or correlation matrix105. Depending on the type of covariance matrix that is generated, PLS can be used to identify how brain activity covaries with task (Task PLS), behaviour performance (Behaviour PLS), functional connectivity (Seed PLS), and brain-behaviour correlations

(Correlation PLS). Task PLS is subsequently used in this thesis as described below. In general, PLS is used to uncover the relations between two input matrices (X and Y) that are used to generate a covariance or correlation matrix, by identifying sets of paired latent variables (LVs) derived from X and Y that show maximized covariance. For task PLS as implemented in this thesis, the covariance matrix is calculated from brain data of different task conditions assigned as the X matrix, and task conditions considered as the Y matrix (loading vectors). Specifically, the X matrix is formed by EEG time-frequency data (ordered as (task condition × observation) × [time-frequency variables]) and the Y matrix is formed by binary values denoting the task type for each observation (ordered as (task condition × observation) × task conditions].

After the X and Y matrices are input to the PLS algorithm, the analysis procedure is conducted in multiple steps. First, for each data column, the grand mean is subtracted from the X matrix (i.e. to “mean-center” the data), and the result is then divided by the standard deviation. The Y matrix is then normalized by dividing elements in a given column by the sum of each data column. Then, the effect space E is derived using the XT × Y cross product matrix, which contains mean- centered task-related effects in X matrix as task condition means. Singular value decomposition (SVD) is used to factorize E into its most statistically descriptive factors with constraints on task conditions, by finding the eigenvalues and eigenvectors of E × ET and ET × E. The SVD projects the effect space into a coordinate system where the covariance matrix is diagonal and decomposes the effect space into three matrices U, L, and V:

푈 × 퐿 × 푉푇 = 푋푇 × 푌 where the kth component has a latent variable pair:

퐿푥(푘) = 푋 × 푢(푘), 퐿푦(푘) = 푌 × 푣(푘)

The left singular vector U is a unitary matrix in the spatial dimension, whose conjugate transpose is equal to its inverse. The right singular vector V is a unitary matrix in the task dimension. The singular value (L) is a rectangular diagonal matrix with non-negative real numbers on descending order on the diagonal. In PLS parlance, vectors u(k) and v(k) from U and V contain the eigenvectors of E × ET and ET × E, indicating “spatial and task saliences”, respectively, whereas the singular values L(k) in L are the square roots of the eigenvalues from E × ET or ET × E, reflecting the variance explained by saliences in the LVs. These saliences are typically

14 interpreted as the underlying brain network pattern of task conditions. Notably, the PLS algorithm employed is distinct from permutation resampling, where PLS is repeated with randomized data, as well as bootstrapping, where PLS is repeated after resampling participants with replacement.

Second, resampling is conducted to obtain non-parametric statistics describing LVs and saliences such as p-values, mean loadings, and standard deviations, by permuting eigenvalues and by bootstrapping eigenvectors and eigenvalues. It is necessary to determine which LVs are statistically significant by calculating the permutation p-value of variance using permutation testing. For each resampling iteration, the permutation test constructs a non-parametric null distribution on some test statistic. This null distribution is specifically designed to match the dataset of interest, by scrambling the relationship between X and Y data, and calculating the test statistic, repeated many times. The test statistic is then compared between the “unscrambled” real data to the null distribution, to determine the likelihood of such change to occur by chance. In this case, we examine the eigenvalue which summarizes how much variance is captured by the eigenvectors. The p-value is the fraction of “null” eigenvalues that are at least as large as the “unscrambled” value. These p-values were then averaged across resamples to indicate the statistical significance of each LV.

By generating statistics, bootstrap resampling is used to assess whether nonzero saliences within significant LVs are significant across the participants studied, thus estimating population effects. By resampling from dataset across participants with replacement many times, bootstrapping allows us to non-parametrically obtain the 95% confidence intervals (the interval containing 95% of resamples), bootstrap ratios (BSRs, mean loadings divided by standard deviations) and p- values of statistical significance, based on the fraction of resamples that overlaps with zero effect. These statistics were then averaged across resamples to indicate the reliability of spatial and task saliences. Notably, the mean loadings of saliences are unitless because the multiplication between eigenvalues (variance explained) and eigenvectors (spatial and task saliences) gives the proportion of the covariance and as such, the saliences are inherently normalized by the procedure identified above.

Interpreting task PLS results requires multiplication between spatial and task BSRs. The sign and magnitude of this operation reflect the direction and size of the task condition expression relative

15 to the grand mean of the overall covariance matrix (the effect space). For instance, negative spatial BSRs multiplied by positive task BSRs in a task condition produce negative expression of covariance. In the case of EEG average time-frequency power, negative expression indicates increased desynchronization. The desynchronization in EEG oscillations such as event-related desynchronization has been associated with higher cognitive processes such as long-term memory, sensory perception, and motor movement46,106. On the other hand, synchronous oscillations between neuron assemblies (type 1 synchronization) indicate a resting state or an inhibition state, whereas that between neuronal networks (type 2 synchronization) reflect mental activity such as short-term memory106.

1.2 Trail Making Test (TMT)

With EEG signals and analysis briefly described, the introduction of the thesis next discusses how this functional neuroimaging modality can be used to probe the neural correlates of a specific behavioural task, the trail making test (TMT) - and why this is of scientific interest. Typically administered by pen and paper, the TMT is a neuropsychological test (NPT) originally developed by psychologists in the U.S. military in 1944 for assessing general intelligence107. Later, the TMT became widely used in behavioural neuroscience and the clinic as part of neuropsychological test batteries to assess frontal lobe function, cognitive impairments due to brain injury, and to assist in diagnosis of brain disease108–115. The test assesses cognitive processes such as visual search, visual planning, visuomotor control, as well as attention and memory113,115–118. As a clinical tool to probe cognition, the sensitivity of the TMT is excellent but the specificity is somewhat limited113. The sensitivity likely arises because damage to one or more brain regions, or their connections, could lead to performance decrements; the specificity problem arises because the tests probe many of the same brain regions, and many of these regions can be damaged by more than one brain disease.

As shown in Figure 1.2, the TMT consists of two parts (A and B), each which involve linking a total of 25 randomly-placed items in an ascending order: part A involves linking numbers (i.e. 1- 2-3-4-5...); and part B, which is more challenging, involves linking numbers alternating with letters (i.e. 1-A-2-B-3-C...). The test administrator instructs participants to complete the test as fast and accurately as possible without lifting the pen from the paper113,117,119–122. Although a specific time limit is not usually imposed, an upper limit of 300 s could be used to terminate the

16 test in practice122. Throughout, the test administrator monitors the task performance and corrects errors when they occur, so that the participant can continue.

Figure 1.2 Example of paper-based Trail Making Test (TMT). Left = TMT-A, Right = TMT-B.

Each TMT part is then typically scored by recording the completion time and the number of errors. Notably, TMT-B typically requires more time to complete than TMT-A. As a baseline measure for psychomotor speed, visual search, and motor tracking, TMT-A is administered before TMT-B. On the other hand, TMT-B also includes these elements with increased demands on higher cognitive functions (such as set-switching), and has been suggested to be more sensitive to impairments in prefrontal and frontal regions as a consequence123. To isolate the factor of higher cognitive functions at the behavioural level and probe cognitive deficits, derived TMT indices are commonly employed such as the difference (B-A), the ratio (B/A), and the proportion [(B-A)/A]116,120,122,124. The difference score (B-A) eliminates the speed component from TMT scores113 and reflects higher cognitive processes more accurately by suppressing the contribution of low-level processes such as visual perception and motor movements125. A large TMT difference score may reflect the impaired ability to modify an action plan and maintain two response sets simultaneously123. Similarly, the ratio score (B/A) also captures cognitive flexibility by minimizing the influence of motor movements and accounting for within- participant variability126. The proportional score [(B-A)/A] has been thought to sensitively reflect prefrontal functions (although it should be recognized that this score should have the same

17 behavioural assessment properties as B/A)116. Whereas TMT completion time scores are susceptible to influences from between-participant factors such as age, education, intelligence quotient (IQ), as well as environmental and demographic factors113,117,127, these factors have much less effect on the derived TMT indices126,128.

Normative data on direct scores129, derived indices130, or both131 have been collected to assist neuropsychological assessment using TMT, and to identify abnormal TMT performance in relation to the associated population distributions. In addition to the completion time data, the TMT errors are an important measure of performance accuracy. Although not mandated in the standard scoring protocol122, researchers have developed many outcome measures that account for the type or number of errors, such as TMT-B shifting errors (failure to alternate between numbers and letters, i.e. 1-2 or A-B), TMT-B sequencing errors (failure to alternate in the correct order, i.e. 1-A-3)132, and seconds per link (average time per correct link)133. However, there are mixed beliefs regarding the clinical utility of TMT error analysis. Some researchers report that TMT error type increases the specificity of detecting cognitive impairment in clinical populations116,134,135, whereas other researchers remain skeptical132,136. Besides the traditional TMT with part A and part B, other variants of the TMT have also been developed and employed, such as the Delis-Kaplan Executive Function System (D-KEFS) TMT137 and the Oxford Cognitive Screen (OCS) TMT138. In the D-KEFS TMT, distinct cognitive processes mixed in the traditional TMT are divided into separate subtasks for improved detection, and distractor stimuli are added to emphasize response inhibition (which may be underrepresented in the traditional TMT-B). The D-KEFS consists of five subtasks: visual scanning, number sequencing, letter sequencing, number-letter switching, and motor speed; with a larger stimulus presentation area allowing longer trails and more distractor stimuli than the traditional TMT. As the names suggest, visual scanning involves crossing out all the number 3s distributed across the response sheet; number sequencing requires linking numbers from 1 to 16 under the distraction of distractor letters (analogous to the traditional TMT-A); letter sequencing involves linking letters from A to P under the distraction of distractor numbers; number-letter switching involves linking numbers alternating with letters (i.e. 1-A-2-B-…-16-P) with a total of 32 items (analogous to the traditional TMT-B); while motor speed requires tracing a dotted line to link circles on the sheet as fast as possible. Scoring is achieved using completions times of the individual subtasks. As a variant of the TMT, the D-KEFS TMT has been demonstrated to detect cognitive impairments

18 related to the frontal lobe in various clinical populations such as fetal alcohol syndrome139, autistic and Asperger’s disorder140, frontal lobe epilepsy141, and focal frontal lesion142,143.

Alternatively, the OCS TMT is specifically designed to assess cognition in stroke patients. The OCS TMT replaces letters and numbers in the traditional TMT with shape-based stimuli such as circles and squares, to increase detection sensitivity on set-switching impairments while reducing the demands for numerical and language processing (i.e. number and letter sequencing). The OCS TMT involves linking 7 target shapes (circle/square/triangle) from large to small under the influence of non-target shape distractors in the two baseline tests (analogous to TMT-A), and linking both target shapes in baseline tests alternately with a total of 14 shapes from large to small in the set-switching test (analogous to TMT-B). Importantly, the test is administered in a self-paced manner to ensure that a low performance accuracy purely reflects cognitive impairments. All tests in the OCS TMT are scored by the number of correct links (i.e. 0-6 for the baseline tests, 0-13 for the set-switching test).

1.2.1 Behavioural studies

By examining the relations among outcome measures of TMT and other NPTs, the neuropsychological correlates and validity of the TMT were indirectly evaluated in many behavioural studies. Regarding the similarities between TMT part A and B, significant correlations were observed between their test performance in healthy individuals (n = 34)123, patients with possible cognitive deficit (n = 121)144, and stroke (n = 106)124, indicating similar low-level mental processes in both TMT parts, such as visual perception and motor movement. In terms of the differences between TMT part A and B, researchers also proposed several contributing factors to their difference in difficulty, such as symbolic complexity, spatial arrangement, number-letter interactions145, which led to an increased demand of visual search and motor movement in TMT-B compared to TMT-A121. More importantly, the differences in cognitive demands between the two TMT parts are the most clinically relevant factor that contributes to the difference in difficulty117,121. Therefore, to further isolate the higher cognitive components, the different demands in visual search and motor movement between the two parts can be addressed by creating a test counterpart with same item spatial arrangement for each test. For example, a new TMT-B counterpart can be created by switching the numbers only items in the TMT-A with items alternating between numbers and letters and vice versa. The cognitive

19 demands of TMT-A and TMT-B, and the difference in these demands between TMT parts, has been extensively studied over the past decades. The role of visuospatial abilities in the increased cognitive demand of TMT-B was supported by the relations between TMT-B performance and the Block Design Test, an NPT that assesses visuospatial abilities, in neuropsychiatric patients (n = 112)146 and healthy old adults (n = 94)147. Regarding cognitive flexibility and executive control, the set-switching or task switching in TMT-B was validated by correlations between TMT-B scores and set-switching measures in healthy young adults123 and patients with suspected cognitive impairment144. Researchers have also suggested the involvement of working memory in both parts146,148. A previous study investigating the role of task-switching, working memory, inhibition control and visuomotor abilities in TMT with 41 healthy old participants employed correlation and regression analyses to examine the relations between scores of TMT (both direct and derived scores) and other NPTs for these cognitive processes125. Among the cognitive constructs of interest, their results indicated that TMT-A primarily involves visual search and perception, whereas TMT-B mainly reflects working memory as well as task-switching to a lesser extent. In addition, the difference score diminished the contribution of visual perception and working memory, thus reflecting executive control more accurately than other derived indices such as the ratio score. Although these findings were inconsistent with previous work123, this was attributed to the difference in task-switching expectations introduced by different task- switching paradigms125.

Another TMT study employing regression analysis on outcome measures of NPTs collected from 172 healthy revealed that executive function, working memory, episodic memory, speed and attention were all strong predictors of TMT-B completion times and the difference scores149. In addition, executive function was also significantly associated with the ratio score and TMT-B shifting but not sequencing errors, whereas working memory was also significantly associated with TMT-B sequencing but not shifting errors. Among these neuropsychological predictors, executive function was characterized as a composite measure of fluency, response inhibition, set- shifting, and planning ability149. Therefore, the author concluded that the ratio score is the purer measure of executive function than other TMT scores149, consistent with previous TMT research123.

1.2.2 Lesion studies

To advance understanding of human cognition in both healthy and diseased states and directly validate the TMT as a clinical tool, the neuroanatomical correlates underlying TMT performance have been characterized by examining the behavioural and neural differences between patients with brain lesions due to various causes, and healthy controls. Most of these behavioural studies involving patients with brain lesions investigate the relative dependence of TMT performance on the frontal lobe compared to the rest of the brain, and more specifically, the effect of laterality on the frontal lobe dependence. A meta-analysis study comparing the performance of TMT-A and TMT-B between 321 patients with frontal lesions and 305 patients with non-frontal lesion showed that the effect of lesion location was only significant in TMT-A, not TMT-B150. Moreover, given the small weighted effect size that was found, the authors concluded that the behavioural difference in TMT-A alone was insufficient to distinguish patients with frontal lesions from those with non-frontal lesions150. Other researchers failed to demonstrate any significant difference in TMT performance between groups of patients with frontal and non- frontal lesions for individuals with stroke (n = 167)151 and traumatic brain injury (n = 68)152. With regards to laterality, the TMT performance of patients with exclusively left lateralized lesions (n = 27), right lateralized lesions (n = 29), as well as bilateral lesions (n = 31) was compared to that of healthy controls (n = 34), revealing that the bilateral lesion group exhibited longer TMT-A completions times than the control group153. However, similar comparisons of TMT completion times between healthy controls (n = 60) and patients with various lesion locations [left anterior (n = 6) , right anterior (n = 13), left posterior (n = 4), right posterior (n = 8)] failed to produce any significance difference154, likely due to the small sample size of the patient groups. Other studies found that patients with left (n = 11) and right frontal lesions (n = 9) exhibited significantly longer TMT-A completions times than IQ-matched healthy controls (n = 24), and that patients with left frontal lesions exhibited significantly longer TMT-B completion times than those with right frontal lesions and healthy controls155. In addition to completion times, patients with left frontal lesions were also found to produce more errors in TMT-B compared to healthy controls155. A subsequent study has also shown that patients with left frontal lesions exhibited significantly more errors in TMT-B than those with non-frontal lesions and healthy controls, whereas the same trend was also observed in the right frontal lesion group, without statistical significance156.

Given that this literature identifies the frontal lobes as general brain regions associated with TMT performance, there are several approaches to extend these findings and thus determined more specific frontal structures that are implicated. These approaches include lesion overlap and subtraction analysis, as well as lesion-symptom mapping. The lesion overlap analysis involves superimposing brain regions that are damaged among patients with similar functional impairments157. However, the overlap regions contains not only the implicated areas of functional impairments but also areas with common blood supply; therefore, if patients without similar impairments exhibit lesions within the overlap regions, then such lesions may be subtracted from the overlap regions during lesion subtraction analysis158. Lesion studies of TMT that utilizes overlap and subtraction analysis are relatively scarce but informative. A lesion study involving patients with focal frontal lesions (n = 49), non-frontal lesions (n = 13) and healthy controls (n = 19) found that compared to healthy controls, patients with lesions in the bilateral inferior frontal region (orbitofrontal cortex/ventromedial prefrontal cortex) made the fewest errors in both TMT parts, whereas patients with lesions in dorsolateral frontal lobe (dorsolateral prefrontal cortex) made the most errors in TMT-B116. Additionally, only patients in the frontal lesion group (but not all) made more than one error in TMT-B. In addition, compared to healthy controls, patients with left frontal lesions showed significant differences in both direct and derived TMT scores; whereas patients with right frontal lesions only showed significant differences in log transformed scores; and no significant differences were observed between left and right frontal lesion groups116, failing to support the left frontal laterality theory159. Later, a patient with a focal lesion in the bilateral ventromedial prefrontal cortex only exhibited minor impairment in the completion times of the five subtasks of the D-KEFS TMT, but significantly increased error rate was observed in the number-letter switching subtask142. Such contradiction with previous research using the traditional TMT116 may be attributed to the difference in TMT format, sample size, and the lack of a control group in the case report. A subsequent lesion study conducted in 12 patients with lateral prefrontal cortex (LPC) lesions and 11 age- and education- matched healthy controls examined the effect of LPC lesion on the performance of the D-KEFS TMT143. They found that patients with LPC lesions exhibited longer completion times for all subtasks than healthy controls - but only letter sequencing, number-letter switching, and motor speed showed significant differences143, consistent with the previous finding that dorsolateral frontal lesions are related to heavily impaired TMT-B performance116. However, patients with non LPC lesions were not included in the study for comparison, limiting the direct elucidation of

22 the prominent functional role LPC plays in TMT performance159. Supported by the functional connectivity between the prefrontal cortices and the contralateral posterior cerebellum160, the role of the cerebellum in TMT performance has also been implicated in several lesion studies161–163. Specifically, stroke patients with lesions in the posterior inferior cerebellar artery (n = 6) exhibited longer TMT completion times than healthy controls (n = 11)161. Another study revealed that patients with lesions in the right cerebellum (n = 11) exhibited longer TMT completion times and a larger difference score than healthy controls (n = 21)162. However, in a somewhat larger study, the TMT performance of both left (n = 15) and right (n = 17) cerebellar lesion groups was not significantly different than that of healthy controls (n = 36)163.

Alternatively, lesion-symptom mapping identifies neuroanatomical correlates by linking certain behavioural impairments with the lesioned brain regions164,165, with methodological advantages over lesion overlap and subtraction analysis. Rather than grouping patients based on the general lesion location, it determines the predictive relationships between brain regions and behavioural impairments to improve the anatomical precision of the associated structures, allowing direct comparison with functional neuroimaging results (see section 1.2.3 below). In lesion-symptom mapping, brain regions can be defined by regions of interest (ROIs) in magnetic resonance imaging (MRI) data over a broad spatial extent, or over individual voxels to resolve detailed anatomical structures. A lesion symptom mapping study of 106 stroke patients (61 chronic, 45 acute) employed predefined ROIs in brain regions associated with executive functions including dorsolateral prefrontal cortex and ventrolateral prefrontal cortex, reporting that ROI lesions were neither significantly associated with derived TMT scores nor accuracy measures such as TMT-B shifting and sequencing errors124. Compared to ROIs, the voxel-based approach for lesion- symptom mapping may be more promising because it allows the discovery of brain-behavioural relations across the entire brain with improved spatial resolution. A voxel-wise lesion symptom mapping (VLSM) study conducted in 344 patients with focal lesions (165 with lesions in frontal regions such as prefrontal cortex, supplementary motor area or premotor cortex) found that the left rostral anterior cingulate cortex was significantly associated with impaired TMT performance (B-A)166. The differences in lesion analyses and sample sizes may contribute to the inconsistency of this finding with previous lesion studies that identified the dorsolateral prefrontal cortex as a neural correlate of TMT performance116,143. Volumetric overlap analysis has also shown that the left rostral anterior cingulate cortex was significantly associated with set-

23 switching in the WCST, but not verbal fluency in the Controlled Oral Word Association test, whereas the dorsolateral prefrontal cortex was implicated in response inhibition during the Stroop test. Such findings are consistent with previous behavioural studies demonstrating the role of set-switching in TMT-B123,144. Patients with lesions in dorsomedial prefrontal cortex were found to exhibit longer TMT-B completions times than healthy controls in a VLSM study involving 27 patients with frontal lesions and 30 healthy controls167. Another VLSM study in 182 war veterans with cortical lesions due to penetrating head injuries revealed that left lateralized, regionally non-specific lesions including the lateral frontopolar cortex, anterior prefrontal cortex, dorsolateral prefrontal cortex, superior and inferior parietal cortex were significantly associated with computationally determined executive scores of the D-KEFS TMT168. In addition to chronic brain injuries, a VLSM study of TMT performance of 30 acute stroke patients with right lateralized lesions revealed that lesions in the right dorsolateral prefrontal cortex were associated with TMT-B sequencing and shifting errors134. Further, by administering the OCS TMT in 144 acute stroke patients, a recent VLSM study demonstrated that lesions in the left insular cortex were associated with decreased accuracy scores in the set- switching test169.

1.2.3 Neuroimaging studies

Although lesion studies have indicated several key structures associated with TMT performance, they are limited by the high inter-individual variability in lesion location and extent, rarely conforming to functionally homogenous neuroanatomical regions. This is reflected in reports of similar test results between different lesion locations, and different test results between similar lesion locations113. In addition, patients with multiple functional deficits due to lesion may engage compensatory mechanisms to maintain behaviour. Therefore, neuroimaging studies can fill the gap by corroborating and interpreting the neural correlates of TMT derived from lesion studies.

In addition to studying the neuroanatomical correlates of TMT performance using the brain lesion approach, it is possible to use various neuroimaging approaches to investigate behavioural-structural relationships. For example, MRI is the primary clinical imaging modality for depicting the soft tissues of the brain – enabling studies of both brain structure and physiological function. A previously mentioned TMT study using MRI of elderly people also

24 examined the correlations between TMT performance and several structural imaging findings: periventricular hyperintensities, deep white matter hyperintensities, and medial temporal lobe atrophy - revealing that medial temporal lobe atrophy was the strongest predictor of TMT-B performance (completion times, errors) and derived TMT scores (B-A, B/A)149. Recently, a similar study in stroke patients revealed that damage to lateral cholinergic pathways was associated with all set-switching metrics such as the derived TMT indices and TMT-B shifting errors, whereas damage to the left superior longitudinal fasciculus was only associated with the difference score124. Additionally, an abovementioned VLSM study involving MRI showed that increased cortical thickness in bilateral prefrontal regions including the left superior frontal gyrus, left anterior inferior temporal cortex, right superior and middle frontal gyri were associated with longer TMT-B completion times167. Diffusion tensor imaging (DTI), a variant of MRI that allows for assessment of white matter integrity170 (quantified as fractional anisotropy) and for three-dimensional mapping of fiber tracts within the brain171,172, was also employed to study the anatomical structures underlying TMT performance in healthy aging adults173. Age- related effects were found, including declines in set-shifting performance and decreased fractional anisotropy in the corpus callosum and cortical association tracts that connect frontal cortex to posterior brain regions173.

Although such structural-behavioural studies are useful, studying the neurophysiological correlates of the TMT with functional neuroimaging methods is crucially important to improve understanding of how brain activity supports TMT performance. As summarized in section 1.1, EEG records electrophysiological signals from the cerebral cortex, enabling measurements of brain activity with millisecond temporal resolution. In 1986, a seminal EEG study was conducted involving 14 healthy young adults to study the underlying cortical activity associated with TMT performance174. Five electrodes were placed in the left anterior (F3), right anterior (F4), left posterior (P3), and right posterior (P4) locations with Cz used as a reference electrode. The EEG signals were recorded in the 2-30 Hz range using an AIM 65 microcomputer, using seven frequency bins, each with a bandwidth of 4 Hz (i.e. 2-6 Hz, 6-10 Hz, etc.). A four-way repeated measures analysis of variance (ANOVA) revealed that increased high frequency activity was observed in the two anterior electrodes compared to the posterior electrodes, especially the left anterior electrode. Additionally, a left lateralized activation observed during TMT was thought to indicate contralateral motor movements of the right hand. The comparison between cortical

25 activation patterns between part B and part A indicated that a left lateralized cortical activation in the posterior electrodes was only evident in part B. The researchers also concluded that the brain activity during TMT performance is complex; widely distributed across the cortex; different between parts; and therefore that the TMT is effective in detecting neuropsychological impairments but not in localizing the underlying brain regions174. To date, this is the only EEG study of TMT performance that has been conducted.

The development of functional MRI (fMRI) has enabled researchers to study the neuroanatomical and neurophysiological correlates of the TMT by depicting brain activity at millimeter spatial resolution throughout the brain. Based on neurovascular coupling, fMRI measures neural activation indirectly through blood oxygenation level dependent (BOLD contrast). There are two major categories of fMRI experiments: resting-state imaging and task- based imaging. Resting-state fMRI is the simpler of the two experiments to execute, as imaging data are collected in the absence of an explicit behavioural task; participants are typically instructed to lie still and not focus on anything in particular. From these data, the predominant analysis approach is to measure the temporal correlations (or “functional connectivity”) of spontaneous time-varying BOLD signals from different brain regions, thus revealing functional brain networks175. Functional connectivity indicates the potential for information sharing between brain regions, such as those that having common functional roles (e.g. bilateral motor connectivity175) and those that form networks to control emergent human behaviours. Numerous resting-state networks have thus been identified and linked to sensory, motor and cognitive domains176. One example is the default mode network (DMN), which is found to be more active during wakeful rest and self-reflection, but shows reduced activity during goal-directed tasks21. The DMN is influenced by numerous factors including normal aging and numerous brain diseases177,178 but is not the main topic of interest in this thesis. Alternatively, and directly pertinent to the discussion that follows, task-based fMRI utilizes tasks that evoke sensory, motor or cognitive responses, to image the brain activity associated with these aspects of mental processing. The task-based brain activity is typically identified by a statistical test of the contrast between the BOLD response during repeated versions of the task condition, and the BOLD response during repeated versions of a control condition (e.g. visual fixation on a cursor located at the center of a display screen).

In the current context, resting-state and task-based fMRI methods can be used to provide “indirect” and “direct” assessment of the brain activity supporting TMT performance. First, resting-state fMRI permits examination of the associations between TMT behavioural performance (administered outside the MRI system) and resting-state functional connectivity patterns. This procedure is similar to how structural-behavioural relationships are performed, as discussed in the beginning of this section. For example, an initial resting-state fMRI study of 21 healthy adults reported that the TMT difference score (B-A) was significantly negatively correlated with functional connectivity between the bilateral intraparietal sulci/superior parietal lobule and other structures of the executive control network (which consists of the bilateral dorsolateral, ventrolateral, dorsomedial prefrontal cortices, lateral parietal cortices, and left frontal insula179). In addition, a subsequent study in 42 healthy young adults revealed the significant negative correlation between resting-state functional connectivity (left superior parietal lobule connection with right ventrolateral prefrontal cortex) and the difference score analogue of the D-KEFS TMT: the difference in completion times between the number-letter switching subtask (TMT-B analogue) and the number sequencing subtask (TMT-A analogue)180.

Second, numerous task-based fMRI studies have investigated the relationship between TMT performance and brain activity more directly. However, adapting the traditional pen-and-paper TMT into the MRI scanner while maintaining the ecological validity remains an engineering challenge. The initial studies developed a number of different TMT adaptations and administrations to overcome logistical and engineering challenges in implementing the standard pen-and-paper version of the task in an MRI system. Eliminating the visual and motor component, the first adaptation of the TMT in 7 healthy young adults involved articulating the number sequence for TMT-A and the alternating number-letter sequence for TMT-B via covert speech in response to the audio stimuli (“count” for verbal TMT-A, “alternate” for verbal TMT- B)181. Due to the covert response, the behavioural measures were determined via self-reporting the highest number or letter achieved within the fixed time limit of 25 seconds for fMRI recording133. Such a time limit was the consequence of undertaking the fMRI study in a "block design" format, such that brain activity was assessed during TMT performance for a constant time duration in comparison to control (i.e. visual fixation) for another constant time duration, for each study participant. In the contrast of verbal TMT-A vs verbal TMT-B, positive activations in the TMT-B were seen in the left dorsolateral and medial prefrontal cortex, left

27 middle frontal gyrus, supplementary motor area (cingulate sulcus), left precentral gyrus and bilateral intra-parietal sulcus, compared to TMT-A181. Later, an fMRI adaptation of the TMT with only a TMT-B equivalent was developed and tested in 32 healthy young adults, which required covert visual search for the next item in the ascending alternating order, and response via a button press once completed, to link the previous item and the correct subsequent item with a straight line182. The task visual stimuli were presented using a back-projection screen and mirrors in the head coil during fMRI. Due to the error correction nature of the paradigm, the behavioural performance was assessed by the number of completed TMT-B trials per test epoch (22 s of response time limit). Despite the absence of TMT-A to contrast the brain activation, group-level BOLD activation and ROI analysis revealed positive activations in the bilateral middle and superior frontal gyri (dorsolateral prefrontal cortex), bilateral inferior frontal gyrus, bilateral anterior insula, bilateral precentral gyrus/premotor areas, ventral and dorsal visual pathways, and the medial pre-supplementary motor area. Specifically, activations in the dorsolateral prefrontal cortex were commonly observed across all participants in the ROI analysis; however, the exact activation loci were not significant in the group model analysis due to inter-individual variability182. Other researchers developed and administered a computerized TMT to 16 healthy young adults, who were instructed to track numbers (pcTMT-A) or numbers alternating with letters (pcTMT-B) via pressing one of three buttons to indicate the position of the black box attached to an item relative to the next item in the ascending order183. During fMRI, the visual stimuli were displayed using a projection screen and mirrors in the head coil. Although the motor component of the pcTMT was drastically reduced compared to the traditional paper TMT, the visual component was preserved, which enabled the accurate recording of TMT response times for each link. Supported by significant correlations between computerized and paper TMT in the TMT-B response times, difference score, and the TMT-B completion times, but not the TMT-A completion times, the behavioural performance of paper and computerized TMT was shown to be highly similar, validating use of the computing TMT in the fMRI study. Similar results were also found when the paper and computerized TMT were both performed in the MRI system. The contrast of pcTMT-B vs pcTMT-A revealed positive activations in the right inferior/middle frontal gyri (dorsolateral prefrontal cortex), right precentral gyrus, left middle temporal/angular gyri, and negative activation in the right fusiform gyrus183, which is consistent with the previous VLSM finding that the right dorsolateral prefrontal cortex was implicated in TMT-B performance accuracy134.

1.3 Tablet Technology

In section 1.2, several task-based fMRI studies were reported that involved adapted versions of the TMT, given that the task is somewhat challenging to implement in realistic (naturalistic) fashion in an MRI system. To appreciate these challenges, a useful starting point is to consider the practicalities of enabling (and also recording) writing and drawing behaviour during an fMRI experiment. The fMRI environment imposes multiple constraints: the imaging participant must lie supine within the narrow bore (60 – 70 cm in diameter) of the magnet, and must remain sufficiently still such that their head motion does not confound the data collection, usually over multiple collection periods each of approximately 10 minutes; and any devices that are present must experience negligible magnetic forces, introduce negligible radiofrequency (RF) heating and most importantly, introduce negligible levels of RF interference such that the device operates properly, with no impact on the temporal signal-to-noise ratio (SNR) of the fMRI data that are collected. It is immediately evident that enforcing real-world behaviour writing and drawing behaviour is impractical in this context. Even if a configuration could be found that allowed the participant to view what they were writing or drawing with pen and paper, this would become cramped and uncomfortable over time and it is not immediately clear how to cue the onset and offset of task performance184, how to record the performance quantitatively, and how to replenish the paper supply intermittently. Given these challenges, it is not surprising that fMRI researchers attempted to bypass them by using other response strategies as indicated above, such as instructing participants to write and draw using their index finger185,186; adapting the behavioural task in question from a visuo-motor to an auditory-motor construct (such that auditory cues elicit certain verbal articulations181, or to enable button press responses181,183.

Two major inter-related problems arise when attempting to interpret the brain activity generated by such tasks. First, it is usually hypothesized that the brain activity associated with the “core elements” of the task in question are somehow preserved even though the response mechanism is not realistic. However, it is not always clear what those core elements are. Second, the lack of a realistic response may turn a simple task, normally performed efficiently by the brain after extensive prior practice in the real world, into a much more novel task that requires significantly more cognitive effort. In addition to creating uncertainty over whether brain regions responsible for processing the core elements of the task are similarly engaged as they would be in the real- world task, this “response mapping” problem can preferentially engage brain areas involved in

29 cognitive control (e.g. the frontal lobes) rather than those involved in highly skilled coordinated movement (e.g. the cerebellum). It is not always straightforward to disentangle these experimental confounds. Consequently, although such bypassing approaches can provide important initial information about brain activity, additional corroborating evidence is required using methods that are much more ecologically valid (i.e. produce behaviour that closely approximates performance in the real world).

This conclusion led researchers to confront directly the challenge of developing a device or approach that approximates the naturalistic writing and drawing during fMRI. An additional objective was to develop a method of quantitative output to promote detailed analysis of brain- behaviour relationships. Early efforts involved a resistance-based pen movement tracer, although this method was limited by calibration issues and the need to track a tracing (rather than freeform drawing) movement along a predefined, one dimensional path184,187; and use of a fibre-optic system to track the position of a stylus tip on a two-dimensional surface, which also suffered from calibration issues and position-tracking limitations184,188. In a task-based fMRI study, the fibre optic-based drawing device was developed and assessed for TMT behavioural recordings during fMRI in 12 healthy young adults188. The device setup allowed the recording of complex handwriting motor movement, real-time performance monitoring on a computer screen, and visual display of task stimuli via a pair of fMRI-compatible goggles. Measured by number of links within the 45 s test block time limit, the study found that significantly less links were made in TMT-B than TMT-A. Brain activation results revealed increased left lateralized brain activity in the middle and superior frontal gyri (dorsolateral prefrontal cortex189), medial frontal gyrus, precentral gyrus, cingulate gyrus, and insula in the Part B vs. Part A contrast188, consistent with previous TMT findings166,181.

Building on this work and attempting to overcome the limitations of previous devices, a touch- sensitive digitizing tablet was subsequently developed in the Graham Laboratory to enable more realistic writing and drawing behaviour during fMRI, including detailed behavioural recording133,184. Like keyboards, touch-sensitive tablets are computer input devices that record two or more coordinate values (i.e. x, y) representing the position of stylus contact on a touch- sensitive screen. After initial signal processing, the written/drawn trajectory can be reproduced on the computer display. Consumer electronics versions of computer tablets are now ubiquitous, with the touch screen and display integrated together.

The fMRI-compatible tablet system developed in the Graham laboratory does not integrate the touch screen and display (although this has been attempted by others for fMRI applications190). Instead, the touch-sensitive surface and stylus are mounted in isolation on an elevated support platform that can be attached to the patient table, together with a controller box and the necessary connecting cables, drivers, and software to record responses. This system is meant to be used in conjunction with an fMRI-compatible projection system (such as a projector mounted at the back of the magnet bore to display task-relevant cues and visual stimuli on a rear-projection screen). The participant views the screen through an angled mirror while lying supine, and in this manner can perceive tablet interactions overlaid on the display.

Figure 1.3 Tablet prototype 2 (A) with a stylus (B) and a video camera (C) to enable visual feedback of stimulus response and hand position (D).

In this thesis, the second-generation tablet prototype (prototype 2, Figure 1.3) was employed to provide visual feedback of hand position (VFHP). The tablet employed a polyester laminate (PL) resistive four-wire transparent touchscreen (Microtouch, Model #RES-6.4-PL4, 3M, St. Paul, MN,

USA; 16 cm diagonal; 13 cm × 10 cm active area) along with its matching controller board (Microtouch, Model #SC400, 3M, St. Paul, MN, USA). In addition to its excellent accessibility and affordability (touchscreen and controller cost less than USD100), as well as straightforward assembly, this panel was selected for a number of technical reasons. First, by using indium tin oxide resistive coatings and a glass substrate, the PL coversheet is not ferromagnetic and connects very easily to magnetically shielded and filtered cables, providing fMRI-compatibility. Second, the touch-sensitive accuracy (0.005 inches) and report rate (180 reports/s) provide very accurate and detailed capture of writing and drawing behaviour. Third, touch and movement registration are not limited to an fMRI-compatible tool such as a stylus, and can be performed by any reasonable appendage that can be moved into contact with the touch-sensitive surface, such as a finger, knee or foot. In such circumstances, tablet contact could be optimized using wearable fMRI-compatible apparel with a protruding point of contact.

To protect against surface damage and unintentional touches, the touchscreen is mounted into a plastic holding frame which attaches to an optional plastic tablet support platform. The platform can be tilted (up to 90 degrees) and adjusted in height (20 to 40 cm above the patient table) for comfort within the magnet bore, while minimizing interference from the torso and from respiratory motion. A junction box under the tablet frame provides connection to the optional stylus (2-pin) and a shielded cable (Type 9539, Belden Inc., St. Louis, MO, USA) running to the penetration panel in the MRI system. The optional stylus consists of a modiﬁed plastic pen barrel (approximately 12 cm in length) with a microswitch on the tip to detect contact between the stylus and touchscreen surface. Pushing with moderate force on the touchscreen with the stylus activates the microswitch, thus producing a small amount of tactile feedback and registering a button press. This optional button input is suitable for response recording or as a crude pressure indicator during tablet-based tasks. The recorded tablet and stylus signals pass through an EMI (electromagnetic interference) ﬁlter (56-705-005-LI, Spectrum Control, Fairview, PA, USA) at the penetration panel and transmit to the tablet controller box outside the MRI suite via shielded cables. The controller box contains the electronic logic in the tablet controller board, power conditioner, and receptacles for USB (universal serial bus) connections to the fMRI stimulus/response computer. The controller box connects to a computer using USB connections for data transmission from tablet and stylus to the computer.

To provide VFHP, this prototype includes a video recording and processing platform, and an augmented reality environment that enables the participant to view the display of the tablet computer overlaid with live video of hand/stylus/touch-surface interactions. Using this approach, participants have increased awareness of their hand position in real time and thus can perform tablet interactions with enhanced ecological validity. The additional hardware consists of an MRI-compatible “TabletCam” colour video camera (12M-i with 4.3 mm lens, MRC Systems, GmbH, Heidelberg, DEU) and its mounting frame; light emitting diodes (LEDs) to illuminate the tablet field of view; and outside the magnet room, an additional computer for video processing with two video capture cards. One capture card is for the TabletCam feed, and the second card, with an appropriate video format converter, is for the stimulus/response computer feedback. Whereas the original stimulus/response computer is still utilized to present task-related stimuli with precise timing, and to log tablet responses (x, y coordinates and force data as a function of time), drivers and software on the video processing computer are used to fuse segmented video of hand/stylus interactions on the touch-sensitive surface of the tablet, task-related visual stimuli, and graphical representation of the interactions as inkmarks, thus creating an interactive augmented reality environment for subsequent display to the user. The segmented video can be created by various mechanisms, although the simplest approach is a “blue-screen” approach analogous to what is used when announcers stand in front of weather maps on television. The tablet surface can be covered in blue tape, which enables a video “mask” to be created of the hand and stylus only (zero signal intensity elsewhere) by segmenting the acquired video based on colour content. In addition, during fMRI experiments, the MRI system emits pulse triggers in synchrony with the imaging sequence. These triggers are then converted to keyboard events via a trigger conversion box to initiate the behavioural task onset, thus time-locking the behavioural tracking data with the fMRI acquisition.

The Graham tablet, as well as other versions of touch sensitive technology have been developed for a wide range of fMRI applications in basic and clinical neuroscience including language processing and learning disorders. This body of research has been the topic of a recent detailed review that I authored191. In the context of the thesis, however, the application of the Graham tablet that is most pertinent involves a series of fMRI studies investigating brain activity associated with TMT performance, as summarized in the next section.

1.4 Application of Tablet Technology in TMT

In the initial development of a prototype tablet in the Graham laboratory, the TMT was used as a proof-of-concept task-based fMRI experiment to demonstrate tablet capabilities184. Brain activation was reported for two participants, showing that the measured mean behavioural response times for both TMT parts were largely improved compared to a past fMRI study on TMT using fibre optic-based drawing device184,188, with values approaching the median scores for the pen and paper TMT184,192,193 suggesting enhanced ecological validity. Consistent with past fMRI research using the fibre optic-based device, the TMT-B vs. TMT-A contrast revealed left hemisphere activations in middle frontal gyrus, superior frontal gyrus, middle temporal gyrus, and the superior parietal lobule, which are associated with executive function, spatial attention, and visuomotor control184,188,194. Additionally, the tablet prototype and TMT task were used in a methodology study to optimize pre-processing pipeline choices for de-noising fMRI data, thus enhancing the sensitivity to detect brain activation195,196.

A subsequent study investigated the neural correlates of the TMT in young healthy adults, and to investigate the effect of tablet prototype on brain and behavioural measures (i.e. whether or not the TabletCam provided enhanced realism in the behavioural performance and caused an associated change in brain activity)133. The experiments were conducted on two separate groups of participants, with identical TMT administration with the tablet except that one group performed while using the TabletCam to provide visual feedback (described as the "visual feedback of hand position (VFHP)" condition; and the other group did not (no VFHP)133. In addition, both groups performed the pen-and-paper version of the TMT so that the ecological validity could be assessed for both modes of the tablet133. Rather than evaluating behavioural response times, as done previously184,188,192,193, the study employed a new metric called seconds per link (SPL) to account for situations where participants did not complete a TMT trial within the fixed time limit133. The time limit was a consequence of undertaking the fMRI study in a commonly used format known as a "block design", such that brain activity was assessed during TMT performance for a constant time duration in comparison to visual fixation for another constant time duration, with multiple blocks of TMT performance and visual fixation for each study participant.

The SPL values obtained with the tablet TMT were shown to be increased (i.e. performance was slower) compared to the values obtained for the standard pen-and-paper TMT, which was attributed to the inherent difference between the two test administrations and environment133. In addition, tablet SPL values were significantly correlated with standard SPL values in TMT-A independent of VFHP, but not in TMT-B, which indicates stronger naturalistic approximation in TMT-A than TMT-B133. Factors contributing to such results could be the increased performance variability in TMT-B and the low sample size. Regarding brain activity, irrespective of the tablet mode, the most common activation in both TMT-A vs. baseline (visual fixation) and TMT-B vs. baseline contrasts was found in the bilateral superior parietal lobule, inferior parietal lobule (IPL), inferior frontal gyrus (IFG), supplementary motor area (SMA), premotor region of the middle frontal gyrus (MiFG), middle temporal gyrus (MiTG), primary visual and visual association areas, as well as the left lateralized pre and post-central gyri. Collectively, these regions are associated with somatosensory and motor processes, visual perception, imagined movement and visual search197,198, as expected.

Interestingly, both TMT-A and TMT-B vs. baseline contrasts also revealed “negative activation” (i.e. significant activation for the baseline minus TMT contrast) in brain regions that are part of the DMN, including the posterior cingulate and angular gyri21. Past research has established that the DMN is suppressed during goal-directed tasks, and active during wakeful rest21, such as during a baseline period of visual fixation. Activation was also more left-lateralized for the TMT-B vs. TMT-A contrast with VFHP, compared to the case without VFHP133. The active regions with VFHP in this case involved the dorsal lateral prefrontal cortex, MiFG, bilateral IFG, superior parietal lobule, and anterior cingulate, which are associated with executive function, motor planning, language processing, set-switching, visual search, and performance monitoring113,181,182,188,198–203. Further, positive activations in dorsolateral prefrontal cortex were only observed in TMT-B, which includes executive functions such as refocusing attention during set-switching204 and perceptual decision-making205. Such results indicate that these mental processes are all at a higher demand in TMT-B compared to TMT-A116,133. Generally, the behavioural performance and brain activity of both interaction modes were found to be very similar; therefore, both are suitable to study the fMRI brain activity associated with TMT. Intuitively, the deployment of the TabletCam is advisable as it makes the tablet TMT a more naturalistic approximation to the paper TMT.

Most recently, the tablet and TabletCam (prototype 2) were used in an fMRI study of TMT performance in a group of healthy elderly adults119, which again included administration of the standard TMT for comparison. In this case, significant positive correlations were found between SPL values for the tablet TMT and the standard TMT, for both TMT parts119, indicating good ecological validity. However, the SPL values were also significantly different between tablet and standard TMT, indicating that the tablet version and test environment was not a perfect replication of the standard test conditions119. Poorer performance was observed with increasing age for both TMT parts, as expected. The fMRI activation maps showed activation patterns in both contrasts (TMT-A vs. control and TMT-B vs. control) that were generally consistent with previous TMT studies116,133,184. When exploring how these patterns of brain activity covaried with age, the older adults demonstrated less TMT-related brain activity in the bilateral occipital lobes, middle temporal lobes, cerebellum, and right parietal lobes, including the postcentral gyri and the inferior parietal lobe, compared to younger adults119. The significance of this work is that it provides additional support for the usage of the TMT as a clinical tool to screen for dementia, as well as providing normative data for future studies of the TMT-related brain activity in elderly patient populations along the cognitive impairment spectrum (i.e. from mild cognitive impairment to more severe cases like Alzheimer’s Disease).

1.5 Hypotheses and Thesis Outline

This thesis aims to investigate the temporal and spectral characteristics of EEG signal content during TMT performance using the fMRI-compatible tablet technology developed in the Graham laboratory, as a supplement to previous fMRI findings revealing only averaged brain activations during the task. Understanding the detailed brain activity associated with the TMT is pertinent in both clinical and research usage of the TMT. Specifically, by examining the behavioural kinematics and neurophysiological activity that varies as participants engage in different mental processing during the test, the sensitivity and specificity of the TMT may be improved for detecting brain impairments. Before studying patient populations with cognitive impairments, acquisition of behavioural kinematic and EEG data in healthy young adults is an important initial step.

Chapter 2 characterizes the temporal dynamics of behavioural kinematics and brain activity during TMT performance using EEG and the fMRI-compatible tablet technology. Although an

EEG study could be conducted with other tablet technology, or perhaps using the standard paper version of the TMT, the use of the fMRI-compatible tablet enables direct comparison with fMRI literature that adopted the same methods. It is hypothesized that the spatial patterns of electrode activation are significantly different 1) across TMT parts with a left frontal lateralization, consistent with previous functional neuroimaging literature investigating a similar study population133; and 2) across time periods with significantly different aspects of performance during each TMT part, such as visual search and linking behaviour, as quantified from tablet- based kinematic metrics.

Finally, Chapter 3 discusses the limitations and future directions of this research direction, including possible data analysis strategies and tablet applications in simultaneous EEG-fMRI studies, as well as studies involving patient populations with cognitive impairments.

Chapter 2 2 EEG Brain Activity during the Trail Making Test 2.1 Introduction

The Trail Making Test (TMT) is widely used in behavioural neuroscience and in the clinic as part of neuropsychological test (NPT) batteries, to assess frontal lobe function and to assist in diagnosis of brain disease113,115. This pen-and-paper test assesses cognitive processes such as visual search, visual planning, visuomotor control, as well as attention and memory113,115–118. There are two parts (A and B), each that involve linking a total of 25 randomly-placed items in ascending order. Part A (TMT-A) involves linking numbers (1-2-3-4-5…) and Part B (TMT-B), which is more challenging, involves linking numbers alternating with letters (1-A-2-B-3-C…). Standardized protocols require the test recipient to perform the TMT as fast as possible without lifting the pen from the paper113,117,120–122. Each part is typically scored by directly measuring the completion time and recording the number of errors; whereas derived scores of the completion times, such as the difference (B-A) and the ratio (B/A) have also been employed to de-emphasize the visuo-motor aspects of performance and to emphasize the cognitive aspects116,120,122,124.

Despite that the TMT is widely used, the underlying brain activity that supports the performance of this test is not understood in detail. Early TMT studies focused on behavioural measures as an indirect indicator of brain state and cognition. Through correlations in behavioural measures with other NPTs, the neuropsychological correlates of the TMT have been demonstrated including visuospatial abilities127,146, set-switching123,144, and working memory125, establishing test validity and sensitivity to brain damage. By examining the behavioural deficits associated with TMT performance in patients with brain lesions, key neuroanatomical correlates such as the left dorsolateral prefrontal cortex116,143,167,168 and anterior cingulate cortex166 have also been identified.

Non-invasive tools for measuring brain activity, such as functional magnetic resonance imaging (fMRI), are starting to provide new opportunities for deeper, more comprehensive investigation of the neuroanatomical and neurophysiological correlates of the TMT (and other NPTs). However, it is challenging to adapt pen-and-paper NPTs so that they can be administered in an MRI system while maintaining ecological validity (i.e. with behavioural performance that

37 38 generalizes to the real-world setting). To study the brain activity associated with TMT performance in healthy adults using fMRI, researchers initially developed several different strategies such as a verbal version of the TMT using covert speech181, a motor TMT using a fiber optic-based drawing device188, a visual TMT involving covert linking responses182, and a computerized TMT using button press responses183. Although all four studies confirmed the involvement of the dorsolateral prefrontal cortex, only the former two identified the anterior cingulate cortex. Although the evidence from these studies is not definitive, it strongly suggests that simplification of visual and motor components of the TMT failed to be fully representative of the naturalistic writing and drawing performance in the real test – and of the associated brain activity.

To address this limitation, fMRI-compatible tablet technology was developed to approximate naturalistic TMT responses during fMRI184. Using a resistive touchscreen and a stylus with force sensor, the first tablet prototype provided visual feedback of tablet interactions with excellent utility. However, the test recipient was unable to see their hand grasping and manipulating the stylus while interacting with the tablet, which was thought to impact behavioural performance under some circumstances133. A second-generation prototype was developed to address this problem by providing visual feedback of hand position (VFHP), enabling the participant to view the display of test stimuli overlaid with live video of hand/stylus/touch-surface interactions in an augmented reality environment206. This approach provides increased awareness of the hand and stylus position in real time, and thus enables test recipients to perform tablet interactions and undertake the TMT with enhanced ecological validity. Furthermore, as the tablet interactions are digitized and recorded on a computer, kinematic metrics can be developed for a much more nuanced characterization of TMT performance than is achievable with the standard scoring procedures.

Recent fMRI research has employed the second-generation tablet prototype to study the brain activity of TMT performance in different populations of healthy adults. Studying young healthy adults, Karimpoor et al. identified bilateral brain activations in regions associated with somatosensory and motor processes, visual perception, imagined movement and visual search in TMT-A versus Control (visual fixation) and TMT-B versus Control contrasts133,197,198. Left lateralized brain activations in regions associated with executive function, motor planning, visual search and performance monitoring were found in the TMT-B versus TMT-A contrast with

VFHP113,181,182,188,198,199, including the dorsolateral prefrontal cortex and anterior cingulate cortex, consistent with previous findings116,166. More recently, Talwar et al. demonstrated age- related decrements in behaviour and brain activation during TMT performance119. Compared to younger adults, older adults exhibited poorer task performance and reduced TMT-related brain activity in the bilateral occipital, temporal, and parietal lobes, consistent with previous reports of age effects in TMT behaviour119,127,129–131.

Functional MRI signals provide a detailed picture of brain activity at millimeter spatial resolution, but typically with low temporal resolution (approximately seconds) because the signals arise from sluggish neurovascular coupling effects. Alternatively, electroencephalography (EEG) uses scalp electrodes to record voltage fluctuations from the ionic currents produced by neural activity, with high temporal resolution (approximately milliseconds). The EEG method thus holds promise for revealing the dynamic mental processes involved in TMT performance. The spatial resolution for localizing neural activity with EEG is typically much lower (centimeters) than that achievable by fMRI, and is primarily limited to regions immediately below the scalp. Nevertheless, it should be possible to use EEG recordings to fill a gap in knowledge about the time and frequency features of neural activity responsible for TMT performance, and to determine whether the EEG signals are localized in a manner that is consistent with fMRI results.

One EEG study of TMT performance was conducted approximately 35 years ago. Signals were recorded from only four electrodes that divided the skull surface into quadrants174. Both TMT parts were found to generate high frequency (14-30 Hz) oscillatory activity, with left lateralized posterior brain activity more evident in TMT-B174, consistent with subsequent TMT studies116,119,133. With modern EEG recording and data analysis technology, as well as better understanding of TMT-related brain activity, the goal of the present study was to examine in young healthy adults the differences in the temporal aspects of behavioural performance and in the EEG signal content between TMT-A and TMT-B. As part of this goal, tablet technology incorporating VFHP was used to provide quantitative kinematic recording of TMT performance to inform the interpretation of EEG signals. Specifically, it was hypothesized that the spatial patterns of electrode activation are significantly different 1) across TMT parts, consistent with previous functional neuroimaging literature investigating a similar study population; and 2)

40 across time periods with significantly different aspects of performance during each TMT part, such as visual search and linking behaviour, as quantified from tablet-based kinematic metrics.

2.2 Methods 2.2.1 Participants

The study was approved by the Research Ethics Board of Sunnybrook Health Science Centre in Toronto. All research was performed with the free and informed consent of the participants, who were right-handed based on self-report and behavioural monitoring, were English-speaking, were free from EEG exclusion criteria (e.g. discomfort with gel on the scalp), and were free from past or present neurological and psychiatric impairments. Participants were recruited from graduate students at the University of Toronto and research personnel at Sunnybrook Research Institute in Toronto.

Sixteen healthy young adults participated in the study (8 male, 8 female, age range: 19-27, mean age 21.3 ± 2.8). All participants performed the tablet-based TMT with EEG in an acoustically shielded room, supervised by an experienced test administrator (Z.L.).

2.2.2 TMT Design

Figure 2.1 Task design for the EEG experiment. See text for details. Fix = fixation, TMT = Trail Making Test.

The TMT was administered according to a format (Figure 2.1) commonly adopted in the literature113,118,120,121 using a stimulus/response computer (Intel i5-2500 4-core CPU, 16 GB RAM). The TMT-A involved linking encircled number stimuli from 1 to 25, which were distributed pseudo-randomly across the screen. The distribution was based on the standard TMT with a 90o rotation to fit the “landscape” format of the display. The TMT-B involved linking number stimuli (1-13) alternating with letter stimuli (A-L) in another pseudo-random spatial distribution. Based on the standardized and previous TMT instructions used in the field113,117,119– 122, participants were asked to connect the circles from “Begin” to “End” as fast and as accurately as possible, without lifting the stylus from the tablet. Prior to the EEG experiment, the TMT performance was also demonstrated to participants using test samples of TMT-A and TMT-B, as a supplement to oral instructions. Similar to previous fMRI studies of the TMT that adopted a block design119,133, the design (one run) contained four trials of control tasks (8 repeats, approximately 19 s), TMT-A (40 s), and TMT-B (60 s), separated by visual fixation (10 s). The control tasks involved linking two items from “Begin” to “End” (1-2). Starting with one TMT-A and one TMT-B version derived from the standard TMT arrangement, the variants for each trial were created by either rotating the stimulus distribution by 180o or swapping between number only stimuli and number-letter stimuli, or both. This procedure was undertaken to minimize the contribution of different stimulus distributions to the performance difference between TMT-A and TMT-B121. The visual fixation task served as the baseline condition in the present study, in which participants focused on a black crosshair located centrally on a white display. Each participant underwent two runs of the task design, completing a total of 8 trials for TMT-A and TMT-B, respectively. The task design was implemented and administered by a custom program written using E-Prime Software (version 2.0.10.356, Psychology Software Tools, Sharpsburg, PA, USA) on the stimulus/response computer, which received and interpreted the stylus position recorded on the tablet, providing task-related performance feedback as inkmarks superimposed on the task stimuli. During EEG recording, tablet behavioural performance was recorded as time-varying (x, y) coordinates indicating stylus position on the tablet. The coordinates were sampled at a rate of approximately 40 Hz in E-Prime and logged into a computer text file upon trial completion.

2.2.3 Tablet Technology

The present study incorporated the same tablet technology used in a number of fMRI studies of tablet-based TMT performance119,133,184, facilitating comparison with the literature. The tablet contained a patented resistive transparent touchscreen (Microtouch, Model #RES-6.4-PL4, 3M, St. Paul, MN; 16 cm diagonal; 13 cm × 10 cm active area) along with its matching controller board (Microtouch, Model #SC400, 3M, St. Paul, MN, USA). Connected to the stimulus/response computer as an input device, the tablet system detected stylus contact force and recorded stylus coordinates, for subsequent display as inkmarks. The stylus tip was also equipped with a force sensor (FSR 400, 30-49649, Interlink Electronics, Carmarillo, CA, USA) to detect the presence and magnitude of the exerted force. However, the contact force was not part of the scope of the study and thus the presence of the force was logged, rather than the magnitude.

As shown in Figure 2.2, the tablet technology and stimulus/response computer were configured for EEG experiments in an acoustically shielded room (Industrial Acoustics Company, The Bronx, NY, USA). The tablet with stylus was placed on the desk in front of the participant to ensure a naturalistic and comfortable writing posture. In addition to the task-related stimuli captured by the stimulus/response computer, the tablet was mounted with a colour video camera

(12M-i with 4.3 mm lens, MRC Instruments GmbH, DEU) and a battery-powered illuminator to enable VFHP. The touchscreen was calibrated on the stimulus/response computer to ensure the spatial accuracy of the task-related stimuli. Enabled by drivers and software on an additional video processing computer (Intel i5-3570 4-core CPU, 8 GB RAM), segmented video of hand/stylus interactions on the touch-sensitive surface of the tablet; the task-related visual stimuli; and the graphical representation of the interactions as inkmarks were superimposed to create an interactive augmented reality environment for subsequent display to the participant via a monitor (LG L1718S, 17-inch diagonal, 1280 × 1024 resolution, 60 Hz refresh rate). The monitor was positioned approximately 65 cm in front of the participant, creating a subtended visual angle of approximately 16º. The surface of the tablet was covered in blue tape, enabling a video “mask” to be created of the hand and stylus only (zero signal intensity elsewhere) by segmenting the acquired video of task-related performance based on colour content. The video camera was angled appropriately to ensure that the blue taped touchscreen was fully captured for seamless superimposition of task-related visual feedback.

2.2.4 EEG Recording

The EEG data were recorded with a sampling rate of 5000 Hz using an actiCAP active electrode system (32 channel), actiCHamp amplifier system, and BrainVision Recorder software (version 1.21.0402, Brain Products GmbH, Gilching, DEU) on a laptop computer (Intel i7-3610QM 4- core CPU, 8 GB RAM). The EEG amplifier collected and amplified electrophysiological signals recorded by scalp electrodes and transmitted them to the recording laptop over a USB (universal serial bus) cable. The stimulus/response computer was also programmed using E-Prime to send parallel port triggers marking the onset of each task phase (control, fixation, TMT-A, TMT-B), to a dedicated trigger port on the EEG amplifier. The head circumference was measured for each participant and the 32 electrodes along with a ground electrode were mounted on the electrode cap of appropriate head size (EASYCAP GmbH, Herrsching, DEU). The participant was then outfitted with the cap, which was adjusted by the experimenter (Z.L.) to ensure that electrode Cz was at the topographic center of the head. Conductive electrolyte gel (SuperVisc 1000 gr.; EASYCAP GmbH, Herrsching, DEU) was then injected (using a blunt needle) through the electrode aperture between the individual electrode and the scalp to minimize the electrical impedance to less than 25 kOhm. The impedance was actively monitored during the gel injection, but the values were not recorded.

2.2.5 Tablet Data Analysis

Using a custom MATLAB program (The MathWorks, Inc., Natick, MA, USA), the TMT performance in each individual trial was rendered as a video file of tablet ink marks superimposed on the test stimuli. The video files were then carefully visually inspected to obtain the completion times, number of errors, correct links, and total link during the inspection, an error link was identified based on the sequence of linked items. For example, in TMT-A, 2-3 is the only correct link that starts with 2, whereas 2-1, 2-4 and others are error links. The intentional target item of a link was judged based on the pause and turn at the item. For example, if link 2-3 crossed 6 in the link path with no noticeable pause and turn at 6, then no errors were logged and 2-3 was counted as a correct link. The descriptive statistics were computed for the completion times, number of errors, correct links, and total links in TMT-A and TMT-B separately. Wilcoxon signed rank tests were also used to evaluate the effect of TMT part in these metrics. The seconds per link (SPL) of each TMT trial was calculated by dividing the completion time (either the block duration, or the time required to complete all links) by the number of correct links.

Several initial analyses were conducted to direct the course of statistical testing and to ensure data consistency. First, the overall SPL distributions in TMT-A and TMT-B were visualized using histograms and each was assessed for normality using the one-sample Kolmogorov- Smirnov test. As both SPL distributions were significantly different from a normal distribution (TMT-A: p < 0.001, TMT-B: p < 0.001), subsequent SPL analyses employed non-parametric statistical tests. Kruskal-Wallis tests were conducted on SPLs in TMT-A and TMT-B separately to assess the effect of trials, which revealed significant effects (TMT-A: p < 0.01, TMT-B: p < 0.001). Using Tukey’s Honestly Significant Difference Procedure to perform a post-hoc multiple pairwise comparison of the trial means, the SPL value of Trial 1 was found to be significantly larger than that of Trials 5-8 (Trial 1-5: p < 0.001; Trial 1-6: p < 0.01; Trial 1-7: p < 0.05; Trial 1-8: p < 0.05). No other significant differences in SPL value were detected among the remaining trials for TMT-A. For TMT-B, a similar effect was observed (Trial 1-5: p < 0.001; Trial 1-6: p < 0.05; Trial 1-7: p < 0.01; Trial 1-8: p < 0.05). Thus, to report EEG results from as many trials of self-consistent data as possible, the behavioural and EEG recordings of Trial 1 in both TMT parts were excluded from all subsequent analyses. After these exclusions, the effect of trial was absent across the remaining seven trials for both TMT parts. The overall SPL distributions were then

45 compared across participants using the two-sample Kolmogorov-Smirnov test. In addition, as part of visually inspecting these distributions, bootstrapping was used to estimate 95% confidence intervals of the mean value at each sampling bin. The skewness of the individual distributions was computed for subsequent Wilcoxon signed rank tests between TMT-A and TMT-B. Lastly, the SPLs were averaged across the seven trials for each participant for both TMT parts, and another Wilcoxon signed rank test was performed to assess the effect of TMT part on the SPL averages.

Additional kinematic analysis was conducted to characterize tablet interactions during TMT performance. Specifically, the time derivative of stylus position coordinates (x, y) was estimated using finite differences divided by the sample period, resulting in plots of speed versus time for each participant for both TMT parts. This method of estimating derivatives is prone to spikes or outliers and other noise; thus the speed plots were further processed using a custom Sigma filter for spike removal207, and a low pass filter with 5 Hz cut-off frequency. The Sigma filter was a sliding time window operator (window length = 151 ms) that provided smoothing by substituting outlier values outside the range of two standard deviations with the average of adjacent non- outlier values. The 5 Hz cut-off frequency for the low pass filter was chosen by observing that 95 % of the spectral content of speed time-courses was below 5 Hz. The speed data were then interpolated using cubic splines and resampled at a rate of 1000 Hz to match the EEG sampling frequency.

The speed time-courses primarily showed two characteristic behavioural features (Figure 2.3): 1) periods of slow stylus speed, presumably due to preoccupation with visual search activities and subsequently called “non-linking periods”; and 2) periods of acceleration to peak speed, followed by similar deceleration, characteristic of purposeful movements to link stimuli and subsequently called “linking periods”. Due to the absence of other EEG and behavioural time markers to identify the onsets and offsets of these periods, the linking and non-linking periods were separated using a speed amplitude threshold. The speed threshold was established manually by slowly increasing the candidate speed threshold from 0 px/s with a constant step for each participant until the linking periods were readily identifiable (i.e. showing a consistent pattern of rapid acceleration and deceleration across individual links), and the number of linking periods matched the total number of links previously determined by visual inspection of the video recordings. Subsequently, time indices specifying linking and non-linking periods were obtained for every trial, enabling the average speed of individual links, non-linking periods, and linking periods to be calculated within-test for each participant and for both TMT parts. Similar to SPL, the distributions of average link speed, non-linking period and linking period were quantified and visualized using histograms, with 95 % confidence intervals estimated as described above. The overall distributions across participants were compared between TMT parts via two-sample Kolmogorov-Smirnov tests. The distributions for individual participants were also examined to compute the 95% confidence intervals for the mean values represented in the overall distributions, and the skewness for Wilcoxon signed rank tests between TMT parts. In addition, the average link speed, non-linking period, and linking period were averaged across seven trials for both TMT parts with the first trials excluded. Wilcoxon signed rank tests were subsequently performed to assess the effect of TMT part on these averages.

2.2.6 EEG Data Analysis

Using EEGLAB208 and custom MATLAB software, the EEG data were down-sampled to 1000 Hz, passed through a band-pass finite impulse response filter with 0.1-100 Hz bandwidth, and decomposed using independent component analysis (ICA) to remove spurious eye blinks, eye movements, muscle noise and channel noise. Identification of artifactual ICs was conducted using an automated classifier algorithm based on machine learning209, which was trained using large scale EEG data and expert labels to reliably and accurately label the signal content of an IC (i.e. brain, artifact, noise) as percentages210. The labelled ICs with more than 90% artifact or

47 noise were excluded from further analysis. To maximize the effects of interest in the frontal electrodes, the EEG data were re-referenced to the mean mastoids using the average signal from electrodes TP9 and TP10, situated close to the earlobes. Because artifacts were removed on all electrodes before the re-reference, the EEG signal quality was not compromised by the re- reference and no artifacts were projected back into the signals. With the first trials removed, the EEG data for each participant were subsequently parsed and concatenated into time-courses for TMT-A and TMT-B. Each TMT trial was parsed along with a 10 s pre-stimulus baseline period, consisting of visual fixation. Given that the TMT stimulus onset was defined as time 0, each trial segment was from -10 to 42 s for TMT-A, and -10 to 62 s for TMT-B. The extra 2 s beyond the task time limit was included in each case to avoid “edge artifacts” during time-frequency decomposition (see immediately below). The analysis included only the EEG power over the available time duration that participants were performing each TMT part within the block duration (as the completion time was variable, and some participants completed a given TMT part faster than the time allotted).

Using custom MATLAB software based on published methodology17, individual trials underwent time-frequency decomposition over the frequency range of 0.1-50 Hz using complex Morlet wavelet convolution. Within the frequency range, wavelets were generated with 20 frequencies that increased exponentially. All wavelets had 6 cycles, irrespective of frequency. Based on their frequencies, the wavelets were then assigned to the five major EEG frequency bands: delta (0.3-4 Hz), theta (4-8 Hz), alpha (8-13 Hz), beta (13-30 Hz), gamma (30-50 Hz). Four wavelets with frequencies closest to the boundary frequencies of EEG frequency bands (i.e. at 4 Hz and 8 Hz) were assigned to both the slower and faster frequency bands, which slightly expands the frequency band ranges to account for individual variability in frequency bands. The total power at each frequency as a function of time was then baseline-corrected using decibel (dB) normalization. The baseline period selected for the analysis was the 10 s visual fixation before each TMT trial. Only 7 s (-8 to -1 s) of the period was used to avoid edge artifacts produced by wavelet convolution. In the time-frequency total power within the completion time of each trial, average time-frequency total power was computed across predetermined linking and non-linking time indices as well as wavelet frequencies assigned to individual frequency bands. For example, in each trial, average power was extracted for the “linking period delta band”, “non-linking period delta band”, “linking period alpha band”, etc. The average time-

48 frequency power was then averaged across the seven trials for each frequency band, time period, and electrode.

Figure 2.4 Task Partial Least Squares (PLS) algorithm for the average EEG time-frequency power.

To assess and interpret the multivariate task-related effects on EEG time-frequency power using non-parametric statistics, task-based partial least square (PLS) regressions were performed using MATLAB104,211,212 (Figure 2.4). Task PLS was chosen for its ability to identify distributed brain networks from noisy and highly correlated EEG data. The method uncovers the relations between two input matrices (X and Y) that are used to generate a covariance or correlation matrix, by identifying sets of paired latent variables (LVs) derived from X and Y that show maximized covariance. The EEG power data of different task conditions were assigned to the X matrix, whereas the task conditions were considered as the Y matrix or loading vector. The number of potentially significant LVs was set equal to the number of task conditions. To evaluate the main effects of TMT parts and time periods, an omnibus task PLS analysis was first conducted involving the time-frequency power in four task conditions: linking period of TMT-A (Link A), linking period of TMT-B (Link B), non-linking period of TMT-A (Nonlink A), and non-linking period of TMT-B (Nonlink B). The dimension of the X matrix was thus 64 (16

49 participants × 4 task conditions) by 150 (30 electrodes × 5 frequency bands), whereas the dimension of Y matrix was 64 (16 participants × 4 task conditions) by 4 (4 task conditions). The effect space E was derived using the XT × Y cross product matrix, which was then decomposed using singular value decomposition to generalize eigenvectors and eigenvalues indicating the spatial and task saliences of each latent variable with variance explained. The statistical significance of the LVs was assessed by 1000 permutation resamples. The p-values and bootstrap ratios (BSRs, mean loadings divided by standard deviations) were obtained for all saliences using bootstrapping with 1000 resamples. In addition, the saliences of paired task contrasts were also assessed to compare the strength of contrasts. These saliences were computed by subtracting the salience resamples of one task condition with the permuted salience resamples of another task condition across all combinations of task conditions. The associated mean loadings, standard deviations, and BSRs were then subsequently calculated. Notably, the multiplication between eigenvalues (variance explained) and eigenvectors (spatial and task saliences) gives the portion of the covariance; thus, the saliences are inherently normalized (i.e. unitless) according to the procedure described above.

To further differentiate the spatial pattern for each effect across electrodes, follow-up task PLS subtests were conducted involving paired conditions such as Link B vs. Link A, Nonlink B vs. Nonlink A, Link A vs. Nonlink A, and Link B vs. Nonlink B. In addition, the general effect of time period in TMT-A and TMT-B was assessed in a combined comparison by adding the time- frequency power of Link A with Link B, and the analogous computation for the Nonlink conditions, permitting a subtest of Link (A+B) vs. Nonlink (A+B). The PLS analysis procedures were conducted in analogous fashion to those described above. Non-significant BSRs, as determined after correction for multiple comparisons according to the false discovery rate (FDR) method using q = 0.05, were assigned a value of zero for representation in the maps. An additional threshold of |BSR| > 2 was employed to exclude any results that were found to be unstable at a given electrode during the PLS resampling procedure.

2.3 Results 2.3.1 TMT Performance

Table 2.1 summarizes behavioural metrics of TMT performance across the participants for the experimental design. The results are averaged over the seven trials and sixteen participants. The

Wilcoxon signed rank tests compared the trial averages of TMT-A and TMT-B. Although the mean completion times are smaller than the block duration of the TMT part, there is variability across trials and participants, as some trials were completed within the block duration, but some were not. In addition, the number of errors and correct links indicate that participants performed both TMT parts well. A participant can complete a maximum of 24 correct links, while the number of total links may exceed 24 as it contains both correct links and errors.

Table 2.1 Behavioural metrics for participants (n = 16) performing the tablet TMT*. TMT Part Mean SD Range Signed Rank A 29.3 6.6 [16.1, 39.9] p < 0.001 Completion times (s) B 37.1 10.0 [20.5, 59.2] A 0.1 0.5 [0, 4] p < 0.05 Number of errors B 0.5 1.6 [0, 7] A 23.5 1.9 [10, 26] p = 0.2 Number of total links B 23.9 0.8 [18, 26] A 23.4 1.9 [10, 24] p = 1 Number of correct links B 23.4 1.9 [12, 24] *First trials of each TMT part were excluded from the calculation. SD = standard deviation.

As indicated by the means and signed rank p values, it is evident that participants exhibited significantly longer completion times, and higher numbers of errors in TMT-B than TMT-A. The variability in completion times and number of errors also increased in TMT-B than TMT-A. Smaller mean difference between TMT parts was observed in number of total links, because both parts have the same stimulus spatial pattern and the same number of links available. The number of total links is more variable in TMT-A than TMT-B, which suggests acclimation during TMT- A as it was administered before TMT-B. As suggested by the means and standard deviations, difference between parts was also small for number of correct links, because most of the trials were completed with no errors (i.e. 24 correct links out of 24 total links).

As shown in Figure 2.5, participants exhibited significantly larger values in TMT-B than in TMT-A for SPL (p < 0.001), linking period (p < 0.001), and non-linking period (p < 0.01) but not for average link speed (p = 0.079). The distributions for TMT-A were found to be significantly different than those for TMT-B for the variables SPL (p < 0.001), average link speed (p < 0.001), non-linking period (p < 0.001), and linking period (p < 0.001), respectively (Figure 2.6). In terms of the skewness of participant distributions, only average link speed (p <

0.01) and non-linking period (p < 0.05) demonstrated significant differences between TMT-A and TMT-B.

52 representing data points of the same participant are connected using a straight line. SPL = seconds per link, * = p < 0.05, ** = p < 0.01, *** = p < 0.001, n.s = p > 0.05.

2.3.2 EEG Time-Frequency Power

The omnibus task PLS with four conditions including linking period of TMT-A, linking period of TMT-B, non-linking period of TMT-A, and non-linking period of TMT-B revealed distinct EEG spatial patterns in each frequency band in the first latent variable, associated with pronounced effects of task (Figure 2.7). The first latent variable accounted for 64% of the data variance, with no others reaching statistical significance. Electrodes with significant BSRs reflect brain regions that are consistently part of a functional neural network that has the identified task

53 condition loadings across resamples. Among the five frequency bands, delta, theta, and alpha bands exhibited significant bilateral activity patterns across most of the electrodes. Specifically, the delta band showed a widespread spatial pattern with highly negative BSRs around the anterior and central regions, whereas the theta band showed highly negative BSRs in the temporal and occipital lobes as well as central posterior part of the brain. The alpha band showed highly negative BSRs in four frontal electrodes, whereas the beta band only exhibited marginally significant negative BSRs in two frontal electrodes. All BSRs in the gamma band were neither significant after FDR correction nor above the |BSR| > 2 threshold, and thus were not displayed.

54 band is not shown due to lack of statistical significance. (b) Mean loadings of task condition weights. Error bars indicate standard deviations. (c) Mean loadings of task contrast weights. Error bars indicate standard deviations. L = left, R = right, BSR = bootstrap ratio, Link A = linking period of TMT-A, Link B = linking period of TMT-B, Nonlink A = non-linking period of TMT-A, Nonlink B = non-linking period of TMT-B.

Associated with these spatial patterns, the mean loadings of task condition weights exhibited an increasing trend across the tasks arranged in the order (Nonlink A, Nonlink B, Link A, Link B) with Nonlink A showing the strongest negative weight, and Link B showing the strongest positive weight (Figure 2.7b). An important feature of this weighting pattern was the different influence of time periods and TMT parts to the variance in EEG power (Figure 2.7c). Specifically, the Link B vs. Nonlink B and the Link A vs. Nonlink A contrasts were the strongest contributors, followed by Link B vs. Link A, whereas Nonlink B vs. Nonlink A was the weakest.

Figure 2.8 The BSRs of the first latent variable in the Link (A+B) vs. Nonlink (A+B) task PLS subtest. All electrode BSRs were FDR-corrected and thresholded as described in the text.

The omnibus PLS results of Figure 2.7 motivated three reduced analyses, which are presented in ranked order from largest effect to smallest effect. First, to investigate the effect of time period, task PLS analysis of the Link vs. Nonlink periods in both TMT parts revealed widespread BSR spatial patterns after FDR correction and thresholding. The subtask of Link (A+B) vs. Nonlink (A+B) is shown in Figure 2.8 for conciseness. Constrained to fixed values by the task PLS algorithm, the significance (p-value) and explained variance of latent variables are not meaningful in a two task PLS analysis, thus only the first latent variable was reported. Similarly, the mean loadings are not meaningful in this scenario and thus were not reported. Highly negative electrode BSRs were detected across the scalp in the delta band, with exceptions mostly in the occipital regions. The theta band consistently demonstrated highly negative BSRs across the scalp, whereas the alpha band exhibited highly negative BSRs in the bilateral frontal regions. Beta and gamma bands revealed moderately negative BSRs localized in the bilateral frontal regions with a slightly right-lateralized spatial pattern.

Second, investigating the effect of TMT parts on EEG power, PLS sub-analysis of only the Link B and Link A data (thus investigating the Link B vs. Link A contrast) revealed electrode activation in the delta and theta bands (Figure 2.9). In the delta band, a slightly left lateralized pattern of negative BSRs was observed in the frontal and temporal regions, also including one

56 central posterior electrode. In the theta band, the effect of parts was localized in one medial posterior and one medial occipital electrode.

Third and finally, task PLS analysis of the Nonlink B vs. Nonlink A conditions failed to detect any electrode activation after FDR correction and thresholding.

2.4 Discussion

Enabled by the digitizing tablet technology, this is the first study to examine the intra-test behaviour and associated EEG findings for TMT performance in young healthy adults. The study findings support the hypotheses, demonstrating a significant effect of time periods (non-linking, linking); and of TMT part (A, B) during TMT performance – as quantified by tablet-based kinematic metrics and EEG time-frequency power. The spatial dependence of the EEG findings is consistent with the existing fMRI and EEG literature investigating the TMT and reporting differences in brain activity between parts A and B. The overall findings are subsequently discussed in detail below, focusing first on behavioural results obtained from tablet-based kinematic recordings, and then on the EEG results. Limitations of the study are also indicated, as these have a potential influence on how the results are interpreted.

2.4.1 Behaviour

Following fMRI studies of the TMT that have used a block design, the present study investigated fixed durations of TMT performance in comparison to a control task performed during another fixed duration. This type of design provides good sensitivity for detecting brain activation in the fMRI context, and was adopted here so that the behavioural and EEG results could be compared well with the existing fMRI literature. The fixed block durations necessitated a slightly different method of scoring TMT performance, however, which is normally based on completion time. In the present work, it was necessary to account for the fact that not all of the participants completed the TMT parts within the chosen block duration. The seconds per link variable, SPL (completion time / number of correct links) was adopted here, and was previously developed to assess TMT performance for each participant based on their full or partial performance of the test133. Excluding the first trials, 87.5% of the TMT-A trials were completed within the 40 s block duration, whereas 97.3% of the TMT-B trials were completed within the 60 s block duration. For the incomplete trials, the median total number of links was 21 (range: 10-23) in

TMT-A, whereas the median was 19 (range: 16-23) in TMT-B. In such cases, the completion time can be estimated by multiplying the SPL value by the correct number of links to be performed in each part (24). Significant correlations have been shown between tablet and paper SPL values; although tablet SPLs were shown to be larger than paper SPLs119,133. Such results indicate that tablet SPL values are a reasonable approximation of paper TMT performance, but that inherent differences are still evident between tablet- and paper-based test administrations: in environment, posture, sensation and subsequent motor performance.

In the present study, SPL values for TMT-B were significantly larger than the values for TMT- A, consistent with previous behavioural results119,133. This supports the elevated cognitive demand of TMT-B compared to TMT-A. In addition, the distribution of SPL values was also evaluated by binning the SPL values for each trial for all participants. The distributions were both found to be right-skewed, indicating that participants took extended time to perform some of the trials in TMT-A and TMT-B, compared to most of the trials. The distributions were found to be statistically different, in keeping with the effect of TMT part on SPL values, although no difference in skewness was found for the sample of participants studied. It is possible that a difference in skewness could be found if a study with a larger sample of young healthy adults was undertaken, as there was a substantial difference in the histogram frequency value for both parts in the bin range from approximately 0.7-1.0 s, corresponding to the fraction of trials which were conducted the fastest. For these trials, the median and 95% confidence intervals for part A and part B did not overlap – with TMT-A exhibiting the higher frequency, again consistent with the increased cognitive demand occurring in TMT-B.

The digitizing tablet also permitted a more detailed study of within-test behavioural performance of the TMT. Inspecting the speed time-courses of tablet interaction enabled two prominent features to be identified during TMT performance: linking periods characterized by rapid acceleration and deceleration, indicating when participants used the stylus in purposeful movements to connect two stimuli; and non-linking periods characterized by negligible or low stylus motion, when participants conducted visual search and cognitive processes as required to form motor plans prior to purposeful movement. These periods were observed in all participants, especially in the later part of performing TMT-A or TMT-B, indicating that the predominant strategy consisted of visual search for the next link, executing the link, then proceeding with the next one. (Other completion strategies were also observed much less frequently. For example,

58 some participants visually searched as far as they could initially and then executed many links consecutively - however, they resorted to the former strategy as their working memory buffer exhausted.)

Both the non-linking and linking periods were found to be longer in TMT-B than TMT-A. Taken together with the SPL findings, these results indicate the elevated processing demands of TMT-B compared to TMT-A as longer time was required to search, plan, and execute individual links. In terms of average link speed, no significant difference was found between TMT parts, despite that the links in TMT-A were performed slightly faster than that in TMT-B. This is an interesting result, suggesting that participants performed intentional movements quite similarly in the two TMT parts – i.e. once they identified which stimuli were to be linked, the linking movement was performed without much influence from whether it was number-number linking or number-letter linking. It is also notable that the effect of TMT part was larger for the linking period than for the non-linking period, suggesting that the difference in cognitive challenge between the two TMT parts was pronounced in this time interval for the participants studied. In other words, higher cognitive processes were found to engage more during linking movement execution than preparation.

It was also found that the distributions for average link speed, non-linking period and linking period values across the participants were different when comparing performance of TMT-A and TMT-B, consistent with the interpretations given above regarding the effect of TMT part on the median values of each metric. In particular, the non-linking period values demonstrated a small but significant difference in skewness of the distribution between TMT-A and TMT-B performance, with the distribution for TMT-B skewed to slightly longer non-linking periods. This reflects the variable cognitive demands for visual search, motor planning, and executive functioning between non-linking periods of TMT-A and TMT-B across participants. In addition, the average link speed values showed a similar but larger difference in skewness, with the distribution for TMT-B skewed to slightly slower speeds. This provides a potential qualification to the argument given above about the similarity of movements during linking periods for both TMT parts, suggesting that the linking period during TMT-B may require slightly more cognitive resources than during TMT-A. The sample size and variability in behavioural results across participants suggest that a larger experiment should be conducted in the future to provide a clearer interpretation. Given that the distribution for SPL showed more variability than that for

59 average link speed, linking, and non-linking periods (as indicated by the 95% confidence intervals for each TMT part at each sampling bin), significant effects of TMT part on skewness were not observed for SPL.

For average link speed, linking period and non-linking period, it should also be recognized that the width of the histograms partly depends on the spatial arrangement of the stimuli in TMT-A and TMT-B, and across the different trials. For example, in the case of a “perfect responder” performing parts A and B correctly with no cognitive effort, the histogram for average speed would be a single spike at the speed of execution; the histogram for non-linking period would be a single spike at a value of zero; and the histograms for the linking period would have the median values and widths reflecting the spatial arrangement of the stimuli and the range of distances between stimuli to be linked. Thus, these histograms could potentially be different for TMT-A and TMT-B; and it is notable that the present task design is based on a previous TMT study demonstrating that the differences in spatial arrangement of TMT stimuli (i.e. longer link distance in TMT-B than TMT-A) contribute to the difference in difficulty between the two parts121. To investigate whether this was a confound in the present task design, the performance data from the perfect responder were simulated across all the trials of TMT-A and TMT-B that were implemented with different stimulus patterns. The resulting distributions of the SPL value and linking period were subsequently found to be very similar between TMT-A and TMT-B, with negligible difference in the means and medians. Therefore, the observed differences between TMT parts in the metrics distributions can be attributed to the differences in cognitive demand, as intended by the task design, and not differences in the spatial arrangement of the stimuli.

In summary, the digitizing tablet has enabled the development of multiple new metrics of TMT performance (SPL, average link speed, non-linking period, and linking period) that provide more insight than the traditional method of scoring the test using completion time. It was found that SPL, linking and non-linking periods all showed a significant effect of TMT part whereas for the average link speed, effects were only detectable when examining distribution skewness. These results are specific to the population studied: young healthy adults. It may be that one of the metrics, or another not yet derived from the tablet data, is particularly sensitive and specific for characterizing patients with deficits in certain aspects of brain function. For instance, patients with traumatic brain injuries may experience difficulties in sustained attention, and set switching,

60 leading to elevated non-link periods and larger effect size between TMT parts for this measure. Supporting the use of tablet-derived behavioural metrics, another commonly used neuropsychological test - the Clock Drawing Test - has been shown to have improved diagnostic accuracy and detection sensitivity in patients with early dementia and mild cognitive impairment when implemented on a tablet, and quantified using a “time-in-air” metric (defined as the stylus transition time from completing one stroke to starting the next without contacting the tablet surface)213. Tablet-based neuropsychological tests are also amenable in principle to a data-driven approach to discriminate populations with different brain health characteristics, rather than using the a priori definition of specific kinematic variables.

2.4.2 Brain Activity

During linking periods of TMT-A and TMT-B, participants moved their stylus rapidly on the tablet, which required focus on coordinating and executing the motor movements to link items. During non-linking periods, the stylus was held stationary or was moved slowly, suggesting the involvement of multiple cognitive processes in preparation for subsequent links, such as visual search, motor planning, working memory and set-switching. Identifying the non-linking and linking periods was important for improved characterization of TMT behavioural performance, and the same was true for interpreting the associated EEG signals. The effect of time period was sufficiently strong that an analysis without considering this effect would have led to misleading results. The use of task PLS analysis critically enabled the effects in the EEG recordings to be interpreted with respect to time period, TMT part, and electrode.

The omnibus PLS lead to a ranking of effects (from largest to smallest: time period; TMT part during linking period; TMT part during non-linking period) that motivated subsequent two- condition PLS analyses for more revealing evaluation of the patterns of activity across EEG electrodes, than was obtainable from the omnibus analysis alone. The two-condition PLS analyses re-affirmed the effect of time period (linking period versus non-linking period) and TMT part (TMT-B versus TMT-A), but indicated that the latter was only significant during the linking period. Despite the arbitrary sign of the condition mean loadings in two-condition PLS analyses, increased negative signal strength (i.e. highly negative BSR) is interpreted as decreased frequency band power (i.e. greater desynchronization). Event-related desynchronization in EEG signals has been associated with higher cognitive processes such as memory, sensory perception,

61 and motor movement46. Consistent with these observations, linking periods showed greater desynchronization than the non-linking periods, and TMT-B exhibited greater desynchronization compared to TMT-A during linking periods. These specific contrasts are subsequently discussed below in relation to the spectral power and spatial characteristics of the associated EEG signals.

As shown in Figure 2.7c, both link vs. nonlink contrasts had larger mean loadings than the two TMT-B vs. TMT-A contrasts. The effect of time period demonstrated different electrode spatial patterns in all five frequency bands examined: delta, theta, alpha, beta, and gamma. Compared to linking periods of both parts, non-linking periods exhibited a widespread increase in delta band power, with the exceptions of a few frontal and occipital electrodes. Delta oscillations have long been implicated in attention processes214, especially as the mechanism underpinning selective attention215. Increased delta power in the resting-state is also associated with increasing activity in the default mode network (DMN)216. Therefore, the results suggest varying attentional demands and distinctively engaged functional networks during visual search and focused motor action. The sources underlying the detected delta band power remain to be directly determined. Based on previous simultaneous EEG and fMRI research20, the difference in delta activity may be originated from the precuneus, posterior cingulate, inferior frontal cortex, and medial prefrontal cortex, which were also shown to be involved during TMT performance in previous fMRI studies119,133.

With regards to theta band activity, non-linking periods demonstrated uniformly increased theta band power across the scalp, when compared to linking periods. Commonly found in the hippocampus24, sensory cortex25, and cingulate cortex27, theta activity has been associated with working memory25,28–31 and long-range synchronization33. Some EEG-fMRI research supports the association between the hippocampus, the cingulate and theta activity34, but some does not35. An increase in theta activity has also been found to be negative correlated with fMRI brain activity in regions that overlap the DMN36, indicating heightened alertness37, and theta and gamma waves have been found to be phased-coupled throughout the cortex217 especially during visual working memory tasks218. Therefore, the widespread increased theta power in non-linking periods is suggestive of a globally heightened alert state during visual search and motor planning, prior to motor execution.

Compared to linking periods, a widespread increase in alpha activity was evident in non-linking periods, especially in bilateral frontal regions. Originating from cortical and thalamic sources2,52,53, alpha activity exhibits negative fMRI signal correlations in cortical and DMN regions, and positive correlations in the thalamus37,53,58–60. Prefrontal alpha synchronization is indicative of internally-oriented cognitive processing and attention, which is typically seen during mental imagery and planning219,220, whereas decreased occipital alpha synchronization is associated with externally oriented attention221 and visual perception222. Alpha band power is also implicated in memory28,223, and negatively correlated to fMRI signals in deep and superficial layers of the visual cortex224. Synthesizing these findings from the literature, therefore, increased alpha band power in non-linking periods likely reflects the internally oriented cognitive processing triggered by visual perception of (and visual attention to) the test stimuli. Working memory during TMT performance may also contribute to the increased alpha band power. Given the widespread spatial pattern with a frontal projection, the detected difference in alpha activity maybe to be the product of frontal cortical and thalamic generators as well as their interactions. This is somewhat consistent with fMRI TMT results, with the caveat that thalamic activity is not a strong effect with this modality and required exploratory analysis with a relaxed cluster-size threshold119,133. In addition, such results also suggest that mu rhythms (the alpha waves found in motor-related areas) were in turn suppressed during linking periods, consistent with previous EEG research225,226.

Compared to linking periods, non-linking periods showed increased and slightly right-lateralized beta band power in several frontal and prefrontal electrodes. A previous study has reported increased beta power immediately after stimulus onset, which is then followed by a power decrease during movement preparation and execution227. Beta waves originating from the deep layers of prefrontal cortex are associated with working memory encoding, retention, retrieval, and reallocation70–72; whereas beta waves generated by the primary motor cortex during motor activity play a role in associating sensory input with the motor command47,73,74. In the present context, the increased beta band power in non-linking periods is consistent with more involvement of working memory (and not sensorimotor processes) during which the participants were actively engaged in visual search and forming motor plans to link stimuli with specific numeric and alphabetic relationships. The right prefrontal lateralization may be related to TMT- B sequencing and shifting errors, as previously shown to be associated with lesions in the right

63 hemispheric dorsolateral prefrontal cortex134. This suggests that the right lateralization may be important to maintain performance accuracy in TMT. Furthermore, a simultaneous EEG-fMRI study identified a positive correlation between beta power and fMRI signals in the posterior cingulate, precuneus, and prefrontal cortex35. In the present study, only the latter region showed elevated beta power in non-linking periods compared to linking periods, likely because the posterior cingulate and precuneus overlap with the DMN21 and are generally suppressed during fMRI studies of the TMT119,133. The beta power results of Figure 2.8 also suggest that the DMN is similarly suppressed during non-linking and linking periods of TMT performance.

Compared to linking periods, non-linking periods showed a slightly right lateralized gamma band power increase in a few prefrontal electrodes. In animal studies, gamma waves have been associated with sensory and complex cognitive processes such as perception81, attention82, and memory83. The sources of gamma waves have yet to be identified in human brain but may be located in the somatosensory cortex and primary visual cortex, inferring from animal studies94,95. Therefore, a high neuronal excitability (i.e. increased sensitivity to synaptic inputs) during visual search and cognitive processes such as memory and set-switching is probable. In addition, the decrease in right lateralized prefrontal gamma activity in linking periods might represent a mechanism to inhibit neural activity that interferes with linking-related sensorimotor and cognitive processes228. It is also important to note the ongoing debate around the functional roles of gamma waves. Some studies report that gamma waves are associated with feature binding mechanisms86,87 and intrinsic network properties88,89, whereas others that there originate simply from oculomotor90 and muscle artifacts91,92. Further source estimates and topographic analyses are required to identify sources of gamma oscillations while excluding non-neuronal electrical activity in the EEG signal. In addition, it is interesting that the gamma and strongest alpha signals are largely from the same brain regions; therefore, improved SNR and data quality of EEG are needed to examine phase amplitude coupling between gamma and alpha signals in the future TMT studies.

Regarding the effect of TMT parts in linking periods, TMT-A demonstrated increased delta power with a modest left lateralization compared to TMT-B in three left temporal electrodes, two right frontal electrodes, and one electrode near the primary motor cortex. Such left lateralization suggests the elevated executive demand in TMT-B performance, which was also shown in the previous TMT studies of brain activity116,119,133,174. However, no effect of part was detected in

64 non-linking periods. The EEG data are consistent with the interpretation of less attentional demand and slightly more DMN activity in TMT-A than in TMT-B. During TMT performance, attention plays an important role in both linking and non-linking periods, and notably the left lateralization was only detected in the linking periods rather than the non-linking periods. A plausible interpretation of these findings is as follows: less cognitive effort is required to execute motor actions in linking periods of TMT-A; whereas in those of TMT-B, the higher cognitive processes such as set-switching and working memory remain engaged to maintain both letter and number sets. On the other hand, analogous visual search and motor planning may dominate the EEG signal in non-linking periods of both TMT parts, thus diluting the signal differences due to executive functioning.

The EEG signals for TMT-A performance demonstrated increased theta band power in central posterior and occipital regions compared to those for TMT-B. Such results support the interpretation that TMT-A is associated with decreased working memory demand in relation to TMT-B. Given the involvement of working memory in the TMT149 and previous fMRI findings119,133, the detected difference in theta waves is likely to originate from sources such as the hippocampus, sensory cortex, and cingulate cortex, as theta waves from all three regions play a role in working memory.

It is premature to draw direct comparisons between the EEG findings of the present study and brain regions identified at approximately 1 mm spatial resolution in fMRI studies of TMT performance. Nevertheless, it is interesting that the EEG spatial saliences are consistent with what would be expected given the fMRI literature. Specifically, the delta power increase was observed in the proximity of the right SMA (medial frontal gyrus), right premotor cortex, left inferior frontal gyrus, and left precuneus, suggesting that TMT-B performance requires more executive functioning and motor-related processing than TMT-A. This result is consistent with previous simultaneous EEG-fMRI research showing association between BOLD signals and delta band activity in the right medial frontal gyrus and left inferior frontal gyrus20. The theta power increase was found near the precuneus, posterior cingulate cortex, and visual cortex, reflecting an increased memory demand and suppression of DMN during TMT-B performance compared to TMT-A. This interpretation is supported by past simultaneous EEG-fMRI experiments showing the relation between theta band activity and the cingulate cortex34,36. These regions have all been shown to be involved during TMT performance in previous fMRI

65 studies119,133, but a direct relationship between the results of EEG and fMRI is not presently supported as EEG source localization has not been conducted. This can be undertaken in the future.

It is interesting that the EEG results during linking periods show most agreement with the previous fMRI literature. The effect of time period is very strong in the EEG results, in keeping with the biophysical mechanism of signal contrast and postulated neural processing, whereas this effect has not previously been investigated in TMT-related fMRI research. As the linking period is longer than the non-linking period, it probably predominates the fMRI results that have been reported to date. However, future research could be undertaken using very high temporal resolution fMRI to study differences in hemodynamic responses in the non-linking and linking periods229,230.

The present EEG results somewhat agree with the past EEG study of the TMT174. In terms of high frequency oscillations in both TMT parts, beta and gamma band power were found to be more active in the frontal regions than posterior regions of the brain when comparing time periods, which is consistent with the past EEG study. No significant difference between TMT parts was found in beta and gamma band power, which somewhat agrees with the similar spatial patterns between parts observed in the past EEG study. A more left lateralized posterior brain activity in TMT-B than TMT-A was suggested in the past EEG study but was not detected in the effect of part. In the present work, the effects of time period and part were evaluated, not the spatial pattern of brain activity in TMT-A and TMT-B. Due to the differences in test design, experimental hardware, and data analysis techniques, many EEG results were not directly comparable between the past EEG study and the present study.

There are few limitations to the present study. First, the TMT is administered only once in the clinic to assess cognition; however, the EEG data analysis required multiple TMT trials of self- consistent data to provide sufficient signal-to-noise ratio (SNR), inevitably introducing practice effects. This was subsequently confirmed in preliminary statistical analysis showing that participants performed the first trial of TMT-A and TMT-B more slowly than the other trials. In the future, advanced functional neuroimaging techniques should be developed to extract meaningful interpretation from a single TMT trial for enhanced clinical relevance. Although this may be difficult to do with EEG, it may be possible to achieve with fMRI conducted at ultra-high

66 fields, such as 7 T and above. Second, because the cognitive processing was asynchronous, complex, and continually varying during TMT performance, and across participants, it was challenging to classify and discriminate the linking and non-linking periods with complete certainty while ensuring that participants were not pursuing other strategies (such as searching for the next stimuli to be linked while performing the current link). Future studies combining EEG recordings with eye-tracking would be useful to characterize the visual search performed by participants during TMT performance, enabling periods with consistent behaviour to be better identified. An alternative is to adapt the TMT paradigm for EEG into separate individual cognitive components, similar to the Delis-Kaplan Executive Function System (D-KEFS) TMT, a popular TMT variant widely used clinically137. Third, the modest sample size of the present study placed a limit on the effect sizes that could be detected with statistical significance, and introduced uncertainty about whether the results are fully representative of a larger population of young healthy adults. Conducting an additional EEG study of TMT performance with a larger sample size would be useful to investigate whether the present findings can be replicated, and whether smaller yet relevant effects are detectable. Fourth, the study did not include a comparison between paper- and tablet-based versions of the TMT to assess ecological validity of the behavioural and EEG findings. Previous comparisons of paper- and tablet-based TMT performance are available in the literature for fMRI settings in which participants used the tablet lying down, reporting good ecological validity119,133. As participants interacted with the tablet while in a sitting posture for the present study, ecological validity is expected to be improved over what was reported in these previous studies – although this remains speculation. Fifth and last, the present study used a modern but relatively simple 32-channel EEG system, and enhanced EEG technology could be used to enhance SNR in future studies. For example, higher electrode count, in conjunction with electromyography and electrooculography would be useful to identify and suppress non-neuronal electrical signal artifacts. Simultaneous fMRI-EEG studies would also be useful to assist source localization and direct characterization of the relation between neural oscillations and fMRI signals in detailed neuroanatomical structures, advancing beyond the EEG lead analysis reported here.

In conclusion, longer SPL, linking and non-linking periods are evident in TMT-B compared to TMT-A, reflecting the increased cognitive demand in TMT-B, while link speed only exhibited such effect when comparing distributions, indicating a modest but dynamic sensitivity to

67 cognitive processing. Regarding brain activity, TMT-A exhibited increased left lateralized delta band power and posterior midline theta band power compared to TMT-B, indicating less attentional and working memory demands in TMT-A, as expected. Non-linking periods demonstrated widespread increases in slow oscillatory activities in the delta, theta and alpha bands, suggesting a decreased attentional demand when the stylus moved slowly or was stationary, as well as a heightened alert state for increased internal processing to prepare for subsequent linking behaviours. In terms of fast neural oscillations, non-linking periods showed increases in beta and gamma band power in frontal regions with a slight right lateralization, which reflects working memory involvement, performance accuracy maintenance, and interference inhibition. These observations contribute to increased understanding of the neural activity associated with TMT performance.

Chapter 3 3 Conclusion

As a supplement to previous functional magnetic resonance imaging (fMRI) studies of the Trail Making Test (TMT), which revealed averaged task-related brain activations with limited temporal dynamics, this thesis has investigated the temporal characteristics of electroencephalography (EEG) signal content during TMT performance using the fMRI- compatible tablet technology developed in the Graham laboratory. The improved understanding of TMT-related behavioural performance and brain activity that has come from this EEG work is beneficial to both clinical and research applications of the neuropsychological test. The following chapter discusses the hypotheses and findings of the thesis, as well as the implications, limitations, and future directions of the research.

3.1 Summary

Chapter 2 employed the fMRI-compatible tablet technology and concurrent EEG to characterize the temporal dynamics of behavioural kinematics and brain activity during TMT-A and TMT-B performance in 16 young healthy adults. For each TMT trial and participant, the tablet data were processed to establish a threshold speed to determine linking and non-linking periods of the test: the former primarily comprising purposeful movements to link two specific stimuli; and the latter primarily comprising memory retrieval and visual search to determine which stimuli should be linked at a given point in time. The EEG signals were decomposed using time-frequency analysis and averaged total power was computed based on the predetermined time periods. The two major hypotheses tested were 1) that differences in tablet-based metrics of TMT performance and in the spatial patterns of EEG frequency bands are expected between TMT-A and TMT-B, consistent with more cognitive demand in TMT-B and with existing functional neuroimaging literature relevant to TMT performance; and 2) that analogous differences are expected between linking and non-linking periods.

The tablet-based behavioural results indicated longer seconds per link (SPL), non-linking and linking periods in TMT-B compared to TMT-A, reflecting the increased cognitive demand in TMT-B, whereas average link speed only exhibited such effect when comparing distributions, indicating a more modest sensitivity. As for EEG-related brain activity, which was analyzed

69 using non-parametric multivariate task partial least squares (PLS), TMT-A exhibited increased left lateralized delta band power and posterior midline theta band power compared to TMT-B, consistent with less attentional and working memory demands in TMT-A as expected. Non- linking periods demonstrated widespread increases in slow oscillatory activities of the delta, theta and alpha bands, suggesting a decreased attentional demand when the stylus moved slowly or was stationary, as well as a heightened alert state for increased internal processing to prepare for subsequent links. Regarding fast oscillations, non-linking periods were associated with increases in beta and gamma band power in frontal regions with a slight right lateralization, consistent with working memory involvement, performance accuracy maintenance, and interference inhibition.

Overall, the results confirmed the hypotheses, provided useful experimental data of TMT performance and brain activity in young healthy adults, and showed how tablet and concurrent EEG experiments have the potential to inform and improve the current TMT usage in clinical and research settings.

3.2 Limitations

Several limitations of the experiments in this thesis are noteworthy. First, because the mental processing during TMT performance is continuous, complex, and varying in different individuals, the separation of linking and non-linking periods should be considered an approximate model of the underlying behaviour. The separation was achieved by setting a trial- specific speed threshold for the stylus and although this was a reasonable approach, it does not guarantee that all participants were perfectly adopting the linking and non-linking strategies. As mentioned in Chapter 2, there was anecdotal evidence of some participants adopting other strategies as part of their TMT performance. This may have reduced the effect size and the spatial pattern of effect when comparing oscillatory EEG signals from linking and non-linking periods. Second, the detected neural oscillations were complex due to the multiple aspects of mental processing inherent to TMT performance, and difficult to disentangle because the oscillations have multiple underlying mechanisms that may interact with each other. This adds uncertainty to the suggested connections between certain neural oscillations and certain behaviours or cognitive attributes of TMT performance. Third, it is likely that the low sample size (n=16) of the present study limited the effect size of TMT part in the detected EEG electrode

70 signal power. Fourth, a 32-channel EEG system was used in the experiment with no electromyography (EMG) and electrooculography (EOG), thus limiting the scope for suppressing signal artifacts from muscle or eye sources.

3.3 Future Directions

The present work sheds light on the use of the tablet and EEG to characterize the temporal dynamics in TMT behaviour and brain activity as a supplement to previous fMRI findings. Many research directions stem from this work, some of which are briefly discussed below.

Initially, a number of methodological refinements should be made to facilitate further interpretation of TMT-related EEG signals, addressing limitations of the present study. For example, event triggers indicating the onset and offset of linking behaviour may be added using machine learning algorithms to improve the calculation of link-based measures, and the separation of linking and non-linking periods, as well as the associated EEG time-frequency power differences. A machine learning approach is reasonable, as adding such triggers to the current tablet TMT paradigm would require real-time monitoring of the stylus speed to differentiate visual search and movement execution, as well as precise tracking of the task progress to determine the current link – elements that are challenging to program robustly across individuals with variable behaviour.

Additionally, the TMT paradigm used for EEG recording should be modified to separate individual cognitive components of the TMT, such as planned movements, number processing and set-switching, similar to the Delis-Kaplan Executive Function System (D-KEFS) TMT, a TMT variant in wide clinical use137. It would then be possible to compare the EEG signal content from both the D-KEFS TMT variant and that of the present study, to validate the interpretation of TMT-related EEG signals given in this thesis.

To better understand the effect of TMT parts on EEG signals, exploratory power analyses may be conducted using the present data. More participants may also be included to the data set to examine the effect size of TMT part in EEG electrode spatial pattern. Without greatly lengthening the experimental setup time, increasing the number of EEG electrodes to 64 or higher is also recommended in a low-noise measurement environment to improve the otherwise somewhat limited spatial resolution9, which may improve the accuracy of subsequent source

71 localization231 and facilitate functional connectivity analysis232. Although the optimal methodology and data analyses are still under development, such EEG results are promising as they may validate previous TMT-related brain activation and functional connectivity results from fMRI. In addition, EMG and EOG recordings should be acquired in conjunction with future EEG, as an effective approach to minimize non-neuronal electrical signal artifacts that are otherwise difficult to isolate in EEG, and that are a potential confound in the present study.

3.3.1 Simultaneous EEG-fMRI of TMT Performance

Despite the ability to examine the entire brain including subcortical structures with excellent spatial resolution (~mm), fMRI has poor temporal resolution (typically ~s) which does not provide meaningful information about neuronal activity that evolves on the millisecond time scale233. In comparison, EEG offers excellent sampling of electrophysiological brain signals at sufficient temporal resolution (~ms) – but primarily from cortical regions, with limited spatial resolution (~cm). Simultaneous EEG-fMRI studies aim to combine the advantages of both neuroimaging studies in a single recording session, and have become more popular in neuroscience research with more than 900 publications over the past 20 years, covering applications such as source localization, cognitive neuroscience, neuropsychopharmacology, as well as epilepsy detection and characterization.

In the context of the TMT, simultaneous EEG-fMRI studies may assist in localizing the sources of neural oscillations and direct characterization of the relation between behaviour, neural oscillations and BOLD signals in detailed neuroanatomical structures35,58,59. An advantage of simultaneous EEG-fMRI is that both sets of functional neuroimaging data are collected from the same time-course of behavioural performance, which is particularly relevant for tasks that inherently include significant performance variability. Simultaneous fMRI-EEG of the TMT would be useful given that the test performance is quite variable across young healthy adults, as shown in this thesis. In addition, fMRI studies reveal that many neural sources are involved to support TMT-related behaviour119,133. Given that EEG source localization becomes more difficult as the number of sources increases, the fMRI results could potentially be very useful to constrain EEG source localization in a simultaneous EEG-fMRI study of the TMT. In the future, characterizing the brain activity underlying TMT performance with the enhanced spatiotemporal

72 resolution provided by this approach may reveal clinically relevant information to assist the diagnosis and treatment planning of cognitively impaired individuals.

Simultaneous EEG-fMRI experiments are non-trivial to implement, however. Challenges include ensuring that high quality multimodal data are collected, and in the case of potential TMT experiments, ensuring that behavioural performance is ecologically valid. Whereas the present thesis speaks to the latter issue, some discussion of the former issue is of interest to the planning of future work.

The predominant artifacts that must be dealt with in simultaneous EEG-fMRI is the large spurious voltages induced in EEG recordings from the switching of the gradient coils during fMRI. The gradient artifacts are approximately two orders of magnitude larger than the EEG signals of interest, and thus are of significant concern. Many approaches are developed to remove gradient artifacts “offline”, once the EEG data have been collected. A common method for offline removal involves subtracting a template of the average artifact signal from the EEG signal, although this can leave troublesome residual artifacts behind due to the frame-wise variability of the gradient-related signals in the EEG data234. Further adaptive artifact filtering can be performed to remove the residual artifacts based on fMRI slice timing, although with some compromise to the EEG signals of interest beyond the slice acquisition frequency. These processing steps are typically challenging to implement so that robust results are achieved, due to the large magnitude of the gradient artifacts.

The challenges associated with offline removal of gradient artifact have led to the search for other alternatives. The sparse acquisition approach is one such solution: the key concept involves leaving “quiet” periods within the repetition time (TR) of fMRI data collection, during which no gradients are switched and EEG data are recorded without gradient artifacts. All image slices can be collected at the beginning of each TR interval35,58,235; or spread evenly spaced 236,237; the former approach produces longer stretches of artifact-free EEG data and is therefore advantageous.

An optimized version of the sparse acquisition approach (fast fMRI-EEG) was recently proposed238. For demonstration purposes, researchers focused on the steady-state evoked potentials in visual cortex, which was generated by presenting flashing checkerboard patterns to a cohort of 9 healthy young adults while they maintained visual fixation. To minimize the

73 gradient artifact in EEG signal, fMRI scans were performed with a TR value of 2 s, with imaging of the entire brain with isotropic 5-mm resolution in 0.1 s using a method known as simultaneous multi-slice inverse imaging. Compared to standard fMRI-EEG, fast fMRI-EEG reduced the standard deviation of EEG signals over the fMRI acquisition period by more than half, indicating substantially decreased signal artifacts. After baseline correction, the EEG oscillatory activity from the visual evoked potentials was 2.5 times stronger in fast fMRI-EEG than conventional echo planar imaging (EPI)-EEG, reflecting improved EEG data quality238. Such methodological development is directly applicable to study TMT-related brain activity with predetermined event timing using fMRI-compatible tablet technology in the future.

Additional data processing is required in simultaneous EEG-fMRI to maintain data quality and reduce artifact produced by cardiac pulsations, also known as the pulse artifact or “ballistocardiogram”. This artifact is commonly removed by adding electrocardiography (ECG) recording in conjunction with simultaneous fMRI-EEG, such that the artifact can be controlled online and removed offline in various ways. Commonly used offline removal procedures include template subtraction and adaptive artifact filtering. Online control approaches include EEG- fMRI clock synchronization239,240 and maintaining stable EEG cable positions – for example by placing the head in a vacuum cushion. However, head movements still remain a concern for both EEG and fMRI recordings and should be carefully controlled to obtain high quality multimodal data.

Conversely, the impact of EEG on fMRI data quality is less problematic. The magnetic susceptibility of EEG electrodes and leads can produce “B0-Inhomogeneity” effects in fMRI, causing signal distortions or signal loss. The magnitude of such effect is heavily dependent on the composition of the EEG electrodes: electrodes with ferromagnetic nickel plating (AI990G,

Jari Electrode Supply, Gilroy, CA, USA) have a B0 perturbation coefficient 10 times higher than those with only diamagnetic materials such as silver, gold, copper, and plastic (E5GH, Astro- Med, Grass Instruments, West Warwick, RI, USA)241. These effects can be mitigated by proper selection of modern fMRI-compatible EEG hardware such as E5GH and EEG lycra caps (EasyCap, Herrsching, DEU).

In addition, a “B1-Inhomogeneity” artifact can result from radiofrequency (RF) coupling in the copper leads that connect EEG electrodes to EEG amplifiers, rather than in the electrodes

74 themselves. Such artifacts can cause fMRI signal degradation in the underlying brain areas. The

B1-Inhomogeneity artifact can be avoided by screening the EEG amplifier electronics and removing electrical connections between the amplifier and the recording system. The latter can be achieved by using wireless EEG systems, fiber-optic links, or encoding EEG signals as RF waves which can be detected by the MRI scanner242.

In addition to data acquisition and hardware developments, recent progress has greatly expanded the multimodal analysis options for simultaneous EEG-fMRI data243. For example, asymmetric data integration involves utilizing the maximum information possible from one modality to inform the analysis of the other. Despite providing a powerful method to localize the source of scalp EEG activity with excellent spatiotemporal resolution, fMRI-informed EEG can fail to differentiate cognitive processes engaged during the experimental task, limiting meaningful implications of brain functions. This limitation may be important when considering simultaneous EEG-fMRI of TMT performance. Alternatively, EEG-informed fMRI links neurophysiology with cognition and behaviour in detailed neuroanatomical structures to reveal functional brain networks based on a linear relationship between the two modalities in single trials. However, the inconsistent temporal precision in this technique makes the relative timing between fMRI brain activations and EEG rhythm generators difficult to disentangle. Nonetheless, this approach is applicable to study the brain activity associated with TMT performance by directly linking EEG signals with fMRI brain activations and tablet-based behavioural metrics of the TMT. The implementation of EEG-informed fMRI typically involves data quality improvement, EEG feature extraction, and prediction of EEG-related BOLD signal changes using a BOLD signal predictor244.

Last, neurogenerative models focus on the underlying biophysical and physiological mechanisms of EEG and fMRI signals and their interactions245. However, the lack of application to complex experimental paradigms such as the TMT and the dependence of spatiotemporal resolution on mathematical modeling and computational power indicate the need for further technical development and model optimization in this field of research. With the rapid advancement of machine learning, multivariate data-driven analyses have become increasingly powerful and flexible for revealing cross-modal relations and common hidden factors, as well as predicting behaviour using multimodal data246. Therefore, for applications in the tablet-based TMT, open-

75 source multimodal fusion algorithms may be employed as explorative analyses to supplement EEG-informed fMRI.

3.3.2 TMT Performance in Aging and Patient Populations

As a clinical tool to probe cognition, the TMT is commonly used to screen for and assess brain injuries114,120,166, neurodegenerative disease108,111 and schizophrenia135. Despite an excellent sensitivity, the specificity of the TMT is limited113. The digitizing tablet technology expanded the behavioural outcome measures of the TMT, including but not limited to the seconds per link (SPL), average link speed, non-linking period, and linking period. In addition to the conventional TMT scores (completion times and number of errors), the tablet performance metrics are available for comparison and analysis among aging and patient populations, which may improve the sensitivity and specificity of the TMT.

To explore the possibilities of such improvement, the performance metrics of the tablet TMT must be examined in patient populations with various brain diseases and injuries. In patients along the spectrum from Mild Cognitive Impairment (MCI) to Alzheimer’s Disease (AD), temporal dynamics in tablet TMT performance and spatiotemporally resolved neuroimaging measures may provide novel insight to characterize the deficits, assist diagnosis, and plan appropriate treatments. In older adults with neurodegenerative diseases, cognitive decrements may be caused by healthy aging, the diseases, or both. As a supplement to previously collected SPL data in healthy young133 and elderly adults119, it will be important to obtain normative data on other performance metrics of the tablet TMT to differentiate the effect of disease and injury from the effect of age in the patient cohorts. It would also be of interest to study tablet TMT performance metrics and related EEG and fMRI findings in patients with brain lesions (e.g. caused by traumatic brain injury (TBI), stroke or surgical intervention). Given the extensive lesion studies of the paper-based TMT involving such patient populations, as described in Chapter 1, this would enable interesting comparisons with (and improved interpretation of) the prior literature, as well as another means to assess the ecological validity of tablet-based TMT.

As demonstrated in this thesis, there is tremendous clinical potential in EEG recoding during TMT performance. Currently, resting-state EEG has identified neurophysiological biomarkers associated with different brain diseases such as AD247, major depressive disorder (MDD)248, and schizophrenia249. Specifically, patients with AD are characterized by slowed mean frequency in

76 spectral power slowness (increased delta and theta power, decreased alpha and beta power), reduced complex activity measured by approximate entropy, and less functional connectivity assessed by EEG coherence analysis247. A more recent study used these EEG biomarkers for AD along with TMT scores to evaluate the outcome of aerobic exercise in patients with MCI, both of which revealed significant improvements in cognition250. Similarly, better TMT performance was shown to be correlated with increased theta phase-gamma amplitude coupling in the left posterior cingulate cortex, the EEG biomarkers of schizophrenia, which indicates that this hyperactive coupling is a compensatory mechanism in areas within the DMN249. In addition, it was shown that TMT-B scores combined with EEG biomarkers of MDD such as asymmetrical frontal alpha power improved the discrimination accuracy between MDD patients and healthy controls251. It is apparent that as a separate measure of cognition, TMT performance is closely related resting-state EEG biomarkers of brain diseases. In the future, the detailed tablet behavioural metrics demonstrated in this thesis may enable concurrent EEG and TMT assessment to assist in differential diagnosis of brain diseases and to improve clinical workflow. By combining resting-state EEG and task-related EEG with tablet-based TMT, the neurophysiological biomarkers can be further characterized with task-related dynamics, towards capturing the full picture of cognitive functioning in healthy or diseased brain. It is also possible for these TMT-related EEG biomarkers to be localized in certain brain regions involved during the task, which may help clinicians to not only assess the degree of cognitive impairment, but also locate the corresponding brain regions in a timely manner.

3.4 Improvements to the Tablet Technology

The existing fMRI-compatible tablet device demonstrates powerful capabilities and strong potential to expand the behavioural repertoire of task-based fMRI for discovery neuroscience and clinical neuroscience research. However, technical improvements to the tablet may be considered in the future, such as to circumvent how the hand obstructs visual stimuli during visual feedback of hand position (VFHP), and to minimize input lag, increase sampling rate, increase temporal sampling, and increase spatial resolution of the touchscreen according to need.

For example, the hand of the participant may obstruct the visual stimuli during tablet task performance with VFHP. There are several options to circumvent this problem. One is to increase the complexity of the augmented reality environment, such as by “tilting” the writing

77 surface to avoid a strictly top-down viewing format for the visual stimuli, similar to typical writing on a tabletop. Subsequently, the visual obstruction could be partially removed by three- dimensional rendering of the hand. Additionally, a previous study examining the role of hand transparency in performance on grasping objects and writing in an augmented reality environment found that transparency helps with the visual obstruction at the cost of depth perception and proprioception issues252. Future technical development needs to establish an optimal level of hand transparency to reasonably circumvent the visual obstruction without interfering with user perceptions. It is worth noting that in the real world, some amount of visual obstruction is normal; therefore, the transparent hand is not necessarily the most appropriate solution.

3.5 Final Remarks

Full understanding of the interactions between brain activity and behaviour during TMT performance is very important to inform and improve how the TMT is used in research and clinical settings. With more quantifiable measures at both behavioural and brain levels, clinicians may be able to exploit untapped potential in existing neuropsychological tests, such as the TMT, to assist in detection, diagnosis, and treatment of cognitive impairments. The existing neuroimaging literature has only just begun to provide detailed examination of complex neuropsychological and behavioural tasks such as the TMT. Nonetheless, the continual development of tablet technology for detailed behavioural metrics and investigations to reveal the underlying brain activity associated with TMT performance using multiple neuroimaging modalities, including EEG, promise to advance understanding of this popular clinical tool. Temporal dynamics in behavioural and brain activity during TMT performance will be important to uncover key differences between healthy and diseased brain states. The present work sheds light on this promising direction of research.

Bibliography

1. Caton, R. Electrical Currents of the Brain. J. Nerv. Ment. Dis. 2, 610 (1875).

2. Lopes da Silva, F. Neural mechanisms underlying brain waves: from neural membranes to

networks. Electroencephalogr. Clin. Neurophysiol. 79, 81–93 (1991).

3. Bullock, T. & Horridge, G. A. Structure and function in the nervous systems of invertebrates.

(San Francisco, 1965).

4. Ullsperger, M. & Debener, S. Simultaneous EEG and fMRI: Recording, Analysis, and

Application. (Oxford University Press, 2010).

5. Henze, D. A. et al. Intracellular features predicted by extracellular recordings in the

hippocampus in vivo. J. Neurophysiol. 84, 390–400 (2000).

6. Nunez, P. L. & Srinivasan, R. Electric fields of the brain: the neurophysics of EEG. (Oxford

University Press, 2006).

7. Britton, J. W. et al. Electroencephalography (EEG): An Introductory Text and Atlas of

Normal and Abnormal Findings in Adults, Children, and Infants. (American Epilepsy

Society, 2016).

8. Klein, S. & Thorne, B. M. Biological Psychology. (Macmillan Learning, 2007).

9. Ryynanen, O., Hyttinen, J. & Malmivuo, J. Study on the spatial resolution of EEG - effect of

electrode density and measurement noise. in The 26th Annual International Conference of

the IEEE Engineering in Medicine and Biology Society vol. 2 4409–4412 (2004).

78 79

10. Berger, H. Über das Elektrenkephalogramm des Menschen. Arch. Für Psychiatr.

Nervenkrankh. 87, 527–570 (1929).

11. Varela, F., Lachaux, J. P., Rodriguez, E. & Martinerie, J. The brainweb: phase

synchronization and large-scale integration. Nat. Rev. Neurosci. 2, 229–239 (2001).

12. Buzsáki, G. Rhythms of the Brain. (Oxford University Press, 2006).

doi:10.1093/acprof:oso/9780195301069.001.0001.

13. Niedermeyer, E. & Silva, F. H. L. da. Electroencephalography: Basic Principles, Clinical

Applications, and Related Fields. (Lippincott Williams & Wilkins, 2005).

14. Buzsáki, G. & Draguhn, A. Neuronal oscillations in cortical networks. Science 304, 1926–

1929 (2004).

15. Wang, X.-J. Neurophysiological and computational principles of cortical rhythms in

cognition. Physiol. Rev. 90, 1195–1268 (2010).

16. Steriade, M. Grouping of brain rhythms in corticothalamic systems. Neuroscience 137,

1087–1106 (2006).

17. Cohen, M. X. Analyzing neural time series data: theory and practice. (The MIT Press,

2014).

18. Basile, L. F. H. et al. Interindividual variability in EEG correlates of attention and limits of

functional mapping. Int. J. Psychophysiol. Off. J. Int. Organ. Psychophysiol. 65, 238–251

(2007).

19. Murphy, M. et al. Source modeling sleep slow waves. Proc. Natl. Acad. Sci. 106, 1608–1613

(2009).

20. Dang-Vu, T. T. et al. Spontaneous neural activity during human slow wave sleep. Proc. Natl.

Acad. Sci. U. S. A. 105, 15160–15165 (2008).

21. Raichle, M. E. et al. A default mode of brain function. Proc. Natl. Acad. Sci. 98, 676–682

(2001).

22. Fox, M. D. & Raichle, M. E. Spontaneous fluctuations in brain activity observed with

functional magnetic resonance imaging. Nat. Rev. Neurosci. 8, 700–711 (2007).

23. Raichle, M. E. & Snyder, A. Z. A default mode of brain function: A brief history of an

evolving idea. NeuroImage 37, 1083–1090 (2007).

24. Sirota, A. et al. Entrainment of neocortical neurons and gamma oscillations by the

hippocampal theta rhythm. Neuron 60, 683–697 (2008).

25. Raghavachari, S. et al. Theta oscillations in human cortex during a working-memory task:

evidence for local generators. J. Neurophysiol. 95, 1630–1638 (2006).

26. Tsujimoto, T., Shimazu, H. & Isomura, Y. Direct Recording of Theta Oscillations in Primate

Prefrontal and Anterior Cingulate Cortices. J. Neurophysiol. 95, 2987–3000 (2006).

27. Onton, J., Delorme, A. & Makeig, S. Frontal midline EEG dynamics during working

memory. NeuroImage 27, 341–356 (2005).

28. Klimesch, W. EEG alpha and theta oscillations reflect cognitive and memory performance: a

review and analysis. Brain Res. Rev. 29, 169–195 (1999).

29. Mitchell, D. J., McNaughton, N., Flanagan, D. & Kirk, I. J. Frontal-midline theta from the

perspective of hippocampal ‘theta’. Prog. Neurobiol. 86, 156–185 (2008).

30. Vertes, R. P. Hippocampal theta rhythm: a tag for short-term memory. Hippocampus 15,

923–935 (2005).

31. Gevins, A., Smith, M. E., McEvoy, L. & Yu, D. High-resolution EEG mapping of cortical

activation related to working memory: effects of task difficulty, type of processing, and

practice. Cereb. Cortex 7, 374–385 (1997).

32. Huerta, P. T. & Lisman, J. E. Heightened synaptic plasticity of hippocampal CA1 neurons

during a Cholinergically induced rhythmic state. Nature 364, 723–725 (1993).

33. Stein, A. von, Chiang, C. & König, P. Top-down processing mediated by interareal

synchronization. Proc. Natl. Acad. Sci. 97, 14748–14753 (2000).

34. Sammer, G. et al. Relationship between regional hemodynamic activity and simultaneously

recorded EEG-theta associated with mental arithmetic-induced workload. Hum. Brain Mapp.

28, 793–803 (2007).

35. Laufs, H. et al. Electroencephalographic signatures of attentional and cognitive default

modes in spontaneous brain activity fluctuations at rest. Proc. Natl. Acad. Sci. 100, 11053–

11058 (2003).

36. Scheeringa, R. et al. Trial-by-trial coupling between EEG and BOLD identifies networks

related to alpha and theta EEG power increases during working memory maintenance.

NeuroImage 44, 1224–1238 (2009).

37. Tyvaert, L., Levan, P., Grova, C., Dubeau, F. & Gotman, J. Effects of fluctuating

physiological rhythms during prolonged EEG-fMRI studies. Clin. Neurophysiol. Off. J. Int.

Fed. Clin. Neurophysiol. 119, 2762–2774 (2008).

38. Klimesch, W., Sauseng, P. & Hanslmayr, S. EEG alpha oscillations: the inhibition-timing

hypothesis. Brain Res. Rev. 53, 63–88 (2007).

39. Worden, M. S., Foxe, J. J., Wang, N. & Simpson, G. V. Anticipatory biasing of visuospatial

attention indexed by retinotopically specific alpha-band electroencephalography increases

over occipital cortex. J. Neurosci. Off. J. Soc. Neurosci. 20, RC63 (2000).

40. Jensen, O., Gelfand, J., Kounios, J. & Lisman, J. E. Oscillations in the alpha band (9-12 Hz)

increase with memory load during retention in a short-term memory task. Cereb. Cortex N.

Y. N 1991 12, 877–882 (2002).

41. Jokisch, D. & Jensen, O. Modulation of Gamma and Alpha Activity during a Working

Memory Task Engaging the Dorsal or Ventral Stream. J. Neurosci. 27, 3244–3251 (2007).

42. Michels, L., Moazami-Goudarzi, M., Jeanmonod, D. & Sarnthein, J. EEG alpha

distinguishes between cuneal and precuneal activation in working memory. NeuroImage 40,

1296–1310 (2008).

43. Andersen, P. & Andersson, S. Physiological basis of the alpha rhythm. /paper/Physiological-

basis-of-the-alpha-rhythm-Andersen-

Andersson/d2df08f6fce3ce286c7bbd8afbb8a13efe3b1041 (1968).

44. Schürmann, M., Başar-Eroglu, C. & Başar, E. A possible role of evoked alpha in primary

sensory processing: common properties of cat intracranial recordings and human EEG and

MEG. Int. J. Psychophysiol. Off. J. Int. Organ. Psychophysiol. 26, 149–170 (1997).

45. Pfurtscheller, G., Neuper, C., Andrew, C. & Edlinger, G. Foot and hand area mu rhythms.

Int. J. Psychophysiol. Off. J. Int. Organ. Psychophysiol. 26, 121–135 (1997).

46. Pfurtscheller, G. & Andrew, C. Event-Related changes of band power and coherence:

methodology and interpretation. J. Clin. Neurophysiol. Off. Publ. Am. Electroencephalogr.

Soc. 16, 512–519 (1999).

47. Hari, R. & Salmelin, R. Human cortical oscillations: a neuromagnetic view through the skull.

Trends Neurosci. 20, 44–49 (1997).

48. Weisz, N., Hartmann, T., Müller, N., Lorenz, I. & Obleser, J. Alpha Rhythms in Audition:

Cognitive and Clinical Perspectives. Front. Psychol. 2, 73 (2011).

49. Thut, G., Nietzel, A., Brandt, S. A. & Pascual-Leone, A. α-Band Electroencephalographic

Activity over Occipital Cortex Indexes Visuospatial Attention Bias and Predicts Visual

Target Detection. J. Neurosci. 26, 9494–9502 (2006).

50. Palva, S., Linkenkaer-Hansen, K., Näätänen, R. & Palva, J. M. Early neural correlates of

conscious somatosensory perception. J. Neurosci. Off. J. Soc. Neurosci. 25, 5248–5258

(2005).

51. Doesburg, S. M., Green, J. J., McDonald, J. J. & Ward, L. M. From local inhibition to long-

range integration: A functional dissociation of alpha-band synchronization across cortical

scales in visuospatial attention. Brain Res. 1303, 97–110 (2009).

52. Lopes da Silva, F. H., Vos, J. E., Mooibroek, J. & van Rotterdam, A. Relative contributions

of intracortical and thalamo-cortical processes in the generation of alpha rhythms, revealed

by partial coherence analysis. Electroencephalogr. Clin. Neurophysiol. 50, 449–456 (1980).

53. de Munck, J. C. et al. The hemodynamic response of the alpha rhythm: An EEG/fMRI study.

NeuroImage 35, 1142–1151 (2007).

54. Suffczynski, P., Kalitzin, S., Pfurtscheller, G. & Lopes da Silva, F. H. Computational model

of thalamo-cortical networks: dynamical control of alpha rhythms in relation to focal

attention. Int. J. Psychophysiol. Off. J. Int. Organ. Psychophysiol. 43, 25–40 (2001).

55. Sigala, R., Haufe, S., Roy, D., Dinse, H. R. & Ritter, P. The role of alpha-rhythm states in

perceptual learning: insights from experiments and computational models. Front. Comput.

Neurosci. 8, 36 (2014).

56. da Silva, F. H., van Lierop, T. H., Schrijer, C. F. & van Leeuwen, W. S. Organization of

thalamic and cortical alpha rhythms: spectra and coherences. Electroencephalogr. Clin.

Neurophysiol. 35, 627–639 (1973).

57. Ehrhart, J. et al. Alpha activity and cardiac correlates: three types of relationships during

nocturnal sleep. Clin. Neurophysiol. Off. J. Int. Fed. Clin. Neurophysiol. 111, 940–946

(2000).

58. Laufs, H. et al. EEG-correlated fMRI of human alpha activity. NeuroImage 19, 1463–1476

(2003).

59. Gonçalves, S. I. et al. Correlating the alpha rhythm to BOLD using simultaneous EEG/fMRI:

inter-subject variability. NeuroImage 30, 203–213 (2006).

60. Laufs, H. et al. Where the BOLD signal goes when alpha EEG leaves. NeuroImage 31,

1408–1418 (2006).

61. Buzsaki, G. et al. Nucleus basalis and thalamic control of neocortical activity in the freely

moving rat. J. Neurosci. 8, 4007–4026 (1988).

62. Marco-Pallares, J. et al. Human oscillatory activity associated to reward processing in a

gambling task. Neuropsychologia 46, 241–248 (2008).

63. Martin, C., Gervais, R., Hugues, E., Messaoudi, B. & Ravel, N. Learning modulation of

odor-induced oscillatory responses in the rat olfactory bulb: a correlate of odor recognition?

J. Neurosci. Off. J. Soc. Neurosci. 24, 389–397 (2004).

64. Berger, H. Über das Elektrenkephalogramm des Menschen. Arch. Für Psychiatr.

Nervenkrankh. 108, 407–431 (1938).

65. Hong, L. E., Buchanan, R. W., Thaker, G. K., Shepard, P. D. & Summerfelt, A. Beta (∼16

Hz) frequency neural oscillations mediate auditory sensory gating in humans.

Psychophysiology 45, 197–204 (2008).

66. Gross, J. et al. Modulation of long-range neural synchrony reflects temporal limitations of

visual attention in humans. Proc. Natl. Acad. Sci. 101, 13050–13055 (2004).

67. Kaufman, D. M., Geyer, H. L. & Milstein, M. J. Kaufman’s Clinical Neurology for

Psychiatrists. (Elsevier Health Sciences, 2018).

68. Kilner, J. M., Baker, S. N., Salenius, S., Hari, R. & Lemon, R. N. Human cortical muscle

coherence is directly related to specific motor parameters. J. Neurosci. Off. J. Soc. Neurosci.

20, 8838–8845 (2000).

69. Kopell, N., Ermentrout, G. B., Whittington, M. A. & Traub, R. D. Gamma rhythms and beta

rhythms have different synchronization properties. Proc. Natl. Acad. Sci. 97, 1867–1872

(2000).

70. Lundqvist, M., Herman, P., Warden, M. R., Brincat, S. L. & Miller, E. K. Gamma and beta

bursts during working memory readout suggest roles in its volitional control. Nat. Commun.

9, 394 (2018).

71. Miller, E. K., Lundqvist, M. & Bastos, A. M. Working Memory 2.0. Neuron 100, 463–475

(2018).

72. Bastos, A. M., Loonis, R., Kornblith, S., Lundqvist, M. & Miller, E. K. Laminar recordings

in frontal cortex suggest distinct layers for maintenance and control of working memory.

Proc. Natl. Acad. Sci. 115, 1117–1122 (2018).

73. Jackson, A., Spinks, R. L., Freeman, T. C. B., Wolpert, D. M. & Lemon, R. N. Rhythm

generation in monkey motor cortex explored using pyramidal tract stimulation. J. Physiol.

541, 685–699 (2002).

74. Baker, S. N. Oscillatory interactions between sensorimotor cortex and the periphery. Curr.

Opin. Neurobiol. 17, 649–655 (2007).

75. Schmidt, R. et al. Beta Oscillations in Working Memory, Executive Control of Movement

and Thought, and Sensorimotor Function. J. Neurosci. 39, 8231–8238 (2019).

76. Hammond, C., Bergman, H. & Brown, P. Pathological synchronization in Parkinson’s

disease: networks, models and treatments. Trends Neurosci. 30, 357–364 (2007).

77. Mallet, N. et al. Disrupted Dopamine Transmission and the Emergence of Exaggerated Beta

Oscillations in Subthalamic Nucleus and Cerebral Cortex. J. Neurosci. 28, 4795–4806

(2008).

78. da Silva, F. L. EEG: Origin and Measurement. in EEG - fMRI (eds. Mulert, C. & Lemieux,

L.) 19–38 (Springer Berlin Heidelberg, 2009). doi:10.1007/978-3-540-87919-0_2.

79. Neuenschwander, S. & Singer, W. Long-range synchronization of oscillatory light responses

in the cat retina and lateral geniculate nucleus. Nature 379, 728–732 (1996).

80. Stopfer, M., Bhagavan, S., Smith, B. H. & Laurent, G. Impaired odour discrimination on

desynchronization of odour-encoding neural assemblies. Nature 390, 70–74 (1997).

81. Gray, C. M., König, P., Engel, A. K. & Singer, W. Oscillatory responses in cat visual cortex

exhibit inter-columnar synchronization which reflects global stimulus properties. Nature

338, 334–337 (1989).

82. Fries, P., Reynolds, J. H., Rorie, A. E. & Desimone, R. Modulation of oscillatory neuronal

synchronization by selective visual attention. Science 291, 1560–1563 (2001).

83. Tallon-Baudry, C., Bertrand, O., Peronnet, F. & Pernier, J. Induced gamma-band activity

during the delay of a visual short-term memory task in humans. J. Neurosci. Off. J. Soc.

Neurosci. 18, 4244–4254 (1998).

84. Melloni, L. et al. Synchronization of neural activity across cortical areas correlates with

conscious perception. J. Neurosci. Off. J. Soc. Neurosci. 27, 2858–2865 (2007).

85. Wespatat, V., Tennigkeit, F. & Singer, W. Phase sensitivity of synaptic modifications in

oscillating cells of rat visual cortex. J. Neurosci. Off. J. Soc. Neurosci. 24, 9067–9075

(2004).

86. von der Malsburg, C. & Schneider, W. A neural cocktail-party processor. Biol. Cybern. 54,

29–40 (1986).

87. Singer, W. Neuronal synchrony: a versatile code for the definition of relations? Neuron 24,

49–65, 111–125 (1999).

88. Steriade, M. & Amzica, F. Intracortical and corticothalamic coherency of fast spontaneous

oscillations. Proc. Natl. Acad. Sci. U. S. A. 93, 2533–2538 (1996).

89. Steriade, M., Amzica, F. & Contreras, D. Synchronization of fast (30-40 Hz) spontaneous

cortical rhythms during brain activation. J. Neurosci. 16, 392–417 (1996).

90. Yuval-Greenberg, S., Tomer, O., Keren, A. S., Nelken, I. & Deouell, L. Y. Transient induced

gamma-band response in EEG as a manifestation of miniature saccades. Neuron 58, 429–441

(2008).

91. Whitham, E. M. et al. Scalp electrical recording during paralysis: quantitative evidence that

EEG frequencies above 20 Hz are contaminated by EMG. Clin. Neurophysiol. Off. J. Int.

Fed. Clin. Neurophysiol. 118, 1877–1888 (2007).

92. Whitham, E. M. et al. Thinking activates EMG in scalp electrical recordings. Clin.

Neurophysiol. Off. J. Int. Fed. Clin. Neurophysiol. 119, 1166–1175 (2008).

93. Fries, P., Nikolić, D. & Singer, W. The gamma cycle. Trends Neurosci. 30, 309–316 (2007).

94. Chrobak, J. J. & Buzsáki, G. Gamma Oscillations in the Entorhinal Cortex of the Freely

Behaving Rat. J. Neurosci. 18, 388–398 (1998).

95. Kerkoerle, T. van et al. Alpha and gamma oscillations characterize feedback and

feedforward processing in monkey visual cortex. Proc. Natl. Acad. Sci. (2014)

doi:10.1073/pnas.1402773111.

96. Fabiani, M., Gratton, G. & Federmeier, K. D. Event-related brain potentials: Methods,

theory, and applications. in Handbook of psychophysiology, 3rd ed 85–119 (Cambridge

University Press, 2007). doi:10.1017/CBO9780511546396.004.

97. Picton, T., Lins, O. & Scherg, M. The recording and analysis of event-related potentials.

Handb. Neuropsychol. 10, 3–3 (1995).

98. Beres, A. M. Time is of the Essence: A Review of Electroencephalography (EEG) and

Event-Related Brain Potentials (ERPs) in Language Research. Appl. Psychophysiol.

Biofeedback 42, 247–255 (2017).

99. Kaan, E. Event-Related Potentials and Language Processing: A Brief Overview. Lang.

Linguist. Compass 1, 571–591 (2007).

100. Gabor, D. Theory of communication. (Institution of Electrical Engineering, 1946).

101. Bruns, A. Fourier-, Hilbert- and wavelet-based signal analysis: are they really different

approaches? J. Neurosci. Methods 137, 321–332 (2004).

102. Huang, F. L. MANOVA: A Procedure Whose Time Has Passed? Gift. Child Q. 64, 56–60

(2020).

103. Puryear, J. S. & Kettler, T. Rural Gifted Education and the Effect of Proximity. Gift.

Child Q. 61, 143–152 (2017).

104. McIntosh, A. R., Bookstein, F. L., Haxby, J. V. & Grady, C. L. Spatial pattern analysis of

functional brain images using partial least squares. NeuroImage 3, 143–157 (1996).

105. Jöreskog, K. G. & Wold, H. O. A. Systems under indirect observation Part II. Part II.

(North-Holland, 1982).

106. Klimesch, W. Memory processes, brain oscillations and EEG synchronization. Int. J.

Psychophysiol. 24, 61–100 (1996).

107. Army Individual Test Battery. Manual of Directions and Scoring. (War Department,

Adjutant General’s Office, 1944).

108. Morris, J. C., Mohs, R. C., Rogers, H., Fillenbaum, G. & Heyman, A. Consortium to

establish a registry for Alzheimer’s disease (CERAD) clinical and neuropsychological

assessment of Alzheimer’s disease. Psychopharmacol. Bull. 24, 641–652 (1988).

109. Dikmen, S., Machamer, J., Temkin, N. & McLean, A. Neuropsychological recovery in

patients with moderate to severe head injury: 2 year follow-up. J. Clin. Exp. Neuropsychol.

12, 507–519 (1990).

110. Buchanan, R. W. et al. Neuropsychological Impairments in Deficit vs Nondeficit Forms

of Schizophrenia. Arch. Gen. Psychiatry 51, 804–811 (1994).

111. Ashendorf, L. et al. Trail Making Test errors in normal aging, mild cognitive impairment,

and dementia. Arch. Clin. Neuropsychol. 23, 129–137 (2008).

112. McKhann, G. M. Changing concepts of Alzheimer disease. JAMA 305, 2458–2459

(2011).

113. Lezak, M. D., Howieson, D. B., Bigler, E. D. & Tranel, D. Neuropsychological

assessment. (Oxford University Press, 2012).

114. Stuss, D. T., Stethem, L. L., Hugenholtz, H. & Richard, M. T. Traumatic brain injury: A

comparison of three clinical tests, and analysis of recovery. Clin. Neuropsychol. 3, 145–156

(1989).

115. Halstead, W. C. Brain and intelligence; a quantitative study of the frontal lobes. xiii, 206

(University of Chicago Press, 1947).

116. Stuss, D. T. et al. The trail making test: A study in focal lesion patients. Psychol. Assess.

13, 230–239 (2001).

117. Strauss, E., Sherman, E. M. S. & Spreen, O. A compendium of neuropsychological tests:

administration, norms, and commentary. (Oxford University Press, 2006).

118. Reitan, R. M. Trail making test results for normal and brain-damaged children. Percept.

Mot. Skills 33, 575–581 (1971).

119. Talwar, N. et al. Functional magnetic resonance imaging of the trail-making test in older

adults. PLOS ONE 15, e0232469 (2020).

120. Corrigan, J. D. & Hinkeldey, N. S. Relationships between parts A and B of the Trail

Making Test. J. Clin. Psychol. 43, 402–409 (1987).

121. Gaudino, E. A., Geisler, M. W. & Squires, N. K. Construct validity in the trail making

test: What makes part B harder? J. Clin. Exp. Neuropsychol. 17, 529–535 (1995).

122. Bowie, C. R. & Harvey, P. D. Administration and interpretation of the Trail Making Test.

Nat. Protoc. 1, 2277–2281 (2006).

123. Arbuthnott, K. & Frank, J. Trail Making Test, Part B as a measure of executive control:

Validation using a set-switching paradigm. J. Clin. Exp. Neuropsychol. 22, 518–528 (2000).

124. Muir, R. T. et al. Trail making test elucidates neural substrates of specific poststroke

executive dysfunctions. Stroke 46, 2755–2761 (2015).

125. Sánchez-Cubillo, I. et al. Construct validity of the Trail Making Test: Role of task-

switching, working memory, inhibition/interference control, and visuomotor abilities. J. Int.

Neuropsychol. Soc. 15, 438–450 (2009).

126. Lamberty, G. J., Putnam, S. H., Chatel, D. M., Bieliauskas, L. A., & et al. Derived Trail

Making Test indices: A preliminary report. Neuropsychiatry. Neuropsychol. Behav. Neurol.

7, 230–234 (1994).

127. Robins Wahlin, T.-B., Bäckman, L., Wahlin, Å. & Winblad, B. Trail Making Test

performance in a community-based sample of healthy very old adults: effects of age on

completion time, but not on accuracy. Arch. Gerontol. Geriatr. 22, 87–102 (1996).

128. Christidi, F., Kararizou, E., Triantafyllou, N., Anagnostouli, M. & Zalonis, I. Derived

Trail Making Test indices: demographics and cognitive background variables across the

adult life span. Aging Neuropsychol. Cogn. 22, 667–678 (2015).

129. Tombaugh, T. N. Trail Making Test A and B: Normative data stratified by age and

education. Arch. Clin. Neuropsychol. 19, 203–214 (2004).

130. Drane, D. L., Yuspeh, R. L., Huthwaite, J. S. & Klingler, L. K. Demographic

characteristics and normative observations for derived-trail making test indices.

Neuropsychiatry. Neuropsychol. Behav. Neurol. 15, 39–43 (2002).

131. Giovagnoli, A. R. et al. Trail making test: normative values from 287 normal adult

controls. Ital. J. Neurol. Sci. 17, 305–309 (1996).

132. Klusman, L. E., Cripe, L. I. & Dodrill, C. B. Analysis of errors on the Trail Making Test.

Percept. Mot. Skills 68, 1199–1204 (1989).

133. Karimpoor, M. et al. Tablet-Based Functional MRI of the Trail Making Test: Effect of

Tablet Interaction Mode. Front. Hum. Neurosci. 11, 496 (2017).

134. Kopp, B. et al. Errors on the Trail Making Test Are Associated with Right Hemispheric

Frontal Lobe Damage in Stroke Patients. Behav. Neurol. 2015, 1–10 (2015).

135. Mahurin, R. K. et al. Trail Making Test Errors and Executive Function in Schizophrenia

and Depression. Clin. Neuropsychol. 20, 271–288 (2006).

136. Ruffolo, L. F., Guilmette, T. J. & Willis, W. G. Comparison of time and error rates on the

trail making test among patients with head injuries, experimental malingerers, patients with

suspect effort on testing, and normal controls. Clin. Neuropsychol. 14, 223–230 (2000).

137. Delis, D. C., Kaplan, E. & Kramer, J. H. Delis-Kaplan Executive Function System.

PsycTESTS® (2001) doi:10.1037/t15082-000.

138. Demeyere, N., Riddoch, M. J., Slavkova, E. D., Bickerton, W.-L. & Humphreys, G. W.

The Oxford Cognitive Screen (OCS): Validation of a stroke-specific short cognitive

screening tool. Psychol. Assess. 27, 883–894 (2015).

139. Mattson, S. N., Goodman, A. M., Caine, C., Delis, D. C. & Riley, E. P. Executive

functioning in children with heavy prenatal alcohol exposure. Alcohol. Clin. Exp. Res. 23,

1808–1815 (1999).

140. Kleinhans, N., Akshoomoff, N. & Delis, D. C. Executive functions in autism and

Asperger’s disorder: flexibility, fluency, and inhibition. Dev. Neuropsychol. 27, 379–401

(2005).

141. McDonald, C. R., Delis, D. C., Norman, M. A., Tecoma, E. S. & Iragui-Madoz, V. J. Is

impairment in set-shifting specific to frontal-lobe dysfunction? Evidence from patients with

frontal-lobe or temporal-lobe epilepsy. J. Int. Neuropsychol. Soc. 11, 477–481 (2005).

142. Cato, M. A., Delis, D. C., Abildskov, T. J. & Bigler, E. Assessing the elusive cognitive

deficits associated with ventromedial prefrontal damage: a case of a modern-day Phineas

Gage. J. Int. Neuropsychol. Soc. JINS 10, 453–465 (2004).

143. Yochim, B., Baldo, J., Nelson, A. & Delis, D. C. D-KEFS Trail Making Test

performance in patients with lateral prefrontal cortex lesions. J. Int. Neuropsychol. Soc. 13,

704–709 (2007).

144. Kortte, K. B., Horner, M. D. & Windham, W. K. The Trail Making Test, Part B:

Cognitive Flexibility or Ability to Maintain Set? Appl. Neuropsychol. 9, 106–109 (2002).

145. Fossum, B., Holmberg, H. & Reinvang, I. Spatial and symbolic factors in performance on

the Trail Making Test. Neuropsychology 6, 71–75 (1992).

146. Larrabee, G. J. & Curtiss, G. Construct validity of various verbal and visual memory

tests. J. Clin. Exp. Neuropsychol. 17, 536–547 (1995).

147. Robins Wahlin, T. B., Bäckman, L., Wahlin, A. & Winblad, B. Trail Making Test

performance in a community-based sample of healthy very old adults: effects of age on

completion time, but not on accuracy. Arch. Gerontol. Geriatr. 22, 87–102 (1996).

148. Crowe, S. F. The differential contribution of mental tracking, cognitive flexibility, visual

search, and motor speed to performance on parts A and B of the Trail Making Test. J. Clin.

Psychol. 54, 585–591 (1998).

149. Oosterman, J. M. et al. Assessing mental flexibility: neuroanatomical and

neuropsychological correlates of the trail making test in elderly people. Clin. Neuropsychol.

24, 203–219 (2010).

150. Demakis, G. J. Frontal Lobe Damage and Tests of Executive Processing: A Meta-

Analysis of the Category Test, Stroop Test, and Trail-Making Test. J. Clin. Exp.

Neuropsychol. 26, 441–450 (2004).

151. Tamez, E. et al. Assessing Executive Abilities Following Acute Stroke with the Trail

Making Test and Digit Span. Behav. Neurol. 24, 177–185 (2011).

152. Anderson, C. V., Bigler, E. D. & Blatter, D. D. Frontal lobe lesions, diffuse damage, and

neuropsychological functioning in traumatic brain-injured patients. J. Clin. Exp.

Neuropsychol. 17, 900–908 (1995).

153. Heilbronner, R. L., Henry, G. K., Buck, P., Adams, R. L. & Fogle, T. Lateralized brain

damage and performance on trail making A and B, digit span forward and backward, and

TPT memory and location. Arch. Clin. Neuropsychol. 6, 251–258 (1991).

154. Channon, S. & Crawford, S. The effects of anterior lesions on performance on a story

comprehension test: left anterior impairment on a theory of mind-type task.

Neuropsychologia 38, 1006–1017 (2000).

155. Gouveia, P. A. R., Brucki, S. M. D., Malheiros, S. M. F. & Bueno, O. F. A. Disorders in

planning and strategy application in frontal lobe lesion patients. Brain Cogn. 63, 240–246

(2007).

156. Hoerold, D., Pender, N. P. & Robertson, I. H. Metacognitive and online error awareness

deficits after prefrontal cortex lesions. Neuropsychologia 51, 385–391 (2013).

157. Rorden, C. & Brett, M. Stereotaxic display of brain lesions. Behav. Neurol. 12, 191–200

(2000).

158. Karnath, H.-O., Himmelbach, M. & Rorden, C. The subcortical anatomy of human spatial

neglect: putamen, caudate nucleus and pulvinar. Brain 125, 350–360 (2002).

159. MacPherson, S. E., Sala, S. D., Cox, S. R., Girardi, A. & Iveson, M. H. Mental flexibility.

Handbook of Frontal Lobe Assessment (Oxford University Press, 2015).

160. Allen, G. et al. Magnetic resonance imaging of cerebellar–prefrontal and cerebellar–

parietal functional connectivity. NeuroImage 28, 39–48 (2005).

161. Exner, C., Weniger, G. & Irle, E. Cerebellar lesions in the PICA but not SCA territory

impair cognition. Neurology 63, 2132–2135 (2004).

162. Gottwald, B., Wilde, B., Mihajlovic, Z. & Mehdorn, H. M. Evidence for distinct

cognitive deficits after focal cerebellar lesions. J. Neurol. Neurosurg. Psychiatry 75, 1524–

1531 (2004).

163. Alexander, M. P., Gillingham, S., Schweizer, T. & Stuss, D. T. Cognitive impairments

due to focal cerebellar injuries in adults. Cortex 48, 980–990 (2012).

164. Bates, E. et al. Voxel-based lesion-symptom mapping. Nat. Neurosci. 6, 448–450 (2003).

165. Rorden, C., Karnath, H.-O. & Bonilha, L. Improving Lesion-Symptom Mapping. J.

Cogn. Neurosci. 19, 1081–1088 (2007).

166. Gläscher, J. et al. Lesion mapping of cognitive control and value-based decision making

in the prefrontal cortex. Proc. Natl. Acad. Sci. 109, 14681–14686 (2012).

167. Miskin, N. et al. Prefrontal lobe structural integrity and trail making test, part B:

converging findings from surface-based cortical thickness and voxel-based lesion symptom

analyses. Brain Imaging Behav. 10, 675–685 (2016).

168. Barbey, A. K. et al. An integrative architecture for general intelligence and executive

function revealed by lesion mapping. Brain 135, 1154–1164 (2012).

169. Varjačić, A. et al. The role of left insula in executive set-switching: Lesion evidence from

an acute stroke cohort. Cortex 107, 92–101 (2018).

170. Basser, P. J. Inferring microstructural features and the physiological state of tissues from

diffusion-weighted images. NMR Biomed. 8, 333–344 (1995).

171. Basser, P. J., Pajevic, S., Pierpaoli, C., Duda, J. & Aldroubi, A. In vivo fiber tractography

using DT-MRI data. Magn. Reson. Med. 44, 625–632 (2000).

172. Mori, S. & Van Zijl, P. C. M. Fiber tracking: Principles and strategies - A technical

review. NMR Biomed. 15, 468–480 (2002).

173. Perry, M. E. et al. White matter tracts associated with set-shifting in healthy aging.

Neuropsychologia 47, 2835–2842 (2009).

174. Lorig, T., Gehring, W. J. & Hyrn, D. L. Period analysis of the EEG during performance

of the trail making test. Int. J. Clin. Neuropsychol. 8, 97–99 (1986).

175. Biswal, B., Yetkin, F. Z., Haughton, V. M. & Hyde, J. S. Functional connectivity in the

motor cortex of resting human brain using echo-planar mri. Magn. Reson. Med. 34, 537–541

(1995).

176. Laird, A. R. et al. Behavioral interpretations of intrinsic connectivity networks. J. Cogn.

Neurosci. 23, 4022–4037 (2011).

177. Damoiseaux, J. S. et al. Reduced resting-state brain activity in the “default network” in

normal aging. Cereb. Cortex 18, 1856–1864 (2008).

178. Broyd, S. J. et al. Default-mode brain dysfunction in mental disorders: A systematic

review. Neurosci. Biobehav. Rev. 33, 279–296 (2009).

179. Seeley, W. W. et al. Dissociable Intrinsic Connectivity Networks for Salience Processing

and Executive Control. J. Neurosci. 27, 2349–2356 (2007).

180. James, G. A. et al. Functional independence in resting-state connectivity facilitates

higher-order cognition. Brain Cogn. 105, 78–87 (2016).

181. Moll, J., De Oliveira-Souza, R., Moll, F. T., Bramati, I. E. & Andreiuolo, P. A. The

cerebral correlates of set-shifting: An fMRI study of the trail making test. Arq.

Neuropsiquiatr. 60, 900–905 (2002).

182. Allen, M. D., Owens, T. E., Fong, A. K. & Richards, D. R. A Functional Neuroimaging

Analysis of the Trail Making Test-B: Implications for Clinical Application. Behav. Neurol.

24, 159–171 (2011).

183. Jacobson, S. C., Blanchard, M., Connolly, C. C., Cannon, M. & Garavan, H. An fMRI

investigation of a novel analogue to the Trail-Making Test. Brain Cogn. 77, 60–70 (2011).

184. Tam, F., Churchill, N. W., Strother, S. C. & Graham, S. J. A new tablet for writing and

drawing during functional MRI. Hum. Brain Mapp. 32, 240–248 (2011).

185. Katanoda, K., Yoshikawa, K. & Sugishita, M. A functional MRI study on the neural

substrates for writing. Hum. Brain Mapp. 13, 34–42 (2001).

186. Makuuchi, M., Kaminaga, T. & Sugishita, M. Both parietal lobes are involved in

drawing: a functional MRI study and implications for constructional apraxia. Cogn. Brain

Res. 16, 338–347 (2003).

187. Reithler, J., Reithler, H., van den Boogert, E., Goebel, R. & van Mier, H. Resistance-

based high resolution recording of predefined 2-dimensional pen trajectories in an fMRI

setting. J. Neurosci. Methods 152, 10–17 (2006).

100

188. Zakzanis, K. K., Mraz, R. & Graham, S. J. An fMRI study of the Trail Making Test.

Neuropsychologia 43, 1878–1886 (2005).

189. Rajkowska, G. & Goldman-Rakic, P. S. Cytoarchitectonic definition of prefrontal areas

in the normal human cortex: II. Variability in locations of Areas 9 and 46 and relationship to

the Talairach coordinate system. Cereb. Cortex 5, 323–337 (1995).

190. Vinci-Booher, S., Sturgeon, J., James, T. & James, K. The MRItab: A MR-compatible

touchscreen with video-display. J. Neurosci. Methods 306, 10–18 (2018).

191. Lin, Z., Tam, F., Churchill, N. W., Schweizer, T. A. & Graham, S. J. Tablet Technology

for Writing and Drawing during Functional Magnetic Resonance Imaging: A Review.

Sensors 21, 401 (2021).

192. Stuss, D. T., Stethem, L. L. & Poirier, C. A. Comparison of three tests of attention and

rapid information processing across six age groups. Clin. Neuropsychol. 1, 139–152 (1987).

193. Stuss, D. T., Stethem, L. L. & Pelchat, G. Three tests of attention and rapid information

processing: An extension. Clin. Neuropsychol. 2, 246–250 (1988).

194. Corbetta, M. & Shulman, G. L. Spatial Neglect and Attention Networks. Annu. Rev.

Neurosci. 34, 569–599 (2011).

195. Churchill, N. W., Raamana, P., Spring, R. & Strother, S. C. Optimizing fMRI

preprocessing pipelines for block-design tasks as a function of age. NeuroImage 154, 240–

254 (2017).

101

196. Churchill, N. W., Spring, R., Afshin-Pour, B., Dong, F. & Strother, S. C. An Automated,

Adaptive Framework for Optimizing Preprocessing Pipelines in Task-Based Functional

MRI. PLOS ONE 10, e0131520 (2015).

197. Richter, W. et al. Motor Area Activity During Mental Rotation Studied by Time-

Resolved Single-Trial fMRI. J. Cogn. Neurosci. 12, 310–320 (2000).

198. Nobre, A. C., Coull, J. T., Walsh, V. & Frith, C. D. Brain Activations during Visual

Search: Contributions of Search Efficiency versus Feature Binding. NeuroImage 18, 91–103

(2003).

199. Brown, J. W. & Braver, T. S. Learned Predictions of Error Likelihood in the Anterior

Cingulate Cortex. Science 307, 1118–1121 (2005).

200. Friederici, A. D., Rüschemeyer, S.-A., Hahne, A. & Fiebach, C. J. The role of left inferior

frontal and superior temporal cortex in sentence comprehension: localizing syntactic and

semantic processes. Cereb. Cortex N. Y. N 1991 13, 170–177 (2003).

201. Winhuisen Lutz et al. Role of the Contralateral Inferior Frontal Gyrus in Recovery of

Language Function in Poststroke Aphasia. Stroke 36, 1759–1763 (2005).

202. Brass, M. et al. When the same response has different meanings:: recoding the response

meaning in the lateral prefrontal cortex. NeuroImage 20, 1026–1031 (2003).

203. Dreher, J.-C. & Berman, K. F. Fractionating the neural substrate of cognitive control

processes. Proc. Natl. Acad. Sci. 99, 14595–14600 (2002).

102

204. Dove, A., Pollmann, S., Schubert, T., Wiggins, C. J. & Yves von Cramon, D. Prefrontal

cortex activation in task switching: an event-related fMRI study. Cogn. Brain Res. 9, 103–

109 (2000).

205. Heekeren, H. R., Marrett, S., Ruff, D. A., Bandettini, P. A. & Ungerleider, L. G.

Involvement of human left dorsolateral prefrontal cortex in perceptual decision making is

independent of response modality. Proc. Natl. Acad. Sci. 103, 10023–10028 (2006).

206. Karimpoor, M. et al. A computerized tablet with visual feedback of hand position for

functional magnetic resonance imaging. Front. Hum. Neurosci. 9, 150 (2015).

207. Solomon, O. M., Larson, D. R. & Jr, N. G. P. Comparison of Some Algorithms to

Estimate the Low and High State Level of Pulses. 96–101 (2001).

208. Delorme, A. & Makeig, S. EEGLAB: an open source toolbox for analysis of single-trial

EEG dynamics including independent component analysis. J. Neurosci. Methods 134, 9–21

(2004).

209. Pion-Tonachini, L., Kreutz-Delgado, K. & Makeig, S. ICLabel: An automated

electroencephalographic independent component classifier, dataset, and website.

NeuroImage 198, 181–197 (2019).

210. Pion-Tonachini, L., Kreutz-Delgado, K. & Makeig, S. The ICLabel dataset of

electroencephalographic (EEG) independent component (IC) features. Data in Brief 25,

104101 (2019).

103

211. Lobaugh, N. J., West, R. & McIntosh, A. R. Spatiotemporal analysis of experimental

differences in event-related potential data with partial least squares. Psychophysiology 38,

517–530 (2001).

212. Grady, C. L. et al. A Multivariate Analysis of Age-Related Differences in Default Mode

and Task-Positive Networks across Multiple Cognitive Domains. Cereb. Cortex 20, 1432–

1447 (2010).

213. Müller, S., Preische, O., Heymann, P., Elbing, U. & Laske, C. Increased Diagnostic

Accuracy of Digital vs. Conventional Clock Drawing Test for Discrimination of Patients in

the Early Course of Alzheimer’s Disease from Cognitively Healthy Individuals. Front. Aging

Neurosci. 9, 101 (2017).

214. Knyazev, G. G. EEG delta oscillations as a correlate of basic homeostatic and

motivational processes. Neurosci. Biobehav. Rev. 36, 677–695 (2012).

215. Lakatos, P., Karmos, G., Mehta, A. D., Ulbert, I. & Schroeder, C. E. Entrainment of

Neuronal Oscillations as a Mechanism of Attentional Selection. Science 320, 110–113

(2008).

216. Meerwijk, E. L., Ford, J. M. & Weiss, S. J. Resting-State EEG Delta Power is Associated

with Psychological Pain in Adults with a History of Depression. Biol. Psychol. 0, 106–114

(2015).

217. Canolty, R. T. et al. High Gamma Power Is Phase-Locked to Theta Oscillations in

Human Neocortex. Science 313, 1626–1628 (2006).

104

218. Schack, B., Vath, N., Petsche, H., Geissler, H.-G. & Möller, E. Phase-coupling of theta-

gamma EEG rhythms during short-term memory processing. Int. J. Psychophysiol. Off. J.

Int. Organ. Psychophysiol. 44, 143–163 (2002).

219. von Stein, A. & Sarnthein, J. Different frequencies for different scales of cortical

integration: from local gamma to long range alpha/theta synchronization. Int. J.

Psychophysiol. Off. J. Int. Organ. Psychophysiol. 38, 301–313 (2000).

220. Knyazev, G. G. Motivation, emotion, and their inhibitory control mirrored in brain

oscillations. Neurosci. Biobehav. Rev. 31, 377–395 (2007).

221. Cooper, N. R., Croft, R. J., Dominey, S. J. J., Burgess, A. P. & Gruzelier, J. H. Paradox

lost? Exploring the role of alpha oscillations during externally vs. internally directed

attention and the implications for idling and inhibition hypotheses. Int. J. Psychophysiol. 47,

65–74 (2003).

222. Klimesch, W., Doppelmayr, M., Russegger, H., Pachinger, T. & Schwaiger, J. Induced

alpha band power changes in the human EEG and attention. Neurosci. Lett. 244, 73–76

(1998).

223. Klimesch, W. Alpha-band oscillations, attention, and controlled access to stored

information. Trends Cogn. Sci. 16, 606–617 (2012).

224. Scheeringa, R., Koopmans, P. J., van Mourik, T., Jensen, O. & Norris, D. G. The

relationship between oscillatory EEG activity and the laminar-specific BOLD signal. Proc.

Natl. Acad. Sci. 113, 6761–6766 (2016).

105

225. Muthukumaraswamy, S. D., Johnson, B. W. & McNair, N. A. Mu rhythm modulation

during observation of an object-directed grasp. Cogn. Brain Res. 19, 195–201 (2004).

226. Pfurtscheller, G., Brunner, C., Schlögl, A. & Lopes da Silva, F. H. Mu rhythm

(de)synchronization and EEG single-trial classification of different motor imagery tasks.

NeuroImage 31, 153–159 (2006).

227. Zaepffel, M., Trachel, R., Kilavik, B. E. & Brochier, T. Modulations of EEG Beta Power

during Planning and Execution of Grasping Movements. PLOS ONE 8, e60060 (2013).

228. Ishii, R. et al. Frontal midline theta rhythm and gamma power changes during focused

attention on mental calculation: an MEG beamformer analysis. Front. Hum. Neurosci. 8, 406

(2014).

229. Lin, F.-H. et al. Event-related Single-shot Volumetric Functional Magnetic Resonance

Inverse Imaging of Visual Processing. NeuroImage 42, 230–247 (2008).

230. Lin, F.-H. et al. Increasing fMRI Sampling Rate Improves Granger Causality Estimates.

PLOS ONE 9, e100319 (2014).

231. Holmes, M. D. Dense array EEG: methodology and new hypothesis on epilepsy

syndromes. Epilepsia 49 Suppl 3, 3–14 (2008).

232. Hassan, M., Dufor, O., Merlet, I., Berrou, C. & Wendling, F. EEG Source Connectivity

Analysis: From Dense Array Recordings to Brain Networks. PLOS ONE 9, e105041 (2014).

233. Logothetis, N. K. What we can do and what we cannot do with fMRI. Nature 453, 869–

878 (2008).

106

234. Allen, P. J., Josephs, O. & Turner, R. A method for removing imaging artifact from

continuous EEG recorded during functional MRI. NeuroImage 12, 230–239 (2000).

235. Bonmassar, G., Anami, K., Ives, J. & Belliveau, J. W. Visual evoked potential (VEP)

measured by simultaneous 64-channel EEG and 3T fMRI: NeuroReport 10, 1893–1897

(1999).

236. Goldman, R. I., Stern, J. M., Engel, J. & Cohen, M. S. Acquiring simultaneous EEG and

functional MRI. Clin. Neurophysiol. 111, 1974–1980 (2000).

237. Goldman, R. I., Stern, J. M., Engel, J. & Cohen, M. S. Simultaneous EEG and fMRI of

the alpha rhythm. Neuroreport 13, 2487–2492 (2002).

238. Lee, H.-J. et al. Concurrent electrophysiological and hemodynamic measurements of

evoked neural oscillations in human visual cortex using sparsely interleaved fast fMRI and

EEG. NeuroImage 217, 116910 (2020).

239. Mandelkow, H., Halder, P., Boesiger, P. & Brandeis, D. Synchronization facilitates

removal of MRI artefacts from concurrent EEG recordings and increases usable bandwidth.

NeuroImage 32, 1120–1126 (2006).

240. Mullinger, K. J., Morgan, P. S. & Bowtell, R. W. Improved artifact correction for

combined electroencephalography/functional MRI by means of synchronization and use of

vectorcardiogram recordings. J. Magn. Reson. Imaging JMRI 27, 607–616 (2008).

241. Stevens, T. K., Ives, J. R., Klassen, L. M. & Bartha, R. MR compatibility of EEG scalp

electrodes at 4 tesla. J. Magn. Reson. Imaging 25, 872–877 (2007).

107

242. Hanson, L. G., Lund, T. E. & Hanson, C. G. Encoding of electrophysiology and other

signals in MR images. J. Magn. Reson. Imaging 25, 1059–1066 (2007).

243. Huster, R. J., Debener, S., Eichele, T. & Herrmann, C. S. Methods for Simultaneous

EEG-fMRI: An Introductory Review. J. Neurosci. 32, 6053–6060 (2012).

244. Abreu, R., Leal, A. & Figueiredo, P. EEG-Informed fMRI: A Review of Data Analysis

Methods. Front. Hum. Neurosci. 12, 29 (2018).

245. Valdes-Sosa, P. A. et al. Model driven EEG/fMRI fusion of brain oscillations. Hum.

Brain Mapp. 30, 2701–2721 (2009).

246. Mitchell, T. M. et al. Predicting human brain activity associated with the meanings of

nouns. Science 320, 1191–1195 (2008).

247. Jeong, J. EEG dynamics in patients with Alzheimer’s disease. Clin. Neurophysiol. Off. J.

Int. Fed. Clin. Neurophysiol. 115, 1490–1505 (2004).

248. Henriques, J. B. & Davidson, R. J. Left frontal hypoactivation in depression. J. Abnorm.

Psychol. 100, 535–545 (1991).

249. Lee, T. H. et al. Relationship between resting-state theta phase-gamma amplitude

coupling and neurocognitive functioning in patients with first-episode psychosis. Schizophr.

Res. 216, 154–160 (2020).

250. Amjad, I. et al. Therapeutic effects of aerobic exercise on EEG parameters and higher

cognitive functions in mild cognitive impairment patients. Int. J. Neurosci. 129, 551–562

(2019).

108

251. Koo, P. C. et al. Combined cognitive, psychomotor and electrophysiological biomarkers

in major depressive disorder. Eur. Arch. Psychiatry Clin. Neurosci. 269, 823–832 (2019).

252. Buchmann, V., Nilsen, T. & Billinghurst, M. Interaction with partially transparent hands

and objects. in Proceedings of the Sixth Australasian conference on User vol. 40 17–20

(Australian Computer Society, Inc., 2005).