NEUROERGONOMIC ASSESSMENT OF THE

ROBOTIC ENHANCEMENT OF SURGERY

David Roland Christopher James MRCS (Eng), MBBS, BSc (Hons)

Hamlyn Centre for Robotic Surgery Department of Surgery and Cancer Imperial College of Science, Technology and Medicine University of London

2012

This thesis is submitted to Imperial College in partial fulfilment of the requirements for the degree of Doctor of Philosophy. Except for where indicated, it presents entirely my own work and describes the results of my own research.

ABSTRACT

Advances in surgery have led to marked improvements in patient care, in particular, indexed by minimally invasive surgery (MIS). With the assistance of technology, progressively more complex procedures are undertaken with the aim of diminishing the impact of surgery on the patient. However, this increases demands placed on the surgeon in terms of technical ability. Surgical robotics may overcome this by enhancing surgeon capability through a variety of means. However, it is imperative that novel equipment is scrutinised not only for its performance effects but as to its impact on the user. This is to ensure that not all a surgeon’s resources are devoted to using a new instrument rendering them unable to focus on other aspects of surgery such as dealing with unexpected events e.g. bleeding. Neuroergonomics encompasses the study of the brain and behaviour at work and entails appreciating the cortical correlates of work-related tasks in order to understand both task demands and how the work environment may be modulated to facilitate performance. This paradigm has been applied to safety critical industry such as aviation and air traffic control, and is wholly applicable to surgery especially as the increasing demands placed on surgeons may be more cognitive in nature and as such not be as readily detected with conventional ergonomic tools.

This purpose of this thesis is to apply neuroergonomic principles to evaluate brain behaviour associated with complex surgical tasks and how this may be modulated by assistive technology. Using functional near infrared spectroscopy (fNIRS), the cortical substrate for undertaking a complicated navigational task, of the nature likely to benefit from robotic enhancement, is assessed. This establishes firstly, a reliance on cortical regions involved in visuospatial working memory; and secondly demonstrates that experts elicited greater cortical activity than novices. Subsequently, assistive technology known to enhance performance is assessed as to its impact on prefrontal cortical (PFC) activity. A randomised controlled trial across six sessions is then undertaken with the aim of appraising how this technology may affect the learning process in terms of frontoparietal cortical activity. Brain behaviour is assessed in terms of activity and network behaviour determined using graph theory. Finally, a novel tool for aiding collaborative surgery is investigated demonstrating a modulation of search strategy and underlying cortical activity affording subsequent performance improvement. This work sheds light on the neurocognitive aspects of undertaking surgical tasks and how this information can be applied within the paradigm of neuroergonomics to evaluate and assess novel instrumentation in surgery.

2 ACKNOWLEDGEMENTS

“A very dangerous state of mind: thinking one understands.”

Paul Valéry

I would like to extend my sincere thanks to my supervisors Professor the Lord Ara Darzi of Denham and Professor Guang-Zhong Yang for their continued support and motivation throughout this thesis. The degree of guidance and supervision that I received was outstanding, ensuring that even when faced with the challenges that research of this nature generates, I was always able to call upon their wisdom and experience for encouragement, assistance and advice. Coupled with this, undertaking research in such a venerable academic environment that they both have crafted has been both an honour and a privilege.

This research was also made possible thanks to a great deal of support from my colleagues and friends within the Hamlyn Centre for Robotic Surgery. In particular, George Mylonas, Ka-Wai Kwok and Mike Sun, all three of whom tirelessly helped in designing, developing and maintaining the intricate equipment that comprised my task paradigms. I must also thank Daniel Elson, Neil Clancy, Vincent Sauvage, Louis Atallah, Kenko Fuji, Lichao Wang, Darren Patten and Gabriella Yongue. Their help and assistance was most gratefully received.

Throughout this period of research I have been privileged enough to be able to turn to Professor Thanos Athanasiou for statistical advice and general guidance. His research ability is outstanding and I am very grateful for all the time he devoted in helping me analyse and interpret the data. Much of my time over the last three years was spent in room 307 with Mikael Sodergren, James Clark, Vahe Karimyan, Richard Newton and more latterly Colin Sugden. They have all provided support and solid friendship throughout this time.

Undertaking this period of research in neuroscience and functional imaging as a surgical trainee is one of the greatest challenges I have faced. Two of my colleagues in particular made this possible. Firstly, Daniel Leff has tirelessly motivated me throughout my whole time at Imperial. His enthusiasm for this research field is infectious and his approach to this complex area all the way from study inception to publishing results is meticulous,

3 rigorous and based on a vast knowledge base. I am truly grateful for Daniel’s continued support and friendship. Secondly, I am indebted to the vast help that Felipe Orihuela- Espina devoted to my research. From my first day on this project, Felipe supported me and I am continually in awe of his ability to teach the most challenging of mathematical concepts with such clarity that any topic was rendered comprehensible to a surgeon. Felipe was a great source of knowledge and advice to many in the department. However, he still always found time for a cup of coffee and a lengthy discussion about fNIRS. I am honoured to have had him as a colleague and friend. I must also thank all those staff and students who were subjects in my experiments. I am truly grateful that they devoted so much of their precious time to my research.

Throughout this period of research, my parents have provided a consistent foundation of support and advice. Their devotion has always been recognised and greatly appreciated. My sister Bronwen, provided reliable and sound advice throughout this research and was always able to add perspective when needed. Above all, I must thank my wife and children. Poppy and Georgie who have not known life without me grappling with this thesis. Yet they provide me with the love, joy and motivation to keep going. Ultimately, I owe all this to the unwavering support and love of my wife Lulu. I will always be grateful for all she has done for me. Without her, none of this would have been possible.

I dedicate this thesis to the memory of my sister Mair Elizabeth James, whose bravery in the face of adversity was truly inspirational.

4 CONTENTS

ABSTRACT ACKNOWLEDGEMENTS CONTENTS LIST OF FIGURES LIST OF TABLES LIST OF ACRONYMS

CHAPTER 1 INTRODUCTION ...... 21 CHAPTER 2 SURGERY AND ERGONOMICS ...... 30 2.1 INTRODUCTION ...... 30 2.2 ASSESSMENT OF SURGICAL ERGONOMICS ...... 31 2.2.1 Performance and Outcome Related Measures ...... 32 2.2.1.1 Observational Techniques ...... 33 2.2.1.2 Non-Observational Techniques ...... 33 2.2.2 Physiological Related Measures ...... 34 2.2.2.1 Electromyography ...... 34 2.2.2.2 Posture ...... 35 2.2.2.3 Motion Analysis ...... 36 2.2.2.4 Self Reporting Questionnaire ...... 37 2.3 STRESS AND PERFORMANCE ...... 38 2.3.2 Heart Rate and Respiratory Rate ...... 40 2.3.3 Neuroendocrine Response ...... 41 2.3.4 Galvanic Skin Conductance ...... 41 2.3.5 Electrooculogram ...... 42 2.3.6 Self Reporting Measures of Stress...... 42 2.4 NEUROERGONOMICS ...... 43 2.4.1 Introduction ...... 43 2.4.2 Applications ...... 44 2.4.2.1 Mental Workload ...... 44 2.4.2.2 Vigilance and Fatigue ...... 46 2.4.2.3 Driving ...... 47 2.4.2.4 Aviation ...... 48 2.4.2.5 Medical Safety...... 48 2.5 NIRS AND NEUROERGONOMICS ...... 49 2.6 CONCLUSIONS ...... 52 CHAPTER 3 FUNCTIONAL NEUROIMAGING AND CORTICAL CORRELATES OF SURGICAL TASKS………… ...... 54 3.1 INTRODUCTION ...... 54 3.2 BIOLOGICAL BASIS OF FUNCTIONAL NEUROIMAGING ...... 55 3.3 NEUROIMAGING MODALITIES ...... 56 3.3.2 Direct Neuroimaging Modalities ...... 57 3.3.3 Indirect Neuroimaging Modalities ...... 58

5 3.3.3.1 Positron Emission Tomography (PET) and Single Photon Emission Computed Tomography (SPECT) ...... 58 3.3.3.2 Functional Magnetic Resonance Imaging (fMRI) ...... 58 3.4 FUNCTIONAL NEAR INFRARED SPECTROSCOPY ...... 59 3.4.1 Background: the ‘Optical Window’ ...... 59 3.4.2 Modified Beer-Lambert Law...... 61 3.4.3 Instrumentation ...... 62 3.4.4 Limitations of Functional Near Infrared Spectroscopy ...... 65 3.5 CORTICAL CORRELATES OF SURGICAL TASKS ...... 65 3.5.1 Neuroimaging and Surgery ...... 66 3.5.2 Motor Learning ...... 69 3.5.3 Visuospatial Ability and Working Memory ...... 71 3.5.4 Error Management and Attention to Action ...... 72 3.5.5 Visuomotor Tasks and Visual Search ...... 73 3.6 CORTICAL CONNECTIVITY...... 73 3.6.2 Anatomical Connectivity ...... 74 3.6.3 Functional Connectivity ...... 74 3.6.4 Effective Connectivity...... 75 3.7 GRAPH THEORY ...... 76 3.7.1 Background ...... 76 3.7.2 Graph Theory: Graph Generation and Terminology ...... 79 3.7.3 Factors Influencing Cortical Graphs ...... 84 3.7.4 Comparison of Graphs ...... 87 3.7.5 Graph Theory and Functional Near Infrared Spectroscopy ...... 88 3.8 CONCLUSION AND HYPOTHESES ...... 89 3.8.1 Prefrontal Cortical Behaviour and Complex Surgical Tasks ...... 90 3.8.2 Modulation of Cortical Behaviour by Assistive Technology ...... 90 3.8.3 Cognitive Burden Estimation ...... 91 CHAPTER 4 THE ROLE OF THE PREFRONTAL CORTEX IN A COMPLEX NAVIGATIONAL TASK…………...... 93 4.1 INTRODUCTION ...... 93 4.2 MATERIALS AND METHODS ...... 96 4.2.1 Subjects ...... 96 4.2.2 Task and Training ...... 97 4.2.3 Behavioural data...... 100 4.2.4 Functional Near Infrared Spectroscopy ...... 100 4.2.5 Systemic Effect and Stress ...... 102 4.3 DATA ANALYSIS ...... 103 4.3.1 Behavioural Data ...... 103 4.3.2 Functional Near Infrared Spectroscopy ...... 103 4.3.2.1 Data Pre-processing ...... 103 4.3.2.2 Statistical Analysis of Cortical Haemodynamic Data ...... 103 4.3.2.3 Manifold Embedding ...... 104 4.3.3 Systemic Effect and Stress ...... 105 4.4 RESULTS ...... 106 4.4.1 Behavioural Data ...... 106 4.4.2 Functional Near Infrared Spectroscopy ...... 107 4.4.2.2 Statistical Analysis of Haemodynamic Activity...... 107 4.4.2.3 Dimensionality Reduction ...... 111 4.4.3 Systemic Effect and Stress ...... 112 4.5 CONCLUSIONS ...... 113 CHAPTER 5 GAZE-CONTINGENT MOTOR CHANNELLING...... 118 5.1 INTRODUCTION ...... 118 5.1.1 Gaze Contingent Motor Channelling ...... 119 5.2 METHODS ...... 121 5.2.1 Subjects ...... 121

6 5.2.2 Task and Training ...... 122 5.2.3 Behavioural Data ...... 124 5.2.4 Functional Near Infrared Spectroscopy ...... 124 5.2.5 Systemic Effect and Stress ...... 126 5.3 DATA ANALYSIS ...... 126 5.3.1 Behavioural Data ...... 126 5.3.2 Cortical Haemodynamics ...... 126 5.3.2.1 Data pre-processing ...... 126 5.3.2.2 Statistical Analysis of Cortical Haemodynamics ...... 126 5.3.3 Systemic Effect and Stress ...... 127 5.4 RESULTS ...... 127 5.4.1 Technical Performance ...... 127 5.4.2 Cortical Haemodynamic Activity ...... 128 5.4.2.1 Influence of GCMC Assistance on Cortical Haemodynamic Behaviour ...... 128 5.4.3 Systemic Effect and Stress ...... 129 5.5 CONCLUSION ...... 133 CHAPTER 6 THE INFLUENCE OF GAZE ASSISTED FORCE FEEDBACK ...... 136 6.1 INTRODUCTION ...... 136 6.2 MATERIALS AND METHODS ...... 140 6.2.1 Subjects ...... 140 6.2.2 Task and Training ...... 140 6.2.3 Randomisation ...... 142 6.2.4 Blinding ...... 142 6.2.5 Functional Near Infrared Spectroscopy ...... 142 6.2.6 Systemic Effect and Stress ...... 142 6.3 DATA ANALYSIS ...... 143 6.3.1 Behavioural Data ...... 143 6.3.2 Cortical Activation ...... 143 6.3.3 Graph Theoretical Analysis ...... 143 6.3.3.2 Graph Construction ...... 145 6.3.3.3 Graph Theory Metrics ...... 146 6.3.4 Systemic Effect and Stress ...... 147 6.4 RESULTS ...... 147 6.4.1 Participant Flow ...... 147 6.4.2 Numbers Analysed ...... 148 6.4.3 Behavioural Data ...... 148 6.4.4 Cortical Activity ...... 149 6.4.5 Frontoparietal Network Activity ...... 152 6.4.6 Systemic Effect and Stress ...... 156 6.5 CONCLUSIONS ...... 157 6.5.1 Methodological Considerations ...... 158 CHAPTER 7 COLLABORATIVE GAZE CONTROL FOR SURGICAL ROBOTICS ...... 160 7.1 INTRODUCTION ...... 160 7.2 MATERIALS AND METHODS ...... 164 7.2.1 Subjects ...... 164 7.2.2 Task and Training ...... 164 7.2.3 Behavioural Data ...... 166 7.2.4 Gaze Behaviour ...... 166 7.2.5 Functional Near Infrared Spectroscopy ...... 167 7.2.6 Systemic Effect and Stress ...... 167 7.3 DATA ANALYSIS ...... 167 7.3.1 Behavioural Data and Gaze Behaviour...... 167 7.3.2 Cortical Haemodynamic Activity ...... 167 7.3.3 Visuoparietal Network Activity ...... 167 7.3.4 Systemic Effect and Stress ...... 168

7 7.4 RESULTS ...... 169 7.4.1 Behavioural Data ...... 169 7.4.2 Gaze Behaviour ...... 170 7.4.3 Cortical Activity ...... 170 7.4.4 Visuoparietal Network Activity ...... 173 7.4.5 Systemic Effect and Stress ...... 175 7.5 CONCLUSIONS ...... 177 CHAPTER 8 CONCLUSIONS AND FUTURE PERSPECTIVES ...... 181 8.1 ACHIEVEMENTS OF THIS THESIS ...... 181 8.2 FUTURE PERSPECTIVES ...... 183 8.2.1 Effect of Task Training on Prefrontal Cortical Activity ...... 183 8.2.2 Fatigue and Surgical Errors ...... 184 8.2.3 Stress and the Systemic Effect ...... 185 8.2.4 fNIRS as a Means to Enhance Perceptual Docking ...... 185 8.2.5 Environment and Team Behaviour ...... 186

APPENDICES A. PREFRONTAL CORTICAL BEHAVIOUR IN THE NAÏVE PHASE OF COMPLEX TASK LEARNING ...... 188 B. INFLUENCE OF HEART RATE AND STRESS ON CORTICAL HAEMODYNAMICS… ...... 193

8 List of Figures

Figure 1.1 Flow diagram indicating the structure of the thesis implemented in order to achieve a means of understanding of how surgical tasks are impacted by assistive technology...... 23 Figure 2.1 Example of optical tracking system (Optotrak Certus®, Northern Digital Inc., Canada) in use during simulated flexible endoscopic surgical task. The subject (left) has optical trackers (see subplot) on the right side of head, right shoulder, forearm and wrist. The video motion tracking camera (labelled on right side of figure) detects infrared light emitted from the trackers and accordingly determines motion...... 36 Figure 2.2 Localised experienced discomfort (LED) scale [36]. Subjects utilise diagram to indicate sites of discomfort at designated intervals whilst undertaking a task (adapted from [36])...... 38 Figure 2.3 Diagrammatic representation of the stress system. Stress evokes the HPA axis (left side of image) by inducing corticotropin-releasing hormone (CRH) to be secreted from the paraventricular nucleus of the hypothalamus. CRH acts on the anterior pituitary gland leading to adrenocorticotropic hormone (ACTH) release. ACTH subsequently induces cortisol release from the zona fasciculata of the adrenal cortex. The right side of the system depicts the locus coeruleus/sympathetic nervous system component of the stress system. This leads to activation of the sympathetic nervous system and noradrenaline release. Multiple feedback inhibitory and stimulatory pathways exist in this system...... 39 Figure 2.4 Heart rate variability as a measure of subject stress. The electrocardiogram (ECG) (upper panel, blue) for subject when relaxed (left) and stressed (right). The corresponding tachogram (central panel, purple) comprises the beat to beat change in the heart rate expressed as the R to R interval and demonstrates less variability in the stressed condition. The lower panel schematically represents stress-related changes within the frequency domain. Stress and the corresponding sympathetic nervous system response causes a peak in the low frequency (LF) component and a relative decrease in the high frequency (HF) component...... 40 Figure 3.1 Evaluation of major classes of neuroimaging modalities in terms of temporal and spatial resolution [redrawn from Strangman et al. [94]]...... 56 Figure 3.2 Schematic representation of EEG machine (left) and MEG (right), not to scale. The EEG machine consists of electrodes (a) positioned over the subject’s scalp which are recorded and processed (b). MEG (c) necessitates the patient to be seated with the head placed suitably close to the sensors...... 57 Figure 3.3 Sample time course depicting activation-induced changes in HbO2 (red) and HHb (blue). Following task onset (shaded green region), HbO2 increases after approximately 6 to 8 seconds and after a slight increase, HHb decreases. Lines and shaded regions of time course represent mean and standard deviation of an averaged signal respectively)...... 60 Figure 3.4 Absorption spectra of HbO2 (red), HHb (blue) and water (black). The subplot indicates the isobestic point where the absorption spectra cross. At higher wavelengths progressively more light is absorbed by water (black line)...... 61 Figure 3.5 Hitachi ETG4000. The main unit (right) contains the processing unit, a liquid crystal display (LCD) and an optical transmitter. The optode hanger (protruding from left hand side of main unit) suspends the fibre optic cables and the optode emitters and detectors. The latter are depicted in the enlarged subplot (left). The 3D digitizer is situated beneath the LCD display...... 63

9 Figure 3.6 Overlay and display in relation to reference MRI atlas. Initially the head mesh is generated using the 3D digitizer (top panel, channel numbers have been added to aid interpretation). The 3D coordinate file is exported to a composite display unit where the head mesh and relative optode positions can be displayed overlying a reference MRI atlas (lower panel). This enables appreciation of relative optode position in relation to underlying cortical structures...... 64 Figure 3.7 Depiction of small-worldness as defined by Watts and Strogatz [164]. Panel a represents a regular lattice with a high clustering coefficient and a long path length. The graph is then rewired randomly in order to generate a random graph (c). The random graph has a short path length and a low clustering coefficient. Following a minimal rewiring, the regular lattice becomes small-world (b), in which the clustering coefficient is still high however the path length is much shorter than the corresponding regular lattice. As depicted in (d), minimal rewiring rapidly shortens the normalised path length whilst preserving the clustering representative of small-worldness. [Redrawn from Watts and Strogatz [164]]...... 77 Figure 3.8 The first description of graph theory in 1736 [160]. Panel a depicts Euler’s original diagram of the bridges of Konigsberg. In order to solve the problem as to whether it was possible to traverse all bridges only once, he represented the map as a graph, as redrawn in panel b. The nodes of the graph represent the land and the edges depict the bridges. Euler determined that in order to cross each only once, the land (nodes) could only have an even number of connections (edges). If two nodes had an odd number of edges, it would be possible if they were the beginning and end of the route. [Figure reproduced and redrawn from [160]]...... 77 Figure 3.9 Overview of graph generation. Neuroimaging data is acquired and nodes are defined (ROI, electrode, sensor, voxel, channel). Functional association of nodes is determined using one of the methods (a). Subsequently a threshold (b) is applied converting the association matrix into a binary adjacency matrix. This serves as the input to generate the network...... 80 Figure 3.10 Diagrammatic representation of network clustering coefficient and pathlength. The clustering coefficient of a node represents the number of connections its nearest neighbours (pathlength = 1) have with each other as a proportion of the maximum number of connections they could have. For example, for node A, the maximum number of connections its neighbours could have is 3 (B – C, B – D and C – D). The number of connections that exist is 1 (B – C). Therefore the clustering coefficient of node A is 1/3. The pathlength is shortest distance between two nodes. The pathlength between node E and F is 4 (depicted red). The pathlength indicates how well a graph is connected and the clustering coefficient is a measure of local graph structure [164, 172, 196]...... 84 Figure 4.1 Images from endoscope acquired during transvaginal cholecystectomy. Panel a displays dissection of gallbladder from the gallbladder bed with the insulated tip (IT) knife. Panel b depicts the specimen following complete removal prior to transvaginal extraction. (Images courtesy of Mr James Clark)...... 94 Figure 4.2 Picture of colonoscope (model 13907PKS, Karl Storz GmbH & Co, Tuttlingen, Germany) used by subjects during the task to navigate between targets...... 97 Figure 4.3 Simulated surgical environment (right) in which the task was performed displayed in relation to anatomical reference (left).Targets can be appreciated within the simulated abdominal cavity and subjects navigated repeatedly between two targets (highlighted in blue) that could not be visualised within the same field of view, necessitating navigation...... 98 Figure 4.4 View through colonoscope as viewed by the subject navigating from one target to the other (panels a to c respectively)...... 98

10 Figure 4.5 Task setup. The subject (left) performs the task by manipulating the tip of the colonoscope with the right hand (panel a) whilst operating the controls of the colonoscope with the left hand (panel b). NIRS optodes can be appreciated attached to the forehead...... 99 Figure 4.6 Haemoglobin time courses from pilot studies demonstrating how the response varies depending on the length of task (green vertical bars) and rest periods (white vertical bars). HbO2 and HHb (red and blue lines respectively) are displayed. Changes in cortical haemodynamics have not returned to baseline prior to task onset predominantly indexed in the HbO2 signal (indicated with arrows) when either a 30s (a) or a 40s (b) rest period is employed...... 100 Figure 4.7 Approximate optode location obtained by transferring topographic data from a representative subject to a 3D cortical surface of an MRI atlas as viewed from right (a), front (b) and left (c). Channels (blue numbered circles) are displayed in relation to UI 10/20 locations [210] (yellow circles)...... 101 Figure 4.8 Systemic effect and stress measurement. Salivette and vial for cortisol measurement (a and b respectively). Wireless receiver (c) for detection of signal from portable ECG (d) worn under subject clothing...... 102 Figure 4.9 Boxplot illustrating the number of times target was visualised by the two groups during the task as assessed by the median of three independent reviewers’ score. The median, 95% confidence interval (CI) and outliers are represented by the box and whiskers. * denotes significance (p < 0.001)...... 106 Figure 4.10 Time courses from excluded subject. Task and rest periods (green and white vertical bars respectively) and HbO2 and HHb (red and blue lines respectively) can be appreciated. Panel a depicts channel 2, in which numerous movement artefacts are present however do not appear to disrupt the overall trend of the time course too markedly. In Panel b (channel 5), it is apparent that movements at the onset of task block 1 and 2 have considerably altered the haemodynamic data...... 107 Figure 4.11 Haemodynamic time course for a representative novice and expert subject (panels a and b respectively). A coupled task-evoked increase in HbO2 and decrease in HHb (red and blue lines respectively) can be appreciated. Activity appears to be greatest in the expert...... 108 Figure 4.12 Channels (large red circles) demonstrating a statistically significant increase in HbO2 and a decrease in HHb and channels (large yellow circles) not activating projected on top of a brain atlas for novices (a) and experts (b). Group differences show a predominantly lateral response in the both groups however this appears to be more pronounced in experts. Small circles represent optode position (emitters in red and detectors in blue)...... 110 Figure 4.13 Manifold embedding of fNIRS data. Experts (blue) and novices (red) are projected into the experiment space. Each point represents the HbO2 and HHb values from a single subject, during a single task, at a single channel. The subplots (a to i) illustrate the reconstructed haemodynamic response at selected points throughout the manifold. Within the subplots, the HbO2 (red) and HHb (blue) time courses are displayed. It can be seen that towards the top left of the embedded space (subplot a), the pattern is most consistent with activation. Towards the centre of the cluster (subplot e) the pattern is consistent with no activity. Points that are closer to together are more similar in haemodynamic behaviour [111, 112]...... 111 Figure 4.14 Embedded space in which all points of the cluster are initially labelled by expertise and channel. The centroids of these clusters is depicted. Experts (blue circles) and novices (red crosses) are displayed and the respective channels are labelled. Points that are closer together are likely to have similar signal similarity, therefore channels whose centroids are closely related to others are likely to have similar behaviour...... 112

11 Figure 4.15 Dendrograms of the hierarchical cluster of channels based on EMD computed over the manifold embedding for novices (left) and experts (Right). It can be appreciated that different pattern of channel clustering occurs between the two groups with relatively more channels clustering at a lower EMD in experts compared to novices...... 112 Figure 5.1 Schematic illustration of GCMC. The surgeon fixates on the target (black dot on screen) and their point of regard is detected with the eyetracker. This positional information is used in conjunction with the location of the tool tip (black line on screen) in order to calculate the force (F) required to be exerted via the haptic manipulator in order to diminish the distance (d) between the two...... 119 Figure 5.2 Screen shot depicting boundary of haptic constraint (conical mesh) constraining tool (green stylus) to target (blue dot) on surface of simulated heart...... 120 Figure 5.3 The subject (seated) can be appreciated holding the haptic manipulator in their right hand which controls the virtual tool (yellow) whilst regarding the target (black dot) on screen. Their gaze (green dashed line) is detected with the portable eyetracker situated beneath the monitor. The tool tip position is known (red dashed line) and is used to calculate the distance (d) between it and the target. Accordingly, a force (F) is generated that is a function of that distance and exerted via the haptic manipulator in order to localise the target. An optode array can be appreciated over the left PFC...... 122 Figure 5.4 Panel a: the tool (yellow) and target (black circle) are visible within the task scene. Trajectory and fixation point (blue ellipse and white cross) are not visible to subject. Subject fixates on target allowing calculation of force needed to displace tool tip to target (b). Target localised (c) and as fixation point remains on target, successful tracking continues (d)...... 123 Figure 5.5 Screen shot of GCMC training game. Targets (can and blue face) crossed the screen in horizontal and vertical directions and had to be localised in the cross-hairs (white circle, centre of image). The graphical user interface (lower right) enabled switching between the training modalities and was hidden from view during the task...... 123 Figure 5.6 Experiment flow. Following randomisation, subjects undertook both tasks. Task periods (20s) were interspersed with rest periods (30s)...... 124 Figure 5.7 Schematic depiction of optode placement in relation to UI 10/20 coordinates. Optode emitters and detectors (red and blue numbers) are positioned according to UI 10/20 positions (depicted at corners of arrays). Channels are accordingly located (black numbers 1 – 24)...... 125 Figure 5.8 Channel location displayed over reference MRI atlas using 3D composite overlay system (Hitachi Medical Corp., Japan). Channel locations have been labelled (numbered yellow circles)...... 125 Figure 5.9 Median performance accuracy in terms of distance from the tool tip to the target (pixels) for control (blue) and GCMC (red) task...... 128 Figure 5.10 Sample time course from representative subject under control (panel a) and GCMC (panel b) conditions. Task and rest periods (green and white vertical bars respectively) are indicated. It can be appreciated that more marked task-evoked activity is demonstrated under GCMC assistance. This is predominantly indexed by an increase in HbO2 (red line); however the task-related decrease in HHb (blue line) is less prominent with the exception of tasks 3 and 5...... 129 Figure 5.11 Figure depicting haemodynamic behaviour for a single subject for control and GCMC assistance (a and b respectively) at each of the 24 channel lcoations. Haemodynamic behaviour is averaged across all 5

blocks of the task. The mean and SD of the HbO2 and HHb (M x cm) signals (red and blue lines and shaded regions respectively) can be appreciated. The task period (green vertical bar) is associated with more task-evoked activity during the GCMC assisted condition. Black numbers represent channel number and the x axis is time (samples)...... 130

12 Figure 5.12 Activation matrix for subjects under control (left) and GCMC (right) conditions. Channels demonstrating a coupled increase in HbO2 and decrease in HHb in which both species reach significance are displayed red. Channels demonstrating a coupled increase in HbO2 and decrease in HHb with one species reaching significance are depicted pink. Remaining channels are displayed yellow. A predominantly right lateralised response can be appreciated and this is spatially broader under GCMC assistance...... 131 Figure 6.1 Flow diagram describing the stages of graph construction from node definition to econometric analysis...... 138 Figure 6.2 Subject performing task. The haptic manipulator (right hand) is used to control the virtual tool whilst gaze behaviour is detected with the portable eyetracker (beneath the monitor). Optodes can be appreciated overlying the left PFC and PC. Figure used with kind permission of subject...... 140 Figure 6.3 Pilot data across 6 sessions of performance accuracy for control (blue) and GCMC (red) users. Error bars indicate 95% CI...... 141 Figure 6.4 Approximate optode location obtained by transferring topographic data from a representative subject to a 3D cortical surface of an MRI atlas for left PFC (a) and left PC (b). Channels (yellow numbered circles) are displayed in relation to UI 10/20 and 10/10 locations [210]. Optode emitters and detectors (red and blue circles respectively) can also be appreciated...... 142 Figure 6.5 Flow of data analysis and graph construction. Following pre-processing and conversion to relative changes in HbO2 and HHb, haemodynamic data is averaged across the five task blocks and over all subjects in each group for each session. This yields a grand averaged timecourse (1). The five sample baseline is compared with the task-averaged Hb value in order to determine task-evoked increases/decreases in HbO2 and HHb. This is subsequently displayed overlying channel locations on a reference MRI atlas (a). For graph generation, both Hb species in the grand averaged time course are bidimensionally cross-correlated in order to generate the association matrix (2). In turn, this is pruned to generate the adjacency matrix (3) using the threshold. In the current study, the association matrix is pruned according to the connection density of the graph with least significant connections removed first. From the adjacency matrix (3), the undirected, weighted cortical network (4) is generated. In order to generate subject-specific cortical networks, stages of analysis are similar with the exception that individual subject time courses are cross-correlated and the association matrix is subsequently pruned according to the relevant subject specific activation matrix...... 144 Figure 6.6 Consort diagram, indicating flow of subjects through the experiment. No subjects were excluded; however, 1 withdrew after the second session...... 148 Figure 6.7 Subject performance across six task sessions and the retention test (session 7, which occurred a median of 71 days after session 6). Data represent mean and 95% CI (error bars) for subjects in control (blue) and GCMC (red) groups. Initially, performance is better in the control group until the learning curves intersect after the second session. Subsequently, GCMC users demonstrate improved performance. There is no significant difference between performance in session 6 and 7 in either group indicating retention of the task...... 149 Figure 6.8 Session averaged haemodynamic data from representative control and GCMC subjects (a and b respectively). Mean HbO2 and HHb (red and blue bold lines) signal can be appreciated with the corresponding SD (red and blue shaded regions). The task and rest periods (green and white bars respectively) can be appreciated...... 150 Figure 6.9 Groupwise statistical analysis of longitudinal changes in cortical activity across sessions 1, 3 and 6 for control (left) and GCMC (right) subjects. Approximate channel locations are indicated by colours representative of the pattern of activity: (a) ΔHbO2 increase and coupled ΔHHb

13 decrease (both reaching significance); (b) ΔHbO2 increase and coupled ΔHHb decrease (one reaching significance); (c) ΔHbO2 increase and coupled ΔHHb decrease (neither reaching significance); (d) No paired increase in ΔHbO2 decrease in ΔHHb. Attenuation in PFC activity and focussing of activity within the PPC can be appreciated...... 152 Figure 6.10 Longitudinal changes in F-P networks for representative control (top) and robotic assisted (bottom) subject. Approximate channel locations (black circles) are nodes within the graph and connections (red) are weighted according to their strength. More connections are apparent with robotic assistance. It can be appreciated that the number of connections varies with session...... 153 Figure 6.11 Evolution in performance and econometric parameters across practice for control (red) and GCMC (blue). By session 6 both groups have migrated to a region indicative of a high efficiency and low cost...... 154 Figure 6.12 Longitudinal changes in F-P networks for representative control (top) and robotic assisted (bottom) subject. Approximate channel locations (black circles) are nodes within the graph and connections (black) are weighted according to their strength. It is apparent that a greater number of inter-regional connections between the PFC and PC are present in earlier tasks...... 154 Figure 6.13 Cortical network econometrics for control (blue) and GCMC (red) subjects across the 6 task sessions in graphs normalised for connection density. It can be appreciated that cognitive burden and efficiency is improved with GCMC and cost is lower. The small-world index is greater with GCMC and both groups display a small-world index > 1 until the final session. Efficiency, economy and the small-world index display a decreasing trend across the six sessions and cost increases. This occurs in tandem with attenuation in PFC activation...... 155 Figure 7.1 Schematic representation of CGC for intra-operative guidance. The trainee and trainer can be appreciated (left and right respectively) operating in a shared robotic surgical environment with portable eyetracking (beneath each monitor) detecting gaze behaviour. Under conventional guidance (upper panel), the trainer verbalises direction to the trainee. With CGC (lower panel), the trainer’s point of regard is detected in real time (blue dashed line) and displayed on the trainee’s monitor (represented as white cross) where it can be easily seen...... 161 Figure 7.2 Schematic illustration of visual saliency. Searching for red oblongs is likely to be reliant on bottom-up search due to salience of scene. A search for the number of vertical green oblongs is likely to necessitate a top- down search in order to identify the less salient target...... 163 Figure 7.3 Task setup. Both trainee and trainer (left and right respectively) control virtual instruments, each with two haptic manipulators (Phantom Omni, SensAble Tech, USA). The trainer’s right hand manipulator is highlighted (yellow). Gaze behaviour is detected with portable eyetracker (x 50 eyetracker, Tobii Technologies, Sweden) situated below both monitors (trainer eyetracker highlighted yellow). The optical topography system can be appreciated (left of screen highlighted yellow) with optodes positioned over the trainee’s V-P cortices...... 164 Figure 7.4 Task images as they appear on trainee monitor. The trainee’s instruments are located inferiorly. Initially, the trainee sees the blue cross indicating the intended biopsy target (i). The trainee then grasps the target nodule (black circle) with their right instrument (ii) and passes it to the trainer’s instrument (iii – iv). Trainer monitor (not displayed) is identical with the exception that the target nodule for biopsy is indicated. .... 165 Figure 7.5 Schematic representation of optodes as positioned in the 4 x 4 array (a). Emitters and detectors (red and blue numbers) and channels (black numbers) are positioned with the penultimate row of optodes overlying Oz. The subject (b) undergoes optode registration to generate a 3D mesh (c). This is subsequently co-registered with a reference MRI scan (d), in

14 which channels (yellow numbered circles) can be appreciated overlying bilateral visuoparietal cortices...... 166 Figure 7.6 Performance in terms of number of biopsies retrieved (left panel) and instrument path length (right panel). Box plots indicate mean and error bars represent 95% CI...... 169 Figure 7.7 Gaze plots from a representative subject under control (left) and CGC (right) guidance demonstrate more focussed fixations during the task as evidenced by more tightly confined regions of hotspots as compared to the control condition where fixations are more widespread...... 169 Figure 7.8 Mean gaze latency for control (blue) and CGC (red) indicating that under CGC guidance the time taken for the trainee’s fixation point to reach the trainer’s fixation point was significantly less. Error bars indicate 95% CI...... 170 Figure 7.9 Grand averaged time courses HbO2 and HHb (M x cm) (red and blue lines respectively) for control (a) and CGC (b) conditions. Data is averaged across the 5 task blocks (represented by green bar) for each of the 24 channels (numbered yellow)...... 171 Figure 7.10 Topogram depicting task-averaged HbO2 response for control (a) and CGC (b) conditions from a representative subject. A broader response extending beyond the visual cortex is evident under the control condition...... 172 Figure 7.11 Figure depicting group averaged V-P channel activation for control (left) and CGC (right) guidance. Level of activation is indicated by colour: (a) red: significant increase in HbO2 coupled to significant decrease in HHb; (b) spots: increase HbO2 and decrease HHb (one species reaching significance); (c) stripes: increase HbO2 and decrease HHb (neither species reaching significance) and (d) black: no coupled increase HbO2 and decrease HHb. A greater number of activating channels is appreciated under verbal guidance (control vs. CGC = 19/24 vs. 11/24)...... 172 Figure 7.12 Activity-guided cortical networks for representative control (a) and CGC (b) subject. Approximate channel locations (black circles) are overlain onto reference MRI atlas. Network edges can be appreciated with strength of functional association between channels represented by line thickness and darkness of colour. It is apparent that the network is broader under the control task (due to increased number of activating channels under this condition). However connections appear stronger with CGC. The latter is reflected in the lower network cost and cognitive burden under CGC guidance...... 173 Figure 7.13 Mean network econometric outcomes for control (blue) and CGC (red) guidance for: number of cortical connections, network cost, network efficiency and task induced cognitive burden when graph generated utilising activity guided network thresholding. Error bars indicate 95% CI...... 174 Figure 7.14 Group-derived cortical networks for control (a) and CGC (b) conditions using the normalised connection density approach. Approximate channel locations (black circles) are overlain onto reference MRI atlas. Network edges can be appreciated with strength of functional association between channels represented by line thickness and darkness of colour. It is apparent that node isolation occurs and is more marked in the CGC group network. In this group, network connections appear weaker possibly accounting for the increased network cost and decreased efficiency of this network...... 174 Figure 8.1 Hypothesis for PFC activity associated with complex task acquisition. An initial phase of poor proficiency and minimal PFC activity is demonstrated (Naïve phase). As task demands are grasped, performance improves with a necessary reliance on the PFC (Novice). Performance continues to improve as PFC activity attenuates (Trainee) until expertise is attained. (Image courtesy of Professor Guang-Zhong Yang)...... 183

15 List of tables

Table 2.1 Short form of state trait anxiety inventory (STAI) [56]. Subjects are asked to rate how they feel based on the questions above. The result is determined by adding up the scores from the three positive statements (e.g. ‘I feel calm’) with the results of negative statements (e.g. ‘I feel tense’) each subtracted from five. Therefore the maximum score indicating no stress equals 24 and the lowest score indicating maximal stress equals 6...... 43 Table 2.2 Summary of fNIRS study investigating variation in mental workload with task demand and the brain behaviour associated with work-related tasks. HbO2 – oxyhaemoglobin, HHb – deoxyhaemoglobin...... 51 Table 3.1 Overview of advantages and disadvantages of the three main NIRS modalities [94, 103]...... 63 Table 3.2 Neuroimaging studies investigating cortical activity of surgical tasks...... 67 Table 3.3 Stages of motor learning as described by Halsband and Lange [117]...... 70 Table 3.4 Example of MRT for matching (a) and non-matching (b) shapes, rotated by 180 degrees (adapted and redrawn from [125])...... 72 Table 3.5 Categories of cortical connectivity and respective definitions...... 74 Table 3.6 Graph constituents in terms of node number and definition, edge calculation and graph threshold. Node definition: (1) Anatomical parcellation; (2) Grey matter volume; (3) Cortical thickness; (4) Voxel; (5) MEG sensor; (6) EEG region of interest (ROI) defined using standardised low resolution brain electromagnetic tomography (sLORETA). Edge calculation: (a) Wavelet correlation; (b) Correlation; (c) Partial correlation; (d) Mutual information; (e) Tractographic; (f) Synchronisation likelihood; (g) Coherence. Threshold: (1) Significance of association; (2) Thresholded to connection density; (3) Thresholded to r value of correlation; (4) false discovery rate (FDR) to correct for multiple comparisons; (5) No threshold; (6) Thresholded according to synchronisation likelihood; (–) Not stated...... 81 Table 3.7 Graph theory terminology...... 83 Table 3.8 Impact that tasks, intelligence, age, pharmacology and pathology have on network properties ...... 85 Table 4.1 Results of multivariate random effect model for ΔHbO2. Questionnaire refers to the STAI response. Significant p values are highlighted in bold...... 109 Table 4.2 Results of multivariate random effect model for ΔHHb. Questionnaire refers to the STAI response. Significant p values are highlighted in bold...... 109 Table 4.3 Results of multivariate random effect model for ΔHbT. Questionnaire refers to the STAI response. Significant p values are highlighted in bold...... 110 Table 5.1 Univariate random effects analysis of impact of session (control versus GCMC) on performance accuracy...... 127 Table 5.2 Results of multivariate random effect model for ΔHbO2. Laterality refers to left or right PFC. Significant p values are highlighted in bold...... 131 Table 5.3 Results of multivariate random effect model for ΔHHb. Laterality refers to left or right PFC. Significant p values are highlighted in bold...... 132 Table 5.4 Results of multivariate random effect model for ΔHbT. Laterality refers to left or right PFC. Significant p values are highlighted in bold...... 132 Table 5.5 The influence of study group (Control versus GCMC) on HR. Significant p value highlighted bold...... 133 Table 5.6 The influence of study group (Control versus GCMC) on HR. Significant p value highlighted bold...... 133 Table 6.1 Results of multivariate random effect model for ΔHbO2. Significant p values are highlighted in bold...... 150 Table 6.2 Results of multivariate random effect model for ΔHHb. Significant p values are highlighted in bold...... 151 Table 6.3 Network econometrics of average clustering coefficient (ACC) and mean pathlength (PL) normalised to equivalent random network and utilised to calculate small-world index, normalised (norm) global efficiency,

16 normalised cost, economy and the number connections between prefrontal and parietal regions...... 156 Table 7.1 Performance, changes in cortical haemodynamics, cortical network metrics, HR and HRV values for both tasks (mean  SD)...... 171 Table 7.2 Results of multivariate REM for ΔHbO2. Significant p values are highlighted in bold...... 175 Table 7.3 Results of multivariate REM for ΔHHb. Significant p values are highlighted in bold...... 176 Table 7.4 Results of multivariate REM for ΔHbT. Significant p values are highlighted in bold...... 176

17 List of Acronyms

2D Two-dimensions 3D Three-dimensions ACC Average Clustering Coefficient Ach Acetylcholine ACTH Adrenocorticotropic hormone AESOP Automated Endoscopic System for Optimal Positioning AP Action Potential ATP Adenosine Triphosphate BOLD Blood Oxygenation Level-Dependent BP Blood Pressure BCI Brain Computer Interface CBF Cerebral Blood Flow CGC Collaborative Gaze Control CI Confidence Interval cMDS Classical Multidimensional Scaling CRH Corticotropin-releasing hormone DCM Dynamic Causal Modelling DLPFC Dorsolateral Prefrontal Cortex DOT Diffuse Optical Topography DPF Differential Pathlength Factor DTI Diffusion Tensor Imaging ECG Electrocardiogram EEG Electroencephalogram EMD Earth Mover’s Distance EMG Electromyography ENT Ear Nose and Throat EOG Electrooculogram FDR False Discovery Rate FEF Frontal Eye Fields fMRI Functional Magnetic Resonance Imaging fNIRS Functional Near Infrared Spectroscopy F-P Frontoparietal GCMC Gaze-Contingent Motor Channelling GL Gaze Latency GSC Galvanic Skin Conductance HALS Hand-Assisted Laparoscopic Surgery Hb Haemoglobin HbT Total Haemoglobin

18 HF High Frequency HPA Hypothalamic-Pituitary-Adrenal HR Heart Rate HRV Heart Rate Variability Hz Hertz ICA Independent Component Analysis ICSAD Imperial College Surgical Assessment Device IT Insulated Tip JSI Job Strain Index LCD Liquid Crystal Display LED Localised Experienced Discomfort LF Low Frequency LPFC Lateral Prefrontal Cortex LREC Local Research Ethics Committee MATB Multi-Attribute Task Battery MEG Magnetoencephalography MIS Minimally invasive surgery MRI Magnetic Resonance Imaging MRT Mental Rotation Task MVC Maximum Voluntary Contraction NIR Near Infrared NIRS Near Infrared Spectroscopy NMJ Neuromuscular Junction Norm Normalised NOSsE Natural Orifice Simulated surgical Environment NOTES Natural Orifice Translumenal Endoscopic Surgery OCHRA Observational Clinical Human Reliability Assessment OSATS Objective Structured Assessment of Technical Skill PET Positron Emission Tomography PFC Prefrontal Cortex PL Pathlength PMC Premotor Cortex PPC Posterior Parietal Cortex PSD Postural Stability Demand REM Random Effects Model ROI Region of Interest RULA Rapid Upper Limb Assessment S Seconds SCI Spinal Cord Injury SD Standard Deviation

19 SDRR Standard Deviation of the R to R Interval SEM Structural Equation Modelling SEMG Surface Electromyography sLORETA Standardised Low Resolution Brain Electromagnetic Tomography SPECT Single Photon Emission Computed Tomography SPSS Statistical Package for Social Sciences STAI State Trait Anxiety Inventory TCA Tricarboxylic Acid TLE Temporal Lobe Epilepsy UI Unambiguously Illustrated V1 Visual Cortex VLPFC Ventral Lateral Prefrontal Cortex V-P Visuoparietal VR Virtual Reality

20

Chapter 1

Introduction

Technological innovation in recent years has enabled the translation of computing and engineering tools to advances in surgery. This echoes the general trend in medicine, for which technology has heralded vast improvements in patient care mediated through enhanced diagnosis, patient monitoring and the treatment of illness. Specialties heavily reliant on instrumentation such as surgery and anaesthesia have witnessed improvements in patient care consequent to technological progress, pertinently indexed within the arena of minimally invasive surgery (MIS). In diminishing patient trauma, MIS has expedited post operative convalescence and improved outcomes for a multitude of procedures [1]. However, ergonomic limitations of instrumentation still pose challenges that obviate the use of MIS technology in every operative setting. Specifically, MIS utilises long instruments with poor force transfer from handle to tip [2] inserted into the abdominal or thoracic cavities via trocars with fixed locations on the patient. This engenders a fulcrum effect necessitating large extracorporeal movements in order to exact small displacement of the instrument tip all of which is coupled with a relatively fixed surgeon position [3]. Additionally, MIS operations are often longer than open procedures and therefore, unsurprisingly, have been shown to be more stressful for the surgeon [4, 5].

Accordingly, robotic systems were developed in order to address limitations of MIS, specifically: unstable camera platform, reduced mobility of laparoscopic instruments, 2- dimensional (2D) visualisation and poor ergonomics [6]. One of the early surgical robots was Automated Endoscopic System for Optimal Positioning, AESOP (Computer Motion, Santa Barbara, CA, USA). AESOP was designed as a camera holder controlled initially by foot pedals, but later models are voice activated. The Endoassist (Armstrong Healthcare Ltd., UK) is another robotic camera holder that is operated by an infrared emitter worn on the surgeon’s head. These systems are thought to provide a more stable image for the surgeon and to obviate the need for an assistant [7].

Telerobotic surgical robots are based on the master-slave concept, whereby the surgeon controls the surgical robot (slave) from a remote console (master). The prime example of

21 this is the da Vinci® robot (Intuitive Surgical, Sunnyvale, CA, USA). Attributes of the da Vinci® system include 3-dimensional (3D) visualisation (accordingly improving depth perception); motion scaling, so that a movement by the operating surgeon does not translate into such a large movement of the robot; and tremor filtration, which aids precision of movement. da Vinci® instruments also have an Endowrist™ (Intuitive Surgical, Sunnyvale, CA, USA) offering 7 degrees of freedom and as such enables procedures such as suturing to be performed relatively easily. This feature also makes it possible to operate within confined body cavities such as the pelvis and thorax; areas which can be problematic with conventional MIS instrumentation. Supplementary to this, in contrast to MIS, the operating console for da Vinci® is more ergonomically designed in that the surgeon is seated with armrests and looks downwards onto the viewing area with a forehead rest.

Despite these benefits of surgical robotic systems, several drawbacks exist, including a lack of haptic feedback. This means that the surgeon has to rely on tissue deformation to determine how hard they are grasping structures and therefore there is the risk of tissue damage. However, evolution in this area is continuously addressing limitations whilst systems are being developed to further extend the capabilities of the surgeon through innovative novel technology. Examples include haptic feedback, motion stabilisation of images, augmented reality and co-registration to pre-operative imaging. Notwithstanding the obvious potential of such innovation, it is still imperative that any novel equipment is rigorously scrutinised not only for performance effects but as to its impact on the user. This is encompassed by ergonomics.

Ergonomics is the study of how people interact with their work environment and tools and was first applied to surgery to highlight potentially hazardous postures of operating room staff [8]. In tandem with the proliferation of MIS, an increase in the ergonomic evaluation of minimally invasive tasks and instrumentation occurred. Numerous methodologies including electromyography (EMG), skin conductance and motion tracking have been utilised to critically appraise such factors as the nature of the instrument grip to the height of the operating table and the laparoscopic monitor. However, with robotic surgical tools, the impact that they have on the surgeon may be more subtle and may not yield great postural differences or vast performance effects within current experimental paradigms. Therefore, it is likely to become progressively more relevant to investigate the brain behaviour of the surgeon in order to not only

22 determine unique task demands, but also to guide design and development of instrumentation to overcome these challenges.

A specific area of ergonomics is Neuroergonomics which is a convergence of neuroscience and ergonomics and is the study of brain behaviour of subjects at work [9]. This field has developed as a consequence of advances in functional neuroimaging which allows the structures of the brain to be defined according to their function. Neuroergonomics therefore allows the impact of performing a task on the brain to be assessed. This is pertinent to surgery due to the increasing intricacy of surgical procedures undertaken and the concomitant increase in complexity of surgical equipment being developed in order to do so. It is important to demonstrate that a new technology not only improves accuracy, but does not increase the demands placed on user cognitive capacity.

Figure 1.1 Flow diagram indicating the structure of the thesis implemented in order to achieve a means of understanding of how surgical tasks are impacted by assistive technology.

Accordingly, the purpose of this thesis is to apply the paradigm of neuroergonomics to understand the cortical correlates of complex surgical tasks and how this may be modulated by assistive technology. In order to realise this, the following steps are taken (also depicted in Figure 1.1):

23  Employment of fNIRS to evaluate the cortical correlates of a complex surgical task within a Natural orifice translumenal endoscopic surgical (NOTES) environment.  Exploration of how performance enhancement may be achieved with assistive technology.  Development of a means to assess whether surgical equipment exacts a beneficial or detrimental effect on cortical network behaviour.  Application of this approach in order to investigate how task learning is modulated by the use of haptic constraints and by the use of a tool to enhance collaboration between surgeons.

These steps of the thesis are further discussed below in relation to individual chapters. Initially, an overview of current surgical ergonomics methodology is given in Chapter 2 followed by a discussion of stress and how this can be used to assess task demands. However, these methods of task evaluation may not be able to detect the subtle demands of complex surgical tasks and how this may be modulated by assistive technology. In response to this, the concept of neuroergonomics is discussed and how this has been applied to safety critical industry. Subsequently, the existing use of functional near infrared spectroscopy (fNIRS) in the evaluation of work-related tasks is reviewed.

Chapter 3 begins with an overview of functional neuroimaging modalities paying particular attention to fNIRS. Then a review of the current literature on neuroimaging of surgical tasks is undertaken as a platform to identification of neurocognitive skills on which surgical tasks are likely to require. In order to realise neuroergonomic evaluation of surgical tasks, it is likely that multiple cortical regions are involved and accordingly connectivity is discussed followed by a detailed review of graph theoretical analysis of functional neuroimaging data.

The first experimental chapter serves to elucidate the cortical response evoked by a complex surgical navigational task and how this is influenced by expertise. The task appears to be subserved by a reliance on lateral prefrontal cortical regions, an area key to visuospatial working memory. However, attenuation in the cortical response was not demonstrated in experts. Rather they displayed greater activity, raising questions about the role of the prefrontal cortex (PFC) in complex task execution.

24 The impact of assistive technology on PFC activity and performance is evaluated in Chapter 5. Gaze-contingent motor channelling (GCMC) constrains motor control to user fixation point and enhances surgical accuracy. In this study, GCMC is evaluated within a neuroergonomic paradigm. Performance is improved but with greater PFC activity possibly indicative of greater attentional demands or users developing a strategy for task execution that is initially PFC-dependent. This chapter raises the question as to how this performance-enhancing instrument may impact on task learning at brain level. In particular, whether it impacts on ‘functional integration’ between regions over time.

Accordingly, Chapter 6 comprises a randomised controlled longitudinal study investigating learning-related frontoparietal (F-P) network changes with or without GCMC. A concept for measuring the impact of task execution on subject brain behaviour is defined: ‘cognitive burden’ and graph theory is applied to the data in order to try and measure this from task-evoked cortical networks and how they are influenced by GCMC and learning. GCMC enhances task performance and at brain level, causes an earlier shift to PFC independence and also alleviates the task-induced ‘cognitive-burden.’

In Chapter 7, neuroergonomic evaluation of ‘collaborative gaze control’ (CGC) is undertaken. CGC is a novel tool that aids collaboration between two surgeons operating in a shared surgical environment. This study elucidates that the method by which CGC improves performance is through a modulation in surgeon search strategy and a subsequent decrease in visuoparietal (V-P) cortical activity. Again, the task-induced cognitive burden is decreased with CGC.

The thesis is concluded in Chapter 8 with a summary of the research findings. Future directions of research in neuroergonomics and surgery are also outlined.

The major contributions of this thesis include:

1) The application of a neuroergonomic approach to the assessment of surgical tasks and the impact of assistive technology on the brain of the surgeon. Much work has been devoted to ergonomic evaluation of surgical technology; however, this thesis is the first to investigate how surgeon brain behaviour is affected by robotic technology and whether this information can be used to determine if it is beneficial or not.

25 2) The investigation of cortical activity associated with a complex navigational surgical task. Activation of visuospatial centres is demonstrated in line with the nature of the task demands have been demonstrated. However, anticipated PFC independence associated with expertise was not demonstrated, rather the opposite which raises the possibility of a novel role of the PFC in the naïve stages of complex task performance.

3) The exploration of cortical correlates of performance enhancement with a robotic surgical tool (GCMC) in terms of PFC behaviour. Improvements in technical performance associated with robotic assistance were demonstrated and associated with increased PFC activity. A greater PFC response may indicate greater attentional demands and subject vigilance.

4) Definition of the concept of ‘cognitive burden.’ This thesis defines the concept of cognitive burden as ‘any deviation from the most efficient neurocognitive pathway of performing a task.’ and quantifies this using graph theoretical metrics.

5) The application of graph theoretical analysis to fNIRS data. This thesis demonstrates the first application of graph theory to fNIRS-derived neuroimaging data. Cortical networks subserving performance with and without GCMC were qualified in terms of efficiency, cost, small-worldness and task- induced cognitive burden. This displayed a beneficial impact of GCMC on the activated frontoparietal network.

6) The neuroergonomic evaluation of collaborative gaze control for robotic surgery. Utilising a neuroergonomic paradigm, CGC is evaluated and its mode of action and performance-enhancement is determined. This study delineates that in utilising CGC, subject gaze behaviour is modulated and underlying visuoparietal cortical activity is reduced. All of which are indicative of a modulation of search strategy from ‘top-down’ to ‘bottom-up’ which is associated with less cortical activity.

26 The research presented in this thesis has resulted in the following publications in peer- reviewed journals, conference proceedings and at international conferences:

 James DR, Orihuela-Espina F, Leff DR, Sodergren MH, Athanasiou T, Darzi AW, Yang GZ. The ergonomics of natural orifice translumenal endoscopic surgery (NOTES) navigation in terms of performance, stress, and cognitive behaviour. Surgery. 2011; 149(4): 525-33

 Mylonas GP, Kwok KW, James DR, Leff DR, Orihuela-Espina F, Darzi AW, Yang GZ. Gaze-Contingent Motor Channelling, Haptic Constraints and associated Cognitive Demand for Robotic MIS. Medical Image Analysis. 2010 doi: 10.1016/j.media.2010.07.007 [Epub ahead of print]

 James DR, Orihuela-Espina F, Leff DR, Mylonas GP, Kwok K-W, Darzi AW, Yang G-Z. Cognitive burden estimation for visuomotor learning with fNIRS. Lecture Notes in Computer Science. 2011; 13(Pt3): 319-26.

 Orihuela-Espina F, Leff DR, James DR, Darzi AW, Yang GZ. Quality Control and Assurance in Functional Near Infrared Spectroscopy (fNIRS) Experimentation. Physics in Medicine and Biology. 2010; 55(13): 3701-24

 Leff DR, James DR, Orihuela-Espina F, Yang GZ, Darzi AW. The frontal cortex is activated during learning of endoscopic procedures (Ohuchida et al., Surgical Endoscopy, January, 2009). Surgical Endoscopy. 2010; 24(4): 968-9

 James DRC, Leff DR, Orihuela-Espina F, Kwok K-W, Sun LW, Athanasiou T, Darzi AW, Yang G-Z. Neuroergonomic assessment of collaborative gaze control for robotic surgery: a functional near infrared spectroscopy (fNIRS) study. Oral communication at Hamlyn Symposium for Robotic Surgery. Royal Geographical Society, London, 2011

 James DRC, Leff DR, Sun LW, Orihuela-Espina F, Kwok KW, Mylonas GP, Athanasiou T, Darzi AW, Yang GZ. Neuroergonomic assessment of collaborative gaze control for robotic surgery: a functional Near Infrared Spectroscopy (fNIRS) study. Poster presentation at Organisation for Human Brain Mapping, Quebec. 2011

27  James DRC, Patten DK, Leff DR, Orihuela-Espina F, Athanasiou T, Darzi AW, Yang GZ. The role of the prefrontal cortex (PFC) in naïve complex motor skills learning: a functional Near Infrared Spectroscopy (fNIRS) study. Poster presentation at Organisation for Human Brain Mapping, Quebec. 2011

 James DRC, Leff DR, Orihuela-Espina F, Kwok KW, Mylonas GP, Gohil S, Athanasiou T, Darzi AW, Yang GZ. Influence of heart rate and stress on cortical haemodynamics associated with motor learning: a longitudinal functional Near Infrared Spectroscopy (fNIRS) study. Poster presentation at Organisation for Human Brain Mapping, Quebec. 2011

 James DRC, Orihuela-Espina F, Leff DR, Mylonas G, Kwok KW, Darzi AW, Yang GZ. Neuroergonomic assessment of the robotic enhancement of surgery. Oral presentation at the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES). Maryland, USA, 2010

 James DRC, Leff DR, Orihuela-Espina F, Kwok KW, Mylonas GP, Darzi AW, Yang GZ. Frontoparietal activation during learning of a visuomotor tracking task: a longitudinal fNIRS study. Poster presentation at Organisation for Human Brain Mapping, Barcelona, 2011

 Orihuela-Espina F, Leff DR, Paggetti G, James DRC, Darzi AW, Yang GZ. Influence of temporal window selection on activation detection variation for fNIRS. Poster presentation at Organisation for Human Brain Mapping, Barcelona. 2010

 James DRC, Orihuela-Espina F, Sodergren M, Darzi AW, Yang GZ. Prefrontal cortical blood flow during a complex navigational task: a fNIRS study. Poster presentation at Organisation for Human Brain Mapping, San Francisco. NeuroImage. 2009; 47(Suppl 1): S172

 Orihuela-Espina F, Leff DR, James DRC, Darzi AW, Yang GZ. ICNA: a software tool for manifold embedded based analysis of functional near infrared spectroscopy data. Poster presentation at Organisation for Human Brain Mapping, San Francisco. NeuroImage. 2009; 47(Suppl 1): S59

28  Caproni M, Orihuela-Espina F, James DRC, Menciassi A, Dario P, Darzi AW, Yang G-Z. An Analysis Framework for Near Infrared Spectroscopy Based Brain Computer Interface and Prospective Application to Robotic Surgery. IEEE/RSJ International Conference on Intelligent Robots and Systems. USA, 2009

 James DRC, Orihuela-Espina F, Sodergren MH, Clark J, Yang GZ, Darzi AW. Performance, stress and cortical activity during navigation in a natural orifice translumenal endoscopic surgery (NOTES) environment. Poster presentation at Association of Surgeons of Great Britain and Ireland, Glasgow. British Journal of Surgery. 2009; 96(S4): 0729

 James DRC, Orihuela-Espina F, Leff DR, Mylonas G, Kwok KW, Darzi AW, Yang GZ. Neuroergonomic assessment of the robotic enhancement of surgery. Poster presentation at Imperial College Surgical Symposium. Imperial College, 2009.

 James DRC, Orihuela-Espina F, Darzi AW, Yang GZ. Prefrontal cortical activity in a complex surgical task evokes a differing response according to expertise: a fNIRS study. Poster presentation at Hounsfield Meeting. Imperial College, 2009.

29

Chapter 2

Surgery and Ergonomics

2.1 Introduction

The advent of MIS heralded marked patient benefits consequent to reducing the invasiveness of surgical procedures. Whilst laparoscopy had been utilised throughout the 20th century [1], it was only after the first laparoscopic cholecystectomy in 1987 that the uptake and dissemination of MIS truly occurred. The ensuing years have witnessed a proliferation of instrument design and development in order to meet the challenges yielded by undertaking complex procedures minimally invasively. It was soon realised that MIS could have deleterious effects on the operating surgeon [2] and as such instrument design has occurred in concert with research into how surgeons interact with and use laparoscopic equipment. Accordingly, the ergonomics of MIS became an important area of surgical research utilising numerous tasks and models in order to appreciate the effect that this paradigm may have on the surgeon [10].

However, it is plausible that as MIS evolves to incorporate robotic technologies enabling progressively more complex procedures to be tackled, further challenges will emerge that may not be readily appreciable with standard ergonomic metrics. Principally, as tasks rely on greater cognitive resources, it is intuitive that user brain behaviour is a natural place to investigate in order to assess the impact of novel performance enhancing tools. Consequently, existing ergonomic techniques are discussed followed by a review of neuroergonomics.

Ergonomics is concerned with how people interact with their work environment and tools. The term is derived from the Greek words ergon – work and nomos – natural laws [11]. It involves studying anthropometry (the human body size), biomechanics (how the body responds to internal and external forces), physiology, the human response to work and adapting the work environment and equipment to human use [11]. One of the first people to study workers and how they interact with their environment was Bernardino Ramizzini (1633-1714) who investigated the role of work in causing diseases [12]. He

30 characterised particular conditions that were attributable to undertaking certain jobs; for example that tailors were prone to sciatica due to the way they sat when working.

Ergonomic research has been applied to many areas such as the military, aviation industry and to office work. In relation to surgery, one of the first ergonomic studies was by Kant et al., who assessed the posture of operating theatre staff. This study highlighted that the posture adopted by both general surgeons and theatre nurses was potentially hazardous [8]. However, this study assessed open surgery and not MIS which is associated with longer operating times and its own ergonomic challenges as outlined in the introduction. Since the advent of MIS, ergonomic assessment of tasks has become progressively more common and sophisticated [10]. This has lead to increasing research utilising many technologies ranging from questionnaire evaluation of an instrument [13] to multimodal assessments utilising EMG, skin conductance, postural information and performance data [14].

The purpose of this chapter is to summarise existing tasks and models utilised in surgical ergonomics and to outline limitations of their application. Subsequently, surgeon stress response is reviewed and how these parameters may be determined and used to indicate how difficult a task or procedure may be. The concept of neuroergonomics is then introduced and discussed with regard to the potential that this paradigm holds for gaining an understanding of the cortical correlates of performing surgery and how this information may be utilised to aid the design and development of assistive technology.

2.2 Assessment of Surgical Ergonomics

Numerous methods have been employed in surgical ergonomic studies including EMG, motion tracking, and force plate systems to assess posture [10]. Methods of ergonomic assessment can be divided into two main categories:

 Performance and outcome related measures  Physiological related measures

The former pertains to the end product of task performance and how this may be modulated under different conditions. Surgeon related measures concern how the task impacts directly on the user and may be assessed with a variety of techniques (discussed in 2.2.2).

31 Paradigm selection in ergonomic studies is motivated by the endpoints of interest. For example, a comparison of laparoscopic equipment requires an appropriate MIS task [15]. However the exact nature of the task is influenced by the measurement equipment. This may inhibit its use in a true operative environment thus necessitating laboratory studies using isolated components of surgical tasks. Lee et al. [10], divide this aspect of ergonomic studies into tasks and models. The task can be static, e.g., opening and closing an instrument, it can be navigational such as a 2 point touching task or it can be a manipulation task, e.g., suturing. The model is the environment in which the task is tested and this can be synthetic, e.g., box trainers for laparoscopic surgery, virtual reality (VR), animal models, or intra operative models.

The choice of task and model will therefore be determined by the outcome of interest and also by the feasibility of performing the task with all the necessary measurement equipment. This may preclude the use of many of these technologies in the operating theatre due to space, the issue of sterility of the equipment and that the experiment cannot hinder the progress of the procedure.

2.2.1 Performance and Outcome Related Measures

Performance and outcome measures quantify how successfully a task was completed by a variety of metrics including subjective and objective scores. Task outcome is frequently used as an endpoint in ergonomic studies [16, 17] as it is important to appreciate whether a particular intervention is successful. However, merely scrutinising the endpoint may not delineate how performance enhancement is meted out and therefore further detailed assessment of the task may be necessitated to further appreciate how a factor may contribute to successful execution. For example, Manasnayakorn et al. demonstrated that task execution time was significantly longer when the table height was 15cm above the elbow in hand-assisted laparoscopic surgical (HALS) task. EMG analysis demonstrated a higher workload for arm extensor, trapezius and paraspinal muscles compared to lower table heights [16]. Therefore, the prolonged time to complete the task may in part be modulated by the necessary increase in muscle workload and possible fatigue at the increased table height.

There are many different methods to assess task performance in surgery and these are beyond the scope of this work. However, selected methods are discussed below. These can be divided into observational and non-observational techniques.

32 2.2.1.1 Observational Techniques

Task execution time is commonly used as a measure of performance; however there are some drawbacks with using it in isolation. It does not necessarily take into account the accuracy of the task and therefore a poor performance that was executed quickly would score more favourably than a slow, accurate performance.

Error identification within surgery has been given much attention due to the potential patient consequences. The number of errors committed can be used to assess performance and scoring methods for this have been devised such as the Observational Clinical Human Reliability Assessment (OCHRA) [18]. This technique has been utilised in the ergonomic assessment of table height in HALS [16].

The objective structured assessment of technical skill (OSATS) involves direct observation and assessment on 3 checklists: a task-specific checklist, a 7 item global rating scale and a pass / fail judgement [19]. This has been shown to be reliable and valid as an assessment tool however it is comparatively labour intensive for the examiners. In their original study, Martin et al., required 48 examiners to examine 20 surgical trainees [19].

Further examples of end-product analysis include anastomotic leak pressure [20], assessment of leak pressure of enterotomy closure [16] and evaluation of tensile strength of surgical knots [21].

2.2.1.2 Non-Observational Techniques

Non-observational techniques encompass technologies such as VR systems used in laparoscopic surgical training and assessment and dexterity analysis techniques such as Imperial College Surgical Assessment Device (ICSAD) [22]. ICSAD is discussed in section 2.2.2.3.

VR systems such as the MIST-VR® (Mentice, Gothenburg, Sweden), incorporate abstract tasks such as object grasping and manipulation. These can be used in training and objective measures such as error rate and economy of movement can be recorded as endpoints [23]. More sophisticated systems such as the LapMentor™ (Simbionix, Cleveland, Ohio, USA) allow complete operations such as laparoscopic cholecystectomy to be undertaken. These also provide haptic feedback and allow performance measures of instrument pathlength and number of movements to be recorded. VR systems offer many

33 advantages; however, it may not be possible to incorporate them into all ergonomic studies. For example, knot tying ability can be assessed with breakage forces, or anastomosis quality with leak pressures. However, these physical measures cannot be applied to virtual reality tasks.

Endpoint analysis is clearly important in ergonomic evaluation of novel instrumentation and technology as it is crucial to appreciate whether performance is ultimately improved. However, outcome alone does not necessarily allow a full appreciation of the means by which this improvement is achieved. This is an important factor in order to ensure that performance enhancement in one area of performance is not detriment to other facets of task execution or to the user themselves.

2.2.2 Physiological Related Measures

2.2.2.1 Electromyography

EMG has been utilised in the field of ergonomics for many years and is also well established in surgical ergonomics [10]. It is used to assess muscular activity and fatigue [24] and works by detecting the underlying action potential (AP) associated with muscle contraction. Muscle contraction is triggered by the firing of motoneurones which produces a depolarising AP which is propagated along the nerve to the neuromuscular junction (NMJ). The NMJ is where the motoneurone has a synaptic connection with the sarcolemma of the target muscle. The motoneurone AP reaches the pre-synaptic membrane of the motor neurone, causing release of the neurotransmitter, acetylcholine (ACh) across the synaptic cleft. This binds to ACh receptors on the post-synaptic membrane causing the AP to be propagated throughout the sarcolemma of the muscle fibre resulting in muscle contraction [25]. These muscle fibre APs are detected by EMG [26].

EMG recordings can be made either with surface EMG (SEMG) or with fine wire EMG. The latter requires electrodes to be inserted percutaneously into the muscle. This results in a signal less affected by noise, however is more painful than SEMG and does not allow measurement of the muscle as a whole. Consequently, in surgery SEMG is widely used [10]. Several analysis strategies exist for EMG data based on amplitude or frequency analysis [26]. It can be difficult comparing data between subjects, therefore results may be expressed as percentage of maximal voluntary contraction (MVC). In order to calculate this, the MVC for a particular muscle is ascertained before the experimental task, and subsequent task levels are normalised to this [10]. Various aspects of MIS have

34 been studied using EMG, including optimal table height for performing the operation [14, 16], instrument grip configuration [15] and type of instrument used [17]. EMG is a valuable tool in assessing ergonomic aspects of surgery; however there are confounding factors such as interference from skin and superficial tissues and difficulty ensuring that exactly the appropriate muscle is being assessed. The signal is also affected by muscle fatigue and by the proportion of fast and slow twitch fibres within the muscle. Therefore variability may exist between subjects [26]. It is also important to note that use of EMG may be limited in certain operative circumstances such as in the assessment of hand muscle usage, secondary to interference with the surgical instruments in question.

2.2.2.2 Posture

One of the first surgical ergonomic studies involved assessing posture of surgeons [8]. Kant et al., assessed surgeons, surgical assistants, theatre nurses and anaesthetic staff during general surgical and ear nose and throat (ENT) surgical procedures. They found that in particular, general surgeons demonstrated static postures that were classed as potentially harmful [8]. Various aspects of MIS such as longer operating times, may lead to an exacerbation of postural problems and consequently research has focused on this aspect of ergonomics.

Postural analysis can be assessed with force plate systems. These consist of a rigid platform with sensors in each corner that emit an electrical output in response to the force exerted through it. From this, the ground reaction force can be calculated. This is the force that is equal to, but in the opposite direction to the force exerted by the subject. From this information, the range of movement and centre of pressure can be computed [10]. Postural information can also be obtained from video recordings of surgical procedures.

This technology has been used to assess variation in posture between open surgery and MIS. It has been found that MIS is associated with a more static posture and this may lead to greater muscular fatigue [27]. Postural analysis assessed by video recording and self-reporting questionnaires has also been used to assess surgical teams performing MIS in a dedicated MIS suite compared to a standard operating theatre. It was found that neck extension of the surgeon was less in the MIS suite and surgeons reported improved ergonomic posture, but no difference in levels of discomfort [28]. A more sophisticated analysis of force plate data by Lee and Park derived postural stability demand (PSD) from the centre of movement and centre of pressure. An increased PSD in the anterior- posterior plane was correlated with increased performance time [29]. Observational tools

35 have been designed to assess both posture and exertion of the upper limbs: the rapid upper limb assessment (RULA) [30] and the Job Strain Index (JSI) [31]. These have been used to assess workload in robotic and endoscopic surgery with the finding that whilst task completion time was longer with the robotic system, it provided a more comfortable environment for the surgeons [32].

2.2.2.3 Motion Analysis

Several motion analysis technologies exist, including electromagnetic, optical and those based on video or observational analysis. Electromagnetic tracking systems include ICSAD [22]. ICSAD comprises a commercially available electromagnetic field generator and tracking sensors attached to the subject’s hands (Isotrak II, Polhemus, USA). From this 3D positional data, information such as the distance the hands move (pathlength), the number of movements taken to perform the task and the time taken to complete the task can be obtained. ICSAD has been used in various surgical paradigms and increased economy of hand movement has been linked to improved outcome [20] and also has been correlated to OSATS [33].

Figure 2.1 Example of optical tracking system (Optotrak Certus®, Northern Digital Inc., Canada) in use during simulated flexible endoscopic surgical task. The subject (left) has optical trackers (see subplot) on the right side of head, right shoulder, forearm and wrist. The video motion tracking camera (labelled on right side of figure) detects infrared light emitted from the trackers and accordingly determines motion.

36 However there are some limitations to this technology in that the electromagnetic field is subject to interference from ferromagnetic substances therefore care must be taken designing the task. Also, an expert surgeon may utilise more hand movements to perform a particular manoeuvre compared to a novice, however this may not represent that the novice is demonstrating greater economy of movement [10].

Optical tracking equipment has been used in ergonomic assessment of MIS as represented in Figure 2.1 [10]. The technology incorporates infrared emitters that are positioned on the subject and are detected via an infrared camera. This allows calculation of joint positional data and also the kinematic analysis of movement associated with a particular task. The camera needs to have a clear line of vision to the emitters on the subject otherwise blocking may occur leading to lost data [10]. Emam et al. utilised optical tracking data along with performance and EMG data in a laparoscopic suturing task. It was found that suturing in a vertical plain was easier and more accurately performed than in a horizontal plain. This was associated with a wider angle of excursion at the elbow and a lower angle of velocity and this was associated with smoother movements [34]. However, it has been found that greater postural instability and greater shoulder abduction have been utilised by expert surgeons to compensate for lack of movement at the wrist due to carpal tunnel syndrome [35]. This is relevant as postural and motion analysis studies may elucidate patterns of movement that have been employed by the subject as a compensatory mechanism to aid task performance.

2.2.2.4 Self Reporting Questionnaire

Self reporting questionnaires have also been utilised allowing description of discomfort whilst undertaking surgical tasks. This can be used to assess use of a particular instrument and to enquire as to any problems such as discomfort on use that may be encountered [13]. Specific questionnaires are also utilised that enable detailed description of location of pain or discomfort upon undertaking a particular task such as the localised experienced discomfort (LED) scale as depicted in Figure 2.2 [36]. The LED scale has been utilised in conjunction with heart rate (HR), subjective mental workload and performance metrics in the comparison of robotic versus MIS task performance [37]. This yielded less physical discomfort with the robotic task [37]. Subjective rating of user discomfort has also been utilised allowing users to rate pain whilst using laparoscopic instruments with varied handles and in varying positions [38].

37

Figure 2.2 Localised experienced discomfort (LED) scale [36]. Subjects utilise diagram to indicate sites of discomfort at designated intervals whilst undertaking a task (adapted from [36]).

2.3 Stress and Performance

Stress is the reaction to stimuli that pose a threat to an organism’s homeostasis, and this threat may be real or perceived [39]. Stress may be physical or psychological and culminates in the activation of pathways that lead ultimately to the fight or flight response. When under stress, a neuroendocrine response is triggered that activates both the hypothalamic-pituitary-adrenal (HPA) axis and the sympathetic nervous system as illustrated in Figure 2.3. Activation of these interconnected systems leads to modulation of physiological parameters (e.g. heart rate (HR), heart rate variability (HRV), blood pressure (BP), skin conductance and pupillary size) that can be quantified under experimental conditions.

It has been demonstrated that performing surgery may be stressful [4, 5, 40, 41] defining it as a relevant parameter to quantify; and that stress levels may vary as the mode of task performance is modulated [4, 5, 42]. Specifically, it has been found that MIS induces a comparatively greater stress response than open surgery [4, 5]. In part, this is likely to be due to greater technical challenges and increased mental workload associated with MIS [4].

38

Figure 2.3 Diagrammatic representation of the stress system. Stress evokes the HPA axis (left side of image) by inducing corticotropin-releasing hormone (CRH) to be secreted from the paraventricular nucleus of the hypothalamus. CRH acts on the anterior pituitary gland leading to adrenocorticotropic hormone (ACTH) release. ACTH subsequently induces cortisol release from the zona fasciculata of the adrenal cortex. The right side of the system depicts the locus coeruleus/sympathetic nervous system component of the stress system. This leads to activation of the sympathetic nervous system and noradrenaline release. Multiple feedback inhibitory and stimulatory pathways exist in this system.

Measuring indices of subject stress under various conditions allow it to be utilised as an ergonomic tool. For example it has been used to compare robotic and laparoscopic tasks [32, 42]. It is likely that improved instrumentation makes tasks easier to perform and should reduce stress levels. However, it may be difficult to extract whether this is due to reduced mental stress or due to making the task physically less demanding. There are several techniques available to record subject stress levels and these have been utilised in assessing stress in surgeons as discussed in the following sections.

39

Figure 2.4 Heart rate variability as a measure of subject stress. The electrocardiogram (ECG) (upper panel, blue) for subject when relaxed (left) and stressed (right). The corresponding tachogram (central panel, purple) comprises the beat to beat change in the heart rate expressed as the R to R interval and demonstrates less variability in the stressed condition. The lower panel schematically represents stress-related changes within the frequency domain. Stress and the corresponding sympathetic nervous system response causes a peak in the low frequency (LF) component and a relative decrease in the high frequency (HF) component.

2.3.2 Heart Rate and Respiratory Rate

HR is controlled by the autonomic nervous system and varies according to modulation of the parasympathetic and sympathetic components of this [43]. It has been demonstrated that procedures perceived to be more stressful resulted in a greater surgeon mean HR intra-operatively [44]. However it is possible that mean HR may increase independently of stressors and as such should be interpreted in conjunction with supplementary measures of stress [44] or reviewed in terms of HRV [41, 43]. Under normal circumstances, the HR varies according to factors such as respiratory sinus arrhythmia, vagal and sympathetic tone [43]. HRV is the fluctuation in time between successive heart beats and reflects the degree of sympathetic and parasympathetic activity [43]. Under stressful conditions this variability in successive heart beats decreases and therefore HRV decreases.

40 As represented in Figure 2.4, HRV can be measured by modulation in the spectral components of the HR. In increasing the sympathetic tone (i.e. under mental stress), there is a relative increase in the low frequency component in relation to the high frequency component [5, 41, 43]. Conversely under rest, the high frequency component is relatively greater in power [5, 41, 43]. Other means of calculating the HRV exist such as time domain methods which may be more accurate in the presence of a shorter HR recording

[43]. The standard deviation of the R to R interval (SDRR) is one such measure and this decreases in line with the reduced HRV and therefore stress [45, 46]. Due to the fact that HR is easily measured non-invasively, it has been utilised in numerous studies of stress in surgery [5, 41, 44, 46, 47]. Stress causes an increase in respiratory rate [48] and minute volume of respiration [49]. However, this parameter has not been used to measure the stress associated with undertaking surgical tasks.

2.3.3 Neuroendocrine Response

As previously discussed and displayed in Figure 2.3, stress causes the release of corticotropin releasing hormone (CRH) from the hypothalamus and other cortical regions including the basal forebrain and the limbic system [48]. CRH also activates the sympathetic nervous system [48]. With regard to the HPA axis, CRH causes release of adrenocorticotropic hormone (ACTH) from the anterior pituitary gland, which in turn instigates cortisol release from zona fasciculata of the adrenal cortex [39]. Consequently, cortisol increases with stress and can be detected both in venous blood and in salivary samples [50]. Cortisol has been utilised to assess intra-operative stress and has been shown to increase in line with the other stress measurements such as questionnaire and mean HR [44] and as it can be collected non-invasively, obviates the possible confounding effects of obtaining a blood sample.

2.3.4 Galvanic Skin Conductance

Galvanic skin conductance (GSC) is a measure of sympathetic nervous system activity. Three electrodes (current, reference and measurement) are attached to the subjects palm. As previously discussed and displayed in Figure 2.3, the stress response activates both the HPA axis and the sympathetic nervous system. Activation of the sympathetic nervous system activates sympathetic nerves in the skin resulting in acetylcholine dependent activation of sweat glands to produce sweat [51]. This in turn reduces the skin resistance and therefore skin conductance increases [51, 52]. Sweat glands are only innervated by sympathetic neurones therefore GSC is specific for this system and not the parasympathetic nervous system [52].

41

GSC has been used to monitor pain [51] and cognitive effort, however has also been used in surgical ergonomic studies. It has been demonstrated that a MIS task induces more mental stress than an open surgical task [4] and that robotic surgery may induce less mental stress than MIS [42]. However, in the latter study significance was not reached. It is also noteworthy that due to the positioning of electrodes on the palm, this technique may interfere with task execution, particularly within the sphere of surgery.

2.3.5 Electrooculogram

Electrooculogram (EOG) is an electrophysiological tool capable of measuring subject blink rate, eye closing time and duration of blink [53]. In a flight simulator task, blink interval increased when subjects began to fly however would decrease in line with increasing task difficulty [53]. It is thought that increasing the visual load of the task may increase blink rate confounding a decrease seen with increasing task workload. Within a surgical ergonomic study, it was found that compared to an open surgical task, MIS induced a greater blink rate associated with an increase in GSC and self reported stress indicating a greater overall stress level [4]. Therefore blink data may be difficult to interpret and correlation with other stress parameters is recommended [4].

2.3.6 Self Reporting Measures of Stress

Self reporting measures of stress in surgery have been utilised to determine surgeon stress. These may take the form of structured interview [54, 55]. However, a drawback of this is that it may be difficult to draw comparisons between studies [40]. Validated scales of stress assessment have also been utilised to quantify stress induced by undertaking surgery [44].

The short form of the state trait anxiety inventory (STAI) as displayed in Table 2.1 comprises 6 questions which subjects use to rate how best they feel each one describes their current state. This has been validated both against the full-form STAI and to detect fluctuations in anxiety [56]. A pre-task questionnaire is completed prior to undertaking the relevant study and subsequently during task and post-task questionnaires are completed at the end. This allows fluctuation in anxiety to be assessed as induced by a task. The short form of the STAI been utilised to quantify intra-operative stress levels of surgeons in conjunction with mean HR and salivary cortisol. Increased self-reported stress was associated with a greater rise in cortisol and a higher mean HR [44].

42

Not at all Somewhat Moderately so Very much so I feel calm 1 2 3 4 I feel tense 1 2 3 4 I feel upset 1 2 3 4 I am relaxed 1 2 3 4 I am content 1 2 3 4 I am worried 1 2 3 4

Table 2.1 Short form of state trait anxiety inventory (STAI) [56]. Subjects are asked to rate how they feel based on the questions above. The result is determined by adding up the scores from the three positive statements (e.g. ‘I feel calm’) with the results of negative statements (e.g. ‘I feel tense’) each subtracted from five. Therefore the maximum score indicating no stress equals 24 and the lowest score indicating maximal stress equals 6.

In summary, measurement of these parameters may help determine whether a task is more challenging if this is significant enough to induce a stress response in the user. A potential advantage over the preceding ergonomic metrics is that it is likely to be able to detect a task that is cognitively demanding whereas physical measures (e.g. EMG and posture) may not. However, a potential limitation is that stress measurements are unlikely to determine the exact nature of the task an its demands that are making it difficult. For example, MIS has been shown to be more stressful than open surgery. It is not possible to determine which aspects of MIS lead to this. Therefore, a natural progression in ergonomics is to progress from investigating physical measures, via stress measurement to neurocognitive assessment. The latter may enable a detailed appreciation of the exact nature of task demands.

2.4 Neuroergonomics

2.4.1 Introduction

Neuroergonomics is the study of brain behaviour of subjects at work and is a convergence of neuroscience and ergonomics [9]. This allows the assessment of the neural basis behind physical and cognitive tasks undertaken at work. This is all the more relevant in environments where users interact with complex technology and equipment such as in the aviation industry. In settings such as these, appreciating the impact that task execution has on the user may afford insight into how the environment may be altered in order to make it easier for the subject; or to act as an early warning system in case of fatigue. This is clearly applicable to MIS and robotic surgical settings where surgeons are undertaking progressively more complex procedures with intricate technologies.

43 Research in the field of neuroergonomics has been underpinned by advances in functional neuroimaging that allow the cortical response induced by tasks to be assessed with a variety of modalities including electroencephalography (EEG), functional magnetic resonance imaging (fMRI), transcranial Doppler and eye behaviour [57]. However, there are challenges associated with utilising neuroimaging to assess ergonomic aspects of task performance. Existing neuroimaging research is primarily undertaken in controlled environments and task selection is tempered in order to answer the exact research point in question. The brain behaviour associated with a complex task in a work environment is likely to be influenced by many factors that may be difficult or impossible to control for and as such caution needs to be exercised in the design and interpretation of neuroergonomic experiments [58]. A further factor that may influence experimental outcome is artefact. This is relevant in neuroimaging studies however, can often be controlled for in the laboratory setting by ensuring the subject remains as still as possible. However, if work tasks are undertaken, this may become more of a problem as the aim is still to preserve the natural work environment as much as possible. Tools such as virtual reality may help overcome this problem. The purpose of this section is to outline applications of neuroergonomics. Section 2.5 focuses on the use of near infrared Spectroscopy (NIRS) in neuroergonomic studies.

2.4.2 Applications

There are many applications of neuroergonomics. Broadly, these can be divided into generalised applications such as fatigue detection and mental workload measurement and more specific applications such as driving, aviation and medical safety. These are considered in the following sections.

2.4.2.1 Mental Workload

Within the paradigm of neuroergonomics, mental workload determination is important as it can serve as a baseline to understand the demands that a task places on the user. This information can be utilised to either adjust the working environment accordingly (i.e. adaptive automation [59]) or to identify situations in which a subject is unable to function effectively. Mental workload can be defined in terms of the demands that a given task imposes balanced against the resources of the individual undertaking it [57, 60]. The main methods of determining mental workload include:  Behavioural measures  Subjective measures  Physiological measures [61]

44 Behavioural measures include task performance. However a drawback of these measures is that two people may perform equally according to the performance metric, yet, for one person, the task may be a lot more challenging and as such require a lot more mental effort. This may not be detected if performance alone is scrutinised. Subjective measures include questionnaires such as the NASA task load index, which has been used extensively to determine mental workload [62]. A potential disadvantage of these methods is that they do not give an indication of the underlying brain behaviour underpinning task performance [61]. However, this method of assessing workload may be easily and cheaply administered. Finally, physiological measures may be used. These include pupillometric studies [63], HR, BP [53] and brain behaviour. However, systemic physiological measures may not so directly reflect mental workload [53].

Much research, particularly utilising EEG has been devoted to measuring cognitive workload [64]. Despite lack of spatial resolution, the flexibility and excellent temporal resolution of EEG has allowed it to be used to assess cortical activity in a wide variety of tasks designed to investigate subject cognitive workload. Interest in this area has increased in line with the expanded role of computers and technology in the workplace and the concurrent increased demands placed on the user. A potential confounding factor with EEG in the measurement of mental workload is the presence of artefacts. Therefore, recordings may need to be repeated and considered in tandem with measurement of ocular and facial muscle activity and artefact removal is often considered in post- processing [57].

It has been demonstrated that increasing cognitive workload in working memory tasks leads to an increase in the theta band spectral peak measured at the Fz site and an attenuation of the alpha band peak [65]. Supplementary to this, EEG signals can discriminate differing workload levels within the n-back task [57] and have also been utilised to assess mental workload in more realistic task settings such as with the Multi- Attribute Task Battery (MATB) [66]. The MATB is a computer-based task that is similar to tasks that pilots undertake and entails a multitasking environment that can be adapted in order to manipulate the task load. Therefore this affords a more realistic environment for assessing subject mental workload. As with the n-back task, increasing the task load can be detected with EEG as an increase in the frontal theta band and a decrease in the alpha band [57]. EEG is also used to detect event-related potentials (ERP) which are the neural response to specific events [57]. In particular the P300 component has been linked to the degree of attentional resources required by a task [57].

45 Motivation for the evaluation mental workload has in part been driven by the concept of adaptive automation [59], whereby user cognitive behaviour is detected in situ and utilised to modulate the input from the computer system. This not only applies to mental workload but user fatigue (discussed below). NIRS has been used to assess mental workload [67, 68], however this will be discussed in section 2.5 in relation to NIRS as a neuroergonomic tool. The importance of addressing mental workload is that other aspects used to assess a task such as performance, may not discriminate variability between individual subjects. Despite equivalent performance outcomes, two people may be operating at different levels of capacity; i.e. an experienced user may achieve a similar outcome to an inexperienced one but are using minimal resources to do so and therefore is able to further improve performance or to maintain performance in the presence of distracting factors. For this reason, addressing the brain behaviour underpinning task execution is relevant and important.

2.4.2.2 Vigilance and Fatigue

Any safety critical industry such as surgery, aviation or complex machine operation is likely to rely on operator vigilance and be sensitive to user fatigue. Vigilance necessitates prolonged observation for events that occur infrequently or inconsistently. Sustained vigilance has been linked to blood flow to the right side of the brain [69] and specifically involves activity within right parietal, frontal and thalamic regions [70]. It has been shown that right sided cerebral blood flow decreases with time throughout vigilance tasks [57]; however, if subjects are prompted throughout the task, blood flow and performance remains similar to baseline levels [69]. Both vigilance detection and enhancement demonstrate clear potential within neuroergonomics. This may take the form of assessing surgeons for their ability to remain vigilant, to identify particular tasks that necessitate greater vigilance, to detect when vigilance is impaired (e.g. associated with fatigue) or in the design and development of systems to prompt users in order to aid maintenance of vigilant performance.

Sleep deprivation and resultant fatigue can affect performance in numerous ways including: reduced working memory, impaired reaction times, impaired vigilance, reduced learning and increased errors when working under time constraints [57]. This situation occurs in shift workers due to disruption of the circadian cycle leading to less sleep [57]. Shift work is commonplace throughout many professions and can render workers sleep deprived and more prone to work-place errors and impair safety outside work such as when driving at the beginning and end of shifts. Consequently much research has focussed on the effects of shift work and sleep deprivation. Medicine is a

46 field where shift work is almost universal and within this, surgery reflects a specialty where technical and cognitive skills may be relied upon whilst sleep deprived. Fatigue effects have been investigated in surgeons [71-73]. Well learned motor skills may be retained with fatigue, however greater PFC activity is required for cognitive tasks [71]. Additionally, MIS tasks may demonstrate deterioration especially after the first of sequential night shifts [72].

Neuroergonomics may be applied to work-related sleep deprivation to help predict worker fatigue. This may be achieved by studying particular shift patterns that predispose to excess fatigue. A further application would be in the online detection of excess fatigue whilst subjects are at work allowing interventions such as alarms to prevent adverse events [57].

2.4.2.3 Driving

Motor vehicle driving necessitates many complex skills including motor control, visuomotor integration, vigilance and error monitoring [74]. The cortical correlates of driving have been investigated with fMRI [74, 75] and involve activity within occipital and parietal cortices in relation to visual attention and association, the cerebellum and motor cortex in relation to motor control and the anterior cingulate cortex for error monitoring [74]. However, utilising fMRI for such investigations imposes restrictions on task design such that simulators have to be utilised which may limit conclusions that can be drawn from such studies. For example, a games controller [74] and joystick [75] were utilised to control the simulated vehicle. Thus a subject intending to move left or accelerate would have to translate turning the wheel or putting their foot on the accelerator into the corresponding movements on the controller. It is possible that this may influence the subject brain behaviour. Use of fNIRS confers the benefit of being able to assess the task in a more realistic environment such as a driving simulator [76] and as such a more accurate representation of the cortical activity associated with driving may be attainable.

There are several aspects of driving that are likely to benefit from neuroergonomic evaluation. Firstly, in the assessment of novel technology aimed at facilitating driving; e.g. adaptive cruise control, which reduces PFC activity when driving in a simulator, thought to reflect a reduction in mental workload [76] (further discussed in section 2.5). Secondly, in the assessment of driving capability which is relevant in the presence of ocular disease or cortical impairment such as focal lesions after cerebrovascular accident

47 or dementia [77]. Thirdly, in the development of systems to monitor driving ability and vigilance in situ enabling this information to feed into early warning systems.

2.4.2.4 Aviation

As with driving, similar applications of neuroergonomics to aviation exist. Attention has been directed toward the detection and response to hypovigilance (as discussed in section 2.4.2.2), particularly within the area of console design [69]. Specifically, detection of user neuropsychological indices may enable the detection of increasing mental workload and indicate either when the subject is overloaded by the task or when increasing assistance from the machine is required. This encompasses adaptive automation [59] which monitors user behaviour and accordingly increases or decreases the role of the machine. This approach has been applied in the setting of air traffic control console design [69] and demonstrated that cortical blood flow as assessed with transcranial Doppler reduces with the time spent on the task. However, when the adaptive automation is applied to accurately cue the subject, performance is retained alongside cortical blood flow. As the reliability of the system decreases, performance declines alongside cerebral haemodynamics [69]. This is relevant as adaptive automotive systems may not be 100% reliable and it is noteworthy that in using them, subject brain behaviour is modulated.

In addition to driving, appreciating the underlying cortical substrate of aviation tasks is imperative. This knowledge can subsequently be utilised to guide the role of the computer in undertaking the task (adaptive automation), or aid in the evaluation of technology that may improve performance, safety and possibly enhance task learning.

2.4.2.5 Medical Safety

Medical errors and the ensuing harm caused to patients are a significant cause of morbidity and mortality within healthcare. Broadly, a medical encounter involves perception of the situation by the doctor and subsequently determining the diagnosis. Following this, a plan is formulated with reliance on prior experience and memory. The final stage involves execution of the management plan [78]. Errors can occur at any juncture within this process and may be due to a variety of factors including human- related and related to the system as a whole.

There are many opportunities for the application of neuroergonomics to medicine in order to enhance patient safety. Analogous to aviation and driving, any area of medicine reliant on user interaction with complex equipment is likely to benefit from a detailed assessment of task demands. Anaesthesia is an example of this and utilises safety features

48 also used in aviation. Anaesthetists interact with a complex console intra-operatively which simultaneously displays much patient information that needs to be assimilated in order to allow safe monitoring of the anaesthetised patient. Mental workload as assessed with the NASA task load index has been utilised alongside performance to appraise display design of anaesthetic consoles [79]. In this instance, performance in terms of response accuracy did not differentiate three of the four consoles whereas mental workload was significantly lower in one which also was associated with the lowest reaction times [79]. This illustrates that performance indices alone may not be sensitive enough to detect which technology is optimum. It is also possible that in using functional neuroimaging in order to delineate task demands, further information beyond subjective sensation of workload may be obtained [9].

As discussed in section 2.4.2.2, fatigue is a potential cause of errors within healthcare, especially in the presence of shift work. The ability to detect fatigue in situ would clearly assist in healthcare delivery as this information could guide rota coordination or to act as an early warning system if someone is too tired to operate. This is particularly relevant to surgery where both cognitive and motor skills are depended upon in order to operate. Further examples of the potential application of neuroergonomics in medicine lie within surgery where advances in technological innovation have heralded vast improvements in patient care. However, as outlined in Chapter 1, MIS which has demonstrated marked benefits to patients may lead to surgeon morbidity [2] and there is a clear motivation to assess these complex tasks in order to ensure surgeon ability and performance is optimised.

2.5 NIRS and Neuroergonomics

Section 2.4 has reviewed the principles of neuroergonomics and potential applications of this paradigm. In terms of the method of determining brain behaviour associated with work-related tasks, EEG has been the predominant neuroimaging tool utilised within either neuroergonomic studies or those of vigilance and mental workload. As will be further discussed in section 3.3, several neuroimaging modalities exist, each with their respective strengths and weaknesses. However, certain attributes are important for acquiring cortical information regarding everyday tasks. In particular, the need for flexibility and adaptability to the task in question is paramount. For this reason, fMRI, which offers detailed spatial resolution and is capable of imaging the whole brain may not be applicable due to the preclusion of ferromagnetic substances within the scanner. Additionally, fMRI studies require the subject has to lie within the scanner with their

49 head remaining still therefore certain tasks (e.g. surgery) may not be practical. EEG offers greater flexibility and enhanced temporal resolution; however, spatial resolution is lacking. Consequently optical imaging with fNIRS is likely to be suitable for neuroergonomic studies as it affords good spatial resolution, moderate temporal resolution, relative resistance to motion artefact and studies can be undertaken relatively cheaply. Briefly, light at the near infrared end of the spectrum passes relatively unhindered through tissues such as the skin and skull until it is absorbed by oxyhaemoglobin and deoxyhaemoglobin (HbO2 and HHb). Accordingly, fNIRS detects relative changes in these haemoglobin species as a surrogate for cortical activity. Technical aspects of fNIRS are detailed in section 3.4. Therefore fNIRS can be used in a wide variety of experimental paradigms and is able to localise the region of cortical activity involved in task execution reasonably well. Advantages and disadvantages of neuroimaging modalities are discussed further in section 3.3.

Several studies have utilised fNIRS to study cortical activity associated with cognitive tasks of varying levels of difficulty and to assess brain behaviour in the work environment. These studies are summarised in Table 2.2. Tsunashima [76] initially used simultaneous fNIRS and fMRI to investigate PFC activity associated with mental arithmetic of increasing level of difficulty. This revealed increased PFC activity in line with increased task complexity and corresponded to the fMRI findings. Increasing the task difficulty also led to increased self-reported workload in terms of the NASA task load index. The assessment of mental arithmetic served as a baseline to assess the impact of driving with or without cruise control which was undertaken in a vehicle simulator whilst PFC activity was assessed with fNIRS. Greater PFC activity was demonstrated driving without cruise control. This study affords a greater fidelity of simulator and as such the results may be more reliable in comparison to simulated driving in whilst lying in an MRI scanner which necessitates the use of a joystick and buttons to steer and control speed [74, 75]. However, a disadvantage is that in only studying one cortical region, if the task substrate changes during the learning process, the relevant cortical regions may change and thus activity may be missed.

Similarly, a train simulator has been used to assess occipital and PFC behaviour whilst driving a train with and without an automated driving system [80]. This study only utilised two subjects however, visual inspection of the data demonstrated lower PFC activity when the automated system was in operation.

50 Author Cortical Task Findings [reference] Region Ayaz [81] PFC n-back task Increasing left PFC activity in one channel with increasing task difficulty Air traffic control task Increasing activity in one medial PFC channel with increasing task difficulty Landing an unmanned flight Left PFC channel activity attenuated as simulator task learnt

Izzetoglu [82] PFC Warship commander task Increasing HbO2 with increasing (game) with increasing workload until subject overloaded and complexity level decreased

Sassaroli [67] PFC Working memory task Increasing HbO2 with task difficulty. Classification algorithm used to predict workload level from haemodynamic

data with 44 – 60% (HbO2) and 55 – 72% success (HHb) Kojima [80] PFC and Train driving simulator Minimal activity when automated occipital with/without automated system used in comparison to manual cortex system control (visual inspection of data from 2 subjects only) Tsunashima PFC Mental arithmetic over 3 Increasing PFC activity task difficulty [76] levels of difficulty (corresponds to simultaneous fMRI recording and self-reported workload with NASA task load index) Driving car simulator Lower PFC activity when using cruise with/without cruise control control Harada [83] PFC Driving on roads with Greater PFC activity in inexperienced young (experienced / drivers. Less PFC activity in elderly inexperienced) and old (experienced) drivers

Table 2.2 Summary of fNIRS study investigating variation in mental workload with task demand and the brain behaviour associated with work-related tasks. HbO2 – oxyhaemoglobin, HHb – deoxyhaemoglobin.

This study does not offer analysis of the data; however, the results appear to corroborate the above-mentioned study assessing driving with / without cruise control [76]. From a neuroergonomic standpoint, these studies may imply that cruise control may help in alleviating the workload of a task. But further work is needed to delineate the exact cognitive effect when using assistive technology of this nature. It is possible that a

51 situation of reduced PFC activity may reduce driver vigilance and thus predispose to hazardous behaviour.

The effect of varying mental workload has also been assessed with fNIRS utilising the n- back task [81]. It was found that increasing task difficulty was associated with increasing activity in one near infrared (NIR) channel in the left PFC and in the same study, left PFC activity at the same location decreased as subjects learnt a simulated flying task. However, in an air-traffic control task undertaken within the same study, as the workload was manipulated by increasing the numbers of aircraft being processed, medial PFC activity was found to increase at a different location in comparison to the other tasks [81].

As functional neuroimaging is applied to progressively more complex tasks, findings may not be directly translatable to equivalent standardised cognitive tests (e.g. the n-back task). Therefore it is important that the brain behaviour associated with complex surgical scenarios is addressed in order to ensure that results can be placed in context. fNIRS affords the flexibility to enable task paradigms that are unlikely to be possible with other neuroimaging modalities; e.g. walking [84, 85], running [86], and complex surgical tasks [87, 88]. This demonstrates that whist there may be some limitations to fNIRS the flexibility it offers makes it suitable for neuroergonomics studies [57].

2.6 Conclusions

Patient care has clearly benefited from advances in surgical technology. In particular, MIS has reduced the trauma of surgery leading, amongst other advantages, to a reduction in post-operative pain, length of hospital stay and intra-operative blood loss. However, this has occurred at the cost of increasing the technical demand placed on the surgeon. Accordingly, ergonomic evaluation of surgical tasks and instrumentation has played a significant role in guiding the design and development of MIS instrumentation.

However, as highlighted in this chapter, existing ergonomic methodology may not be capable of detecting the unique demands that surgical tasks impose on surgeons. For example, ergonomic tests are mainly designed to detect endpoints in user behaviour; e.g. motor output of task execution (EMG, postural tools and motion tracking), task outcome (performance metrics) or physiological response to the task (stress). However, in the application of neuroergonomics to surgery and therefore investigating surgeon brain behaviour, a more precise understanding of task demand can be determined. Consequently, it may become apparent that unique neurocognitive aspects of a task may

52 underpin its difficulty, e.g. visuospatial challenges in MIS. This may impact on many ergonomic and stress parameters or these metrics may not be sensitive enough to detect them. This is all the more relevant as progressively more complex tasks are undertaken minimally invasively and also with the advent of surgical robotics whereby surgeon instrumentation is increasingly intricate and as such may increase the cognitive demands on the operator. Supplementary to this, if the cortical demands of a task are appreciated, the impact of assistive technology can be assessed at brain level in order to appreciate whether or not it is helping.

As has been highlighted [57], optical imaging may offer benefits as a neuroimaging modality for neuroergonomic studies. In particular it offers flexibility which enables it to be utilised in assessing the brain behaviour of a wide variety of tasks in their ‘natural’ environment (e.g. driving [83]). This attribute is coupled with a spatial resolution that may enable greater delineation of the cortical regions involved in task execution. Consequently, in section 2.5 studies utilising fNIRS within a neuroergonomic paradigm were reviewed. This demonstrated that fNIRS has been utilised to assess variation in mental workload with increasing task demand and also has been applied in studies of work-related tasks such as automobile and train driving and numerous aviation tasks. This implies that fNIRS offers a degree of flexibility that may lend itself to assessing the cortical behaviour of surgeons.

The following chapter further explores the application of neuroergonomics to surgery. Initially, the principles underpinning functional neuroimaging and the different modalities available are discussed. Subsequently, functional neuroimaging studies within the field of surgery are reviewed and the neurocognitive skills likely to be necessary to undertake surgical tasks are proposed and evaluated. This serves as a baseline in order to appreciate the likely cortical behaviour required to perform surgery. Graph theory, a means of investigating and analysing functional neuroimaging data is discussed and how it may be applied to neuroergonomic studies. Ultimately a strategy is formulated as to how neuroergonomics can be applied to surgery and this is used to define the hypotheses and studies of the ensuing experimental chapters.

53

Chapter 3

Functional Neuroimaging and Cortical Correlates of Surgical Tasks

3.1 Introduction

In the preceding chapters, it has been outlined how advances in surgery have led to marked improvements in patient care, yet have placed increasing demands on the surgeons undertaking these procedures. Current ergonomic methods have been used in many areas (e.g. aviation and manufacturing) and following the proliferation in the uptake of MIS, were applied extensively to surgery. However, ergonomic tools address a variety of endpoints such as performance and stress and may not entirely appreciate the precise demands that a surgical task may impose. This is all the more relevant as progressively more complex surgical tasks are undertaken often with robotic assistance. Surgical robots may enhance certain aspects of performance (e.g. motion scaling and tremor filtration) yet may exact distinct challenges due to increasingly complicated interfaces with the surgeon using them. Accordingly, neuroergonomics, a paradigm applied to other safety critical industry such as aviation, air traffic control and driving is reviewed and proposed as a means to investigate the unique demands that complex surgical tasks may place on the surgeon.

This chapter begins by reviewing the biological basis of functional neuroimaging. Subsequently, a brief overview of the variety of direct and indirect neuroimaging techniques will be discussed and how these may be applied to surgery and consequently why fNIRS is practical for this role. Studies investigating surgeon brain behaviour are reviewed and utilised to predict likely neurocognitive skills that performing surgical tasks are likely to require such as motor skill and visual spatial working memory. The cortical correlates required to accomplish surgery are likely to be multifaceted, involving many brain regions. As such, cortical connectivity and networks are discussed. It is possible

54 that investigating how brain networks are functioning may offer insight not only into task demands, but also how network behaviour may be modulated by assistive technology. Graph theory is a means of analysing complicated networks and has been applied extensively to neuroimaging data. In this chapter graph theoretical methods are reviewed with the intent to appreciate how this technique may be applied to neuroergonomically assess surgical tasks and equipment. Subsequently, hypotheses are generated and tested in the ensuing experimental chapters.

3.2 Biological Basis of Functional Neuroimaging

Functional neuroimaging enables the function of the regions of the brain to be investigated non-invasively. In relation to indirect neuroimaging modalities, this makes the assumption that increased neuronal activity is coupled to increased metabolic activity [89]. Glucose is the main energy source for the brain, and is metabolised via glycolysis into pyruvate or lactate and these in turn can be metabolised via the tricarboxylic acid (TCA) / Kreb’s cycle to produce adenosine triphosphate (ATP). Neurons and glial cells are the predominant cell types within the brain. The glial cells are in the majority and surround the capillaries of the brain, therefore cerebral glucose uptake must be mediated via these cells [89]. Glycolysis occurs within the glial cells generating pyruvate, this in turn is metabolised by lactate dehydrogenase to lactate. The lactate is then released into the extracellular space to be used by the neurons for energy [90].

As local energy demands increase (as a consequence to neuronal activity) there is an increased need for the delivery of glucose and oxygen and the removal of metabolic waste products such as carbon dioxide. In order to meet this, an increase in cerebral blood flow (CBF) occurs. This causes an increase in HbO2 and a decrease in HHb. This forms the basis of the theory of ‘neurovascular coupling’ [91]. This was first noted by Roy and Sherrington who observed an increase in noise from a cranial bruit secondary to a occipital arteriovenous malformation following visual stimulation [92]. The increased noise was due to an increase in blood flow through it. There are several theories as to exactly how cerebral blood flow and metabolism and therefore neurovascular coupling are mediated. The energy requirement of activated neurones may drive this process coupled with the requirement for glucose. Other factors implicated include potassium, intracellular pH and adenosine as by-products of metabolism. It is possible that neurotransmitters with vasodilating and constricting properties may generate a haemodynamic response in the cerebral microvasculature prior to the build up of metabolic by-products.

55 The resultant local changes in cerebral vasculature cause an increase in oxyhaemoglobin

(HbO2) and a decrease in deoxyhaemoglobin (HHb). A 5% increase in demand for HbO2 is met by a 30% increase in supply [93]. Therefore, local cerebral blood flow increases disproportionately in comparison to the requirement for oxygen leading to the drop in HHb. In sum, these effects result in an overall increase in total haemoglobin (HbT). NIRS (discussed below) is able to detect changes in these parameters and fMRI detects the activation-induced reduction in HHb.

3.3 Neuroimaging Modalities

Several neuroimaging modalities exist and these can be classified into two groups according to the principles on which they are based. Direct methods measure the electrical activity of neuronal firing: EEG and magnetoencephalography (MEG) and indirect methods which rely on the neurovascular coupling. These methods include fMRI, positron emission tomography (PET), single-photon emission computed tomography (SPECT) and fNIRS. Each modality has its own advantages and drawbacks and broadly this can be summarised by a compromise between temporal resolution and spatial resolution as displayed in Figure 3.1.

Figure 3.1 Evaluation of major classes of neuroimaging modalities in terms of temporal and spatial resolution [redrawn from Strangman et al. [94]].

The explanation for choice of functional neuroimaging modality is reliant on several factors including the nature of the task in question, the practicality of the method and the resources available. In relation to the assessment of surgical skills, justification has been made for using fNIRS [95]. However, a brief overview of functional neuroimaging modalities is given, lending weight to the choice of utilising NIRS for neuroergonomic studies.

56 3.3.2 Direct Neuroimaging Modalities

Direct or electrophysiological measures include EEG and MEG as represented in Figure 3.2 and detect electrical activity from neuronal depolarisation. EEG is a graphic representation of the electrical activity associated with neuronal depolarisation. It is the summation of the electrical activity generated from numerous neurons recorded from the cortical surface [96]. The subject wears electrodes according to standardised coordinate system such as the unambiguously illustrated (UI) 10/20 system [97] incorporating up to 256 electrodes. Despite excellent temporal resolution (milliseconds), EEG predominantly only detects activity from the cortical surface, therefore is less able to appreciate activity from deeper cortical structures. Furthermore, EEG signal can be contaminated by movement of facial and ocular muscles that often necessitates signal processing in order to filter this out.

Figure 3.2 Schematic representation of EEG machine (left) and MEG (right), not to scale. The EEG machine consists of electrodes (a) positioned over the subject’s scalp which are recorded and processed (b). MEG (c) necessitates the patient to be seated with the head placed suitably close to the sensors.

MEG records the magnetic field generated by neuronal depolarisation in a plane orthogonal to the electrical current. This is a very weak signal and as such, sophisticated equipment is required and needs to be housed in rooms that are magnetically shielded. MEG offers slightly enhanced spatial sensitivity in comparison to EEG; however the equipment is less flexible. Both of these direct neuroimaging techniques offer virtually instantaneous temporal resolution of the neuronal event however spatial resolution is relatively poor due to difficulty in localising the origin of the synaptic activity. This is termed the ‘inverse problem’ and requires the application of algorithms to try and determine this. In relation to neuroimaging and surgery, MEG is unlikely to be suitable due to the reduced flexibility of subject positioning whilst being imaged. EEG is clearly

57 very flexible; however the problems with both signal contamination and the difficulty in localising the source of the cortical activity may make this modality less applicable to surgery.

3.3.3 Indirect Neuroimaging Modalities

Indirect neuroimaging modalities include PET, SPECT, fMRI and fNIRS. They detect changes in the cerebral circulation following cortical activation. As discussed in relation to neurovascular coupling, neuronal activity results in local changes in the cerebral circulation that are detected with indirect neuroimaging techniques.

3.3.3.1 Positron Emission Tomography (PET) and Single Photon Emission Computed Tomography (SPECT)

PET imaging requires administration of a tracer labelled with a positron emitter such as 15O, 11C and 13N. These are injected into the subject prior to performing the task. The positrons collide and cause release a pair of photons that are detected by the PET scanner. 15O-labelled water is commonly used and has a half life of 2 minutes therefore the task needs to be performed in that timeframe [96]. Accordingly, PET is capable of determining changes in cerebral blood flow with radiolabelled water and consumption if radiolabelled glucose is used. The half life of the isotopes is relatively short and therefore need to be produced on site which in conjunction with the PET camera, adds to the expense of this modality.

SPECT utilises radioisotopes including 99Te and 123I. The image resolution is not as good as PET; however, the half-life of the isotopes is longer allowing the study of tasks of greater duration [96]. A drawback with both PET and SPECT is that they are more invasive necessitating injection of a radioactive substance which is less likely to be acceptable to healthy volunteers.

3.3.3.2 Functional Magnetic Resonance Imaging (fMRI)

Magnetic resonance imaging (MRI) has revolutionised clinical imaging of the human body and works by utilising the interaction of hydrogen within the body with a magnetic field. When a magnetic field is applied, the single proton atom within the hydrogen nucleus aligns itself either parallel or anti-parallel to the field. A radio frequency pulse is applied and when removed, atoms return to their original position generating a signal that can be reconstructed to generate the high resolution images.

58 fMRI detects changes in the cerebral circulation due to cortical activation. In particular it detects the decrease in deoxyhaemoglobin concentration accompanying activity and this can be seen as high signal on a T2-weighted image [96]. This is due to the fact that deoxyhaemoglobin is paramagnetic and its presence changes the proton signal in water surrounding the blood vessel. This produces blood oxygenation level-dependent (BOLD) contrast [98]. This is in effect an endogenous contrast agent obviating the need for administering any tracer or contrast agent to the subject. This discovery has revolutionised functional neuroimaging as subjects can be visualised performing tasks with excellent spatial resolution. The subject undergoes baseline scans and subsequent images are acquired during the task. These images can be compared to detect areas undergoing task-induced activation. These results are overlain onto a high resolution anatomical scan in order to identify anatomical regions activating in response to the task. This yields functional images with very high spatial resolution allowing detailed assessment of the cortical substrate underpinning many tasks.

There are several drawbacks with fMRI specifically, that the temporal resolution is of the order of several seconds and therefore relatively slow. Also MRI is sensitive to motion artefact and consequently subjects have to remain very still whilst performing the task. People with pacemakers or any metal implants (except prosthetic joints) are unable to be imaged with MRI and people who suffer from claustrophobia are unlikely to be able to tolerate scanning. Tasks have to be designed to be undertaken in an MRI scanner and therefore the task paradigm cannot include any ferromagnetic substance. Coupled with this, fMRI is relatively expensive due to the cost of the scanner which needs to be housed in a specially designed room to limit the effects of the magnetic field and the need for technical staff required to operate it.

3.4 Functional Near Infrared Spectroscopy

3.4.1 Background: the ‘Optical Window’

Near infrared spectroscopy is a non-invasive indirect neuroimaging modality that detects changes in HbO2, HHb and HbT as a surrogate for cortical activity. As discussed in section 3.2, as a consequence of neurovascular coupling cortical activity leads to an increase in HbO2 and following a small increase, an overall decrease in HHb as displayed in Figure 3.3. The principle underlying NIRS is that light in the near infrared range, especially between 700 to 1000nm [99], passes relatively unhindered through tissues, an effect termed the ‘optical window’. This explains why when a torch is held up to a hand, a red glows emanates from the opposite side due to the red light passing through the tissue

59 whilst light at other wavelengths is scattered or absorbed. It was Franz Jobsis in his 1977 paper: ‘Non-invasive, infrared monitoring of cerebral and myocardial oxygen sufficiency and circulatory parameters’ [100] who applied this principle to the monitoring of myocardial ischaemia in the exposed heart and to the detection of cerebral hypoxia by transillumination through a feline cranium.

The ability for light to pass through a medium is influenced amongst other factors, by the wavelength of the light and the optical properties of the light absorbing molecules (chromophores) that it encounters. One of the major biological chromophores is haemoglobin (Hb) and this has varying optical properties according to whether it is oxygenated or deoxygenated (HbO2 or HHb). The absorption spectra of HbO2, HHb and water are displayed in Figure 3.4. These are the predominant chromophores in tissue between approximately 700 to 1000nm until at higher wavelengths, water absorbs the light and therefore it will not pass so readily through the tissues. Therefore by detecting the level of scattering of the near infrared light at two wavelengths in this range, it is possible to calculate relative changes in concentration of HbO2 and HHb.

Figure 3.3 Sample time course depicting activation-induced changes in HbO2 (red) and HHb (blue). Following task onset (shaded green region), HbO2 increases after approximately 6 to 8 seconds and after a slight increase, HHb decreases. Lines and shaded regions of time course represent mean and standard deviation of an averaged signal respectively).

60

Figure 3.4 Absorption spectra of HbO2 (red), HHb (blue) and water (black). The subplot indicates the isobestic point where the absorption spectra cross. At higher wavelengths progressively more light is absorbed by water (black line).

3.4.2 Modified Beer-Lambert Law

As discussed above, the principle of NIRS is reliant on the ability of light in the near infrared range to pass through the tissues of the scalp, skull and cerebrospinal fluid in order to detect fluctuations in HbO2 and HHb. A light transportation model is applied to reconstruct relative changes in chromophore concentration from the absorbed light. One such model is the Beer-Lambert Law as displayed in equation (1):

I A = logo = ε c d (1) 10 I

In this equation, Io is the intensity incident on the medium and I is the intensity transmitted through the medium. ε is the specific extinction coefficient of the species within the medium, c is the concentration of the absorbing compound and d is the distance between optical emitter and detector. However, the Beer-Lambert law assumes that the light is either absorbed or passes through the medium. Therefore the formula is modified to take into account loss of light due to scattering and the longer pathlength through the brain and tissues. This becomes the modified Beer-Lambert Law [101]:

61 I A = logo = ε c d DPF + G (2) 10 I

DPF is the differential pathlength factor and represents the total distance of photon travel secondary to scattering and G is photon loss due to scattering. The can either be estimated or measured [102]. The path of photons as they pass through the tissues resembles a ‘banana-shape’ from the emitter to the detector and the depth of penetration is determined in part by the inter optode distance [101, 102].

If the extinction coefficients of the chromophores (HbO2 and HHb) are known, then relative changes in their concentrations can be determined. This is also reliant on the fact that the absorption spectra for HbO2 and HHb are different as demonstrated in Figure 3.4.

Measuring at 2 wavelengths enables quantification of the relative changes in HbO2, HHb and consequently HbT. Topographic maps can be acquired by employing multiple sources and detectors. However, by taking multiple measurements at multiple sites utilising varying interoptode distance, or by utilising time resolved information, tomographic images can be acquired. This yields greater lateral resolution and depth information of the cortical region of interest.

3.4.3 Instrumentation

There are 3 main categories of near infrared spectroscopy measurements and these are outlined below.  Continuous wave. Continuous wave systems emit intermittent pulses of light and detect the amplitude decay.

 Time domain. Time domain systems utilise short pulses of light at picosecond rates into the tissues. Following interaction with the tissues of the scalp, skull, brain and chromophores, the light pulses are broadened and attenuated. The temporal distribution of the photons is then detected as they leave the tissues and this yields data regarding absorption and scattering.

 Frequency domain. Frequency domain systems shine light continuously however the amplitude is modulated. Scattering and absorption information is then recovered by detecting the amplitude decay and delay of the signal.

The 3 different modalities offer individual advantages and disadvantages and these are summarised in Table 3.1.

62 System Type Advantages Disadvantages

Continuous Wave Good temporal resolution Less depth of penetration (<1sec) Smaller instrument size and flexibility Lower cost Time Domain Good spatial resolution Lower sampling rate Accurate determination Greater instrument size of absorption and Higher cost scattering Frequency Domain Higher sampling rate Less depth of penetration Accurate determination of absorption and scattering

Table 3.1 Overview of advantages and disadvantages of the three main NIRS modalities [94, 103].

Figure 3.5 Hitachi ETG4000. The main unit (right) contains the processing unit, a liquid crystal display (LCD) and an optical transmitter. The optode hanger (protruding from left hand side of main unit) suspends the fibre optic cables and the optode emitters and detectors. The latter are depicted in the enlarged subplot (left). The 3D digitizer is situated beneath the LCD display.

63 The system utilised in this thesis is a commercially available continuous wave device, the Hitachi ETG4000 (Hitachi Medical Corp., Japan). It is capable of determining relative changes in HbO2 and HHb at 24 channel locations. The ETG4000 is displayed in Figure 3.5. It consists of 18 fibre optic cables consisting of 10 optode emitters and 8 detectors. Light is shone from the emitters at 695 and 830nm and received by the detectors which are coupled to avalanche photodiode detectors. These are phase-locked ensuring no cross talk between channels. Optical signals are sampled at a rate of 10 Hertz (Hz), digitised and converted to relative changes in HbO2 and HHb. The ETG4000 allows visualisation of the data as a time course or topographically either schematically or overlain on a 3D mesh following registration of optode positions.

Figure 3.6 Overlay and display in relation to reference MRI atlas. Initially the head mesh is generated using the 3D digitizer (top panel, channel numbers have been added to aid interpretation). The 3D coordinate file is exported to a composite display unit where the head mesh and relative optode positions can be displayed overlying a reference MRI atlas (lower panel). This enables appreciation of relative optode position in relation to underlying cortical structures.

64 Co-registration is undertaken with a 3D digitiser (Isotrak, Polhemus, USA). Initially, a head mesh is defined following determination of anatomical landmarks (nason, inion, cranial vertex and the tragi of the right and left ears). Optode positions are displayed on this as and can be transferred to a 3D composite display unit (Hitachi Medical Corp., Japan) in order to overlay onto a reference MRI atlas. This process is depicted in Figure 3.6.

3.4.4 Limitations of Functional Near Infrared Spectroscopy

One of the major limitations of NIRS is that only approximately 1cm of the outer cortex is imaged. This is influenced by the intensity of the light, the scattering of light by the tissues and also the inter optode distance. An inter optode distance of 3cm leads to an imaging depth of approximately 1.5cm [94]. Consequently deep brain structures such as the basal ganglia and hypothalamus are not detectable with NIRS. Thus the spatial resolution is limited by the inter optode distance however, using multiple emitters and detectors that overlap can lead to greater resolution. This is the principle behind diffuse optical tomography [104].

A further issue is that anatomical localisation does not occur if NIRS recordings are made in isolation without coregistration with an MRI scan. In order to overcome this, subjects can undergo an MRI scan with fiducial markers at the site of the optodes [105]. However, this necessitates undergoing an MRI scan at extra expense which negates one of the advantages of NIRS that it is relatively cheap to scan multiple subjects. A similar problem exists for EEG and MEG consequently an external reference system was developed known as the International 10/20 system [97]. The probabilistic underlying cortical structures associated with this system have also been determined [105] thus enabling areas of activation to be linked to the underlying anatomical structures.

3.5 Cortical Correlates of Surgical Tasks

In order to undertake neuroergonomic investigation of surgical performance and how this may be influenced by assistive technology, a baseline appreciation of neurocognitive behaviour underpinning execution of these complex tasks is necessitated. With this as a foundation, it is then theoretically possible to determine how brain behaviour is modulated by assistive technology. With regards to existing ergonomic techniques, such as EMG or stress, results can be analysed with the assumption that greater muscular activity or stress is detrimental and therefore equipment or tasks can be compared in relation to this parameter; such that the instrument leading to the least muscular usage or

65 lower stress levels is the optimum. However, cortical activity required to perform surgery is likely to be very complex and influenced by many variables. Therefore a binary response of ‘less activity is optimum’ may not apply.

The objective of this section of the thesis is to review the existing literature on the neuroimaging of surgical tasks and to outline the necessary neurocognitive skills which are likely to be involved in the performing surgery. Subsequently, these skills will be outlined in relation to the cortical regions subserving them. This will form the foundation for determining which cortical areas to study and hypothesising how they may be scrutinised within neuroergonomic studies to ascertain whether assistive technology is helping or not.

3.5.1 Neuroimaging and Surgery

Functional neuroimaging has been utilised to investigate brain behaviour whilst undertaking open [71, 106] and minimally invasive [87, 88, 107] surgical tasks. These studies have been undertaken with fNIRS with the exception of Zhu et al. [107] who utilise EEG and Wanzel et al. [108] who used fMRI. Table 3.2 outlines the existing literature on functional neuroimaging and surgery. These studies illustrate not only that it is possible to use neuroimaging to investigate the brain behaviour associated with surgery, but that a wide variety of surgical tasks can be assessed.

With regard to open surgical knot-tying, it has been demonstrated that in line with cortical plasticity associated with motor learning [109, 110], novice surgeons displayed PFC activity whereas trainees and consultants did not [106, 111]. Notably, with training this activity attenuated [112]. Surgical knot tying is a bimanual coordination task and PFC behaviour attenuated accordingly with expertise, which has been demonstrated for a non- surgical complex bimanual task [113]. In a further study of surgical knot-tying, undertaken in experienced surgeons [71], it was found that this task remained PFC independent despite acute sleep deprivation. Simultaneously, a cognitive task demonstrated increasing PFC activity in line with increasing tiredness [71]. These studies of surgical knot-tying establish that despite, the paradigm and subject selection, the role of the PFC was in line with established motor learning theory [109, 110]. However, as progressively more complex tasks are investigated, it is possible that the parallels to existing neuroscience literature may not stand as a surgery is likely to rely on a broad range of neurocognitive skills.

66 Author Task Region Findings [reference] (modality) Open surgery Leff [106] Bimanual surgical knot PFC (fNIRS) Experts and trainees exhibit lower tying levels of PFC activity than novices. Novices display dynamic changes in activity with increased activity in block two of five task blocks. Leff [111] Bimanual surgical knot PFC (fNIRS) Experts displayed less cortical tying (same dataset as activity than novices. Novice activity [106]) lateralised to left PFC. Trainee and expert behaviour and performance indistinguishable. Leff [112] Bimanual surgical knot PFC (fNIRS) Expert and trainee brain behaviour tying at two time points differ from novices. Novice activity following training attenuates with training. Leff [71] Bimanual knot tying PFC (fNIRS) Knot tying remained PFC- and cognitive task independent throughout night shift. throughout night shift Cognitive tasks required greater PFC activity throughout night shift. MIS Leff [87] Laparoscopic 2 point Left PFC and Novices performed task at 4 time touching task, assessed right parietal points with training. Pattern of at 4 time points cortex (fNIRS) cortical activity converged towards a pattern of minimal haemoglobin change. Ohuchida Laparoscopic knot tying PFC (fNIRS) Experts and novices demonstrated no [88] PFC activation but trainees did. Zhu [107] Laparoscopic tracking T3/4 and Fz T3-Fz coherence reduced in the task, undertaken (EEG) implicit motor learning group. implicitly or explicitly Greater neural efficiency alongside implicit learning. Abstract tasks Wanzel Mental rotation tasks Whole brain Better performance in MRT [108] (MRT) (fMRI) associated with activity in left parietal, left lateral, medial temporal regions, posterior cingulate- precuneus and premotor cortex.

Table 3.2 Neuroimaging studies investigating cortical activity of surgical tasks.

67 MIS is technically challenging and yields the problem of viewing a 3D task in 2D. Leff et al. [87], studied a frontoparietal (F-P) activity during a MIS task. In this study, subjects undertook a 2 point touching task and cortical activity progressed towards an attenuated state across learning. Within this task, both targets were visible within the field of view thus it may have afforded the ability to learn the task as soon as familiarisation with the MIS set up had occurred. In contrast, Ohuchida et al. [88], investigated PFC activity associated with MIS suturing. This is a very complex task which may challenge even those experienced in MIS. Subjects were divided according to experience (novice, trainees and experts). It was found that only the trainee group activated their PFC and furthermore, novices who underwent subsequent task training then activated this region. This is counter to established motor-learning theory however, in this instance it is plausible that novices found laparoscopic suturing so challenging that they were unable to effectively recruit PFC regions to the task. Therefore, it is only when they undergo further training in the task that they are able to activate the PFC to attend to the task. Experts, however, did show an attenuated PFC response as they were clearly familiar with the task [88]. This study highlights that the cortical response to a complex surgical task can be measured however interpretation of the results may not be entirely straightforward and an appreciation of the stage of learning and the nature of the task is critical.

A MIS tracking task was also interrogated by Zhu et al. [107], who employed EEG to explore regional coherence and how this is influenced by the mode of learning. They recruited novice subjects to undertake a tracking task within a MIS environment who either used implicit or explicit learning. They demonstrated a reduced coherence between T3 (verbal analytic region) and Fz (motor planning area) in those subjects training using implicit learning. The authors conclude that this represents a bypassing of the verbal analytic region and leading to greater neural efficiency in the implicit learning group [107]. This study utilises a tracking task which is likely to be less complicated than MIS suturing however does investigate how different brain regions interact. This may become more relevant as progressively more complex tasks are investigated at brain level. If tasks are reliant on many neurocognitive functions that are multi-regional, measures of integration and connectivity may afford an understanding not only of task-induced activity but how this may be modulated by learning or assistive technology.

Finally, Wanzel et al. [108], utilised fMRI to determine the cortical regions involved in mental rotation tasks (MRT). This group had previously linked MRT skills to ability in a surgical task necessitating the ability to visualise the stages of the procedure prior to

68 undertaking it: a Z-plasty [114]. They also correlated visual-spatial ability with economy of hand motion in surgical performance [115]. In relation to the neuroimaging study [108], greater MRT performance was associated with activity in numerous cortical regions. As functional brain data is acquired with fMRI utilising abstract tasks, direct conclusions with regard to real surgery are difficult to draw [116]. However, these studies highlight that visual-spatial skill may be linked to surgical ability in complex procedures and thus provide a target to scrutinise. For example, visual-spatial ability is likely to be highly relevant in navigational tasks. Therefore, these regions could be investigated in the presence of navigational aids to see if this alleviates the burden of the task.

In summary of existing neuroimaging studies of surgical tasks, a broad variety of paradigms have been employed in open and MIS tasks. This clearly demonstrates the feasibility of investigating the cortical correlates of performing surgery especially with modalities such as fNIRS and EEG which enable users to undertake the task in a more realistic environment. However, it is also clearly highlighted that the cortical response needs to be appreciated in terms of the experience of the subjects and the nature of the task. The following sections focus on neurocognitive skills that are likely to be relevant in surgery.

3.5.2 Motor Learning

Motor behaviour involves many subcortical and cortical regions of the brain, which culminate in motoneurones activating skeletal muscle to effect a movement. The motor cortex (M1), located just anterior to the central sulcus, controls voluntary movement and the whole body is somatotopically mapped within this region: the motor homunculus. Activation within M1 results in specific activation of the corresponding muscle group on the contralateral side of the body. The premotor cortex (PMC) lies anterior to M1 and is involved in movement preparation. The prefrontal cortex (PFC) is predominantly involved in executive function such as monitoring performance for errors and is predominantly active in the early phase of motor learning [106, 109, 110]. The basal ganglia is a deeper structure consisting of several regions that help to aid movement and to prevent unwanted movements. The cerebellum is a hind brain structure that is involved in balance, posture and coordination. The thalamus relays information from deeper brain structures such as the basal ganglia to higher cortical regions.

69 Stage Characteristic Initial stage Slow performance, associated with increased sensory control, irregular actions and inconsistent performance time. Intermediate stage During this stage, as the task is being learnt, a sensorimotor map is developing and the task is performed with increasing speed. Late stage Rapid, automised skilful movement.

Table 3.3 Stages of motor learning as described by Halsband and Lange [117].

Motor learning involves the progression of performance from unskilled to skilled stages. Halsband and Lange divide this into 3 stages as described in Table 3.3. During the early phase of learning, when performance maybe erratic, areas such as the PFC are highly relevant. In particular the dorsolateral PFC (DLPFC) which is associated with working memory [117, 118] may be important in integrating sensory and motor information in order to execute the task. The neurocognitive processes involved in these stages alter as the task is learnt, resulting in differences in patterns in activity with task acquisition [109, 119]. These changes are thought to be characterised by either increases or decreases in activation consequently leading to either a redistribution of activity within the brain or a reorganisation [109].

Redistribution is the process by which the same overall cortical areas are involved in the task. However as the task is learnt, these areas play a greater or lesser role. Reorganisation is the process whereby a different cortical network subserves the task when it is well learnt [109]. Therefore, as a task is learnt distinct regions become more or less relevant. An example of this is the role of the PFC in motor learning. It is thought to act as a ‘scaffolding’ for the early stages of task acquisition and as the task is learnt, this ‘scaffolding’ falls away [110] and as discussed in section 3.4.2, this was demonstrated for surgical knot-tying [106]. The process of decreased activation may be secondary to an underlying neural efficiency such that a spatially contracted region may be sufficient to execute the task due to fewer neurons firing more strongly [120]. Further regions associated with practice-related decrease in activity include the anterior cingulate cortex and the posterior parietal cortex [109], however the latter is also implicated in motor memory representing the development of an internal model of the task [121]. Regions associated with a practice-related increase in activity include the motor cortex and supplementary motor cortex [109]. However, as discussed above, in addition to studies

70 demonstrating decreasing activity in the posterior parietal cortex (PPC), there is evidence that its activity may increase in line with practice [121].

Therefore, activity in particular regions of the brain such as the PFC will demonstrate varying activity dependant on experience. This is relevant when assessing surgeons of differing expertise, in that as far as the motor component of the task is concerned, experts may demonstrate less cortical activation [106]. A further issue is that neuroimaging studies tend to afford opportunity for subjects to practice the task. As such, subjects invariably are able to succeed in undertaking the task. Within surgery, tasks may be very challenging and therefore novices may not be able to complete the task or develop an appropriate strategy to approach it. This may explain the results demonstrated by Ohuchida et al. [88], who found no PFC activity in novices in a very complex MIS task. Therefore it is possible that in tackling complex tasks, an initial naïve phase of learning may exist which may be PFC independent.

3.5.3 Visuospatial Ability and Working Memory

Surgical skill has been linked to visuospatial ability [114]. Using MRT, as shown in Figure 3.4, visuospatial ability was assessed and linked to surgical skills [114, 115]. Those with higher scores in the MRT performed better in a surgical task selected as it was thought likely to require a greater visual spatial ability [114]. In a subsequent study, some of the same cohort of surgeons was assessed performing MRT whilst undergoing fMRI investigation of cortical areas associated with visuospatial ability [108]. Better performance was associated with activation of posterior cingulate-precuneus and premotor areas [108]. A further study utilising an endovascular surgical paradigm, did not show any correlation to visual-spatial ability and performance [122]. Within a MIS paradigm, Hedman et al., demonstrated a correlation between high level visuospatial ability and a complex MIS task [123]. It is possible that visuospatial ability may influence surgical performance but only when the task is sufficiently complex [114, 123] and the link may be more pronounced in novices and lost following training [114, 115]. Therefore, these findings may subsequently be more pronounced if extrapolated to more complex MIS procedures especially procedures undertaken with flexible instruments where spatial orientation is likely to be problematic [124]. However, as discussed in 3.5.2, direct comparison of MRT outcomes with surgical tasks may not be trivial as these are abstract tasks as opposed to dynamic changes within a complex surgical environment. Utilising fNIRS, it may be possible to assess visuospatial ability at cortical level whilst surgical tasks are being performed.

71

Table 3.4 Example of MRT for matching (a) and non-matching (b) shapes, rotated by 180 degrees (adapted and redrawn from [125]).

Mental rotation necessitates utilising visuospatial working memory. Working memory is the system that maintains and stores information in the short term [126]. It is composed of a phonological loop and a visuospatial sketchpad [126]. The ability to retain and manipulate object information and spatial location is termed visuospatial working memory [127]. Cortical areas implicated in this include the DLPFC, the frontal eye fields (FEF) and the PPC [118, 128]. Areas such as the DLPFC and PPC therefore may be activated in surgical tasks that necessitate a greater reliance on visual spatial ability. The PFC is also required for successful spatial navigation [129-131] which is relevant for more complex surgical tasks. Activity in these cortical regions may subsequently be modulated by factors such navigational aids that aid spatial orientation during surgical procedures.

3.5.4 Error Management and Attention to Action

Complex tasks require monitoring for errors and subsequent adjustments to performance. This allows the current situation to be related to the overall goals of the task [132]. The PFC is essential for guiding goal directed behaviour [130] and in particular any task which requires ‘top-down’ processing whereby activity is guided by the subject’s intentions as opposed to innate reactions [133]. In particular it is thought that the posterior medial PFC is essential for the monitoring of task performance and in feedback from errors [134]. In response to error detection, performance adjustments need to be made. The lateral prefrontal cortex is thought to play a role in performance modification in light of errors detected by the medial PFC [134]. Furthermore, functional interactions exist between the lateral and medial PFC that regulate action according to external events [135]. This is important in complex surgical tasks that are likely to require constant monitoring for errors and appropriate response in light of errors.

72 3.5.5 Visuomotor Tasks and Visual Search

Visuomotor behaviour entails the assimilation of visual information and motor activity. Within surgery, certain tasks are likely to represent well learnt, internalised skills, such as surgical knot-tying [106]. However, even if the discrete components of forming a knot are undertaken without higher cortical input by an expert surgeon, other factors, such as positioning each throw in relation to surrounding anatomy clearly need to be considered. This is highly relevant as a surgeon has to be aware of important local structures and constant sensory input, especially visual, is imperative to effectively guide the hand to undertake the task in question. Visuomotor behaviour underpins eye-hand coordination and involves numerous cortical regions including the PFC and the parietal cortex. A frontoparietal network subserves reaching to both a stationary and moving target [136]. Furthermore, the PPC is key to integrating eye and hand behaviour and also making continuous corrections to a reach a moving target [136, 137].

Visual search can be divided into ‘bottom-up,’ i.e. search is guided by the saliency of the scene, and ‘top-down,’ whereby search is reliant on voluntary gaze direction [138]. Top- down search is thus dependent on supplementary brain regions (PFC and parietal cortex (PC) [139]). Therefore if a target markedly differs from its background, it is visually salient is more likely to be detected by a bottom-up search; whereas, if a target requires further cognitive input to find it, a top-down search ensues. fNIRS has been utilised to study behaviour within the visual cortex (V1) during visual search and it has been found that during search, changes in visual cortical hemodynamics were greater when the search was more challenging [140]. The neurocognitive skills discussed in this section are likely to apply to many surgical tasks. However, these skills are will be based on activity in many regions. Accordingly, the following section considers cortical connectivity and how this may be measured to aid in the neuroergonomic investigation of tasks.

3.6 Cortical Connectivity

Cortical regions can be described both in terms of specific task-related functions termed ‘functional segregation’ or as to how distinct regions interact: ‘functional integration’ [141-143]. In the context of previous work investigating surgeon brain behaviour, studies investigating activation within discrete cortical regions can be thought of employing a functional segregation approach [88, 106, 111, 112]. However, F-P activity was considered in a MIS task consequently addressing a more integrative approach [87]. To further appreciate functional integration, cortical connectivity needs to be characterised thus affording the ability to investigate properties of networks and how they may be

73 influenced by parameters such as assistive technology and task learning. There are several methods of assessing cortical connectivity as summarised in Table 3.5 and discussed below.

Definition Ref Anatomical Delineates anatomical links between cortical regions [144] Functional A measure of statistical dependence between regions [145] Effective The influence that one cortical regions exerts over another [146]

Table 3.5 Categories of cortical connectivity and respective definitions.

3.6.2 Anatomical Connectivity

Anatomical connectivity can be determined using diffusion tensor imaging (DTI). This utilises MRI to measure the diffusion of water across myelinated axonal tracts in the brain. This affords the delineation of nerve tracts throughout the brain allowing these to be compared in both healthy brains and disease [144].

3.6.3 Functional Connectivity

Functional connectivity refers to temporal correlations between spatially remote regions [145]. This is based on the principle that if the signal from two regions of the brain is similar, there may be a neuronal connection. However, this similarity may also arise from a common afferent input [147]. A variety of measures can be used to calculate this such as coherence [148], mutual information [147] correlation [149] and independent component analysis [150]. Functional connectivity has been utilised extensively to study the default mode network [151] and has also been used to study network changes associated with task-learning. The acquisition of a bimanual motor task was accompanied with a decrease in functional connections as the task is learnt [148]. The default mode network has also been demonstrated using fNIRS, in which the authors gathered resting state data and applied Pearson correlation in order to demonstrate functionally connected regions [149]. However, this study utilises tomographic reconstruction of the data and therefore resembles fMRI data in comparison to optical topography. Functional connectivity has also been applied to fNIRS data acquired from the infant brain and used to explore how cortical networks develop across the first 6 months of life [152].

Lu et al. [153] undertook a resting state functional connectivity study using fNIRS also utilising seed correlation analysis on the 0.04 to 0.15 Hz component of the fNIRS time course. These results validated the use of fNIRS in the measurement of functional

74 connectivity. The authors utilise all Hb species in their analysis however stronger connectivity was demonstrated with HbO2 and HbT, presumed to be due to the higher signal to noise ratio [153]. Independent component analysis (ICA) has also been utilised to determine functional connectivity and has been suggested to have higher sensitivity and specificity than seed correlation analysis [154]. It is also important to note that functional networks are thought to reflect underlying anatomical connectivity [150].

3.6.4 Effective Connectivity

Effective connectivity is the influence that one neuronal system exerts on another [146, 147]. Whereas functional connectivity addresses the similarity of the signal from multiple cortical regions, effective connectivity employs techniques such as Grainger causality [155], structural equation modelling (SEM) [156] and dynamic causal modelling (DCM) [157] which allow determination of the directionality of influence between discrete cortical regions. These studies have demonstrated changes in network dynamics associated with learning [156] and that connection number may vary according to task demand [155]. Effective connectivity has been assessed with NIRS using Grainger causality analysis in anaesthetised rats [158]. However, fNIRS has not yet been utilised to determine effective connectivity in humans. Although, intrinsic transfer entropy has been employed to investigate the influence of the systemic circulation on cortical haemodynamics [159].

This section has summarised the existing surgical neuroimaging literature and has also discussed cortical regions likely to be implicated in undertaking complex surgical tasks. This has highlighted the concepts of functional segregation and functional integration of cortical behaviour. With regard to the existing surgical neuroimaging literature, a segregation approach has broadly been employed (with the exception of [87]). However, it may be necessary to investigate multiple cortical regions in light of the complex nature of the tasks in question. Therefore a framework for investigating cortical networks involved in surgical tasks is requisite in order to firstly describe brain behaviour and secondly, how this may be modulated with learning or assistive technology. Graph theory [160], is a field concerned with the description of networks and has been used extensively in the analysis of functional neuroimaging data [161]. It is likely to be appropriate in the paradigm of neuroergonomics as metrics can be determined enabling the understanding of changes in cortical networks.

75 3.7 Graph Theory

3.7.1 Background

The brain comprises a markedly complicated network of neurones and connections [162] shaped by the interaction of its components with each other [161]. As cortical networks become further understood, it is apparent that they share features with other network systems despite differences at the microscopic level [161]; specifically, cortical networks demonstrate ‘small-world’ properties [163-165]. A small-world network is characterised by having local highly connected areas coupled with long range connections linking to distant regions within the network. Social networks are analogous in that a person tends to have a close group of inter-related acquaintances who each in turn may have links to other groups which may be geographically or socially distant. Accordingly, it is not uncommon to bump into someone in another part of the world and to realise that you have a mutual connection, even if via a few ‘intermediaries’. This small-world concept was investigated by Stanley Milgram [166] who conducted a study in which he posted messages to various people throughout America instructing them to send a letter to a stock broker in Massachusetts (target person). Participants could only forward the letter to people they knew by name. The shortest chain contained only two intermediaries and the median was five [166]. This study demonstrates the small-world nature of social connections amongst the two hundred million population of America.

Thirty years later, small-world networks were quantified by Watts and Strogatz [164]. They generated a computational model of networks as shown in Figure 3.7, which quantified parameters indicative of a small-worldedness. Specifically, they generated a regular lattice in which all network points (nodes) were connected to their nearest neighbours. This yields a high clustering coefficient, indicative of increased local connections (further explained in 3.7.2 in relation to graph theory terminology) but an increased path length due to the number of connections necessary to pass in order to traverse the network. Watts and Strogatz then randomly reconnected the network until it became fully random. Intervening networks were small-world, characterised by regions of dense local clustering with occasional long range connections that reduce the overall network path length as depicted in Figure 3.7, central panel [164]. Importantly, it was seen that by merely reconnecting a few connections, the path length would greatly reduce with only a minor decrement in the clustering coefficient.

76

Figure 3.7 Depiction of small-worldness as defined by Watts and Strogatz [164]. Panel a represents a regular lattice with a high clustering coefficient and a long path length. The graph is then rewired randomly in order to generate a random graph (c). The random graph has a short path length and a low clustering coefficient. Following a minimal rewiring, the regular lattice becomes small-world (b), in which the clustering coefficient is still high however the path length is much shorter than the corresponding regular lattice. As depicted in (d), minimal rewiring rapidly shortens the normalised path length whilst preserving the clustering representative of small- worldness. [Redrawn from Watts and Strogatz [164]].

Figure 3.8 The first description of graph theory in 1736 [160]. Panel a depicts Euler’s original diagram of the bridges of Konigsberg. In order to solve the problem as to whether it was possible to traverse all bridges only once, he represented the map as a graph, as redrawn in panel b. The nodes of the graph represent the land and the edges depict the bridges. Euler determined that in order to cross each only once, the land (nodes) could only have an even number of connections (edges). If two nodes had an odd number of edges, it would be possible if they were the beginning and end of the route. [Figure reproduced and redrawn from [160]].

77 This was applied to three networks from different walks of life that all demonstrated small-world properties: a network of film actors, the power grid of the Western United States of America and the neural network of the worm Caenorhabditis elegans (C. Elegans) which is the sole example of a completely mapped neural network [167]. These examples all demonstrated a shorter path length and greater clustering coefficient to their respective random networks [164]. The quantification of small-worldness and demonstration that the neurological network of C. elegans displayed these properties indicated that anatomically connected neurological networks displayed small-world properties. Subsequently, Stephen et al. established that functionally connected neuroimaging data also displayed features of small-worldness [168]. In their study, functional connectivity data from research investigating epileptiform activity in the macaque cortex was analysed with graph theory revealing a small-world structure in the primate brain. Subsequently, a study of anatomical connectivity demonstrated small- world properties in a human cortical network derived from cortical thickness, thus representing for the first time that anatomical and not just functional networks displayed small-world properties [169]. Accordingly, it appears that the human brain can be described in terms of complex networks with small-world properties. It is thought that this configuration reflects the need for local activity coupled with integration across cortical regions, thus encompassing functional integration and segregation [141, 142, 170]. Crucially these networks may be influenced by various factors such as pathology, task execution and task learning [167]. In the context of surgical neuroergonomics, evaluating how a cortical network is affected by task learning or assistive technology may further shed light on whether it is aiding the user. Therefore a system is required to display and analyse properties of complex networks.

Graph theory is a field of mathematics concerned with the description and analysis of networks in the form of graphs. A graph is a means of representing data as an interconnected network in which the level of connectedness and interactions between different elements within the graph can be investigated. Graph rudiments include a set of nodes (also termed vertices) connected via links (edges) which can be uni- or bidirectional. The origins of this field began with the Swiss mathematician and physicist Leonhard Euler 1707 – 1783. Euler proved that it was impossible to cross the seven bridges that connected two islands in the river Pregel in the city of Konigsberg once only in order to return to where he had set off from as displayed in Figure 3.8 [160]. In order to prove this, he displayed the bridges as a graph. This is thought to be the origins of graph theory and presently it is applied to many areas of modern life in the assessment and

78 investigation of complex networks [171]. In the following section, graph generation and terminology of graph theory metrics will be discussed in the context of methodology employed in functional neuroimaging.

3.7.2 Graph Theory: Graph Generation and Terminology

As discussed, a graph is a means of representing data as a network and is constructed with a set of points termed nodes or vertices, interconnected via connections: edges. Connections can be undirected or directed according to the direction of influence that one node has on another. The resultant network can then be analysed and characterised with regard to graph theoretical parameters (discussed below and summarised in Table 3.3) enabling properties of the graph to be determined. An overview of graph generation from functional neuroimaging data is represented in Figure 3.9. Initially, neuroimaging data is acquired and nodes of the network are defined. Nodes (also termed vertices) need in some way to be independent from each other however, retain a certain homogeneity amongst other nodes [161]. The nature of nodes will vary according to how network data is obtained. They may represent a cellular system, as in C. elegans and accordingly edges represent synapses. With regard to electrophysiological data, nodes can represent EEG electrodes [172] or MEG sensors [173]. This approach sacrifices anatomical resolution and may lead to an inherent bias in the data due to the fact that a single anatomical area could influence several nodes thus influencing how two nodes correlate (volume conduction effect) [174]. With fMRI, nodes can be defined anatomically following coregistration of the images to a parcellated anatomical template. Accordingly nodes represent the average behaviour from all voxels within that region. However, it has been found that topological properties of networks are influenced by the type of anatomical parcellation used [175]. This may have implications in comparing data from different studies. However, an advantage of using anatomically defined nodes lies with the fact that comparisons can then be drawn from prior neuroimaging studies [174].

The number of nodes used and how nodes are defined in graph theoretical neuroimaging studies are summarised in Table 3.6. Once nodes are defined, network edges are calculated. The nature of what an edge represents is determined by how they are calculated. Following edge generation, the graph is connected. If all edges are equal, then it is an unweighted graph and if the strength of the edge is considered, it becomes a weighted graph. Weighted graphs afford greater information to be gleaned from the network. Within neuroimaging, edges are calculated by using a measure of connectivity between regions. Anatomical connectivity can be used, in which case cortical thickness

79

Figure 3.9 Overview of graph generation. Neuroimaging data is acquired and nodes are defined (ROI, electrode, sensor, voxel, channel). Functional association of nodes is determined using one of the methods (a). Subsequently a threshold (b) is applied converting the association matrix into a binary adjacency matrix. This serves as the input to generate the network.

can be determined and correlated with that measured in other brain regions [169]. Functional association between regions is commonly computed as a means of determining an edge. Numerous metrics have been utilised in order to do this as summarised in Table 3.6 and in Figure 3.9. Methods include correlation [176, 177], mutual information [173], wavelet correlation [165, 178, 179], partial correlation [180], phase synchrony [181] and synchronisation likelihood [182].

80 Author [reference] Modality Node (n= ) Node Edge calculation Threshold definition Achard [165] fMRI 90 1 a 1, 3, 4 Achard [178] fMRI 90 1 a 2 Bassett [179] fMRI 112 1 a 1 Eguiluz [183] fMRI 4891 3 b 3 17174 31503 Fornito [184] fMRI 91 – 4320 1 a 2 Lynall [185] fMRI 72 1 a, d 2 Meunier [186] fMRI 90 1 a 2 Salvador [187] fMRI 90 1 c 1, 4 van den Heuvel fMRI 10,000 3 b 3 [177] van den Heuvel fMRI 95,000 3 b - [188] Wang [175] fMRI 70, 90 1 b 2 Whitlow [189] fMRI 116 1 b 2 Bassett [180] MRI 104 2 c 3 Bernhardt [176] MRI 90, 104 3 b 2 Chen [190] MRI 45 3 b 1, 4 He [169] MRI 54 3 b 1, 3, 4 Zalesky [191] MRI 100 – 4000 1, 4 e 5 Bassett [173] MEG 275 5 d 2 Deuker [192] MEG 204 5 d 2 Stam [182] MEG 126 5 f 2 Langer [193] EEG 84 6 g 3 Micheloyannis EEG 30 7 f 2 [194] Stam [172] EEG 21 7 f 6

Table 3.6 Graph constituents in terms of node number and definition, edge calculation and graph threshold. Node definition: (1) Anatomical parcellation; (2) Grey matter volume; (3) Cortical thickness; (4) Voxel; (5) MEG sensor; (6) EEG region of interest (ROI) defined using standardised low resolution brain electromagnetic tomography (sLORETA). Edge calculation: (a) Wavelet correlation; (b) Correlation; (c) Partial correlation; (d) Mutual information; (e) Tractographic; (f) Synchronisation likelihood; (g) Coherence. Threshold: (1) Significance of association; (2) Thresholded to connection density; (3) Thresholded to r value of correlation; (4) false discovery rate (FDR) to correct for multiple comparisons; (5) No threshold; (6) Thresholded according to synchronisation likelihood; (–) Not stated.

81 Term Definition Reference Graph Set of nodes linked by edges to form a network. [161, 164] Nodes Points within graph connected via edges. Variously defined as [161] anatomical location, EEG electrode or MEG sensor. Also termed vertex. Edge Connection between nodes. Can be anatomical, functional or [161] effective. Adjacency Derived from the full cross correlation matrix of nodes [161] matrix following application of threshold retaining edges of graph, displayed in Figure 3.9 Clustering Characterises the ‘cliquishness’ of a network and represents [164, 195] Coefficient the number of edges connecting the nearest neighbours of a node as a fraction of all possible edges as represented in Figure 3.10. It is indicative of local graph structure. Pathlength Pathlength is the shortest path between two nodes, averaged [164, 196] across all pairs of nodes in the networks, as displayed in Figure 3.10. Therefore pathlength represents the global structure of the graph. Cost Network cost is the number of edges in a network in terms of [174, 178] the maximum number of possible edges. Degree Node degree is defined as the number of edges that link it to [174] the rest of the network. Degree This represents the distribution of degrees of all nodes in the [174] Distribution network. This will be Gaussian in a random network however; complex networks tend to skew towards high degree. Also termed centrality. Betweenness Describes the number of shortest paths between any two [174] Centrality nodes in the network that traverse a particular node. Therefore a node with high centrality may be a bottleneck in the network and can be important in efficient transmission. Efficiency A measure of the efficiency of information transfer within a [197]

network. Can be global (Eglob: whole network) or local (Eloc :

node specific). Eglob and Eloc similar to inverse of pathlength and clustering coefficient, however accommodates parallel information transfer and the strength of edges and a relevant route across disconnected graphs. Cost- Cost-efficiency is derived from the global efficiency at a [173] Efficiency given cost minus the cost. This can be calculated at all levels of network cost therefore can obviate need for an arbitrary threshold to prune the network.

82 Small-World Refers to the small worldedness of a network in terms of high [163, 198] Index clustering and a short pathlength. Small world index (σ) can be calculated by comparing pathlength and clustering coefficient to equivalent random graphs ( and  respectively). σ >>1 indicative of small-worldness Has also been defined in terms of global and local efficiency. Network Network scale refers to the degree distribution of all nodes in [177] Scale the network. Brain networks have been described as scale-free due to the presence of a few highly connected nodes. Hub A hub consists of nodes with increased clustering and [161, 165, 174] therefore with a high degree. Can be defined by smallest pathlength and highest degree. Modularity Modularity describes the degree in which a network consists [190] of numerous modules, which are regions characterised by densely interconnected nodes. Robustness Robustness is indicative of how the structure of the network is [161] affected when an edge or a node is removed. The selection of edge or node to be removed can be random or targeted.

Table 3.7 Graph theory terminology.

These methods have various properties; for example, correlation captures linear associations whereas synchronisation likelihood, mutual information and phase synchrony also detect non-linear interactions [174]. Wavelet correlation and coherence are relevant to data in the frequency domain and partial correlation may limit the impact from several nodes being influenced by a single cortical region (volume conduction) [174]. These methods determine functional association between regions. Effective connectivity [147] could also be utilised, however Smith et al., comment on both computational and mathematical limitations in using DCM and SEM [199].

As illustrated in Figure 3.9, cross comparison of all nodes yields a n x n matrix representing the association between all nodes in the network. Self correlations are excluded leaving the total number of possible connections available. A threshold is then applied to this matrix such that if aij = 0, no connection exists between nodes i and j. The threshold applied is clearly very important leading to either a sparsely or a fully connected graph [161]. The threshold can be selected according to the significance (p value) of the association between two nodes [169], or it can be defined by the r value of the correlation [165]. A further approach is to view the network and ensuing network properties over a range of thresholds [178]. A set threshold (r value) risks inclusion of

83

Figure 3.10 Diagrammatic representation of network clustering coefficient and pathlength. The clustering coefficient of a node represents the number of connections its nearest neighbours (pathlength = 1) have with each other as a proportion of the maximum number of connections they could have. For example, for node A, the maximum number of connections its neighbours could have is 3 (B – C, B – D and C – D). The number of connections that exist is 1 (B – C). Therefore the clustering coefficient of node A is 1/3. The pathlength is shortest distance between two nodes. The pathlength between node E and F is 4 (depicted red). The pathlength indicates how well a graph is connected and the clustering coefficient is a measure of local graph structure [164, 172, 196]. spurious links that may impact on the final network by masking its topology, however, statistical measures can overcome this [169]. It is recommended that a network should be visualised over a range of thresholds [174, 196]. This does appear arbitrary yet the small world properties of the network are most obvious at lower levels of connections density (network cost of approximately 20%) [174]. Supplementary to this, varying the threshold affords normalisation of graph connection density and this in turn allows graphs to be compared [174, 200]. Following these steps and displayed in Figure 3.9, the graph is constructed allowing calculation of network parameters. It is also possible to also divide network measures according to whether first- or second-order metrics. First-order rely on one graph property (pathlength, clustering coefficient, efficiency and cost-efficiency) and second order rely on more than one graph property (small-worldness and modularity) [192]. Graph metrics and terminology are summarised in Table 3.7. These are utilised to calculate properties of the network and who these may be influenced by various factors such as age, pathology or task performance.

3.7.3 Factors Influencing Cortical Graphs

The preceding sections describe the history and theory behind graph theory and graph generation. A summary of graph theory metrics and terminology is subsequently described in Table 3.7. The use of graph theory in the analysis of functional neuroimaging data has rapidly expanded and as such, numerous studies have been

84 Factor Effect on Network Reference Working memory Improved task performance correlated with increased cost- [173] task efficiency Frontal and parietal hubs identified. Network metrics [192] repeatable over time Intelligence Increased clustering and reduced path length associated with [193] intelligence. Intelligence associated with greater degree centrality in parietal cortex, anterior cingulate cortex and fusiform cortex Intelligence negatively correlated with path length [188] Verbal fluency Greater small-worldness correlated with verbal fluency [185] Mental arithmetic Small-world network demonstrated with greater [194] synchronisation likelihood in adults during task (theta band). Small-worldness decreased during task (alpha 2 band) Motor learning Network displays modularity, nodal flexibility increases then [179] decreases with learning. Individual nodal flexibility predicted subsequent learning Finger tapping Scale-free small-world network demonstrated [183] Age Global and local efficiency reduced with age [178] No difference in overall modularity with age, but different [186] patterns in module connections. Network less small-world in adults (compared to children) [194] Dopamine Global and local efficiency reduced with dopamine [178] antagonism antagonism Schizophrenia Qualitative difference in hub distribution with schizophrenia [180] Reduced cost-efficiency of network in schizophrenia Greater mutual information in schizophrenia [173] Weaker functional connections in schizophrenia alongside reduced clustering, small-worldness and probability of hubs [185] Alzheimer’s Greater path length and reduced clustering coefficient in [172] disease Alzheimer’s patients Temporal lobe TLE associated with increased clustering coefficient and path [176] epilepsy (TLE) length. Qualitative difference in hub location. TLE patients more susceptible to targeted network attack Spinal cord injury Increased out degree from right and left supplementary motor [201] (SCI) patients area in SCI patients compared to controls and greater local efficiency in SCI patients

Table 3.8 Impact that tasks, intelligence, age, pharmacology and pathology have on network properties

85 undertaken investigating how metrics of network structure may be influenced by factors including psychiatric pathology, age and task performance. These are summarised in Table 3.8. With regard to working memory, improved task performance is associated with an increased cost-efficiency of the network in particular in left parietal, temporal and midline prefrontal cortical areas [173]. This study demonstrates that a relatively modest increase in the density of connections (cost) yields a far larger increase in the efficiency of the network. Therefore improvements in the network as a whole do not necessarily have to be underpinned by vastly increasing the wiring cost.

Parietal and frontal hubs have been demonstrated in a working memory task and this study involved a repeated assessment demonstrating that network parameters were reasonably stable over time indicating a degree of repeatability [192]. A reduced pathlength and increased clustering, indicative of small-worldness have also been correlated to increasing intelligence [188, 193]. Greater degree centrality in the parietal, anterior cingulate and fusiform cortices is similarly associated with higher intelligence [193]. Similarly, verbal fluency is also associated with small-worldness [185].

These studies show the trend that properties indicative of small-worldness are more evident in association with improved performance in these cognitive tasks. Supplementary to this, small-worldness of cortical networks has been established during motor tasks [179, 183]. Bassett et al., investigated network modularity subserving the learning of a motor task [179]. A flexibility of the modular structure was defined and this served to predict forthcoming performance at an individual subject level [179].

It is also apparent that age may lead to a different pattern of network modularity compared to the younger brain [186] and that associated with ageing, there is a reduction in global and local efficiency [178] and less small-worldness of the network [194]. Graph theoretical analysis has also been utilised to investigate cortical network changes associated with pathological conditions. Schizophrenia is associated with weaker functional connections decreased probability of network hubs and reduced small- worldness [185]. Cost-efficiency is also lower associated with schizophrenia [173]. Patients with Alzheimer’s display a greater path length and reduced clustering indicative of reduced small-worldness [172] and administration of sulpiride, a dopamine antagonist also reduces the global and local efficiency of the network [178].

86 In summary, it is apparent that factors that such as pathology, age and task execution modulate cortical networks and that these changes can be detected. A progression towards small-worldness appears to be beneficial for successful task execution and this is a network attribute that appears to diminish in the investigated pathologies. As discussed in section 3.5, successfully performing surgical tasks is likely to rely on many neurocognitive processes including motor skill [106], visuospatial working memory and ability, attention to action and adequate error monitoring. In light of the above findings of enhanced performance in cognitive tasks in the presence of increased small-worldness, it is feasible that improved performance in surgical tasks may be underpinned by a more small-world cortical network. Furthermore, it is plausible that if assistive technology is able to improve performance, the underlying cortical network may be modulated. Thus it is necessary to be able to compare cortical networks both for learning-related changes and also for between group changes to determine if a particular tool is helping.

3.7.4 Comparison of Graphs

As considered above, it is reasonable to infer that graph theoretical metrics may be modulated whilst undertaking surgical tasks and further affected in the presence of assistive technology. In order to establish and quantify differences between networks, it is imperative to understand how this can be undertaken. In contrasting two distinct graphs, any difference in network size or connection density will directly affect graph theoretical metrics, therefore it is essential that both have an equal number of edges and nodes [174]. This may be problematic as the threshold applied to prune a graph may render a varying number of connections and nodes may also be isolated from the network.

In order to counter the effect of varying node number and connection density, graphs can be thresholded variably in order to ensure that all networks are equal [194]. A limitation with this approach is that different thresholds may be applied to subjects. This means that connections with a lower weight may be included in the graph of one subject and not in another [200]. Exploring the graph over a range of thresholds and therefore cost may aid in appreciating underlying network properties [161] and this approach has been utilised to compare graphs [173, 178]. An alternate approach to comparing graphs is to normalise the graph to an equivalent random graph. This approach may still introduce bias as varying the node number or connection density differentially affects a graph according to whether it is random or a regular. If the node number is fixed, adding a connection will affect network path length more in a regular graph than a random one and the impact of adding a node (if connection density is fixed) will also affect the regular graph more. This

87 may be impact on calculation of the small world index and comparing it between groups [200]. In calculating the small-world index, equivalent random graph comparison is used [165, 175, 178, 180, 188, 191, 193]. This approach is also used in conjunction with visualising the network over a range of connection density [178, 180, 188, 193].

Despite the hazards associated with graph comparison, many studies have utilised one of the aforementioned techniques to investigate how a network may be influenced by a specific factor such as age or pathology (as summarised in Table 3.8). It is also noteworthy that longitudinal assessment of graphs has indicated acceptable intra class correlation implying that metrics may be stable for repeated assessment. Although this was most marked for first order metrics [192]. Therefore, it appears that graph theory may be a suitable approach not only for scrutinising cortical network properties underpinning surgical task execution but moreover, may serve as a means to compare changes in cortical behaviour accompanying task learning and the use of assistive technology.

3.7.5 Graph Theory and Functional Near Infrared Spectroscopy

Currently, it appears that graph theory has not been employed in the analysis of fNIRS data. In order to understand whether this mode of indirect functional neuroimaging can be used to construct cortical networks, the individual stages of graph construction are considered in relation to the constraints of fNIRS and also in relation EEG, MEG and fMRI that are routinely used. Initially, the nodes of the graph need to be generated. As previously discussed these routinely represent EEG electrodes, MEG sensors or voxels derived from fMRI. The latter are apportioned by a variety of means yielding anything from approximately 90 to 10,000 nodes depending on whether an anatomical atlas is used [202] or whether individual voxels are utilised [177, 188]. Node number influences the magnitude of graph theory metrics calculated [175]. For example, the small-world index has a 95% difference when calculated with a 90 compared to a 4000 node network in the same subject [191]. However, the binary outcome of whether or not the network is small- world was not affected [191]. With regard to EEG studies, 21 electrodes and therefore nodes have been described [172], therefore it is feasible that a 24 channel optical topography system could suffice. In relation to what the nodes represent, fNIRS reflects relative changes in HbO2 and HHb in a region between the optode emitter and detector. Similarly, fMRI represents a haemodynamic response; however, this is derived on a per- voxel basis, thus allowing regions to be anatomically parcellated. Nodes in EEG derived networks share a similar property to fNIRS optodes in that they may represent cortical

88 activity from regions beyond where the electrode is positioned. A problem with this is the volume-conduction effect. This is the principle that signal derived from more than one node may be influenced by just one cortical area, thus giving rise to spurious strong local connections.

Once the data is acquired, it is used to derive a cross-correlation matrix which in effect represents the functional connections between different cortical regions. Several studies have already utilised to fNIRS to investigate functional connectivity [149, 203] thus this technology appears capable of detecting functional associations between cortical regions and therefore can generate the basis of a graph. Network thresholding and scaling can be undertaken independently of the mode of data acquisition. Therefore, it appears intuitive that fNIRS can be used to construct cortical networks to be analysed with graph theoretical measures. This affords the benefits of optical imaging (portability, cost, relative resistance to motion artefact, no contraindication to ferromagnetic equipment, good temporal resolution and moderate spatial resolution) to be utilised in conjunction with information regarding cortical network behaviour.

3.8 Conclusion and Hypotheses

Neuroergonomics entails examining the brain behaviour of subjects in order to assess the neural correlates of work-related tasks [9]. Many applications exist, in particular within safety critical industry and in environments necessitating interaction with complex equipment. In recent years, developments within surgery have led to an emphasis on minimising the trauma of surgery to the patient. Whilst this is clearly beneficial, it has necessitated a proliferation of equipment design and development in order to realise expectations of MIS. In tandem with this process, an increasing emphasis has been placed on the evaluation of how surgeons interact with minimally invasive technology. This has been spurred on by often inadequate ergonomic attributes of laparoscopic equipment. Supplementary to this, research and development of surgical robotics has increased with the aim of overcoming many of the challenges posed by MIS.

As reviewed in Chapter 2, numerous methods are used in the evaluation of surgical ergonomics; however, none specifically study surgeon neurocognitive behaviour. This is highly important because increasingly complex procedures are being undertaken with more and more intricate equipment and accordingly the neurocognitive behaviour of the user is likely to become more relevant. In particular, the exact nature of task challenges could be delineated (i.e. navigational, motor, visuospatial) and assistive technology can

89 be tailored accordingly. Additionally, the advantages and disadvantages of tools may be assessed at brain level.

In order to evaluate surgical robotics within a neuroergonomic paradigm, it is first necessary to investigate the cortical correlates of performing highly complex tasks. Therefore determining if the same theories apply as in more simple tasks. E.g. will learning-related PFC independence occur as it does in a simple motor task? Secondly, it is necessary to determine if and how assistive technology affects the brain behaviour of the user. Finally, the ability to use a metric to determine whether or not a particular tool has or has not helped at brain level would clearly be favourable.

3.8.1 Prefrontal Cortical Behaviour and Complex Surgical Tasks

In section 3.5, the existing literature investigating brain behaviour associated with performing surgery is reviewed and subsequently, neurocognitive skills that undertaking surgical tasks are likely to be reliant upon are discussed. From this literature, PFC independence has been demonstrated with task expertise however this is likely to be highly task-dependent. The one study investigating a complex MIS task demonstrated PFC independence in both novice and expert states [88]. Despite some methodological considerations due to study design, this finding may bear relevance for highly complex surgical tasks in that subjects may not activate if no strategy for the task exists.

The brain behaviour necessitated to undertake surgical tasks will vary according to the exact task demands. Therefore MIS and robotic surgical tasks are likely to yield a different cortical response to open surgical tasks, possibly laying greater reliance on visuospatial and visuomotor skill. Accordingly it is hypothesised that complex task execution is likely to be more reliant on visuospatial centres and reliance on these regions will attenuate with experience.

3.8.2 Modulation of Cortical Behaviour by Assistive Technology

It is more challenging to predict the impact that robotic technology may have on brain behaviour and clearly this will vary greatly dependent on the task. With regard to neuroergonomic studies and the effect performance enhancing equipment, PFC activity is reduced when driving with cruise control [76], and similar results were found for train driving [80]. However, with regard to a complex surgical task, assistive technology may enhance performance but do so by increasing the attentional demand of the user. In doing so this may ‘force’ a strategy on the user thereby giving them the ability to execute the

90 task. Additionally, increasing the complexity of surgical equipment may increase the vigilance of the user leading to enhanced performance.

3.8.3 Cognitive Burden Estimation

Principle to the paradigm of neuroergonomics is the necessity to understand user brain behaviour and determine whether assistance is needed or whether a particular tool is aiding performance at brain level. As discussed in section 2.4.2.1 in relation to mental workload, various means have been used to study this and one of the main findings is that increasing PFC activity is related to escalating mental workload. As progressively more intricate and complex tasks are assessed, it is likely that this ‘more PFC activity is bad’ may not stand. In response to this it is necessary to develop a concept of how much a task is affecting the user. Accordingly, the term ‘cognitive burden’ is defined as ‘any deviation from the most efficient neurocognitive pathway of performing a task.’ This definition is independent of the means of assessing it.

As reviewed in Chapter 3, graph theory is a mathematical tool for complex network analysis. The influence of many task-related and pathological conditions on graph theory metrics have been investigated. It is plausible that performance enhancement will affect the brain on a multi-regional level and therefore network metrics such as cost and efficiency may shed light on whether performance is being enhanced via improved behaviour of cortical networks. Therefore, it is hypothesised that graph theoretical measures will help discern performance enhancement with assistive technology at brain level.

In summary it is hypothesised that:

 Complex surgical tasks are reliant on brain regions key to visuospatial processing.  Assistive technology can modulate brain behaviour, possibly via an initial increase in task-related activity likely to be due to increasing user vigilance.  Graph theory can be applied to fNIRS data in order to measure network econometrics including the task-induced cognitive burden  Performance enhancement due to GCMC is likely to be underpinned by a more efficient cortical network as determined with graph theory.

91  Improved collaboration with CGC is secondary to a modulation in search strategy from ‘top-down’ to ‘bottom-up’ search and associated with a reduced level of cortical activity in visual centres and an increased network efficiency.

The next chapter addresses the first point by investigating the prefrontal cortical activity associated with a complex navigational surgical task and how this is modulated with expertise.

92 Chapter 4

The Role of the Prefrontal Cortex in a Complex Navigational Task

Work from this chapter has been published: The ergonomics of natural orifice translumenal endoscopic surgery (NOTES) in terms of performance, stress and cognitive behaviour. Surgery, 2011; 149(4): 525-33.

4.1 Introduction

The goal of performing operations with the least impact on the patient has driven the progression from conventional surgery to minimally invasive surgery (MIS). MIS offers marked benefits to patients in terms of reduced post operative pain, less scarring and an earlier time to discharge from hospital. This has also led to an emergent field of surgery: natural orifice translumenal endoscopic surgery (NOTES) [124]. NOTES entails using flexible endoscopes to perform operations via a natural orifice including the mouth, vagina, anus or bladder; ultimately leading to scarless procedures. NOTES procedures have not been widely assimilated into surgical practice secondary to a variety of safety- related and procedural difficulties. This highlights the necessity for a thorough understanding of the unique cognitive and ergonomic demands that this emergent field places on the surgeon. Therefore, the motivation for this chapter is to assess a complex surgical task of this nature in order to begin appreciate the nature of the burden it may place on the surgeon.

Flexible instruments in the form of endoscopes are traditionally utilised for intralumenal investigation of the gastrointestinal tract. Navigation in this environment is dependent on following the course of the bowel and orientation of the endoscope in the operative space is less important. Navigation in an extralumenal 3D environment such as the abdominal cavity is more challenging as it demands the use of additional cues to spatially orientate the endoscope. Additionally, the endoscopic image is relayed to a monitor thus requiring a 3D task to be viewed in 2D, forcing the operator to perceptually reconstruct the scene.

93

Figure 4.1 Images from endoscope acquired during transvaginal cholecystectomy. Panel a displays dissection of gallbladder from the gallbladder bed with the insulated tip (IT) knife. Panel b depicts the specimen following complete removal prior to transvaginal extraction. (Images courtesy of Mr James Clark).

This type of procedure is likely to require technical motor skill in order to operate the instrument, visual spatial ability, spatial orientation and navigational skill to maximise the use of spatial cues, as well as attention to action and performance monitoring for errors. The PFC plays an important role in these executive functions [109, 110, 117, 129-132, 134, 204] and has been previously studied whilst performing MIS and open surgery [88, 106]. However, no study has assessed the cortical response in relation to flexible endoscopic procedures. This is important in relation to neuroergonomics as it is essential to understand the demands of such complex procedures in terms of brain behaviour in order to appreciate how this may be modulated by assistive technology. Performance undertaking this type of procedure may be enhanced by navigational aids such as

94 gyroscopes or off line reorientation of the image relayed to the monitor. Parallels may be drawn to pilots using gyroscopes or the horizon to orientate themselves in space. One method for assessing the benefit of these navigational aids is to determine whether they alleviate the neurocognitive burden of attention demanding tasks. Such an analysis may confirm that neuronal resources are freed up to focus on other aspects of the task or goal, such as decision making, vital in high-end industries such as surgery and aviation. However, prior to evaluating whether cortical activity is modulated by technological enhancement, it is imperative to gain a baseline understanding of the cortical response to complex navigational tasks.

As discussed in section 3.5, the first application of neuroimaging to assess surgeons was by Wanzel et al. [108], who used fMRI to determine the cortical response evoked by MRT in surgical trainees. As the study aimed to correlate cortical responses during MRT to surgical performance outside the scanner, extrapolating these findings to real surgery is challenging. Using fNIRS, Leff et al., [106] assessed the PFC response to open and MIS procedures and observed learning related attenuation during technical skills acquisition [106]. The findings from these studies are consistent with PFC changes associated with motor learning in general [109, 110, 117]. However, these tasks did not require complex spatial navigation. A complex MIS task was utilised by Ohuchida et al. who observed activation only in subjects undergoing task training and not in novices and experts [88]. However, it is possible that finding an absence of PFC activation in experts and novices represents 2 distinct stages of task acquisition. It is plausible that at the most naïve phase, novices had not had sufficient time to develop or deploy a strategy to complete the task. Erratic performance at this stage may be PFC independent. Subsequently following prolonged task exposure, expert surgeons may be able to execute skills with advanced levels of automaticity which again results in PFC redundancy.

The purpose of this chapter is to begin to elucidate the role of the PFC in such complex surgical tasks. In order to do so, a flexible endoscope is used to navigate between 2 designated points in the Imperial College Natural Orifice Simulated surgical Environment (NOSsE) [205]. The task is designed so that the two targets are not visible within the same field necessitating navigation between the two. Therefore, it is likely to require successful spatial navigation and working memory for which the PFC is involved [129- 131]. The posterior medial frontal cortex is implicated in the monitoring of performance and subsequent adjustments are mediated via the lateral PFC (LPFC) [132, 134]. However, activity within certain PFC structures such as the anterior cingulate cortex may

95 be too deep to be reliably detected using fNIRS. Therefore, a LPFC response is predicted, due to visual spatial activity and the need to instigate performance adjustments. As certain elements of endoscopic manipulation may be learnt with increasing exposure, attenuation in PFC activity associated with increasing experience may be anticipated [109, 110, 117]. Significantly, as NOTES is an emergent field, experienced endoscopists who will be capable of expertly operating the endoscope will not necessarily be ‘expert’ in this environment.

The aims of this chapter are: (1) to determine the pattern of prefrontal cortical behaviour associated with performing a complex navigational surgical task; (2) to assess how this pattern of behaviour varies with expertise; and (3) to determine whether changes in cortical haemodynamics occur independent of stress. It is hypothesised that the magnitude of the PFC response will decrease in line with increasing expertise. However, it is anticipated that a LPFC response will predominate; primarily, due to the association of this area with visual-spatial activity and performance adjustment.

4.2 Materials and Methods

4.2.1 Subjects

Following local research ethics committee (LREC) approval, 29 male subjects with no history of, or current neuropsychiatric conditions were recruited from Imperial College staff and students [Mean age  standard deviation (SD) = 31.6  6.5]. Written informed consent was obtained from all subjects prior to enrolment in the study. Subjects were divided into 2 groups according to experience of endoscopy. 18 were novices [Mean age (years)  SD = 28  4.1] and 11 were experts [Mean age (years)  SD = 37.4  5.4]. Experts were defined by having performed more than 50 endoscopic procedures per year and novices had no experience at all with endoscopy. Of the 11 experts, 6 were gastroenterologists (2 consultants and 4 registrars) and 5 were surgeons (3 consultants and 2 registrars).

Handedness was assessed using a validated scale [206]. All gave written consent prior to participation and were asked to refrain from caffeine and alcohol for 24 hours prior to the study. Caffeine is known to reduce cerebral blood flow [207, 208] and its half life is 3 to 8 hours [207] therefore after 24 hours it is likely to have been eradicated from the subjects.

96 4.2.2 Task and Training

The task was performed in a dimly lit, quiet room free from interruptions. Subjects utilised a colonoscope (model 13907PKS, Karl Storz GmbH & Co, Tuttlingen, Germany) as illustrated in Figure 4.2 to perform an endoscopic navigational task between two designated targets within a simulated abdominal cavity, the Imperial College NOSsE [205].

Figure 4.2 Picture of colonoscope (model 13907PKS, Karl Storz GmbH & Co, Tuttlingen, Germany) used by subjects during the task to navigate between targets.

Figure 4.3 displays the targets for navigation within the NOSsE in relation to an anatomical reference and Figure 4.4 displays images of the targets as viewed by the subjects through the colonoscope as they navigated from the left hand target (in position of gallbladder) to the right hand one (on inferior surface of diaphragm).

97

Figure 4.3 Simulated surgical environment (right) in which the task was performed displayed in relation to anatomical reference (left).Targets can be appreciated within the simulated abdominal cavity and subjects navigated repeatedly between two targets (highlighted in blue) that could not be visualised within the same field of view, necessitating navigation.

Figure 4.4 View through colonoscope as viewed by the subject navigating from one target to the other (panels a to c respectively).

The colonoscope was operated by the subjects in a standing position. The left hand held the controls which manipulate the tip in 2 degrees of freedom and the right hand holds the end shaft of the colonoscope allowing withdrawal or advancement and/or rotation for torque steering. This is represented in Figure 4.5. The image from the tip of the colonoscope was relayed to a monitor. During the task periods, subjects were instructed to navigate between two targets in the simulator as many times as possible in the allotted time. The targets were in the position of the gallbladder and on the inferior surface of the diaphragm. These targets were selected as they were in close proximity of each other, however both targets could not be visualised simultaneously in the same field of view. Thus subjects were required to navigate between the two. Initially, subjects underwent a standardised training period consisting of demonstration of the colonoscope and its controls. Subsequently, the colonoscope was introduced into the simulator and target location was demonstrated and subjects had a five minute period to familiarise themselves with the task.

98

Figure 4.5 Task setup. The subject (left) performs the task by manipulating the tip of the colonoscope with the right hand (panel a) whilst operating the controls of the colonoscope with the left hand (panel b). NIRS optodes can be appreciated attached to the forehead.

The experiment was constructed in a block design paradigm and following a baseline rest period of 30 seconds (s) subjects undertook 5 blocks of 20s of task followed by an inter- trial rest period of 50s. The length of the inter-trial rest period was selected following a pilot study demonstrating that this length allowed sufficient time for the task evoked cortical response to return to baseline levels as demonstrated in Figure 4.6. Initially, the task and rest period were both 30s (Figure 4.6, panel a), and subsequently 20 seconds rest and 40s task (Figure 4.6 panel b). These paradigms did not allow enough time for cortical haemodynamics to return to baseline and consequently, a task period of 20s and an inter- trial rest period of 50s was selected. 30s was selected as the length of the pre-trial baseline rest period as it has been demonstrated to elicit a greater subsequent task-induced cortical response [209]. During the baseline and rest periods, subjects were asked to close their eyes. The starting position of the task was identical for each subject.

99

Figure 4.6 Haemoglobin time courses from pilot studies demonstrating how the response varies depending on the length of task (green vertical bars) and rest periods (white vertical bars). HbO2 and HHb (red and blue lines respectively) are displayed. Changes in cortical haemodynamics have not returned to baseline prior to task onset predominantly indexed in the HbO2 signal (indicated with arrows) when either a 30s (a) or a 40s (b) rest period is employed.

4.2.3 Behavioural data

Task videos were retrospectively scored by 3 reviewers independently. Performance was assessed by the number of times that the subjects viewed the targets within the 20 second task period. In order to successfully visualise the target, it had to lie between predetermined markers on the screen. The width of the markers represented the central 20% of the screen in width and height.

4.2.4 Functional Near Infrared Spectroscopy

A commercially available continuous wave optical topography system (ETG4000,

Hitachi Medical Co., Japan) was used to detect changes in cortical HbO2 and HHb, as previously outlined in section 3.4.3. This device employs 10 near infrared light emitters

100 (695 and 830nm) and 8 photodiode avalanche detectors. The 18 optodes were affixed in two 3 x 3 arrays with an inter-optode distance of 3cm. Light signals are sampled at a rate of 10Hz in 24 channel locations. The area of interest was the PFC and the optodes were positioned according to the unambiguously illustrated (UI) 10/20 system [210] with the left inferomedial optode positioned over Fp1 and the left inferolateral optode over F7. On the right side, the inferomedial optode was positioned over Fp2 and the inferolateral optode over F8. The 36cm2 optode array was therefore centred over F3 and F4 on the left and right respectively. Optode location in relation to a reference MRI scan is displayed in Figure 4.7. The projection of this area of the cerebral cortex can be statistically mapped to the PFC [105]. The optode arrays were secured in place with 2 foam strips with velcro fasteners as displayed in Figure 4.5. Following optode placement, subjects were positioned ready to perform the task. Room lighting was dimmed and the optode gain was calibrated. If there was unsatisfactory coupling of emitters and detectors, affected optodes were inspected for any obstruction from hair and calibration was repeated once this had been addressed.

Figure 4.7 Approximate optode location obtained by transferring topographic data from a representative subject to a 3D cortical surface of an MRI atlas as viewed from right (a), front (b) and left (c). Channels (blue numbered circles) are displayed in relation to UI 10/20 locations [210] (yellow circles).

Following each experiment, optode positions and reference points on the subject’s head: nasion, inion, the tragi of the right and left ears and the cranial vertex were determined in space using an electromagnetic digitiser (Isotrak, Polhemeus). A 3D composite display unit (Hitachi Medical Co., Japan) was used to overlay approximate probe position onto an MRI atlas (Hitachi medical Co., Japan). An example of this from a representative subject is displayed in Figure 4.7.

101 4.2.5 Systemic Effect and Stress

Continuous heart rate monitoring was performed throughout the experiment. This was performed with a portable ECG worn on a band positioned under the subject’s shirt (Bioharness v2.3.0.5, Zephyr Technology limited) and transmitted wirelessly to a nearby computer. Salivary cortisol levels were recorded before and after the study. The first reading was taken as soon as consent had been obtained and the second at the end of the experiment after the NIRS optodes were removed. Subjects were instructed to chew on the specifically designed swabs (Salivettes®, Sarstedt Limited, Leicester, United Kingdom). Samples were stored in a – 200C freezer until analysis in batch with a commercially available salivary cortisol enzyme immunoassay kit (Salimetrics®, Philadelphia, USA).

Figure 4.8 Systemic effect and stress measurement. Salivette and vial for cortisol measurement (a and b respectively). Wireless receiver (c) for detection of signal from portable ECG (d) worn under subject clothing.

The short form of the STAI was used to assess subjective stress levels of subjects before during and after the study [56]. Each consists of 6 statements and the subjects rate how well they feel each statement applies to them. The pre-study questionnaire was completed as soon as consent was obtained and whilst the first salivary cortisol reading was taken. The intra-trial and post-trial questionnaires were both completed at the end of the study concurrently with the second salivary cortisol reading. The short form STAI is shown in Table 2.1, section 2.3.6. Figure 4.8 depicts the HR monitor and the salivettes utilised for cortisol measurement. The systemic effect can also be assessed using scalp blood flow recordings. This is determined using laser Doppler flowmetry and can be subtracted from topographic data therefore limiting its influence on the magnitude change in Hb species.

102 4.3 Data Analysis

4.3.1 Behavioural Data

Inter-rater reliability between the 3 independent reviewers was assessed with Cronbach’s alpha and a value of 0.70 or greater taken as reliable. The median value of the 3 reviewers’ scores for each individual block was calculated. The total score for each subject across all 5 blocks was then calculated from this. A univariate random effects analysis was undertaken with performance as the dependent variable in order to investigate the effect of expertise. A p value ≤ 0.05 was deemed significant.

4.3.2 Functional Near Infrared Spectroscopy

4.3.2.1 Data Pre-processing

Time courses of HbO2 and HHb were generated for each channel and each subject from raw light intensities using the modified Beer-Lambert law [101]. No DPF correction was undertaken and the inter optode distance was fixed at 3cm. Haemodynamic data was processed using bespoke software operating out of Matlab® (Mathworks™, USA) [211]. The data was linearly detrended and decimated to 1 hertz to remove system drift and physiological noise. Data was then subject to data integrity checks to detect saturation related problems by means of in-house developed algorithms. The numerical solution of the modified Beer-Lambert law is nullified when light detected at both wavelengths is saturated (apparent non-recording) and related by a negative constant if only one wavelength is saturated (mirroring). Consequently, a thresholded multiscale cross- correlation permits accurate identification of the saturation episodes. Affected channels were excluded. Furthermore, the time course of the haemodynamic signals were independently visually scrutinized by two reviewers independently to assess for optode movement and consensus between the two observers was reached.

4.3.2.2 Statistical Analysis of Cortical Haemodynamic Data

In order to determine the presence or absence of channel-wise activation, the task-induced change in HbO2 and HHb was calculated for each channel (Mann-Whitney,  = 0.05) (Statistical Package for Social Sciences (SPSS) v16, Chicago, Illinois) utilising data averaged across all 5 blocks of the task. The baseline value was calculated from the average of the 5 pre-task samples. The task value was calculated from the average task Hb level which was determined from a fixed window beginning 3s after task onset and which was 20s in duration. The fixed window was selected following assessment of the haemoglobin time courses. It was noted that some of the activations peaked after the task

103 period of 20s had finished. Subsequent analysis found the average time to peak to be for experts: (mean  SD) 28.1  22.2s for HbO2 and 26.7  20.3 for HHb. Novices: (mean 

SD) 32.8  22.5 for HbO2 and 29.4  21.4. This is longer than previously reported (as reviewed in [212]). A 20s window following a 3s delay after task onset was selected. This was to ensure that the window would capture activation associated with the task but would not capture too much activity that may occur during motor rest. A channel was deemed active if a coupled significant change increment HbO2 and change decrement HHb occurred.

A random effects model (REM) analysis was conducted in order to determine the effect of expertise on Hb data. Optical data was expressed as task minus baseline values

(ΔHbO2, ΔHHb and ΔHbT). This was then used to investigate the effect of subject expertise (novice or expert), channel (1-24) and stress levels (HRV and questionnaire) on the dependent variables (ΔHbO2, ΔHHb and ΔHbT). This was conducted with Intercooled Stata (v8.0 for windows, Stata Corporation, USA).

4.3.2.3 Manifold Embedding fNIRS data is intrinsically high dimensional. Dimensionality reduction based on manifold embedding was utilised to aid with data visualisation and analysis. Manifold embedding is the topological generalization of the concept of dimensionality reduction. Under this paradigm, the signals of interest are thought of as conforming to the mathematical entity of a manifold. Metrics of signal similarity in the ambient space impose the topology so that points closer along the manifold are those exhibiting greater similarity or behaviour. In this sense, distances over the manifold, termed the geodesic, are the entelechy of haemodynamic behaviour similarity. Projection of the manifold to lower dimensions facilitates its visualization. An example of this is the display of the world on a map. The world is a sphere and distances along its surface (geodesic distance) between cities or landmarks can be represented in 2D on a map. Accordingly, geographic information that may not be readily visible on a sphere (such as the distance between two countries on opposite sides of the world), can be appreciated.

In this study, an Euclidean space was constructed and Isomap was chosen as the manifold embedding strategy. Isomap [213] is the combination of computing the geodesic and further projection using classical Multidimensional Scaling (cMDS) [214]. The geodesic distance along the manifold surface was estimated using Floyd’s algorithm [215], using 7 neighbours to construct the base distances graph. This number of neighbours has already been shown that can capture the variability of the haemodynamic manifold [111]. Lower

104 numbers of neighbours lead to unconnected components where as much larger numbers will reduce the Euclidean ambient distance. A number of different ambient spaces were defined and analysed by modifying the construction of the patterns or feature vectors so to highlight and explore different aspects of the dataset. An example is constructing one pattern per vector , and including samples from both haemoglobin species during the task window. This particular example will yield a 40 dimensional ambient space (20 samples at 1Hz per Hb species).

Following previous studies of PFC activity associated with surgical knot tying and MIS [95, 111, 112] Earth Mover’s Distance (EMD) [216, 217] was used to establish similarities between groups of interest; namely experts versus novices and per channel analysis. Using EMD, points in a weighted distribution or signature are matched to a destination weighted distribution minimizing the effort required to achieve best possible overlap. Thus enabling comparison between groups. Based on the EMD results it is possible to cluster channels according to their similarity taking into account both HbO2 and HHb. The lower the EMD, the more similar the groups. A classic hierarchical clustering algorithm [218] was applied and channels showing similar behaviour as manifested by EMD were grouped into clusters. This procedure unveils channels with a similar EMD and therefore displaying similar behaviours and this was used to explore differences between subjects according to expertise.

4.3.3 Systemic Effect and Stress

Heart rate data was analysed using bespoke software programmed using Matlab® vR2008a (Mathworks™, USA). The raw ECG was synchronised with the NIRS signal by means of their time stamp (following previous synchronisation of both systems’ clocks). This was subsequently divided into seven 20s segments which represented the first 20s of the rest period (pre-task), the 5 blocks of task (task), and the final 20s of the final rest period (post-task). The R-waves were detected by the software and each ECG recording was inspected to ensure that all R-waves were detected accurately and that no signal noise was. The R to R interval was determined allowing calculation of the HRV by means of the SDRR [43]. A decrease in the SDRR is associated with increased stress [45]. HRV data was incorporated into the REM. The HRV for each trial and the pre- and post trial values were subject to group-wise analysis with the Friedman test to see if there was any significant change in HRV across all trials. Each trial was subsequently compared to the following trial with Wilcoxon sign rank test for any significant increases or decreases in HRV across all trials.

105

Pre- and post- task salivary cortisol levels were compared (Wilcoxon rank sign). The stress questionnaire was scored so that negative statements were negatively scored and added to the score from the positive statements. This yielded a maximum score of 24 indicating no subjective stress. The pre-trial, intra-trial and post-trial scores were explored with Friedman test for significant changes across all three time points. Changes between individual groups were compared with Wilcoxon rank sign test. The questionnaire results were also incorporated into the REM.

4.4 Results

4.4.1 Behavioural Data

All subjects were right handed with a median Edinburgh handedness inventory of 70. Inter rater reliability between the three independent reviewers of task performance was excellent (Cronbach’s alpha = 0.997). Novices visualised the target 12.5  6.8 times (median  SD) whereas experts achieved 65.0  33.4 times (median  SD) over the whole task (p < 0.001) as illustrated in Figure 4.9. When performance was measured in this way it was reflective of expertise (p<0.001). This suggests that the method used to assess technical performance was construct valid.

Figure 4.9 Boxplot illustrating the number of times target was visualised by the two groups during the task as assessed by the median of three independent reviewers’ score. The median, 95% confidence interval (CI) and outliers are represented by the box and whiskers. * denotes significance (p < 0.001).

106 4.4.2 Functional Near Infrared Spectroscopy

Following data integrity checks, channel 22 from subject 2 (novice) was excluded due to optode mirroring. Following assessment for optode movement, 4 subjects (3 novices and 1 expert) were excluded. Figure 4.10 displays representative time courses from one of the excluded subjects (subject 7). Figure 4.11 illustrates haemodynamic time courses for a representative novice and an expert. It is apparent that upon task onset, an obvious increase in HbO2 and decrease in HHb occurs consistent with activation. This time course is from channel 13 in both subjects (lateral PFC). The average time to peak for experts was (mean  SD) 28.1s  22.2 for HbO2 and time to nadir, 26.7s  20.3 for HHb. For novices, average time to peak was (mean  SD) 32.8s  22.5 for HbO2 and time to nadir, 29.4s  21.4 for HHb.

Figure 4.10 Time courses from excluded subject. Task and rest periods (green and white vertical bars respectively) and HbO2 and HHb (red and blue lines respectively) can be appreciated. Panel a depicts channel 2, in which numerous movement artefacts are present however do not appear to disrupt the overall trend of the time course too markedly. In Panel b (channel 5), it is apparent that movements at the onset of task block 1 and 2 have considerably altered the haemodynamic data.

4.4.2.2 Statistical Analysis of Haemodynamic Activity

The results of the REM are shown in tables 4.1 to 4.3. This analysis demonstrates that expertise was not an independent direct predictor of change in HbO2, HHb or HbT. However, channel location and trial number were significant independent predictors of changes in all three Hb species. With regard to the effect of trial on Hb species, the

107 coefficient is negative for HbO2 and HbT and positive for HHb indicating an overall decrease in HbO2 and HbT across the 5 trials and an increase in HHb. A per-channel analysis was also performed to identify which channels demonstrated changes consistent with activation in experts and novices. The results are illustrated in Figure 4.12. Channels highlighted in red demonstrated a statistically significant increase in HbO2 and a decrease in HHb (significant in channels 2 and 5 in experts). It is apparent that the experts had a more pronounced pattern of LPFC activation than the novices.

Figure 4.11 Haemodynamic time course for a representative novice and expert subject (panels a and b respectively). A coupled task-evoked increase in HbO2 and decrease in HHb (red and blue lines respectively) can be appreciated. Activity appears to be greatest in the expert.

108

ΔHbO2 Variable Coefficient s.e. z P>z 95% C.I. Expertise 1.616 2.448 0.66 0.509 -3.182 to 6.415 HRV 0.032 0.0261 1.24 0.213 -0.019 to 0.084

Questionnaire -0.930 0.432 -2.16 0.031 -1.777 to -0.084 Channel 0.068 0.033 2.10 0.036 0.005 to 0.132 Trial -2.017 0.167 -12.10 0.000 -2.343 to -1.690 Constant 21.701 9.242 2.36 0.019 3.587 to 39.815

σu 4.885

σe 10.963 rho fraction of variance due 0.1657

to uj

Table 4.1 Results of multivariate random effect model for ΔHbO2. Questionnaire refers to the STAI response. Significant p values are highlighted in bold.

HHb Variable Coefficient s.e. z P>z 95% C.I. Expertise -0.921 1.064 -0.87 0.387 -3.006 to 1.164 HRV -0.014 0.010 0.010 0.163 -0.035 to 0.006 Questionnaire -0.016 0.188 -0.08 0.932 -0.384 to 0.352 Channel 0.048 0.012 3.77 0.000 0.023 to 0.073 Trial 0.326 0.066 4.96 0.000 0.197 to 0.454 Constant -0.029 4.010 -0.01 0.994 -7.889 to 7.830

σu 2.133

σe 4.319 rho fraction of variance 0.196

due to uj

Table 4.2 Results of multivariate random effect model for ΔHHb. Questionnaire refers to the STAI response. Significant p values are highlighted in bold.

109

HbT Variable Coefficient s.e. z P>z 95% C.I. Expertise 0.700 3.317 0.21 0.833 -5.801 to 7.202 HRV 0.021 0.030 0.69 0.489 -0.038 to 0.079 Questionnaire -0.946 0.585 -1.62 0.106 -2.092 to 0.201 Channel 0.117 0.037 3.17 0.002 0.045 to 0.189 Trial -1.685 0.188 -8.95 0.000 -2.054 to -1.316 Constant 21.561 12.491 1.73 0.084 -2.921 to 46.042

σu 6.669

σe 12.397 rho fraction of variance 0.224

due to uj

Table 4.3 Results of multivariate random effect model for ΔHbT. Questionnaire refers to the STAI response. Significant p values are highlighted in bold.

Figure 4.12 Channels (large red circles) demonstrating a statistically significant increase in HbO2 and a decrease in HHb and channels (large yellow circles) not activating projected on top of a brain atlas for novices (a) and experts (b). Group differences show a predominantly lateral response in the both groups however this appears to be more pronounced in experts. Small circles represent optode position (emitters in red and detectors in blue).

110 4.4.2.3 Dimensionality Reduction

The results of data embedding are illustrated in Figure 4.13. Each individual point in the embedded space represents task HbO2 and HHb of a given subject during a single block at a specific channel. The signal underlying each point can be reconstructed as seen in the subplots (a to i) in Figure 4.13. The embedded space captures the haemodynamic behaviour of all the subjects and clusters them according to similarity. This can be explored in order to locate areas demonstrating patterns consistent with activation. The embedded points clustered according to channel are shown in Figure 4.14. The closer the channels are to each other, the more similar the signal and therefore the more similar the behaviour. It is apparent from this figure that there is a relatively clear delineation between the expert and novice channels. Similarity between the channels can be computed by the EMD. Behavioural similarity was explored using hierarchical clustering as shown in the dendrograms in Figure 4.15, which yielded different patterns between novices and experts. Firstly, the patterns of channel linkages differ between the two groups and secondly, within the expert cluster, more channels are linked together at a lower EMD. This suggests that there is similarity in the behaviour of a greater number of channels in the experts compared to the novices.

Figure 4.13 Manifold embedding of fNIRS data. Experts (blue) and novices (red) are projected into the experiment space. Each point represents the HbO2 and HHb values from a single subject, during a single task, at a single channel. The subplots (a to i) illustrate the reconstructed haemodynamic response at selected points throughout the manifold. Within the subplots, the HbO2 (red) and HHb (blue) time courses are displayed. It can be seen that towards the top left of the embedded space (subplot a), the pattern is most consistent with activation. Towards the centre of the cluster (subplot e) the pattern is consistent with no activity. Points that are closer to together are more similar in haemodynamic behaviour [111, 112].

111

Figure 4.14 Embedded space in which all points of the cluster are initially labelled by expertise and channel. The centroids of these clusters is depicted. Experts (blue circles) and novices (red crosses) are displayed and the respective channels are labelled. Points that are closer together are likely to have similar signal similarity, therefore channels whose centroids are closely related to others are likely to have similar behaviour.

Figure 4.15 Dendrograms of the hierarchical cluster of channels based on EMD computed over the manifold embedding for novices (left) and experts (Right). It can be appreciated that different pattern of channel clustering occurs between the two groups with relatively more channels clustering at a lower EMD in experts compared to novices.

4.4.3 Systemic Effect and Stress

Due to equipment failure, heart rate data was not collected for 5 subjects (1 expert and 4 novices) and for trial 5 in another subject (expert). There was no significant change in HRV across all trials and the pre- and post trial rest periods (p=0.078) and there were no significant differences between any sequential groups. From the REM analysis, expertise was not a predictor for difference in heart rate variability (p=0.670), or stress

112 questionnaire response (p=0.411). Salivary cortisol did not significantly increase from pre- trial to post-trial in novices (p=0.528) and the median value reduced in experts from 4.5 pre-trial to 4.2 post-trial (p=0.026). In sum, this suggests that there was no difference in the stress response between experts and novices.

HRV was not a predictor for change in HbO2, HHb or HbT. Stress questionnaire response was a predictor for ΔHbO2 but not ΔHHb or ΔHbT (Tables 4.1 – 4.3). The STAI responses for novices were 22 to 20 and 22 (median values out of 24) for pre-, during and post study questionnaires respectively. This was a statistically significant overall change (p=0.002). Post hoc analysis demonstrated a statistically significant change when comparing pre-trial to intra-trial questionnaires (p=0.005) and from intra-trial to post-trial questionnaires (p=0.003). For experts, the median responses (out of 24) were 21, 22 and 24 for pre-trial, intra-trial and post-trial questionnaires respectively. This was statistically significant (p=0.048). Post hoc analysis demonstrated a significant difference between pre-trial and post-trial questionnaires (p=0.028).

These results suggest that the systemic effect in terms of HRV did not impact on cortical haemodynamics and it was only the stress questionnaire response that was found to have an influence on HbO2 and not the other Hb parameters. The task-associated decrease in stress questionnaire response noted in the novice cohort may imply that they found the task subjectively stressful, however, this result was not mirrored in the physiological parameters of stress.

4.5 Conclusions

In this chapter, the PFC response evoked by an endoscopic navigational task has been assessed with fNIRS. It has been found that experts performed significantly better than novices and did so using a different pattern of cortical activation. Expertise related differences were not manifest as anticipated, as a variation in the overall changes in cortical haemodynamics, but instead as a difference modulated by activation of distinct channels. Specifically, experts demonstrated more pronounced LPFC activation than novices. Moreover, cortical haemodynamic behaviour was observed to be more co- ordinated across NIR channels in subjects with more endoscopic experience. This was evident as a greater number of channels demonstrating similarity in cortical behaviour as evidenced by the hierarchical clustering of EMD between channels in the embedded space. This implies that experts execute a complex navigational task using a more uniform pattern of PFC activity. In contrast, activation patterns across a bilateral

113 prefrontal network in novices are less coherent, with fewer regions demonstrating activation.

Greater PFC activity associated with expert performance was unforeseen as classically, attenuation within this region is associated with task expertise [109]. The neurocognitive basis for these findings is reliant on the nature of the task and the resources that the subjects draw on, namely visuospatial working memory, action selection and ongoing task monitoring for errors and performance adjustment. The task was a complex endoscopic paradigm which necessitated navigation between two points that were not visible within the same field of view. Therefore, it is likely that subjects had to rely on visuospatial working memory in order to maintain a representation of the desired course throughout the trial. Working memory may be considered as three subcomponents: (1) the phonological loop; (2) the visual spatial sketchpad and (3) the central executive [126]. The central executive acts as a control system supported by the other two components [126]. The DLPFC has been implicated as the location of the central executive [126, 219] and the ventral lateral PFC (VLPFC) maintains task-relevant information in non-human primates [220]. Therefore this region is principle to visuospatial working memory. It is possible that the association between expertise and LPFC activation represents more efficient visuospatial processing. Further task-relevant functions of the LPFC include high level cognitive control subserving the ability to select appropriate actions according to internal goals [133, 135].

It has also been proposed that the PFC is organised in a rostro-caudal, hierarchical manner [221, 222] with anterior regions representing an abstract plan of the task and internal monitoring, and more posterior regions representing concrete action representations [221]. It is possible that both experts and novices have a task plan and are capable of internal monitoring. However, in the current study, experts also demonstrate activation of more caudal LPFC regions. This may be indicative of experts having a concrete representation of the task which is subsequently manifest as greater performance.

The medial PFC plays an important role in monitoring task performance for errors [134, 223]. In particular the dorsal anterior cingulate cortex is thought to detect errors and engage the DLPFC to modulate performance accordingly [134, 135, 223]. However, in the current study, no medial PFC activity was detected in either group. One explanation is that the study was not designed to force or detect errors and subjects were not instructed

114 as to what would constitute error. Alternatively, areas encoding error detection such as the dorsal anterior cingulate cortex, may be too deep to be reliably detected using fNIRS.

These findings differ from previous work investigating the role of the PFC in surgery [106, 112]. This disparity, in part is likely to be representative of the diverse nature of surgical tasks. Leff et al., [106, 112] investigated surgical knot tying which is a bimanual coordination task and the findings were consistent with motor skill learning [109, 110] in that novices demonstrated greater PFC activation that attenuated as the task was learnt. This is consistent with PFC activity associated with the acquisition of non-surgical bimanual coordination tasks [113, 224]. The role of the PFC in MIS has been previously assessed [87]. However, in this study, a simple two point target localisation task in which the targets were persistently in a static field of view. Therefore, it is difficult to extrapolate the findings to the current NOTES paradigm.

The lack of PFC attenuation with expertise demonstrated in the current study may be due to the fact that task complexity impeded the novices from developing a strategy to successfully reach the targets. Reliant more on luck and random movement rather than distinct navigational ability, these subjects demonstrated a lower number of activated channels and a less coherent pattern of PFC behaviour, perhaps indicative of withdrawing from the task. The expert cohort however, demonstrated superior endoscopic ability and were able to focus on navigating between the targets more efficiently. In order to achieve this, greater activation of the LPFC was utilised. It is possible that with further training, this PFC response would follow the predictable course of practice related-attenuation of response. These results in part corroborate the work of Ohuchida et al., who only demonstrated activation in the cohort of subjects undergoing task training; i.e. those who had the ability to complete the task, but were not at the stage of automated performance [88]. Leff et al., demonstrated significantly greater PFC activation in novices compared to experts in a surgical knot-tying task [106]. Novices undertook a standardised training session prior to the study in order to establish a baseline level of ability on the bimanual task. It is possible that within this time frame, novices progressed through naïve phases of task acquisition and were then able to strategically execute knot-tying, albeit in a slower and more effortful manner than experts. Consequently, this ‘integrative phase’ was associated with marked PFC activity. Conversely, expert surgeons well accustomed to surgical knot-tying were at a PFC-independent expert stage of performance [106]. Following this finding in the current chapter, Appendix 1 further explores the theory that a PFC-independent novice phase of performance exists.

115

It has been demonstrated that changes in cortical haemodynamics were likely to be independent of stress. This is highly significant since surgical tasks have been shown to be stress-inducing [4, 5] and that stress can influence PFC responses [225, 226]. It is possible that stress was not a significant factor in the current study as the experiment was undertaken on a simulator in a laboratory environment therefore not inducing as much stress as performing authentic surgery may do.

This paradigm for functional neuroimaging raises important questions about the temporal nature of the NIRS response. In the current study, the time to peak (HbO2) and time to nadir (HHb) were longer than previously reported, as reviewed in [212]. One possibility is that the subjects used their visuospatial working memory to maintain a mental picture of the task into the rest period. This may be analogous to delay-period activity thought to represent the maintenance of information in the mind after the stimulus has been removed in visuospatial tasks [219]. Alternatively, the subject may be contemplating something unrelated, especially given a 50s rest period. However the length of motor rest was essential as it left enough time to allow the cortical response to normalise. Moreover, it has been suggested that it is important to ensure that the cortical response has resolved during a period of rest or inactivity [227].

The aim of this chapter was to elucidate the cortical response associated with a complex navigational endoscopic task, in order to characterise changes associated with varying expertise and to determine whether haemodynamic changes occurred independently of stress. It was hypothesised that a greater LPFC response would be elicited and that novices would have greater overall changes in cortical haemodynamics due to task naivety. Expertise was not a predictor for overall changes in cortical haemodynamics, but a difference in behaviour was unveiled through variations in the pattern of activation. Experts demonstrated greater LPFC activation during a complex navigational task than novice endoscopists, possibly due the application of a strategy in order to successfully achieve the goal with resultant activity in areas associated with visuospatial ability [108] and modification of performance in response to error [132, 134]. In contrast, novices were not able to perform the task with any coherent strategy and as such demonstrated less PFC activation.

Critically, these results establish that it is essential to interpret the level PFC activity in light of stage of the learning process and not simply in terms of less activation being

116 indicative of expertise and automated performance. This was a necessary finding prior to investigating how brain behaviour may be affected by assistive technology. Examples of these technologies include navigational aids, motion stabilisation and GCMC [228], which may help to enhance robotic surgical performance.

117

Chapter 5

Gaze-Contingent Motor Channelling

The work in this chapter has been published: Gaze-Contingent Motor Channelling, haptic constraints and associated cognitive demand for robotic MIS. Medical Image Analysis, 2010 doi: 10.1016/j.media.2010.07.007

5.1 Introduction

In the preceding chapter, a complex MIS navigational task was investigated in terms of PFC activity. Key results included the reliance upon LPFC regions for task execution and also the finding that experts demonstrated greater PFC activity than novices. This differs from the established theory of learning whereby the PFC is integral to the early phases of task acquisition; however, may represent a naïve stage of the learning process (further investigated in Appendix 1). Following on from this, the purpose of this chapter is to assess a technology designed to improve performance: GCMC.

MIS offers clear benefits to patients such as reduced post operative pain and reduced length of stay in hospital. However, MIS places increased demands on the surgeon partly due to visuomotor misalignment and poor instrument ergonomics [4, 5]. Surgical robotics aims to address these limitations of MIS, specifically: the unstable camera platform, reduced instrument mobility, 3D images viewed in 2D and poor ergonomics [6]. Enhancing surgeon capability through the use of technology may be beneficial in many respects such as improving accuracy and limiting erroneous movements. As detailed in the preceding chapter, a reliance on LPFC centres underpinning visuospatial working memory was demonstrated likely due to the navigational demands of the NOTES task. Accordingly, in the present chapter, a device that improves performance accuracy and may lead to a simplification of the navigational process is investigated: GCMC.

118 5.1.1 Gaze Contingent Motor Channelling

With GCMC enabled, information regarding depth perception and tissue deformation is gained in real time from the surgeon’s gaze patterns via eye tracking. This can then be used to constrain the hand of the surgeon within ‘safe boundaries’ thus reducing the risk of damage to surrounding structures [228]. Previously, preoperative registration of images was required to generate such constraints; however, GCMC allows this to be performed in real time without prior knowledge of structures. Thus GCMC allows realignment of the visuomotor axis of the surgeon by detecting gaze behaviour and directly transferring this into motor behaviour.

This streamlining of the human-computer interaction has lead to the concept of ‘perceptual docking’ [229], whereby operator-specific motor and perceptual behaviour is acquired in situ via human-computer interaction. In so doing, optimum aspects of human performance such as gaze accuracy can be utilised in real time to derive information to guide the robotic instruments [229]. This process necessitates a seamless interaction between the surgeon and the robot in order to harness surgeon intent precisely thereby maintaining optimum safety.

Figure 5.1 Schematic illustration of GCMC. The surgeon fixates on the target (black dot on screen) and their point of regard is detected with the eyetracker. This positional information is used in conjunction with the location of the tool tip (black line on screen) in order to calculate the force (F) required to be exerted via the haptic manipulator in order to diminish the distance (d) between the two.

119 GCMC leads to more accurate performance with less accidental tissue penetration whilst operating on a simulated beating heart in 2D and 3D [228]. Therefore this technique requires the surgeon to fixate on the target as opposed to the tool which is a more expert pattern of gaze behaviour in performing a tracking task [230]. A schematic illustration of GCMC is depicted in Figure 5.1. Specific parameters such as the force profile and the boundaries can be varied [228]. An example of this is portrayed in Figure 5.2.

Figure 5.2 Screen shot depicting boundary of haptic constraint (conical mesh) constraining tool (green stylus) to target (blue dot) on surface of simulated heart.

An appreciation of the manner in which novel instruments modulate performance is clearly a critical facet in determining uptake and dissemination. However, it is unlikely that exclusively scrutinising technical performance will elucidate the impact of a technology on the user. As discussed in Chapter 2, the field of ‘Neuroergonomics’ [9] describes brain behaviour at work and is concerned with evaluating novel technology with functional neuroimaging in order to determine how tasks impact on user brain behaviour. Research in this field is motivated by advances in functional neuroimaging that allow the cortical response induced by a variety of tasks to be assessed coupled with the fact that advances in technology are placing increasing demands on humans [231]. In the setting of robotic surgery, it is important to demonstrate that a new technology not only improves accuracy, but that it does not risk sensory overload of the surgeon. Clearly GCMC has vast potential to enhance performance by improving accuracy of the surgeon and safety for the patient. It is possible that this technology may necessitate increased concentration from the surgeon thus increasing cognitive workload.

120 The PFC is implicated in attention to action [232] and the ongoing monitoring of the task with subsequent adjustments to performance [132, 134]. Accordingly, when performing with GCMC, subjects may require these regions to attend to the novel approach to the task. As discussed above, a novice pattern of gaze behaviour is one in which the hand precedes the eye and as expertise is acquired, the eye subsequently leads the hand. Therefore, it is possible that when the GCMC is enabled, the subject has to employ a more ‘expert’ pattern of gaze behaviour. It is likely that this will require more attention and consequently greater PFC activation. However, in being ‘forced’ to regard the target, the subject is in effect employing an appropriate strategy for task execution and therefore performance is improved.

Visual search can be divided into ‘bottom-up’ and ‘top-down’ approach [138, 139]. Bottom-up search is dependent on scene saliency and top-down influenced by extra- striatal centres including the PFC [233]. It is feasible in that forcing the subject to attend to the target may increase the reliance on extra-striatal regions such as the PFC. Additionally, undertaking the task with GCMC assistance may necessitate a greater vigilance from the subject. This is known to increase cortical activity, in particular within the right PFC and parietal regions [70].

The flexibility of fNIRS as a neuroimaging modality was demonstrated by its application to an endoscopic task in the previous chapter. The attributes of portability, relative resistance to motion artefact allow it to be applied in the current paradigm. The aim of this study is to assess the impact of GCMC in a randomised controlled trial on: (1) the cognitive demands of the user in terms of PFC activity and (2) performance. It is hypothesised that GCMC will improve performance necessitating greater subject vigilance and increased attentional demand leading to increased PFC activation.

5.2 Methods

5.2.1 Subjects

Following LREC approval, twenty one subjects (3 females) aged 28.9 years  3.1 (Mean  SD) who were free from neurological and psychiatric illness participated in the study. All had normal or corrected normal vision. Written consent was obtained and handedness was assessed with the Edinburgh handedness inventory [206]. Participants were asked to refrain from caffeine and alcohol for 24 hours prior to the study.

121 5.2.2 Task and Training

The visuomotor task involved manipulating a virtual tool with the aim tracking a target on the surface of a simulated beating heart. The task was designed to represent a cardiac ablation for arrhythmia as this involves localising a point on the beating heart and is a procedure likely to benefit from a technology such as GCMC. Figure 5.3 is a schematic representation of the task setup.

Figure 5.3 The subject (seated) can be appreciated holding the haptic manipulator in their right hand which controls the virtual tool (yellow) whilst regarding the target (black dot) on screen. Their gaze (green dashed line) is detected with the portable eyetracker situated beneath the monitor. The tool tip position is known (red dashed line) and is used to calculate the distance (d) between it and the target. Accordingly, a force (F) is generated that is a function of that distance and exerted via the haptic manipulator in order to localise the target. An optode array can be appreciated over the left PFC.

The task was undertaken in 2D with the virtual tool controlled by a haptic device (Phantom Omni, SensAble Technologies, MA, USA) which is able to provide force feedback in 6 degrees of freedom. Therefore guidance can be delivered back to the surgeon. Eye tracking was performed with a stand-alone infrared video-based eye tracker (Tobii x50, Tobii Technologies, AB, Sweden) that records fixation points on the screen at 50 Hertz. Subject specific calibration of the eye tracker was performed prior to experiment onset and was repeated until it was satisfactorily achieved. During the task, the beating heart background deformed with a regular trajectory at a frequency of approximately 50Hz. Figure 5.4 illustrates the target trajectory.

122

Figure 5.4 Panel a: the tool (yellow) and target (black circle) are visible within the task scene. Trajectory and fixation point (blue ellipse and white cross) are not visible to subject. Subject fixates on target allowing calculation of force needed to displace tool tip to target (b). Target localised (c) and as fixation point remains on target, successful tracking continues (d).

The task and principles of GCMC were explained to all subjects prior to standardised training. Training consisted of a bespoke simulation requiring subjects to fire at targets that crossed the screen using cross-hairs to aim. Initially, subjects guided the cross-hairs with the haptic device, then with their fixation point and then with GCMC enabled. Subjects spent 2 minutes on each amounting to a training period of 6 minutes. This enabled familiarisation with the tool but not the task. A screen shot of the training game is displayed in Figure 5.5.

Figure 5.5 Screen shot of GCMC training game. Targets (can and blue face) crossed the screen in horizontal and vertical directions and had to be localised in the cross-hairs (white circle, centre of image). The graphical user interface (lower right) enabled switching between the training modalities and was hidden from view during the task.

123 Subjects then had a 10s opportunity to trial the task prior to commencement of the experiment. Following a pilot study, a block design paradigm was employed that appeared to capture a PFC response upon task execution. After a baseline rest period of 30s, 5 blocks of 20s of task and 30s of rest were undertaken. Subjects undertook the task under both conditions and experiment flow is depicted in Figure 5.6. The order in which the trials were undertaken (control versus GCMC) was randomised (random number generator). All experiments were undertaken in a quiet, dimly lit room, free from interruptions.

Figure 5.6 Experiment flow. Following randomisation, subjects undertook both tasks. Task periods (20s) were interspersed with rest periods (30s).

5.2.3 Behavioural Data

Performance was scored as the mean Euclidean distance between the virtual tool tip and the target in pixels. A smaller distance equates to better performance [228].

5.2.4 Functional Near Infrared Spectroscopy

Fluctuation in cortical haemodynamics was captured with a continuous wave optical topography system (ETG 4000, Hitachi Medical Co., Japan). As described in the previous chapter, ten near infrared light emitters (695 and 830nm) and 8 photodiode avalanche detectors were positioned in two 3 x 3 arrays over the PFC with an inter-optode distance of 3cm. The arrays were sited according to the UI 10-20 system [97, 210] with the left inferomedial optode positioned over Fp1 and the left inferolateral optode over F7. On the right side, the inferomedial optode was positioned over Fp2 and the inferolateral optode over F8. The 36cm2 optode array was therefore centred over F3 and F4 on the left and right respectively. Optode location is schematically represented in Figure 5.7 and subsequent channel location overlain onto a reference MRI is depicted in Figure 5.8.

124

Figure 5.7 Schematic depiction of optode placement in relation to UI 10/20 coordinates. Optode emitters and detectors (red and blue numbers) are positioned according to UI 10/20 positions (depicted at corners of arrays). Channels are accordingly located (black numbers 1 – 24).

Figure 5.8 Channel location displayed over reference MRI atlas using 3D composite overlay system (Hitachi Medical Corp., Japan). Channel locations have been labelled (numbered yellow circles).

Optode arrays were secured in place with 2 foam strips with velcro fasteners. Following positioning of the optodes, the room lighting was dimmed and the optode gain was calibrated. If any unsatisfactory coupling of emitters and detectors was detected, affected optodes were inspected for any obstruction from hair and calibration was repeated. Optode recalibration was undertaken in between the two task conditions. A black cap was placed over the optodes after they had been positioned with the aim of further reducing ambient light. Following each experiment, optode positions and reference points on the subject’s head (nasion, inion, the tragi of the right and left ears and the cranial vertex)

125 were recorded with a 3D digitiser: as in the previous chapter. Results were subsequently overlain onto an MRI atlas using a 3D composite display unit (Hitachi Medical Co., Japan) as illustrated in Figure 5.8.

5.2.5 Systemic Effect and Stress

Continuous heart rate monitoring was performed throughout the experiment. This was performed with a portable ECG that was worn on a band positioned under the shirt of the subjects (Bioharness v2.3.0.5, Zephyr Technology limited) and transmitted wirelessly to a nearby computer. Subjective stress levels were determined with the short form of the STAI [56]. The first questionnaire was undertaken as soon as consent had been obtained (pre-study) and the remaining were undertaken after the study (during and post-study).

5.3 Data Analysis

5.3.1 Behavioural Data

Behavioural data in terms of distance from the tool tip to the target was averaged across the 5 task blocks and a univariate random effects analysis was conducted in order to investigate the influence of mode of assistance (control versus GCMC) on it ( = 5%).

5.3.2 Cortical Haemodynamics

5.3.2.1 Data pre-processing

Data pre-processing and analysis was similar to the preceding chapter using bespoke software operating out of Matlab® (Mathworks™, USA) [211]. Time courses of HbO2 and HHb were generated for each channel and each subject from raw light intensities using the modified Beer-Lambert law [101]. Subsequently, it was linearly detrended and decimated to 1Hz to remove system drift and physiological noise. As discussed in section 4.3.2.1, data integrity checks were undertaken in order to detect saturation related problems (apparent non-recording and optode mirroring). Affected channels were excluded.

5.3.2.2 Statistical Analysis of Cortical Haemodynamics

Individual channels were assessed for task induced activation in terms of a change in

HbO2 and HHb. Block averaged task data (derived using a window encompassing the 20secs of the task) was compared with the baseline 5 pre-task samples (Wilcoxon rank sign,  = 0.05). In order to investigate the influence of mode of assistance (control versus GCMC), channel (1 to 24), laterality (left or right PFC) and systemic effect in terms of

126 mean HR and HRV on cortical haemodynamics, a REM was conducted. Hb data was expressed as ΔHbO2, ΔHHb and ΔHbT and the influence mode of the dependent variables was accordingly determined (Intercooled Stata, v8.0 for windows, Stata Corporation, USA).

5.3.3 Systemic Effect and Stress

The HR and HRV (expressed as the SDRR) was calculated for each subject under both sessions. This was incorporated into the REM in order to determine if these parameters were found to be predictors for change in cortical haemodynamics. The STAI responses were undertaken before, during and after the whole trial and therefore cannot be utilised to appreciate how subjective stress is influenced by each mode of assistance. Therefore the pre-trial, intra-trial, and post-trial responses were explored for significant changes across all three time points with changes between individual readings determined with Wilcoxon rank sign test ( = 0.05).

5.4 Results

5.4.1 Technical Performance

Performance, as manifest by a diminution of the distance from the tool tip to the target, was better with GCMC enabled as demonstrated in Table 5.1 and illustrated in Figure 5.9 (p < 0.000).

Performance Variable Coefficient s.e. z P>z 95% C.I. Session -24.523 0.794 -30.88 0.000 -26.080 to -22.967 Constant 80.042 4.862 16.46 0.000 70.513 to 89.572

σu 19.896

σe 10.424 rho fraction of variance 0.785

due to uj

Table 5.1 Univariate random effects analysis of impact of session (control versus GCMC) on performance accuracy.

127

Figure 5.9 Median performance accuracy in terms of distance from the tool tip to the target (pixels) for control (blue) and GCMC (red) task.

5.4.2 Cortical Haemodynamic Activity

Following data integrity checks, 257 channels (out of 1008) data were excluded due to saturation artefacts and 7 subject sessions out of 42 were excluded due to optode movement. An example of a sample time course is shown in figure 5.10. This depicts changes in relative concentrations of HbO2 and HHb consistent with task-evoked activation. From visual inspection of this data, task-evoked activation is more prominent under GCMC assistance. Block averaged haemodynamic data from this subject is depicted in Figure 5.11 for all 24 channels under both task conditions. This indicates that activation appears more prominent under GCMC assistance.

5.4.2.1 Influence of GCMC Assistance on Cortical Haemodynamic Behaviour

Groupwise statistical analysis of task-evoked activation accompanying performance with or without GCMC assistance are illustrated in Figure 5.12. It is apparent that when undertaking the task with GCMC assistance, subjects activated a spatially broader PFC region: control versus GCMC: 1/24 channels versus 7/24 channels. The cortical response is lateralised to the right PFC and is a predominantly medial PFC in location.

Outcomes of the REM analysis are depicted in tables 5.2 to 5.4. With regard to ΔHbO2, session (control versus GCMC) was the only predictor for change in this Hb species. The coefficient is positive indicating that the magnitude ΔHbO2 is greater under GCMC assistance. Although not reaching significance, channel location did show a trend towards predicting ΔHbO2 (p = 0.056). ΔHHb is significantly influenced by the laterality of the PFC response (p = 0.037) indicating that the magnitude ΔHHb is greater in channels

128 overlying the left PFC. A decreased ΔHHb is associated with greater cortical activation and therefore this response fits with the lateralisation of the response depicted in the activation matrix (Figure 5.12). In tandem with the ΔHbO2, ΔHbT is significantly influenced by session indicating a greater response with GCMC. However, this chromophore is significantly influenced by PFC laterality (p = 0.026).

Figure 5.10 Sample time course from representative subject under control (panel a) and GCMC (panel b) conditions. Task and rest periods (green and white vertical bars respectively) are indicated. It can be appreciated that more marked task-evoked activity is demonstrated under GCMC assistance. This is predominantly indexed by an increase in HbO2 (red line); however the task-related decrease in HHb (blue line) is less prominent with the exception of tasks 3 and 5.

5.4.3 Systemic Effect and Stress

Tables 5.5 and 5.6 depict the influence of task condition on mean HR and HRV. HR was greater whilst subjects were performing under GCMC assistance however, this did not extend to a modulation of HRV according to mode of task execution (GCMC versus control). Accordingly, it is possible that the manner in which subjects undertook the task may have influenced their systemic circulation; however, it did not induce stress in terms of HRV.

With regard to STAI response, due to the timings of the questionnaire, it was not possible to differentiate whether one mode of assistance was more stressful; however, it was able to indicate whether subjects found the study as a whole subjectively stressful. The STAI responses for all subjects were 21, 20 and 22 (median values out of 24 for pre-, during and post-study questionnaires respectively). This was a significant change across all

129 groups (p=0.001). Post hoc analysis demonstrated that the decrease from pre- to during study was not significant (p = 0.230); however the during to post-study and pre- to post- study responses were (p = 0.002 and 0.014 respectively).

In sum, it can be determined that operating with GCMC assistance has affected the systemic circulation and users possibly found the study as a whole subjectively stressful. However, with regard to the influence of this on cortical haemodynamics: it can be appreciated from Tables 5.2 – 5.4, that stress in terms of HRV was only a predictor for ΔHHb (p = 0.021).

Figure 5.11 Figure depicting haemodynamic behaviour for a single subject for control and GCMC assistance (a and b respectively) at each of the 24 channel lcoations. Haemodynamic behaviour is averaged across all 5 blocks of the task. The mean and SD of the HbO2 and HHb (M x cm) signals (red and blue lines and shaded regions respectively) can be appreciated. The task period (green vertical bar) is associated with more task-evoked activity during the GCMC assisted condition. Black numbers represent channel number and the x axis is time (samples).

130

Figure 5.12 Activation matrix for subjects under control (left) and GCMC (right) conditions. Channels demonstrating a coupled increase in HbO2 and decrease in HHb in which both species reach significance are displayed red. Channels demonstrating a coupled increase in HbO2 and decrease in HHb with one species reaching significance are depicted pink. Remaining channels are displayed yellow. A predominantly right lateralised response can be appreciated and this is spatially broader under GCMC assistance.

ΔHbO2 Variable Coefficient s.e. z P>z 95% C.I. Session 0.768 0.314 2.45 0.014 0.154 to 1.383 Channel -0.078 0.041 -1.91 0.056 -0.159 to 0.001 Laterality 0.993 0.567 1.75 0.080 -0.118 to 2.014 HR 0.005 0.015 0.32 0.748 -0.024 to 0.034 HRV -0.023 0.016 -1.45 0.148 -0.054 to 0.008 Constant -0.009 1.136 -0.01 0.994 -2.235 to 2.217

σu 2.314

σe 3.872 rho fraction of variance 0.263

due to uj

Table 5.2 Results of multivariate random effect model for ΔHbO2. Laterality refers to left or right PFC. Significant p values are highlighted in bold.

131

ΔHHb Variable Coefficient s.e. z P>z 95% C.I. Session 0.094 0.132 0.72 0.474 -0.164 to 0.352 Channel -0.006 0.017 -0.35 0.729 -0.040 to 0.028 Laterality 0.502 0.241 2.08 0.037 0.029 to 0.974 HR -0.007 0.006 -1.21 0.225 -0.018 to 0.004 HRV 0.012 0.006 2.02 0.044 0.001 to 0.023 Constant -1.049 0.443 -2.36 0.018 -1.919 to -0.179

σu 0.793

σe 1.640 rho fraction of variance 0.190

due to uj

Table 5.3 Results of multivariate random effect model for ΔHHb. Laterality refers to left or right PFC. Significant p values are highlighted in bold.

ΔHbT Variable Coefficient s.e. z P>z 95% C.I. Session 0.872 0.370 2.36 0.018 0.147 to 1.598 Channel -0.084 0.048 -1.74 0.081 -0.179 to 0.010 Laterality 1.496 0.670 2.23 0.026 0.183 to 2.808 HR -0.004 0.018 -0.23 0.817 -0.039 to 0.030 HRV -0.008 0.019 -0.43 0.666 -0.045 to 0.029 Constant -1.120 1.337 -0.84 0.402 -3.740 to 1.500

σu 2.699

σe 4.560 rho fraction of variance 0.259

due to uj

Table 5.4 Results of multivariate random effect model for ΔHbT. Laterality refers to left or right PFC. Significant p values are highlighted in bold.

132

HR Variable Coefficient s.e. z P>z 95% C.I. Session 7.985 0.720 11.09 0.000 6.574 to 9.396 Constant 52.338 6.583 7.95 0.000 39.436 to 65.240

σu 27.504

σe 9.455 rho fraction of variance 0.894

due to uj

Table 5.5 The influence of study group (Control versus GCMC) on HR. Significant p value highlighted bold.

SDRR Variable Coefficient s.e. z P>z 95% C.I. Session 0.532 0.484 1.10 0.272 -0.418 to 1.481 Constant 49.405 8.209 6.02 0.000 33.317 to 65.494

σu 34.641

σe 6.352 rho fraction of variance 0.967

due to uj

Table 5.6 The influence of study group (Control versus GCMC) on HR. Significant p value highlighted bold.

5.5 Conclusion

This purpose of this chapter was to employ a neuroergonomic paradigm in order to assess the impact of using GCMC on subject neurocognitive behaviour. The task required users to track a moving target on the surface of a simulated beating heart. The target oscillated necessitating continuous monitoring and performance adjustment. The task undertaken was repeated with the exception of the addition or withdrawal of GCMC assistance. Performance was better with GCMC; however this was associated with a different pattern of cortical activity characterised by a greater degree of right sided PFC activity. It is possible that the task itself induced some subjective stress; however, the impact of the systemic circulation and stress response on cortical haemodynamics was unlikely to have been substantial.

133 This result is important as it demonstrates that ostensibly GCMC is beneficial; however, enhanced task execution may be founded on an increase in attentional resources of the user. When enabled, GCMC forces the subject to fixate on the target. As a task is acquired eye behaviour switches from following the tool to leading it [230]. Presently, subjects are relatively task naïve, yet are required to adopt a distinct strategy to localise the target than they would at this early stage of the learning process. This aspect may require greater attentional control and therefore PFC involvement. In order for subjects to be successful in this task, it was imperative that their gaze behaviour remained on the target. As discussed above this is a strategy that they may not be used to at this stage in the learning process. Accordingly, this (coupled with the continuous performance monitoring) is likely to have necessitated user vigilance. Increasing time on task and vigilance is associated with increased right PFC activity [70]. This may explain the markedly lateralised response.

Throughout the task, the target was constantly moving necessitating continuous positional adjustment of the virtual instrument. The medial PFC is implicated in performance monitoring [134, 223] and therefore is likely to be integral to this process. Supplementary to this, using GCMC is likely to impact on subject visual behaviour. Under the control condition, the target, which was visually salient within the scene, was likely to be detected with a ‘bottom-up’ search, leaving the subject to tend to the manipulation of the tool. Whereas, with GCMC enabled, users had to direct their gaze to, and subsequently pursue the target. Therefore this is likely to employ more ‘top-down’ control of the search which necessitates greater extra-striate cortical activity in centres including the PFC [233]. This approach seemingly appears to increase the cognitive demand of the task. However, once the target is fixated upon, GCMC ‘takes over’ meaning that the user does not have to translate the target locus from visual to motor coordinates therefore making the task easier.

In the previous chapter, a broader area of PFC activity was demonstrated in those with greater endoscopic experience and who subsequently achieved superior performance. It was hypothesised that an initial task-naïve phase exists in which subjects are unable appropriately direct resources to the task in question. Once initial challenges of task execution are explored and determined, it is thought that effortful yet effective PFC dependent performance is possible. Subsequently, this evolves to PFC independence as expertise is acquired [109, 110, 234]. Placing the findings of the current chapter in this context, it is plausible that in the control task, little PFC activity is present as users have

134 not yet defined an approach to execute the task successfully. Accordingly, performance is less accurate. However, with GCMC enabled, subjects are required to employ the ‘expert’ strategy of fixating on the target. Thereafter, performance is enhanced. A potential limitation with activity-based analysis such as that utilised in the present chapter, is that some of the information from the signal is lost, namely the time to peak and time to nadir of the Hb species. Alternate analysis strategies such as statistical parametric mapping have been applied to NIRS data [235] and do enable this aspect of the signal to be incorporated into the analysis.

Haptic guidance has been used to teach subjects novel movements [236]. Therefore learning with haptic constraints may also provide a safe environment for task acquisition during early, error prone phases of learning. Subsequently, performance may remain improved when the constraint is removed. This lends weight to the potential of GCMC in instructing trainees alongside assisting those who are experts. It is also noteworthy that GCMC did not appear to significantly induce subject stress and that systemic circulatory effects did not appear to overtly influence cortical haemodynamics. This is relevant as surgery can be stressful and introducing equipment that may exacerbate this may be detrimental to the surgeon.

In the current study, a neuroergonomic approach to the evaluation of a novel technology has been undertaken. In order to do this subject brain behaviour was determined at a single time point undertaking the task under both conditions. This has in part begun to investigate how GCMC exerts its effect. However, an important aspect is how a tool such as this impacts on the acquisition of the task across time. Specifically how learning- related plastic changes are impacted upon the addition of assistive technology. A visuomotor tracking task as employed here is likely to be reliant on other cortical regions such as the parietal lobe. Therefore in order to further elucidate the impact of this performance-enhancing instrument on the user, in the ensuing chapter, GCMC is investigated in greater detail across task learning and in relation to multi-regional cortical network activity.

135 Chapter 6

The Influence of Gaze Assisted Force Feedback

Work from this chapter has been published: James DR, Orihuela-Espina F, Leff DR, Mylonas GP, Kwok K-W, Darzi AW, Yang G-Z. Cognitive burden estimation for visuomotor learning with fNIRS. Lecture Notes in Computer Science. 2011; 13(Pt3): 319-26.

6.1 Introduction

In the previous chapter, PFC activity associated with performing a visuomotor tracking task with and without the assistance of GCMC was investigated. Whilst a modulation in brain behaviour with this tool was noted, it raised the questions of how this assistive technology may influence the learning process and how it may impact on inter-regional cortical communication. The purpose of this chapter is to investigate the impact of GCMC on task-learning in terms of frontoparietal (F-P) network behaviour. Regarding complex motor skills, task learning is associated with neuroplastic changes within the brain that lead to up-regulation or down-regulation of regional activity [109] and a modulation of the underlying cortical network from a novice to expert state [117]. Longitudinal changes in regional brain activation commensurate with skills acquisition are well described and comprise attenuated responses in the PFC and increased activation in primary and secondary motor regions and the cerebellum consistent with a re- distribution or re-organisation of the activation map [109, 112, 113, 224, 237]. In contrast, there have been few reports characterising temporally evolving functionally connected network architectures.

Only a few studies of motor learning suggest that functional connectivity varies from novice to learned states [148, 238, 239]. This data suggests that early phases of motor learning are associated with greater connectivity with frontal regions compared to late phases [148] and that changes in network integration [238] and modular flexibility [179] accompany skills learning. However, variation in connectivity is likely to be dependent

136 on brain regions of interest and strengthening of connectivity between motor regions (e.g. cerebellum and primary motor cortex) has been observed [239]. Importantly, the majority of these studies incorporated a singular training regimen and were not designed to characterise differences in network behaviour evoked by different training regimens or modes of learning. However, for high risk industries in which motor skills acquisition and performance accuracy are integral to public safety (e.g. aviation, surgery, etc.) it is attractive to compare the influence that tailored training regimens and technologies designed to support or assist learning have on the speed of skills acquisition, evolution in regional activation(s) and efficiency of brain networks. This study addresses the limitations of the current literature and leads on from the preceding chapter by investigating the influence of task acquisition with and without GCMC on learning- related evolution in both functional activation(s) and network communication. In order to realise this, graph theory which enables networks and their properties such as small- worldness to be characterised is utilised.

Cortical systems share features with other complex networks (e.g. biological and social) despite differences at the microscopic level [161]. Specifically, cortical networks demonstrate ‘small-world’ properties [163-165], which are characterised by high local connectivity or cliquishness coupled with long range connections linking distant network regions [164]. Small-worldness corresponds to the brain’s capacity for local processing (functional segregation) coupled with an ability for multi-regional communication, integral for task execution (functional integration) [142, 170].

As discussed in section 3.7, graph theory has been applied extensively to neuroimaging data derived from EEG [172, 193, 194], MEG [173, 182, 192] and both functional [165, 178, 179] and anatomical [176, 180, 191] MRI data, but has not been applied to fNIRS data. However, a brief review of graph construction [161] suggests that this method is appropriate for investigating brain connectivity from data captured using optical methods.

Graph construction is outlined in Figure 6.1. First network ‘nodes’ are defined, which may represent sensors or electrodes (e.g. MEG and EEG), individual voxels [177, 188], cortical volume [180] or anatomical regions of interest [169, 190] (e.g. MRI). Concerning fNIRS experiments, nodes may represent optical data acquired at different cortical loci (‘channels’). Following node definition, network ‘edges’ are calculated by generating a functionally associated matrix, determined using methods such as correlation [176, 183, 190], partial correlation [180, 187], mutual information [173, 192], synchronisation

137 likelihood [172, 182], and coherence [193]. This matrix is a measure of functional connectivity between the nodes [147]. Next the association matrix is pruned by applying a threshold in order to retain the important functional connections. Thresholding techniques vary and include the significance of the functional association [190], the magnitude of the correlation r value [183, 193], the average degree of nodes within the graph [194]; the connection density of the graph [173, 176, 180, 184-186] or a combination of these approaches [165]. Network econometric data are subsequently derived from the pruned adjacency matrix. The reproducibility of results analysed using graph theory provides additional confidence that the metrics are sufficiently reliable to assess changes in network configurations over time [192]. However, comparing graph econometric data at a subject or group level is a challenging area of research [200] and care needs to be taken to ensure that networks are of equal connection density and nodal number [161, 174, 196, 200]. An alternative method is to compare the graph to an equivalent ‘random graph’ in order to derive the extent of network small-worldness [164].

Figure 6.1 Flow diagram describing the stages of graph construction from node definition to econometric analysis.

In comparing graphs, it is possible to assess the impact of independent variables on the derived econometric data. This has been used to assess the effect of Alzheimer’s disease [172], epilepsy [176], dopamine blockade [178], schizophrenia [180, 185], working memory task performance [173, 192], arithmetic [194], intelligence [193] and motor task performance [179, 183] on graph properties. Graph theory and ensuing metrics can quantify cortical network behaviour in disease [180] and under pharmacological treatment [178] and therefore may serve as a means to monitor the effect of drugs on the central nervous system or disease course [161]. Similarly, neuroergonomics [9] is concerned with assessing the brain and behaviour in the work environment; and clearly the ability to determine whether an intervention has benefited the user at brain level would be invaluable. As summarised in section 2.4.2.1, much research has focussed on measuring mental workload. In relation to NIRS research, increasing PFC activity has been associated with increasing cognitive effort [76, 80, 82]. However, as has been realised in Chapters 5 and 6, a greater PFC response was associated with improved performance and may not necessarily reflect mental workload. Accordingly, as

138 hypothesised in section 3.8.3, the term ‘cognitive burden’ is defined in relation to the most efficient use of cortical resources and any deviation thereof. In this setting, graph theory metrics are utilised in order to quantify this concept.

Presently, visuomotor tracking is undertaken with or without GCMC by subjects across 6 sessions on separate days. Following randomisation, subjects practiced the task at each session either with or without the presence of GCMC. When this tracking task is undertaken with GCMC [228], user fixation point is determined in real time and used to exert force feedback to the subject via the haptic manipulator used to control the virtual tool during the task. As demonstrated in the preceding chapter, this aids localisation of the virtual tool to the moving target and leads to a distinct pattern of PFC behaviour indexed by an increased right sided PFC response.

A randomised controlled longitudinal study design is employed in order to determine the influence of GCMC on the evolution of both F-P activation and F-P network communication. Task-naïve subjects were randomised to acquire the task with or without GCMC whilst changes in F-P cortical haemodynamics were detected using fNIRS. Graph theory was applied to cortical haemodynamic data in order to capture the extent of small- worldness of the task-evoked cortical network and to determine if gaze-assistance confers a benefit in terms of efficiency of F-P processing.

Firstly, it is hypothesised that across the six sessions of the task, there will be an evolution in the magnitude of cortical activity within F-P brain regions. As subjects progress from task naivety to expertise, PFC attenuation is predicted consistent with reduced attentional demands accompanying task acquisition [106, 109, 110, 112, 113, 237]. Localisation of the parietal response to the PPC is anticipated commensurate with development of an internal model of task performance [121]. Secondly, a small-world F- P network is predicted which may be modulated with learning-related changes in technical skills acquisition [148, 238]. Thirdly, it is envisaged that learning-related evolution in F-P activation(s) and network architecture may progress more rapidly to the trained state in gaze-assisted learners, since GCMC stabilises user performance and may result in earlier attainment of technical expertise. Finally, network econometrics have been shown to predict superior performance [173], thus it is conceivable that GCMC, in leading to enhanced performance, will reduce network cost whilst improving network efficiency and economy and accordingly reduce the task-induced cognitive burden.

139 6.2 Materials and Methods

6.2.1 Subjects

Following LREC approval, a randomised single blinded trial was conducted. 21 subjects (6 female) with no prior nor current history of neuropsychiatric conditions were recruited from Imperial College London [Mean age  S.D. = 21.2  2.4 years]. Subjects were right handed [Edinburgh handedness inventory [206] score median (range) = 70 (40 – 100)]. Subjects gave written consent prior to participation. All participants were task naïve. Exclusion criteria included a history of or current neuropsychiatric illness, left handedness or prior task experience. Subjects were required to refrain from alcohol and caffeine for 24 hours prior to each data collection point, in order to eliminate their effects on cerebral haemodynamics during the study.

Figure 6.2 Subject performing task. The haptic manipulator (right hand) is used to control the virtual tool whilst gaze behaviour is detected with the portable eyetracker (beneath the monitor). Optodes can be appreciated overlying the left PFC and PC. Figure used with kind permission of subject.

6.2.2 Task and Training

Subjects performed a virtual cardiac ablation task on a simulated beating heart. The task design is outlined in detail in section 5.2.2. Briefly, as illustrated in Figure 6.2, a haptic manipulator (SensAble Technologies, USA) was used to control a virtual tool with the aim of localising the moving target as accurately as possible. Subjects were randomised

140 to perform the task either free hand in the conventional manner (control group) or with robotic assistance (experiment group) in the form of GCMC [228]. With GCMC enabled, subjects performed the task with the exception that user gaze behaviour was extracted and used to guide force feedback via the haptic manipulator in order to constrain the subject’s hand to their fixation point and thus to the target. This technology has been demonstrated to improve operative accuracy in contrast to free hand performance [228].

A block design paradigm was employed with each ‘session’ comprising five blocks of task (20s) interspersed with rest (30s). The purpose of this study was to assess longitudinal changes associated with the acquisition of this task with or without robotic assistance; therefore, subjects repeatedly performed the task on six separate days, resulting in six sessions per subject. A pilot study determined six sessions to be sufficient to demonstrate learning of the task as illustrated in Figure 6.3. All participants received a standardised introduction which precluded any opportunity to practice, ensuring that the most naïve phase of learning was captured. Subjects were blinded to group assignment and to the study objectives. Subjects were recalled for a retention test (session 7) to ensure task learning approximately two months after the original test.

Figure 6.3 Pilot data across 6 sessions of performance accuracy for control (blue) and GCMC (red) users. Error bars indicate 95% CI.

Performance was determined as the distance (pixels) from the tool tip to the target and was averaged over the 5 blocks of the task to yield a session average. Based upon pilot data, the study was powered to detect a difference of 1.5 SD in technical performance between groups at  = 0.05 and 90% power, which required 11 subjects in each group.

141 6.2.3 Randomisation

A random number generator assigned participants to either free hand (control group) or to robotic assistance (experiment group) for the duration of the trial.

6.2.4 Blinding

Subjects were blinded to group assignment and the other group’s task. Due to the delivery of force feedback, it is likely that subjects in the experimental group were aware that they were receiving robotic assistance. There was no practice between sessions for the duration of the study. Learning was self guided and free from external influence.

6.2.5 Functional Near Infrared Spectroscopy

Left F-P activity in terms of relative changes in HbO2 and HHb was detected with fNIRS (ETG4000, Hitachi Medical Corp., Japan) at 24 channel locations as depicted in Figure 6.2 and schematically represented in Figure 6.4. fNRIS setup was identical to the previous experiment, detailed in section 5.2.4, with the exception of optode placement. Channels 1 – 12 were positioned overlying the left PC and Channels 13 – 24 were positioned over the left PFC in relation to the UI 10/20 and 10/10 systems as depicted in Figure 6.4.

Figure 6.4 Approximate optode location obtained by transferring topographic data from a representative subject to a 3D cortical surface of an MRI atlas for left PFC (a) and left PC (b). Channels (yellow numbered circles) are displayed in relation to UI 10/20 and 10/10 locations [210]. Optode emitters and detectors (red and blue circles respectively) can also be appreciated.

6.2.6 Systemic Effect and Stress

Simultaneous HR recording was undertaken with a portable electrocardiogram (Bioharness, Zephyr Technology, USA). HR data was used to derive the HRV in terms of the SDRR [43] as a surrogate for systemic effect and task-induced stress. Concurrently,

142 subject stress was determined with the STAI [56] in order to determine the influence that the systemic circulation and autonomic arousal had on cortical haemodynamics.

6.3 Data Analysis

6.3.1 Behavioural Data

The influence of group and session on performance was determined with a univariate analysis and post hoc Bonferroni correction (SPSS v18, USA). Performance at session 6 was compared with that attained during the retention test (Wilcoxon rank sign) to ensure learning of the task.

6.3.2 Cortical Activation

Raw optical data was linearly detrended and decimated to 1 hertz to remove system drift and physiological noise. Data was then subject to integrity checks in order to detect saturation related problems as detailed in Chapter 4, prior to conversion to relative changes in HbO2 and HHb using the modified Beer-Lambert Law [101]. Cortical activation was defined as a task-related increase in HbO2 coupled to a decrease in HHb (Wilcoxon rank sign). For each Hb species, this was defined as a statistically significant (p<0.05) change between baseline rest (5s data prior to stimulus onset), and task averaged data. This was derived from group averaged data, thereby characterising the activation pattern of the control (free hand) and experimental (GCMC) groups. A variable, ΔHb was derived for each HbO2 and HHb for each channel of data, calculated as the difference between the task and baseline data (as defined above). ΔHb data was incorporated into a REM (Intercooled Stata, v10.0 for windows, Stata Corporation, USA) in order to appreciate the influence of group (control versus experimental) and practice session (1 - 6) on the task-induced changes in Hb data. The REM was also used to determine whether region (PC versus PFC) or sub region (inferior / superior parietal and lateral / medial PFC) influenced ΔHb.

6.3.3 Graph Theoretical Analysis

Graph construction is outlined in Figure 6.5. Two approaches to graph construction were employed. Firstly, subject-specific networks pruned according to channel activity were utilised. Secondly, group averaged data was used and networks were pruned according to graph connection density. These processes are detailed below. The motivation for this approach was a desire to investigate the feasibility of incorporating the activation status of channels into the process of network generation. Thereby channels not involved in the task statistically would not influence the ensuing network.

143

Figure 6.5 Flow of data analysis and graph construction. Following pre-processing and conversion to relative changes in HbO2 and HHb, haemodynamic data is averaged across the five task blocks and over all subjects in each group for each session. This yields a grand averaged timecourse (1). The five sample baseline is compared with the task-averaged Hb value in order to determine task-evoked increases/decreases in HbO2 and HHb. This is subsequently displayed overlying channel locations on a reference MRI atlas (a). For graph generation, both Hb species in the grand averaged time course are bidimensionally cross-correlated in order to generate the association matrix (2). In turn, this is pruned to generate the adjacency matrix (3) using the threshold. In the current study, the association matrix is pruned according to the connection density of the graph with least significant connections removed first. From the adjacency matrix (3), the undirected, weighted cortical network (4) is generated. In order to generate subject-specific cortical networks, stages of analysis are similar with the exception that individual subject time courses are cross-correlated and the association matrix is subsequently pruned according to the relevant subject specific activation matrix.

144 6.3.3.2 Graph Construction

Firstly, the subject-specific approach is discussed. Each haemodynamic timecourse was cross-correlated in order to generate a functionally connected unweighted, bidirectional

graph. Bidimensional cross-correlation Ri, j t, Hb was utilised as shown in Equation 6.1 such that both changes in HbO2 and HHb were simultaneously considered in determining the functional association between two channels. Where t and Hb represent the temporal and haemodynamic lag among the signals and subscripts Hb only refer to the haemodynamic signal index and t to the temporal sample index with T being the length of the signals in samples. Over lined symbols represent mean signal value.

6.1

This generates an association matrix for each subject at each time point. The association matrix is then pruned according to the subject-specific activation matrix (t-test,  = 0.05). Four patterns of cortical activity were considered:

 (A) ΔHbO2 increment and ΔHHb decrement both reaching statistical significance,

 (B and C) ΔHbO2 increment and ΔHHb decrement with only one species reaching significance

 (D) ΔHbO2 increment and ΔHHb decrement with neither species reaching significance.

A threshold was then applied to the connected graph, permitting only those links between any two channels representing the activity patterns (A-D), thereby capturing the evolving relationships within an activated F-P network. The strength of the edges was scaled according to the pattern of activation, therefore lending more weight to associations between significantly activating channels. Scales 1, 0.75, 0.75 and 0.5 were used for patterns A to D respectively. From this approach, a network was generated from every subject as long as they had an activating channel.

145 The second approach to graph construction involved generating the association matrix as detailed above with the exception that the data bidemensionally cross-correlated was the averaged haemodynamic behaviour for each group (control and GCMC) at each time point (1 – 6). This rationalised the econometric data to 12 graphs in total. Cortical networks are then derived from the association matrices following application of a threshold. Too lenient a threshold may result in retention of edges describing spurious associations between cortical regions and that do not truly represent functional connections. However, too aggressive a threshold may result in the rejection of meaningful cortical connections, under population of the graph and / or node isolation making graph comparison problematic [174, 200]. The graphs were scaled to ensure equivalent connection densities, thereby facilitating graph comparison. The connection density was fixed and edges were sequentially removed according to the significance of the functional association (least significant removed first) until the desired density was reached.

6.3.3.3 Graph Theory Metrics

In order to determine small-worldness, each graph was compared to its equivalent random graph [200]. This was calculated by randomly rewiring the graph as previously described [164]. Here the regular graph is built by linking each node (channel) with its topological neighbours in the optode array. From the session and group specific graphs and the equivalent random graphs, the average pathlength L and Lr and average clustering

coefficient C and C r were calculated. This in turn was used to compute the small-world index, as shown in Equation 6.2 [174]. Calculation of small-worldness was only undertaken from networks derived from group-averaged data.

CL 6.2 CLrr

Network cost KG, a measure of the number of connections in a graph in relation to the

total number of possible connections; and global efficiency EGglob , a measure of information transfer within a network; were calculated using Equations 6.3 and 6.4. Where d is the distance between nodes (,)ij and N is the number of nodes within the network. The cost-efficiency of a network is increased in an economical network [178]

146 and therefore is calculated using Equation 6.5 and termed network economy. At a fixed connection density, the number of connections linking the prefrontal and parietal channels was calculated in order to give an indication of connectivity between these two regions where i and j stand for nodes in the graph.

6.3 K() G dij| i j i G j G

11 6.4 EGglob () N( N 1) i G j G dij| i j [198]

norm norm 6.5 Economy()()() G Eglob G K G [178]

The cognitive burden was therefore calculated as the inverse of the network economy. If the cost dominates the equation, the cognitive burden will be high and conversely, if efficiency dominates the equation, cognitive burden will be low.

6.3.4 Systemic Effect and Stress

The influence of SDRR on task-induced change in HbO2 and HHb was determined with a random effects analysis (Intercooled Stata, v10.0 for windows, Stata Corporation, USA). The influence of group, session and timing (pre-, during or post-study) on the STAI questionnaire was assessed with a univariate model (SPSS v18, USA).

6.4 Results

Twenty one subjects were randomised to perform a visuomotor tracking task with / without the assistance of GCMC across 6 task sessions. Task-evoked cortical activity across left F-P regions was determined and graph theory was applied to calculate network econometrics in terms of small-worldness, cost, efficiency and cognitive burden of the resultant network and how this was modulated by learning in the presence / absence of GCMC.

6.4.1 Participant Flow

The flow of subjects is illustrated in Figure 6.6.

147

Figure 6.6 Consort diagram, indicating flow of subjects through the experiment. No subjects were excluded; however, 1 withdrew after the second session.

6.4.2 Numbers Analysed

One subject withdrew from the study after the 2nd session and two of the remaining subjects were unable to return for the retention test. The six sessions were undertaken on separate days over a median of 8 days. Following data integrity assessments [240], 127 channels (out of a total of 2880 channels, 4.4%) were excluded due to system noise or artefacts. For the activity-guided approach to network construction, only data in which all 24 channels had passed integrity checks were included in the analysis (69 out of a total of 122 sessions). However, in the group-averaged approach, following integrity checks, all remaining haemodynamic data was utilised to generate the group-average response and were accordingly included in the network analysis.

6.4.3 Behavioural Data

Performance as determined by the distance from the tool tip to the target is illustrated in Figure 6.7. It is apparent that accuracy increases with time on the task, manifest as a diminution in this distance, across the 6 sessions. Session, but not group was observed to be a predictor for task performance (p<0.05 and p=0.518 respectively). However, the combined influence of group and session did predict a change in performance (p<0.05). Whilst accuracy is better in the control group initially, the learning curves intersect after the second session as GCMC users improve to greater extent than do controls. Therefore, group alone does not predict performance unless session is concurrently accounted for. Eighteen subjects returned for the retention test following [median (range)] 71 (21 – 116)

148 days. At this stage, performance was not significantly different from that during session six (control: p = 0.4307; GCMC: p = 0.9404). This indicates retention of skill at a median of 71 days.

Figure 6.7 Subject performance across six task sessions and the retention test (session 7, which occurred a median of 71 days after session 6). Data represent mean and 95% CI (error bars) for subjects in control (blue) and GCMC (red) groups. Initially, performance is better in the control group until the learning curves intersect after the second session. Subsequently, GCMC users demonstrate improved performance. There is no significant difference between performance in session 6 and 7 in either group indicating retention of the task.

6.4.4 Cortical Activity

A timecourse from a representative subject from each group is depicted in Figure 6.8. This originates from a parietal channel and a pattern consistent with activation is apparent. Broadly, this appears to become more prominent towards the final session. Group-wise statistical analysis of longitudinal changes in cortical activity commensurate with technical skills acquisition are illustrated in Figure 6.9. Naïve performance (first session) was associated with PFC and parietal cortical (PC) activation, regardless of group (control group versus GCMC). However, rapid attenuation in PFC and PC activation was only observed in the GCMC group. In contrast, after limited practice (session 3) persistent PFC activation (channels 14, 16 and 21 – 23) and spatially broader PC activation was observed in learners randomised to the free hand group (control). Attenuation in PFC and PPC activation only occurred toward the final session in unassisted learners. Longitudinal learning-related changes in parietal excitation comprised a spatial contraction in the number of activating channels within the PPC.

149 However, the residual activation in the PPC appeared to be more intense. Longitudinal changes in PC activations transpired more swiftly in gaze assisted learners.

Figure 6.8 Session averaged haemodynamic data from representative control and GCMC subjects (a and b respectively). Mean HbO2 and HHb (red and blue bold lines) signal can be appreciated with the corresponding SD (red and blue shaded regions). The task and rest periods (green and white bars respectively) can be appreciated.

HbO2 Variable Coefficient s.e. z P>z 95% C.I. Group 0.676 0.967 0.70 0.484 -1.220 to 2.572

Channel -0.026 0.033 -0.79 0.428 -0.090 to 0.038

Session -0.137 0.057 -2.42 0.016 -0.248 to -0.026

PC/PFC 2.560 0.465 5.50 0.000 1.647 to 3.472

Sub-region -1.167 0.226 -5.16 0.000 -1.610 to -0.724

HR 0.006 0.005 1.40 0.163 -0.003 to 0.015

SDRR 0.027 0.005 5.00 0.000 0.016 to 0.037

Constant -2.384 1.487 -1.60 0.109 -5.299 to 0.530

σu 2.047

σe 11.541 rho fraction of variance 0.030

due to uj

Table 6.1 Results of multivariate random effect model for ΔHbO2. Significant p values are highlighted in bold.

150

HHb Variable Coefficient s.e. z P>z 95% C.I. Group -0.310 0.531 -0.58 0.560 -1.351 to 0.732

Channel 0.069 0.019 3.68 0.000 0.032 to 0.105

Session 0.051 0.032 1.57 0.116 -0.012 to 0.114

PC/PFC 0.455 0.265 1.72 0.086 -0.064 to 0.974

Sub-region -0.079 0.129 -0.61 0.540 -0.331 to 0.173

HR 0.004 0.003 1.41 0.158 -0.001 to 0.009

SDRR -0.001 0.003 -0.37 0.710 -0.007 to 0.005

Constant -2.376 0.816 -2.91 0.004 -3.975 to -0.777

σu 1.115

σe 6.564 rho fraction of variance 0.028

due to uj

Table 6.2 Results of multivariate random effect model for ΔHHb. Significant p values are highlighted in bold.

As displayed in Table 6.1, session and cortical region (PFC vs PC) were independent predictors of changes in HbO2. The change in HbO2 decreased as the sessions progressed, implying an overall attenuation in the magnitude of the cortical response throughout the study. Region (PC versus PFC) was found to predict ΔHbO2 with a greater response in the PFC. However, when these regions were further sub-divided a greater ΔHbO2 was predicted in parietal sub-regions. This discrepancy is likely due to the complexity of the cortical response across the six sessions in that changes in sub-regions (e.g. posterior PC increase and superior PC decrease) were demonstrated implying that gross changes across whole regions may not capture trends in smaller areas of the PFC and PC. Group was not an independent predictor for changes in cortical haemodynamics. Therefore, differences in cortical haemodynamics were mediated by temporal fluctuations across sessions and modulation according to cortical region (PFC vs PC). As displayed in Table 6.2, Channel was the only predictor for ΔHHb.

151

Figure 6.9 Groupwise statistical analysis of longitudinal changes in cortical activity across sessions 1, 3 and 6 for control (left) and GCMC (right) subjects. Approximate channel locations are indicated by colours representative of the pattern of activity: (a) ΔHbO2 increase and coupled ΔHHb decrease (both reaching significance); (b) ΔHbO2 increase and coupled ΔHHb decrease (one reaching significance); (c) ΔHbO2 increase and coupled ΔHHb decrease (neither reaching significance); (d) No paired increase in ΔHbO2 decrease in ΔHHb. Attenuation in PFC activity and focussing of activity within the PPC can be appreciated.

6.4.5 Frontoparietal Network Activity

Two approaches to network generation were adopted. Initially, subject-specific activity thresholded networks were created. Figure 6.10 displays the F-P networks for representative subjects under control and GCMC conditions generated using this technique. Accordingly, Figure 6.11 displays the progression of econometric parameters across task learning. A limitation with this approach is that 69 networks with varying node number and connection density are generated. Drawing comparison between these networks is problematic [200]. However, upon inspection of learning-related changes in

152 network econometrics (Figure 6.11), it is apparent that early in the learning process, a lower efficiency is demonstrated and this increases throughout task acquisition to a state higher efficiency and low cost at learning termination.

In response to the limitations of the activity-guided approach to graph generation, the second strategy was adopted in which networks were generated from group-averaged data and were normalised for connection density in order to aid comparisons between graphs [174, 200]. If the connection density is too low, the graph may become disconnected and yet overpopulating a graph may impair visualisation of small world properties. Small- world behaviour has been visualised best at a connection density from 0.05 to 0.34 [178]. Therefore a connection density of 0.355 was selected. At this level, some nodes were isolated from the network. However, this was only apparent in sessions 3 and 4 (1 and 2 nodes isolated respectively) in the control group and sessions 1, 3 and 5 (3, 5 and 5 nodes isolated respectively) in the GCMC group. Accordingly, a lower threshold was not selected and above this value small-work behaviour is not likely to be realised [178]. Networks generated by this approach are depicted in Figure 6.12.

Figure 6.10 Longitudinal changes in F-P networks for representative control (top) and robotic assisted (bottom) subject. Approximate channel locations (black circles) are nodes within the graph and connections (red) are weighted according to their strength. More connections are apparent with robotic assistance. It can be appreciated that the number of connections varies with session.

153

Figure 6.11 Evolution in performance and econometric parameters across practice for control (red) and GCMC (blue). By session 6 both groups have migrated to a region indicative of a high efficiency and low cost.

Figure 6.12 Longitudinal changes in F-P networks for representative control (top) and robotic assisted (bottom) subject. Approximate channel locations (black circles) are nodes within the graph and connections (black) are weighted according to their strength. It is apparent that a greater number of inter-regional connections between the PFC and PC are present in earlier tasks.

154

Figure 6.13 Cortical network econometrics for control (blue) and GCMC (red) subjects across the 6 task sessions in graphs normalised for connection density. It can be appreciated that cognitive burden and efficiency is improved with GCMC and cost is lower. The small-world index is greater with GCMC and both groups display a small-world index > 1 until the final session. Efficiency, economy and the small-world index display a decreasing trend across the six sessions and cost increases. This occurs in tandem with attenuation in PFC activation.

The evolution of cortical network econometrics with and without gaze-assistance utilising this approach are displayed in Figure 6.13 and Table 6.3. It is apparent that the normalised global efficiency, normalised cost and cognitive burden of the network are improved with GCMC assistance. This implies that gaze-assisted learners developed a pattern of F-P network communication that was more efficient, economical and less costly than that of free-hand learners. Econometric data of average clustering coefficient (C ) and mean pathlength ( L ) are displayed in Table 6.3. These are normalised according to the equivalent random graph [164] and are used to calculate the small-world scalar: . Small-worldness behaviour is present when 1 . This is apparent in all sessions apart from the 6th session in both groups. The small-World index is also greater under GCMC assistance.

The loss of small-worldness may relate to an increase in mean pathlength coinciding with a simultaneous marked reduction in the number of functional connections between the PFC and PC in the final session in both groups (Table 6.3). Moreover, final session PFC

155 redundancy is evident from the activity-related analysis (see Figure 6.9). It is conceivable that as activity within the PFC attenuates, its functional association with the PC may well decrease, especially as activations within regions of the latter (i.e. the PPC) are magnified. This effect occurs in tandem with a trend towards a reduction in network efficiency, economy and an increase in cost towards the final session accompanied by the loss of small-worldness.

Session ACC* Mean Small- No. Norm. Norm. Cognitive PL* World F-P Global Cost Burden Index Conn- Efficiency ections 4.207 2.937 1.432 41 0.00792 0.203 0.195

1 2 3.742 3.252 1.151 43 0.00735 0.233 0.226

3 4.049 3.479 1.164 40 0.00695 0.249 0.242

4 4.001 3.509 1.140 35 0.00722 0.241 0.234

5 3.171 2.938 1.057 35 0.00755 0.221 0.213 Control 6 3.481 4.071 0.855 25 0.00671 0.263 0.256 1 3.262 1.954 1.670 33 0.00878 0.153 0.144 2 3.861 2.757 1.401 43 0.00822 0.205 0.197 3 4.557 2.601 1.752 42 0.00880 0.155 0.146 4 3.317 2.671 1.242 36 0.00726 0.236 0.229

GCMC 5 4.612 2.386 1.933 39 0.00825 0.198 0.190 6 3.593 4.308 0.834 27 0.00718 0.248 0.241

Table 6.3 Network econometrics of average clustering coefficient (ACC) and mean pathlength (PL) normalised to equivalent random network and utilised to calculate small-world index, normalised (norm) global efficiency, normalised cost, economy and the number connections between prefrontal and parietal regions.

6.4.6 Systemic Effect and Stress

Mean HR was not a predictor for variation in ΔHbO2 or ΔHHb. However, SDRR was a predictor for ΔHbO2 (p < 0.05). With regard to the STAI questionnaire, group and timing (pre-, during, post-study) were predictors of a change in response (p<0.05), and post hoc analysis demonstrated a significant increase from during to post study response. STAI response was (median out of 24) 21.5 and 23 for control and GCMC respectively indicating the potential of more subjective stress in control users. Session was not a predictor for change in response. In sum, HR alone was not a predictor in change in cortical haemodynamics. However, stress in terms of HRV may have influenced brain behaviour but stress response in terms of the questionnaire was stable over the 6 sessions and therefore stress is unlikely to be solely responsible for dynamic changes in brain behaviour.

156 6.5 Conclusions

In the present chapter, left F-P cortical activity has been evaluated in subjects throughout task learning with or without GCMC. fNIRS was utilised to detect local changes in cerebral haemodynamics associated with brain activation. The primary findings are that gaze-assisted motor learning leads to greater improvements in technical skill, more rapid evolution in regional cortical activation indicative of task internalisation, and a more efficient / economical F-P network architecture than free hand learners. Additionally, a loss of small worldness toward practice termination regardless of the mode of learning has been observed. Learning-related loss of small-world properties is understood in the context of a last session increase in the mean pathlength coupled to a reduction in F-P connectedness and simultaneous PFC redundancy. It is conceivable that as involvement of the PFC wanes, so does it’s functional association to the PC, thereby reducing the number of connections and increasing the path required to circumnavigate the network.

These results exhibit congruity with established theory of learning related neuroplasticity which suggests that expertise development is associated with attenuation in PFC activity [106, 109, 110, 224]. One theory is that attention and control centres in the PFC are necessary to support novel task demands during unskilled, effortful performance but are no longer necessary and subsequently ‘pruned’ as a result of practice dependent changes in executive control [110]. The residual PPC activity is indicative development of an internal model of the task [121], dynamic monitoring of arm movements [241] and sensorimotor integration [137, 242]. Therefore, longitudinal redistribution of activation comprising PFC down- regulation and PPC up-regulation suggests expertise acquisition, development of an internal model of the task and residual reliance on the PPC for effectively translating gaze detected target location into precise tracking.

Moreover, it is important to appreciate variation in cortical network behaviour in order to better understand learning-related changes in brain integration associated with skills acquisition. Moreover, there is evidence that modular flexibility within a network may predict subsequent performance whereas activation changes alone may not [179]. The findings in the current study are in line with other research demonstrating reduced connectivity between the PFC and other brain regions in conjunction with motor task learning [148] and reduced cortical network integration with time on task [238]. However, unlike the training protocols in these studies which assessed network behaviour at only two [148] or three time points [238], the current study assessed cortical network behaviour at each and every task session. A more comprehensive analysis such as this

157 enables practice-related changes in brain behaviour to be delineated more precisely. Furthermore, this enables behaviour between the two groups to be contrasted in order to determine whether one paradigm enhances learning at a cortical level. In this study, subjects performing with GCMC were more accurate at learning termination and demonstrated modulation of their cortical activity at an earlier stage in the process of learning manifest as earlier PFC attenuation and focussing of activity within the PPC.

As discussed, a potential application for graph theoretical analysis of cortical networks is to the paradigm of neuroergonomics. In this setting, brain behaviour is studied in order to appreciate the cortical resources required during task execution to ascertain whether or not assistive technology is helpful in ameliorating the burden of task demands. Importantly, in the current study, technology capitalising on the user’s gaze behaviour to constrain motor performance enhanced the progression toward an ‘expert’ pattern of brain activation and did so with an apparent improvement in network economy and a reduction in network cost and normalised global efficiency. This was also associated with a lower cognitive burden under GCMC guidance. The behavioural corollary of improved network architectures are superior learning capabilities manifest as more rapid gains in technical performance.

6.5.1 Methodological Considerations

This study demonstrates one of the first applications of graph theory to brain behaviour as detected with fNIRS, imposing certain methodological considerations. One advantage of fNIRS is the ability to quantify relative changes in both HbO2 and HHb, thereby capturing greater complexity in terms of haemodynamic relationships between brain regions than other imaging modalities.

Two approaches to graph generation were employed in this chapter. Conceptually, the primary approach in which channels were included in the network appears favourable as it is only determining functional associations between regions that are meeting the criteria for task-involvement. However, as was demonstrated, this approach led to multiple graphs of varying connection density and node number that are problematic to compare. Accordingly, the second approach was adopted which is more congruent to established graph theoretical analysis of neuroimaging data [161]. However, this method incorporates all nodes into the network and hence may include spurious connections into the network.

158 In summary, this study is the first to utilise fNIRS to understand cortical networks using graph theory. Subjects performing with GCMC were faster to acquire tracking skills than free hand learners and developed a pattern of cortical excitation that more rapidly progressed to the expert trained state. GCMC not only improved technical performance but resulted in a brain network whose architecture facilitated more efficient functional communication. This finding highlights the potential of this strategy for analysing functional neuroimaging data acquired under varying conditions such as with or without assistance, in order to characterise the effects of assistive technology on the user’s brain within the paradigm of neuroergonomics. This technique is applied again in the following chapter in order to investigate the impact of assisting collaboration within a robotic surgical task.

159

Chapter 7

Collaborative Gaze Control for Surgical Robotics

Work from this chapter has been presented: James DRC, et al. Neuroergonomic assessment of collaborative gaze control for robotic surgery: a functional near infrared spectroscopy (fNIRS) study. Hamlyn Symposium for Robotic Surgery. Royal Geographical Society, London, 2011

7.1 Introduction

In the preceding experimental chapters, principles of neuroergonomics have been applied in order to understand the PFC behaviour supporting a complex surgical task. Subsequently, assistive technology with the ability to stabilise performance and enhance accuracy was tested at brain level in order to appreciate how it may work and whether it is beneficial to the surgeon. The findings of these studies demonstrated a PFC response not entirely predicted. This highlighted the need for greater investigation of the mode of action GCMC and also the need for metrics that can be used to investigate the effect of task modulation by technology on brain function. To this end, graph theory and ensuing metrics were used. The purpose of this chapter is to apply the analysis strategy developed in Chapter 6 to the investigation of a novel system for enhancing collaboration within surgery. This study begins to highlight the mode of action of this tool and also the marked relevance of neuroergonomic evaluation of technology within surgery.

Collaboration is integral to performing surgery, whether operating with an assistant, scrub nurse or co-operating with colleagues from allied specialties. Within robotic surgery, the need for twin operators is in part being addressed by dual console systems such as the da Vinci® Si that enables two surgeons to operate simultaneously. In this scenario, it is important that communication between both surgeons is maximally effective to enable a

160

Figure 7.1 Schematic representation of CGC for intra-operative guidance. The trainee and trainer can be appreciated (left and right respectively) operating in a shared robotic surgical environment with portable eyetracking (beneath each monitor) detecting gaze behaviour. Under conventional guidance (upper panel), the trainer verbalises direction to the trainee. With CGC (lower panel), the trainer’s point of regard is detected in real time (blue dashed line) and displayed on the trainee’s monitor (represented as white cross) where it can be easily seen. seamless flow of information between the two operators and ensure an efficient workflow. Similarly, excellent communication facilitates technical skills training in surgery. During ‘open’ surgery, a variety of methods are employed for communicating with a trainee that includes a combination of verbal instruction, physical pointing or actual demonstration. However, during robotic and minimally invasive surgery (MIS), there may be circumstances in which the trainee or collaborating surgeon is using both instruments simultaneously within the surgical field of view, constraining the trainer / master surgeon and rendering them reliant solely on verbal communication.

Within MIS and robotic surgery techniques exist such as telestration that aid information transfer between surgeons and / or between trainer and trainee. Telestration allows

161 information to be ‘drawn’ onto a monitor at a remote site by the surgeon guiding the procedure. This information is then displayed on the operator’s screen with the aim of guiding performance [243] and may be undertaken either remotely or locally. Remote guidance or telementoring enables surgeons to be guided by a mentor at a location remote from the operation. This form of instruction has been applied in order to enable regional experts to guide surgeons at local centres [244] and to provide assistance and mentoring from surgical experts in other countries [245].

There has been interest in the role that gaze behaviour may have in improving the flow of communication between collaborating subjects. For example, it has been demonstrated that shared gaze during visual collaboration enables a more efficient search strategy when compared to verbal collaboration alone [246]. Therefore, it is anticipated that observing a guiding surgeon’s point of regard instead of, or in conjunction with their verbal instruction will significantly improve the performance of the operating surgeon by providing supplementary cues critical to task success. With this principle at its foundation, an innovative system referred to as ‘collaborative gaze control’ (CGC) was developed to enable an operating surgeon to be directed by visual guidance as opposed to or in conjunction with verbal instruction from an expert. With CGC enabled, the trainer’s gaze behaviour is extracted in real-time. Their point of regard is subsequently relayed to the trainee’s screen which may be in a remote location. Therefore, the trainee’s operative manoeuvres can be directed more precisely, potentially obviating the dependence on verbal instruction. Importantly, in manipulating target salience, visual search is modulated leading to enhanced performance [247]. A schematic representation of CGC is illustrated in Figure 7.1.

As illustrated in Figure 7.2, if a target markedly differs from its background, it is visually salient and is more likely to be detected by a ‘bottom-up’ search strategy guided by the saliency of the scene [138], whereas if a target requires greater cognitive input to be identified, a ‘top-down’ search ensues which is dependent on extra-striate regions including the PFC and PC [139]. Challenging, effortful visual search results in greater visual cortical (V1) excitation [140]. Evaluating the influence of novel technologies that manipulate visual search on the surgeon’s cortical function is important to ensure that performance enhancement is not offset by greater attentional demands at brain level. Therefore, evaluating the brain behaviour associated with performing with / without CGC may shed light on how this device exerts its effect.

162

Figure 7.2 Schematic illustration of visual saliency. Searching for red oblongs is likely to be reliant on bottom-up search due to salience of scene. A search for the number of vertical green oblongs is likely to necessitate a top-down search in order to identify the less salient target.

In order to examine this, fNIRS [94, 100] is utilised to measure task-evoked fluctuations in HbO2 and HHb within cortical tissues which is reflective of cortical activation. As has been demonstrated in the preceding experimental chapters, fNIRS is flexible, relatively resistant to motion artefact and can be used in conjunction with ferromagnetic instruments making it a suitable neuroimaging modality for neuroimaging studies as it enables tasks to be evaluated in a ‘natural’ environment.

The aim of this study is to investigate the influence of CGC on changes in visual search strategies, technical performance, and visuoparietal (V-P) brain behaviour in subjects being guided to perform simulated biopsy using robotic MIS. The neuropsychological experimental evidence for collaboration suggests that collaborative search is more effective when driven by visual guidance [246]. Therefore, it is hypothesised that with CGC enabled increased target saliency will lead to a ‘bottom-up’ search strategy reflected in a more focused pattern of V1 activation and a reduction in the need for recruitment of extra-striatal visual association areas. Conversely, verbal communication (gold standard) is anticipated to lead to a more effortful ‘top-down’ visual search strategy, necessitating recruitment of additional cortical regions outside V1 and manifest as greater excitation in centres of visual attention. Consequently, it is possible that the resultant ‘bottom-up’ search under CGC guidance will yield a higher network efficiency and lower network cost and cognitive burden due to search of this nature being effectively processed within V1 and not requiring as much extra-striatal communication. As CGC is anticipated to improve the perceptual flow of information to the trainee, it is hypothesised that technical procedural skills will be superior during gaze-guidance than during verbal guidance.

163 7.2 Materials and Methods

7.2.1 Subjects

20 subjects (1 female) were recruited (following LREC approval) from Imperial College London, [mean age, (years  SD) = 28.9  1.5]. Subjects with a history of or current neuropsychiatric illness, previous exposure to the task or left handedness were excluded. Of the 20 subjects, 10 were surgical trainees. The task was designed in order to be abstract enough so that prior surgical experience was unlikely to be beneficial. Additionally, all subjects served as their own controls therefore risk of bias was reduced. However, expertise was also considered in the analysis to confirm this (discussed below).

7.2.2 Task and Training

The robotic surgical task entailed the subject (‘trainee’) and an expert (‘trainer’) collaborating in taking virtual biopsies from a simulated gastric mucosa in a shared surgical environment as depicted in Figure 7.3.

Figure 7.3 Task setup. Both trainee and trainer (left and right respectively) control virtual instruments, each with two haptic manipulators (Phantom Omni, SensAble Tech, USA). The trainer’s right hand manipulator is highlighted (yellow). Gaze behaviour is detected with portable eyetracker (x 50 eyetracker, Tobii Technologies, Sweden) situated below both monitors (trainer eyetracker highlighted yellow). The optical topography system can be appreciated (left of screen highlighted yellow) with optodes positioned over the trainee’s V-P cortices.

164 Haptic manipulators (Phantom, Omni, SensAble Technologies, USA) were used to control robotic graspers in the virtual scene. The task necessitated the subject taking a virtual biopsy and passing the specimen to the guiding surgeon (trainer). Both the subject’s and the trainer’s graspers were visible within the same field of view with the former located inferiorly and the latter superiorly as depicted in Figure 7.4. Within the operative field, seven nodules were visible to the subject. The choice of nodule for biopsy was randomly determined and this selection was available only to the trainer. Thus, the appropriate biopsy site had to be conveyed to the subject either visually (CGC) or verbally (control) by the trainer. Once the biopsy was taken by the subject, the specimen was passed to the trainer’s graspers and when successfully transferred to the trainer, it disappeared from the field of view. Task images are represented in Figure 7.4. This process was repeated as many times as possible during the allotted task periods.

Figure 7.4 Task images as they appear on trainee monitor. The trainee’s instruments are located inferiorly. Initially, the trainee sees the blue cross indicating the intended biopsy target (i). The trainee then grasps the target nodule (black circle) with their right instrument (ii) and passes it to the trainer’s instrument (iii – iv). Trainer monitor (not displayed) is identical with the exception that the target nodule for biopsy is indicated.

Prior to commencing the study, all subjects received a standardised task familiarisation period. All subjects performed the simulated biopsy task under verbal (control) and visual instruction (CGC). The order was randomised (random number generator) in order to control for learning effects. Regarding the control task, biopsy site was described by the trainer with verbal instructions. With CGC enabled, a portable eyetracker (x 50 eyetracker Tobii Technologies, Sweden) situated beneath the trainer’s monitor detected their fixation point and conveyed this to the trainee’s screen as a cross. Therefore, with CGC enabled the trainer looked at the target and their selection would be conveyed to the trainee.

Following previous fNIRS studies of open surgical [71, 106], MIS [87] tasks, and the preceding experiments, a block design was employed comprising a baseline rest period (30s) followed by five blocks of task (30s) and inter-trial rest periods (30s). During rest periods, subjects remained still with their eyes open regarding a black screen on the task

165 monitor. The task period was extended to 30s in order to allow more time for subjects to undertake biopsies and therefore making differences in performance between the two groups easier to discern.

7.2.3 Behavioural Data

The number of nodules that the subject was able to successfully biopsy and transfer to the experimenter’s graspers across the task period was used as a performance metric. Additionally, instrument pathlength (metres) was also determined and used as an indicator of technical performance.

7.2.4 Gaze Behaviour

Subject and trainer gaze behaviour was recorded throughout the study with portable eyetrackers (x 50 eyetracker, Tobii Technologies, Sweden) situated beneath the task monitor (as displayed in Figure 7.3). The gaze behaviour of the trainer was utilised to derive their fixation point in order to display this as a cross on the subject’s monitor. Both subject and trainer fixation points were recorded in order to determine the time taken for the subject to fixate on the same area as the expert: gaze latency (GL, seconds).

Figure 7.5 Schematic representation of optodes as positioned in the 4 x 4 array (a). Emitters and detectors (red and blue numbers) and channels (black numbers) are positioned with the penultimate row of optodes overlying Oz. The subject (b) undergoes optode registration to generate a 3D mesh (c). This is subsequently co-registered with a reference MRI scan (d), in which channels (yellow numbered circles) can be appreciated overlying bilateral visuoparietal cortices.

166 7.2.5 Functional Near Infrared Spectroscopy

Brain behaviour was assessed using a 24-channel Optical Topography system (ETG4000, Hitachi Medical Corp., Japan). For the purposes of this experiment the 4 x 4 array was used to position sixteen optodes (8 emitters and 8 detectors) over the occipital cortex as displayed in Figure 7.5. The row above the lowermost row of the array was centred over Oz [210] thus capturing the bilateral visual cortices and adjacent parietal cortices. Initially, cortical data was subject to integrity checks [240], to identify and eliminate data contaminated with saturation-related artefacts.

7.2.6 Systemic Effect and Stress

A portable band ECG (Bioharness v2.3.0.5; Zephyr Technology Limited, USA) was used to acquire continuous heart rate data, from which HRV was derived [43] and used to determine subject stress.

7.3 Data Analysis

7.3.1 Behavioural Data and Gaze Behaviour

The number of nodules biopsied by each subject during the allotted task time and the instrument pathlength (metres) were determined. GL (seconds) was derived from the eye- tracking data stream. These data were incorporated into the REM in order to assess whether the mode of guidance (control versus CGC) predicted technical performance and visual search behaviour. Surgical experience (novice 1, trainee 2) was incorporated into the REM in order to determine if it was a predictor for differences in cortical haemodynamics or performance  = 0.05).

7.3.2 Cortical Haemodynamic Activity

Cortical activity at all 24 channel locations was determined as a statistically significant increase in HbO2 (ΔHbO2) and decrease in HHb (ΔHHb) from baseline (Wilcoxon Rank Sign, =0.05). ΔHb was the dependent variable used to investigate the influence of task (CGC versus control) and stress on brain behaviour using a REM, (Intercooled Stata, v10.0 for windows, Stata Corporation, USA).

7.3.3 Visuoparietal Network Activity

Cortical haemodynamic data was used to construct a task-evoked network of the 24 channels using graph theory [161]. As in the preceding chapter, two approaches were used as follows: (a) Firstly, a 24 x 24 bidemensional cross-correlation matrix was

167 constructed by cross-correlating all channels. This matrix represents the strength of functional association between each of the 24 channels. This is subsequently pruned in order to exclude inactive channels from the ensuing network. Consequently, networks are only generated from “active” channels. This approach renders a network for each subject during each task condition. However, a potential drawback is that multiple networks are produced that may have varying numbers of connections thus making comparison in network architectures between conditions more challenging [200]; (b) Accordingly, the second method of network construction was also employed which involved generating the cross-correlation matrices from group averaged haemodynamic data (one for control task and one for CGC). These were then pruned by removing network connections sequentially based upon the significance of the functional association (least significant first) until both networks had an equal connection number. This latter approach means that all networks to be compared will have an equal number of graph nodes and connections thus making comparison more valid [200].

Econometric data from these networks was then calculated, to derive: (a) the number of network connections; (b) the maximum global efficiency [178]; (c) the normalised cost [178] and (d) the task-induced ‘cognitive burden’ of the network. In the previous chapter and presently, the cognitive burden is the inverse of the network cost-efficiency [178]. Cost-efficiency is derived from subtracting network cost from network efficiency and equates to how economical the network is [178]. If a network is economical the cost- efficiency is high and accordingly the cognitive burden is low. Network measures derived from the activity-guided approach generated results for each subject and were compared between the two groups using a REM analysis to determine whether the mode of guidance (CGC versus control) significantly influenced network econometrics (=0.05).

7.3.4 Systemic Effect and Stress

HRV as calculated by the SDRR [43] was derived from the HR readings. The SDRR decreases under stress [45] and was incorporated into the REM in order to determine that differences in HRV or the mean HR was not an independent predictor of changes in cortical hemodynamics. Furthermore, HRV was utilised to determine whether the task was more stressful under control or CGC guidance.

168 7.4 Results

7.4.1 Behavioural Data

Performance was enhanced with CGC in terms of both an increase in the number of nodules biopsied [mean  SD: control = 5.9  1.6 vs. CGC = 7.2  2.0; p<0.05] and a diminution in instrument pathlength throughout the task [mean  SD: control (metres) = 0.6  0.1 vs. CGC = 0.3  0.1; p<0.05] as depicted in Figure 7.6 and Table 7.1. This implies that trainee’s were able to undertake the task more rapidly and with greater economy of movement in CGC mode as the distance travelled by the virtual instruments was substantially less. Surgical experience was not a predictor for performance in terms of number of nodules biopsied (p = 0.601).

Figure 7.6 Performance in terms of number of biopsies retrieved (left panel) and instrument path length (right panel). Box plots indicate mean and error bars represent 95% CI.

Figure 7.7 Gaze plots from a representative subject under control (left) and CGC (right) guidance demonstrate more focussed fixations during the task as evidenced by more tightly confined regions of hotspots as compared to the control condition where fixations are more widespread.

169 7.4.2 Gaze Behaviour

Figure 7.7 depicts the visual search pattern acquired from a representative trainee under both guidance conditions. It is apparent that whilst operating under CGC guidance, the fixations appear to be more localised to the nodule to be biopsied implying that unnecessary additional target search / random saccadic activity was minimised. As displayed in Figure 7.8 and Table 7.1, GL was shorter with CGC [mean  SD: control (s) = 1.6  0.4 vs. CGC = 0.9  0.2; p<0.05]. This indicates that with CGC enabled, the trainee being guided would fixate more rapidly on the appropriate target nodule.

Figure 7.8 Mean gaze latency for control (blue) and CGC (red) indicating that under CGC guidance the time taken for the trainee’s fixation point to reach the trainer’s fixation point was significantly less. Error bars indicate 95% CI.

7.4.3 Cortical Activity

Following data integrity checks, a total of 22 channels out of 936 were excluded. Grand averaged time courses for each task condition are represented in Figure 7.9. Patterns consistent with task-related activity can be appreciated. Topograms of a representative subject depicting the average change in HbO2 overlying the V-P cortices are displayed in

Figure 7.10 and demonstrate a broader HbO2 response in the control condition that extends beyond the visual cortex. Cortical haemodynamic change evoked by verbal guidance was more diffuse as illustrated in Figure 7.11 (verbal: 19/24 channels active versus CGC: 11/24 channels active) and more likely to involve bi-parietal as well as bi- visual cortices. Tables 7.2 – 7.4 display the REM results in terms of predictors of in Hb species. Session (control versus CGC) was a predictor for ΔHbO2 and ΔHbT but not ΔHHb, signifying a greater response in the control task. Previous surgical experience did not predict a change in any of the three Hb species. Block number (1 – 5) predicted a change in Hb species and channel predicted a change in HHb and HbT. Overall, this data supports the hypothesis that training in CGC mode results in less V-P brain activation.

170

Figure 7.9 Grand averaged time courses HbO2 and HHb (M x cm) (red and blue lines respectively) for control (a) and CGC (b) conditions. Data is averaged across the 5 task blocks (represented by green bar) for each of the 24 channels (numbered yellow).

Variable Control CGC p value Number of nodules 5.9  1.6 7.2  2.0 <0.001 Instrument Pathlength (m) 0.6  0.1 0.3  0.1 <0.001 Gaze latency (s) 1.6  0.4 0.9  0.2 <0.001

ΔHbO2 (Mol x cm) 2.8  12.5 1.5  10.9 <0.001 ΔHHb (Mol x cm) -0.9  5.6 -1.1  5.1 0.188 No. connections 97.6  119.3 81.6  97.3 <0.001 Normalised cost (a.u.) 0.5  1.0 0.3  0.6 <0.001 Cognitive Burden (a.u.) 0.4  1.1 0.3  0.6 <0.001 Global Efficiency (a.u.) 0.1  0.3 0.09  0.3 <0.001 Mean HR (beatsmin-1) 70.1  9.2 74.5  14.2 <0.001

SDRR 64.2  37.9 57.1  29.8 <0.001

Table 7.1 Performance, changes in cortical haemodynamics, cortical network metrics, HR and HRV values for both tasks (mean  SD).

171

Figure 7.10 Topogram depicting task-averaged HbO2 response for control (a) and CGC (b) conditions from a representative subject. A broader response extending beyond the visual cortex is evident under the control condition.

Figure 7.11 Figure depicting group averaged V-P channel activation for control (left) and CGC (right) guidance. Level of activation is indicated by colour: (a) red: significant increase in HbO2 coupled to significant decrease in HHb; (b) spots: increase HbO2 and decrease HHb (one species reaching significance); (c) stripes: increase HbO2 and decrease HHb (neither species reaching significance) and (d) black: no coupled increase HbO2 and decrease HHb. A greater number of activating channels is appreciated under verbal guidance (control vs. CGC = 19/24 vs. 11/24).

172 7.4.4 Visuoparietal Network Activity

Initially, cortical networks were constructed so that only activating channels were included in the analysis. Figure 7.12 depicts the activity thresholded cortical network under control and CGC conditions for a representative subject. Figure 7.13 and Table 7.1 represent results of econometric analysis of the activity-guided networks in terms of number of cortical connections, normalised cost, maximum global efficiency and cognitive burden. Subjects generated significantly fewer functional cortical connections whilst operating with CGC compared to verbal guidance [mean  SD: control = 97.6  119.3 vs. CGC = 81.6  97.3; p<0.001]. The resultant cortical networks for control and CGC performance demonstrated lower network cost (a.u.) [mean  SD: control = 0.5  1.0 vs. CGC = 0.3  0.6; p<0.05] but lower efficiency (a.u.) with CGC enabled [mean  SD: control = 0.1  0.3 vs. CGC = 0.09  0.3; p<0.05]. Importantly, the task-induced cognitive burden when operating with CGC was reduced (a.u.) [mean  SD: control = 0.4  1.1 vs. CGC = 0.3  0.6; p<0.05] as illustrated in Figure 7.13.

Figure 7.12 Activity-guided cortical networks for representative control (a) and CGC (b) subject. Approximate channel locations (black circles) are overlain onto reference MRI atlas. Network edges can be appreciated with strength of functional association between channels represented by line thickness and darkness of colour. It is apparent that the network is broader under the control task (due to increased number of activating channels under this condition). However connections appear stronger with CGC. The latter is reflected in the lower network cost and cognitive burden under CGC guidance.

.

173

Figure 7.13 Mean network econometric outcomes for control (blue) and CGC (red) guidance for: number of cortical connections, network cost, network efficiency and task induced cognitive burden when graph generated utilising activity guided network thresholding. Error bars indicate 95% CI.

Figure 7.14 Group-derived cortical networks for control (a) and CGC (b) conditions using the normalised connection density approach. Approximate channel locations (black circles) are overlain onto reference MRI atlas. Network edges can be appreciated with strength of functional association between channels represented by line thickness and darkness of colour. It is apparent that node isolation occurs and is more marked in the CGC group network. In this group, network connections appear weaker possibly accounting for the increased network cost and decreased efficiency of this network.

174 Group network behaviour as determined by normalised connection density approach is displayed in Figure 7.14. When networks were generated using this approach, efficiency was lower under CGC guidance (0.021 vs. 0.029) and cost was higher 54.84 vs. 39.30.

7.4.5 Systemic Effect and Stress

The mean HR and SDRR are depicted in Table 7.1. Whilst operating with CGC, the mean

HR was significantly higher (p<0.001) and SDRR was significantly lower (p<0.001). The latter indicating that subjects may have found operating with CGC assistance more stressful. Upon REM analysis (Tables 7.2 – 7.4), mean HR and SDRR were both predictors for changes in HbO2 and HHb. However, did not predict a change in HHb. Non-functional changes in cortical haemodynamics mediated through stress-induced variations in the systemic circulation such as HR and blood pressure are known to occur [159] . However, other fNIRS studies have demonstrated that the systemic effect does not necessarily confound task-induced changes in cortical haemodynamics [248].

HbO2

Variable Coefficient s.e. z P>z 95% C.I.

Session -1.659 0.344 -4.83 0.000 -2.332 to -0.986

Block 0.357 0.115 3.10 0.002 0.131 to 0.583

Channel 0.018 0.024 0.78 0.437 -0.028 to 0.064

HR 0.057 0.014 4.02 0.000 0.029 to 0.084

SDRR -0.043 0.017 -2.50 0.013 -0.076 to -0.009

Experience 1.907 1.757 1.08 0.278 -1.538 to 5.351

Constant -0.489 2.898 -0.17 0.866 -6.170 to 5.192

σu 3.835 σe 11.149 rho fraction of variance 0.106 due to uj

Table 7.2 Results of multivariate REM for ΔHbO2. Significant p values are highlighted in bold.

175 HHb

Variable Coefficient s.e. z P>z 95% C.I.

Session -0.128 0.156 -0.82 0.414 -0.434 to 0.179

Block 0.165 0.053 3.13 0.002 0.062 to 0.269

Channel 0.105 0.012 9.74 0.000 0.084 to 0.126

HR -0.002 0.006 -0.28 0.780 -0.014 to 0.010

SDRR 0.012 0.007 1.67 0.094 -0.002 to 0.026

Experience 0.815 0.654 1.25 0.213 -0.467 to 2.097

Constant -4.191 1.099 -3.81 0.000 -6.346 to -2.037

σu 1.412 σe 5.107 rho fraction of variance 0.071 due to uj

Table 7.3 Results of multivariate REM for ΔHHb. Significant p values are highlighted in bold.

HbT

Variable Coefficient s.e. z P>z 95% C.I.

Session -1.523 0.317 -4.80 0.000 -2.145 to -0.901

Block 0.192 0.107 1.79 0.074 -0.018 to 0.401

Channel -0.087 0.022 -3.97 0.000 -0.130 to -0.044

HR 0.557 0.013 4.42 0.000 0.031 to 0.080

SDRR -0.055 0.015 -3.67 0.000 -0.084 to -0.026

Experience 1.073 1.386 0.77 0.439 -1.643 to 3.790

Constant 3.834 2.319 1.65 0.098 -0.711 to 8.379

σu 2.998 σe 10.347 rho fraction of variance 0.077 due to uj

Table 7.4 Results of multivariate REM for ΔHbT. Significant p values are highlighted in bold.

176 7.5 Conclusions

In this study, performance in a simulated surgical task has been improved by modulating the manner in which collaborating surgeons interact with one another. Communicating through collaborative gaze-driven control leads to a greater number of successful biopsies and a reduction in instrument path length, the latter being a measure of dexterity previously shown to reflect skill level in laparoscopic and open surgery [249, 250]. The foundation for this improvement is a change in visual search strategy manifest as a reduced GL indicating that with CGC, trainee fixation points more rapidly reach the expert’s. This is accompanied by a decreased level of cortical activity across primary visual centres in the brain. Moreover, econometric analysis of the cortical network constructed from activating channels demonstrated that the cost and cognitive burden in the brain behaviour was significantly ameliorated when operating with gaze guidance. It is possible that in altering the search strategy and concomitantly alleviating the task- induced cognitive burden, subjects are able to better direct their attention to effectively manoeuvring the instruments in order to execute the task.

CGC utilises eyetracking in order to detect the fixation point of the surgeon guiding the procedure. This is relayed to the trainee as a fixation cross on their monitor and in so doing displays the intended location as a visually salient target. Consequently, the resultant search strategy is changed in keeping with the cortical correlates of streamlined visual search from a ‘top-down’ to ‘bottom-up’ strategy [138, 139]. The latter mode of search is less reliant on input from cortical regions outside V1 and leads to a reduction in activity in that region in accompaniment to search simplification [140]. Presently, this effect has been observed as a reduction in V-P cortical haemodynamic changes with comparatively fewer channels reaching statistical threshold for activation. Parietal cortical activity is also associated with occulomotor intention and attention and may be important in planning eye movements [251]. This is likely to be relevant under both task conditions. However, it is likely to be more relevant under verbal guidance as auditory information needs to be processed in order to generate and detect the desired target location. It is possible that the search is more demanding under verbal guidance, in keeping with Kojima et al. who demonstrated increased occipital activity when a search was more effortful [140].

Graph theory has been utilised to investigate cortical networks in both pathological and non-pathological brains [173, 178] and allows network parameters such as cost and efficiency to be determined [174]. Graph theory was applied to experiment data in order

177 to further appreciate the impact of CGC on brain functional networks. Two approaches were applied: firstly, channels exhibiting task-induced activity form the basis of the cortical network and secondly, all channels irrespective of activation status were included in the network allowing graphs to be equalised for connection number.

From the first (activity-guided) approach, it is evident that verbal guidance from a trainer leads to a greater number of network connections (edges) in the graph owing to spatially broader task evoked activations in the trainee’s brain. The corollary of this is a higher network cost [174]. However, the global efficiency of the network which is measure of information transfer / communication between channels [174] was greater. It is plausible that the lower network efficiency under gaze assistance may be explained by the fact that subjects are experiencing a wholly novel means of guidance. In the preceding chapter, network efficiency was been found to increase with task learning, therefore a learning- related increase may be demonstrated in line with task acquisition. Similarly, greater familiarity with traditional verbal guidance may be responsible for improvements in network efficiency over and above those observed under gaze-assistance. When all channels, irrespective of activation status were included in the network, cost was found to be greater and efficiency lower with CGC. However, a potential limitation with this approach, is that under CGC guidance, less cortical regions were recruited to complete the task and yet would still be included in network construction. This may have led to the greater number isolated nodes appreciated in the CGC network. It is plausible that these ‘irrelevant’ regions impinge on the efficiency and cost of the resultant network and this is likely to be more pronounced under CGC control as the area of task-evoked activity is smaller.

The cognitive burden is the inverse of the global efficiency minus the cost and is therefore equivalent to ‘cost-efficiency’ (CE) [173, 178]. In relation to the activity-guided network analysis, cognitive burden is lower with gaze assistance (equivalent to higher CE). Despite the relatively high network efficiency observed with verbal guidance, the increased cost dominates and this accounts for a comparatively higher cognitive burden when contrasted with gaze-guidance. Therefore, this result is extremely important is it indicates that visual guidance results in a V-P network architecture in trainees that is more cost-efficient than that established with traditional verbal training. A higher CE (and therefore low cognitive burden) is associated with improved task performance [173], whereas a CE reduction has been demonstrated in old age and with dopamine antagonism [178]. Furthermore a higher CE is a advantageous attribute of a cortical network as it

178 indicates that a greater improvement in network efficiency is achieved without disproportionately increasing the number of network connections (and hence the network cost) [173]. Therefore, it can be assumed that under CGC brain behaviour is more effective.

In undertaking this collaborative task with verbal guidance, it is necessary that the trainer locates the target and then verbalises its location to the trainee / assistant. In turn, these instructions have to be interpreted in order to modify the visual search. It is intuitive that verbal instructions regarding target location are time consuming to deliver, more complex to interpret and harder to translate into the ‘visual’ workspace, ultimately relying therefore on greater cognitive work. It is suspected that gaze assistance makes the flow of information between the trainer and trainee more seamless by increasing the perceptual fidelity of the instruction that is given. Extrapolating this effect to the in vivo setting, a reduction in the attentional demands of executing a procedure may liberate attentional resources to be devoted to other safety critical aspects of clinical care (e.g. reacting to unexpected events, multitask decision making, planning subsequent operative steps, etc.). Although not specifically investigated within the confines of this study, it is feasible that in using CGC the need to verbalise the intended target is bypassed and as such the trainer can focus on supplementary aspects of the procedure. For example, if the site of suture placement is already determined and displayed visually, a trainer can then focus verbal instruction on the technical aspects of suturing manoeuvres required to achieve accurate tissue apposition.

This study has demonstrated that by enhancing communication between collaborating surgeons, operator performance can be improved. This is achieved with an improved cost-efficiency of the activated brain network. Therefore, it is feasible that surgeons will have greater cognitive resources to devote to other aspects of the procedure and unexpected events. Moreover, this study reinforces the necessity of employing a neuroergonomic paradigm as part of the appraisal of novel surgical technologies. This is all the more pertinent as advances in technology enables progressively more complex procedures to be undertaken using MIS or robotic assistance and in so doing are likely to place greater cognitive demand on the surgeon. Guidance with CGC represents just one form of telementoring and the results of the present study suggest that providing a trainee with on screen guidance derived from perceptual information from their trainer can further enhance their technical performance.

179 In conclusion, it has been demonstrated that operating with CGC improves surgeon performance. In altering how the expert guides the trainee, search strategy is modified leading to less cortical activity and a reduced task-induced cognitive burden leading to greater performance. This is likely to benefit surgeons who are being guided in that they may have greater resources to deal with other aspects of undertaking surgery.

180 Chapter 8

Conclusions and Future Perspectives

8.1 Achievements of this thesis

Minimally invasive surgery has revolutionised surgical practice. As indexed by cholecystectomy; up until the latter part of the 20th century, was associated with a 7 to 10 day hospital stay and significant complications secondary to a large subcostal incision. Upon the advent of laparoscopic surgery, this procedure is undertaken as a day case and has also been performed using ‘scarless’ techniques such as single incision laparoscopic surgery (SILS) and NOTES. Progressively more complex procedures are undertaken minimally invasively in part, driven by patient expectations, surgeons and underpinning this: advancing technology. This has lead to many challenges, some of which have been met by surgical robotics.

Research into the ergonomics of MIS is abundant due to the poor human factors associated with laparoscopic equipment and procedures. However, as the use of surgical robotics proliferates, the demands of interacting with this complex technology will be more cognitive. Consequently, existing ergonomic research may not unveil the unique nuances of performance modulation that are likely to occur. Accordingly, the motivation of this thesis was to apply the field of neuroergonomics to assess the robotic enhancement of surgery. Surgeon brain behaviour has been previously investigated within the paradigm of skills assessment and learning [95], but to our knowledge, this is the first application of neuroergonomics to evaluate the impact of robotic surgical technology on surgeon brain behaviour.

Initially, brain behaviour underpinning a complex navigational task was assessed in Chapter 4. This work demonstrated that the unique task-demands led to activation of regions key to visuospatial processing. Additionally, experts were found to activate a spatially broader region of the PFC. This finding is not in keeping with the conventional thinking of PFC activity with expertise. However, it is possible that an initial PFC-

181 independent stage of learning exists when performance is very naïve and subjects are unable to formulate a cogent strategy to perform. Accordingly, activity in this region is absent / diminished and it only increases when task demands are explored and a coherent approach to the task is devised which is subsequently reliant on the PFC. This concept is further explored in Appendix A.

With this as a foundation, Chapter 5 investigated how assistive technology, which may simplify navigation, in the form of GCMC impacted on performance and user brain behaviour. It was found that performance was enhanced but coupled to increased PFC activity. This was thought either to be due to increased attentional demand associated with using the tool, or possibly that GCMC ‘forces’ the user to adopt an effective strategy in order to complete the task and that initially necessitates greater vigilance leading to an increase in right sided PFC activity. Following on from this study, the question was posed as to how this technology would modulate the learning process.

Previous work in the field of neuroergonomics and psychology has aimed to measure mental workload associated with tasks, often with questionnaires or with particular cortical changes, e.g. increasing PFC activity in response to escalating cognitive effort. However, as demonstrated in the preceding experimental chapters, greater cortical activity may not be detriment to performance. Accordingly, a general concept of ‘cognitive burden’ has been defined as ‘any deviation from the most efficient neurocognitive pathway of performing a task.’ In order to measure this, graph theory was applied in Chapter 6 to calculate network econometrics of an activated F-P network across 6 learning sessions with or without robotic assistance in the form of GCMC. To our knowledge this is the first application of graph theory to fNIRS-derived neuroimaging data. Network measures of efficiency, cost, small-worldness and task-induced cognitive burden were derived and used to show that GCMC was enhancing task learning at brain level via increased network efficiency and decreased task-induced cognitive burden. This further improved understanding as to how GCMC exerts its effect but also shed light on longitudinal network behaviour associated with learning.

In Chapter 7, a graph theoretical approach was applied to neuroergonomically evaluate CGC, a tool that aids collaboration in surgery. This study demonstrated that CGC modulated surgeon gaze behaviour and led to less V-P activity and a lower task-induced cognitive burden. The relevance of this result is that it may mean the surgeon has greater resources available to devote to other aspects of the procedure such as dealing with

182 unexpected events. This also emphasises the relevance of applying neuroergonomics to the assessment of surgical robotic tools as this paradigm is able to reveal how already complex tasks are impacted by technology and whether this is helping the surgeon or not.

8.2 Future perspectives

In using functional neuroimaging to investigate brain behaviour of such complex tasks, many further research questions were inevitable. Below are outlined specific areas that are clear targets for future research following the work in this thesis.

8.2.1 Effect of Task Training on Prefrontal Cortical Activity

In Chapter 4, greater PFC activity was elicited in those subjects more proficient in the task. It was postulated that an initial task-naïve phase of PFC independence may exist prior to development of an adequate strategy to undertake the task. Figure 8.1 represents a hypothesis of PFC behaviour in complex task acquisition.

Figure 8.1 Hypothesis for PFC activity associated with complex task acquisition. An initial phase of poor proficiency and minimal PFC activity is demonstrated (Naïve phase). As task demands are grasped, performance improves with a necessary reliance on the PFC (Novice). Performance continues to improve as PFC activity attenuates (Trainee) until expertise is attained. (Image courtesy of Professor Guang-Zhong Yang).

Once an appropriate approach to the task has been developed, activity increases prior to attenuating with expertise. This effect was further explored in Appendix A, when novices were given minimal instruction in order to execute a complex bimanual knot tying task. Trials associated with task success were characterised by greater PFC activation. This hypothesis warrants further investigation as this effect is not currently described in the literature. Future work could include a longitudinal evaluation of PFC activity during a

183 complex task such as laparoscopic suturing. Subject brain behaviour would need to be evaluated regularly throughout the learning process. The relevance of this to complex task performance and surgery is that it may be incorporated into competency assessment. In so doing, subject performance can be interpreted in light of PFC activity in order to elucidate at which stage of the learning process they currently reside. It is also feasible that techniques such as mental imagery may hold promise in understanding the cortical correlates of such tasks as studied in chapter 4 that are reliant on visuospatial working memory. This may aid elucidation of cortical regions key to these tasks and hold promise for novel training strategies.

8.2.2 Fatigue and Surgical Errors

A further application for the use of fNIRS in surgery within the paradigm of neuroergonomics is in the assessment of surgeon fatigue. Despite legislative reductions in working hours, shift systems are prevalent in medical practice and a cause of fatigue. As discussed in 2.4.2.2, the ability to detect surgeon fatigue would clearly be beneficial. The PFC response to cognitive tasks increases in line with rising fatigue however, performance is preserved [71]. This implies a compensatory increase in PFC activity to maintain a similar level of performance and furthermore, it is clear that monitoring performance in isolation would not have demonstrated this effect. This could be applied to either ‘screen’ surgeons for fatigue prior to a complex procedure or for the online recognition of it intra-operatively. Wireless technology and miniaturisation of NIRS instrumentation is likely to facilitate this application by reducing the intrusiveness of equipment therefore enabling continuous monitoring of cortical activity in the natural work environment (i.e. intra-operatively).

Additionally, a further avenue for research is in the use of graph theoretical analysis in order to investigate impairment of cortical networks in association with fatigue. As reviewed in 3.7.3, the effect of independent variables such as age and disease state on graph properties have been investigated. However, to our knowledge, the effects of fatigue on network structure are yet to be determined. This thesis represents the first application of graph theoretical analysis to fNIRS data. Future perspectives could include the generation of synthetic data in order to further validate this technique. Synthetic optical data could be utilised to construct cortical networks which can then be manipulated in a specific manner in order to determine if this directly influences econometric parameters of the networks.

184 8.2.3 Stress and the Systemic Effect

Throughout this thesis, the systemic effect has been has been measured using a variety of means. This information was incorporated into data analysis. Subject physiological variables did often predict changes in Hb species, yet no consistent effect was found. Nevertheless, this is clearly a very important area of research, particularly as more complex work-related tasks are undertaken within the paradigm of neuroergonomics. This means that the influence of systemic physiology on cortical haemodynamics may increase in line with tasks that are more stressful or have a greater physical component. Appendix B expands on the exploration of the longitudinal dataset presented in Chapter 6. An avenue for future research in this area is with the use of causality to investigate the directionality and magnitude of influence between these signals. This research will have marked importance particularly with longitudinal datasets, ensuring that temporal fluctuations in brain behaviour across multiple sessions are due to neuroplastic changes in the brain as opposed to a variation in influence of the systemic circulation.

8.2.4 fNIRS as a Means to Enhance Perceptual Docking

Perceptual docking is the concept of deriving operator-specific motor and perceptual behaviour in situ via human-computer interaction. This information can be utilised to guide and improve performance. GCMC is an example of this and the cortical correlates of this tool were investigated in Chapters 5 and 6. fNIRS could be further utilised within the perceptual docking framework to detect specific features of brain behaviour in order to alter performance accordingly. Specifically, as outlined above in 8.2.2, surgeon fatigue and cognitive burden could be determined online and used as a warning system. In a similar vein, particular maladaptive changes in cortical activity have been reliably detected up to 30 s prior to the making of an error [252]. This implies that errors may not solely be due to transient lapses in attention, but due to shifts in cortical activity occurring across a broader time frame. Consequently, this may yield a target for intra-operative detection and action prior to an error taking place.

In addition to the applications outlined above, NIRS could also be utilised as an input to control the robot: i.e. Brain Computer Interface (BCI). fNIRS has been successfully applied as an input to BCI [253-255]; however, limitations exist due to the temporal nature of the haemodynamic response. Notwithstanding, the potential for BCI in surgery is vast either for direct mechatronic control of an instrument such as a retractor or for use as an ‘on / off’ switch to control movement of an instrument that is subsequently guided by other means such as eyetracking. In order to apply BCI to surgical robotics safety and

185 ethical implications need to be addressed in order to ensure that there is no undue risk to patient safety.

8.2.5 Environment and Team Behaviour

Chapter 4 investigated PFC activity underpinning complex task performance within a NOTES environment. This study addressed the demands placed on the surgeon by a particular aspect of this novel and technically demanding field of surgery. However, this did not address one of the wider issues associated with NOTES. Namely, that with current equipment, NOTES procedures demand a large team to perform them. For example, operating the flexible endoscope requires one surgeon adjusting the controls and one manipulating the operating instruments. Supplementary to this, there may be a further surgeon retracting tissues via the transvaginal port and two more operating transabdominally. Gynaecologists are also required in order to achieve safe access into the peritoneal cavity. This already numbers 7 surgeons before anaesthetists and operating room staff are included. Therefore, undertaking what is already a technically challenging procedure, may become more demanding due to coordination of such a large team.

A system for enhancing collaboration between surgeons was investigated in Chapter 7. In this study, trainee brain behaviour was investigated. However, a further application of neuroergonomics is in the assessment of the surgical team as a whole and factors that may facilitate group communication and streamline performance. Further work would entail the assessment of equipment designed to enhance communication on all members of the team. It is possible that improving collaboration would alleviate the burden not only on the operating surgeon (as assessed in Chapter 7) but on the surgeon delivering guidance.

In conclusion, in this thesis fNIRS was used to evaluate the brain behaviour underpinning complex surgical tasks. Moreover, a neuroergonomic paradigm has been applied in order to understand the impact of robotic assistance on surgeon brain behaviour. To this end, the concept of ‘cognitive burden’ was defined and determined by applying graph theory in order to quantify changes in cortical networks yielded by equipment known to enhance surgical performance. Longitudinal modulation in network and haemodynamic behaviour was observed and varied according to mode of learning shedding light on the mode of action of a robotic tool that enhances performance accuracy. This approach was further applied in order to investigate surgical collaboration demonstrating how this process is streamlined by technological assistance. It is hoped that the paradigm of neuroergonomics will continue to be applied to surgery in order to further our understanding of the brain

186 behaviour of the surgeon. This information can be utilised to guide the design and development of surgical technology enabling progressively more complex procedures to be undertaken whilst diminishing the impact of surgery on the patient.

187 Appendix A

Prefrontal Cortical Behaviour in the Naïve Phase of Complex Task Learning

Work from this appendix has been presented: James DRC, et al. The role of the prefrontal cortex (PFC) in naïve complex motor skills learning: a functional Near Infrared Spectroscopy (fNIRS) study. Poster presentation at Organisation for Human Brain Mapping, Quebec. 2011.

A.1 Introduction

In Chapter 4, PFC activity associated with a complex navigational task was investigated. A broader area of cortical activity was observed in the expert cohort. This was an unforeseen result due to the well recognized role of the PFC in task learning [109]. However, this effect has been observed previously in relation to a highly complex surgical task whereby novices and experts did not activate the PFC and only those undergoing task training did [88]. It is feasible that a PFC independent task-naïve phase of the learning process exists, especially in highly complex tasks.

The PFC performs a multitude of roles including performance monitoring for errors [134, 223]. Medial PFC regions (dorsal ACC) are thought to detect errors and engage lateral regions in order to effect appropriate performance alteration [134, 135, 223]. The PFC is also thought to house the central executive for visuospatial working memory [126]. All these roles are principle to the process of task execution. However, it is plausible that in order to engage this region, a baseline level of understanding of the stages and demands of the task is necessary.

Most studies allow pre-scanning task familiarisation and practice enabling subjects to develop a cognitive strategy based on an initial exploration of task demands leading to

188 PFC engagement. Leff et al., demonstrated greater PFC activity in novice subjects in a surgical knot tying task [106] with attenuation in this response alongside task learning following extensive practice [112]. In these studies, subjects were trained to a level of competency such that they were able to successfully complete the task albeit in a slow and effortful manner. This training may have afforded time for them develop the necessary skills to complete the task. However, limited practice on more complex tasks [88] may prevent subjects developing an adequate cognitive strategy and manifest as poor PFC engagement. This hypothesis may account for the lack of PFC activity elicited following limited training on laparoscopic suturing [88] and the lower degree of activity demonstrated in novices in Chapter 4.

In response to this, the purpose of this study is to investigate the naïve phase of motor task acquisition with fNIRS on a complex bimanual co-ordination task. It is hypothesised that with minimal task training, subjects will be unable to successfully complete the task leading to a lower level of PFC activity.

A.2 Materials and Methods

Following LREC, Nine male right handed medical students [mean age  SD = 20.1  2.6] consented to take part in the study. Written informed consent was obtained prior to enrolment. Subjects were excluded if they had any prior experience of surgical knot tying either practical or observational. All subjects were requested to refrain from caffeine and alcohol for 24 hours before the study.

Task training was restricted to 4 task demonstrations (1 observing and 3 with auditory description) with no time to practice the task. Demonstrations were undertaken according to the Royal College of Surgeons Basic Surgical Skills course. Whilst observing the task, subjects were required to place their hands on the table in order to prevent them from mimicking the movements that they were being demonstrated. The training regime was selected following repeated pilot studies in which stages of the training were selectively removed until the subjects were barely able to complete the task.

Subjects were required to complete a four throws of a surgical reef knot as previously described by Leff et al. [106]. The knot was a one handed surgical reef knot that is a core skill of any surgical trainee necessitating complex manipulation of the suture with one hand ensuring that the suture is not dropped at any point during the task. Each task comprised 4 throws of the knot on a specially designed jig designed for the purpose of

189 training surgical knot-tying (Ethicon Ltd, New Jersey, USA). The suture used was 2/0 vicryl (Ethicon Ltd, New Jersey, USA). The task paradigm comprised (as used by Leff et al. [106]) a 30 s baseline rest period followed 5 blocks of the task which was self paced. An inter-trial and post-trial rest period of 30 s was used. Subjects were prompted to start tying the knot at the appropriate time verbally having had their hands flat on the desk either the side of the knot-tying jig before and after each task block.

Performance was determined by time taken to complete the knot, number of movements and total pathlength (m) of movements. These were recorded using ICSAD [22]. Task success was determined when all 4 throws of the knot had been successfully completed.

PFC activity was determined with a 24 channel NIRS instrument (ETG4000, Hitachi Medical Corp, Japan). Optode placement was identical to Chapters 4 and 5. Activation was determined as a coupled task-evoked increase in HbO2 and decrease in HHb

(Wilcoxon rank sign). Task minus baseline data (ΔHbO2 and ΔHHb) was used for comparisons between task success and failure (Mann-Whitney). Number of movements, pathlength of hand movements was recorded using the Imperial College Surgical Assessment Device (ICSAD) [22]. Data was processed as previously described [106, 240].

A.3 Results

Nine male right handed subjects were recruited. Six subjects completed 5 knots, 1 withdrew after failing 2 knots, 1 failed 4 knots and 1 subject failed to complete all 5 knots. Average number of hand movements were: [number of movements, mean (SD) = 53.0  29.5] and pathlength: [metres, mean (SD) = 8.9  5.0]. The cortical haemodynamic time course of a successful subject is displayed in Figure A.1. As shown in Figure A.2,

ΔHbO2 was significantly higher for successful knots [Mol x cm, Mean (SD): successful = 18.3  30.3; unsuccessful = -5.6  52.4], p < 0.05. Whereas ΔHHb was greater for successful knots knots [Mol x cm, Mean (SD): successful = 4.3  10.6; unsuccessful = - 4.4  16.3], p<0.05. Group averaged data demonstrated that no channels reached statistical threshold for activation.

190

Figure A. 1 Haemodynamic time course from subject who successfully completed the task. HbO2 and HHb (M x cm) (red and blue lines respectively) demonstrate changes consistent with activation during the task period (green vertical bars) at most channels (labelled 1 – 24). X axis represents time (samples)

Figure A.2 Bar chart comparing mean PFC ΔHbO2 between successful and unsuccessful trials (red and blue bars respectively).

191 A.4 Conclusions

It is suspected that subjects may fail to sufficiently engage the PFC following limited training on complex tasks [88]. The current experiment demonstrates that PFC activity in subjects who succeed is significantly greater than those who fail. Moreover, it has been observed that limited training (current study) results in significantly less PFC activation versus extended practice [106].

Theoretically, subjects who adequately engage the PFC are better able to attend to actions at the most naïve stage of learning, thereby outperforming those who cannot. Further work will aim to prove the hypothesis that a PFC independent stage of task learning exists. Additionally, progressively increasing task training may unveil which facets of training evoke engagement of the PFC and subsequently improve performance. The implications of this work are relevant not only for skills training but also within the paradigm of neuroergonomics where progressively more complex tasks are investigated at brain level.

192 Appendix B

Influence of Heart Rate and Stress on Cortical Haemodynamics

Work from this appendix has been presented: James DRC, et al. Influence of heart rate and stress on cortical haemodynamics associated with motor learning: a longitudinal functional Near Infrared Spectroscopy (fNIRS) study. Poster presentation at Organisation for Human Brain Mapping, Quebec. 2011.

B.1 Introduction

The influence of ‘systemic physiological noise’ such as heart rate (HR), blood pressure and resultant low frequency oscillations (LFO) on task related changes in cortical haemodynamics detected using fNIRS has been investigated [159, 248]. Systemic physiology may influence task-related cortical haemodynamics, however, filtering out the systemic does not significantly change the task-related activation map [248]. Similarly, in this thesis, HR and stress parameters did influence cortical haemodynamics, but no consistent effect was appreciated.

It is not known whether the influence of the systemic effect and stress on cortical haemodynamics change with time. However, for studies interested in observing longitudinal changes in brain behaviour with learning it is critical to be sure that changes in cortical activity are truly a reflection of neuroplasticity rather than a variation in the systemic effect. The aim of this study was to further investigate the influence of HR on longitudinal changes in cortical haemodynamics.

B.1 Materials and Methods

This study utilises the data from Chapter 6. Initially, HR and HRV data was visualised alongside ΔHbO2 values for each group across the six task sessions. Subsequently, haemodynamic and HR data was converted to the frequency domain using a fast Fourier transformation in order to inspect for similarity or common features as has previously

193 been investigated [159]. Throughout the experimental chapters, raw optical data was converted into relative changes in Hb species and then decimated. For the purposes of this study, decimation was not undertaken as the frequency bands of interest may have been filtered out.

B.3 Results

No obvious visible trends in the data could be appreciated between ΔHbO2 and HR and HRV. Upon conversion to the frequency domain, it is apparent in Figure B.1 that there may be common features between the HR and HbO2 signals.

Figure B.1 Power spectral domain for HbO2 (top) and HR (bottom) from a representative subject. Signal similarity appears to exist between the two as indicated by the similar peaks at around 0.1Hz (highlighted red).

B.4 Conclusions

It can be appreciated from these results that the systemic response in terms of HR and HRV does bear influence over cortical haemodynamics. This was demonstrated in

Chapter 6 statistically in terms of HRV predicting fluctuations in ΔHbO2, yet this effect could not be visualised. However, upon conversion to the frequency domain, it appears that commonality between the 2 signals does occur warranting further scrutiny. This effect has previously been investigated [159], however, not across a longitudinal dataset. There is a clear motivation for further work in this area in order to determine if the influence of the systemic response fluctuates across learning. This study also demonstrates the need for concurrent stress and HR monitoring during functional neuroimaging studies.

194 References

1. Harrell, A.G. and B.T. Heniford, Minimally invasive abdominal surgery: lux et veritas past, present and future. American Journal of Surgery, 2005. 190(2): p. 239-243. 2. Reyes, D.A., B. Tang, and A. Cuschieri, Minimal access surgery (MAS)-related surgeon morbidity syndromes. Surg Endosc, 2006. 20(1): p. 1-13. 3. Berguer, R., Surgical technology and the ergonomics of laparoscopic instruments. Surgical Endoscopy, 1998. 12(5): p. 458-62. 4. Berguer, R., W.D. Smith, and Y. Chung, H., Performing laparoscopic surgery is significantly more stressful for the surgeon than open surgery. Surgical Endoscopy, 2001. 15(10): p. 1204-1207. 5. Bohm, B., et al., A prospective randomized trial on heart rate variability of the surgical team during laparoscopic and conventional sigmoid resection. Archives of Surgery, 2001. 136(3): p. 305-310. 6. Ballantyne, G.H., Robotic surgery, telerobotic surgery, telepresence, and telemontoring. Review of early clinical results. Surgical Endoscopy, 2002. 16(10): p. 1389-1402. 7. Aiono, S., et al., Controlled trial of the introduction of a robotic camera assistant (EndoAssist) for laparoscopic cholecystectomy. Surgical Endoscopy, 2002. 16(9): p. 1267-1270. 8. Kant, I.J., et al., A survey of static and dynamic work postures of operating room staff. International Archives of Occupational and Environmental Health, 1992. 63(6): p. 423- 428. 9. Parasuraman, R., Neuroergonomics: research and practice. Theoretical Issues in Ergonomics Science, 2003. 4(1-2): p. 5-20. 10. Lee, G., et al., Methodological infrastructure in surgical ergonomics: a review of tasks, models, and measurement systems. Surg Innov, 2007. 14(3): p. 153-67. 11. Scheer, S.J. and A. Mital, Ergonomics. Archives of Physical Medicine and Rehabilitation, 1997. 78(3 Suppl): p. S36-45. 12. Franco, G. and L. Fusetti, Bernardino Ramazzini's early obsrevations of the link between musculoskeletal disorders and ergonomic factors. Applied Ergonomics, 2004. 35(1): p. 67-70. 13. van Veelen, M.A. and D.W. Meijer, Ergonomics and design of laparoscopic instruments: results of a survey among laparoscopic surgeons. Journal of Laparoendoscopic and Advanced Surgical Techniques, 1999. 9(6): p. 481-489. 14. Berquer, R., W.D. Smith, and S. Davis, An ergonomic study of the optimum operating table height for laparoscopic surgery. Surgical Endoscopy, 2002. 16: p. 416-421. 15. Berguer, R., et al., An ergonomic comparison of in-line vs pistol-grip handle configuration in a laparoscopic grasper. Surgical Endoscopy, 1998. 12: p. 805-808. 16. Manasnayakorn, S., A. Cuschieri, and G.B. Hanna, Ergonomic assessment of optimum operating table height for hand-assisted laparoscopic surgery. Surgical Endoscopy, 2009. 23: p. 783-789. 17. Manukyan, G.A., et al., Ergonomics with the use of curved versus straight laparoscopic graspers during rectosigmoid resection: results of a multiprofile ergonomic study. Surgical Endoscopy, 2007. 21: p. 1079-1089. 18. Tang, B., et al., Identification and categorization of technical errors by Observational Clinical Human Reliability Assessment (OCHRA) during laparoscopic cholecystectomy. Archives of Surgery, 2004. 139: p. 1215-1220. 19. Martin, J.A., et al., Objective structured assessment of technical skill (OSATS) for surgical residents. British Journal of Surgery, 1997. 84: p. 273-278. 20. Datta, V., et al., Relationship between skill and outcome in the laboratory-based model. Surgery, 2002. 131(3): p. 318-323. 21. Hanna, G.B., Frank T.G., and A. Cuschieri, Objective assessment of endoscopic knot quality. American Journal of Surgery, 1997. 174(4): p. 410-413. 22. Taffinder, N., et al., Can a computer measure surgical precision? Reliability, validity and feasibility of the ICSAD. Surgical Endoscopy, 1999. 13(s1): p. 81. 23. Aggarwal, R., K. Moorthy, and A. Darzi, Laparoscopic skills training and assessment. British Journal of Surgery, 2004. 91(12): p. 1549-1558.

195 24. Koneczny, S. and U. Matern, Instruments for the evaluation of ergonomics in surgery. Minimally Invasive Therapy and Allied Technologies, 2004. 13(3): p. 167-177. 25. Fagerlund, M.J. and L.I. Eriksson, Current concepts in neuromuscular transmission. British Journal of Anaesthesia, 2009. 103(1): p. 108-114. 26. Kamen, G. and G.E. Caldwell, Physiology and interpretation of the electromyogram. Journal of Clinical Neurophysiology, 1996. 13(5): p. 366-384. 27. Berguer, R., et al., A comparison of surgeons' posture during laparoscopic and open surgical procedures. Surgical Endoscopy, 1997. 11(2): p. 139-142. 28. van Det, M.J., et al., Ergonomic assessment of neck posture in the minimally invasive surgery suite during laparoscopic cholecystectomy. Surgical Endoscopy, 2008. 22(11): p. 2421-2427. 29. Lee, G. and A.E. Park, Development of a more robust tool for postural stability analsyis of laparoscopic surgeons. Surgical Endoscopy, 2008. 22(4): p. 1087-1092. 30. McAtamney, L. and E.N. Corlett, RULA: a survey method for the investigation of work- related upper limb disorders. Applied Ergonomics, 1993. 24(2): p. 91-99. 31. Moore, J.S. and A. Garg, The strain index: a proposed method to analyze jobs for risk of distal upper extremity disorders. American Industrial Hygiene Association Journal, 1995. 56(5): p. 443-458. 32. Lee, E.C., et al., Ergonomics and human factors in endoscopic surgery: a comparison of manual vs telerobotic simulation systems. Surgical Endoscopy, 2005. 19(8): p. 1064- 1070. 33. Datta, V., et al., The relationship between motion analysis and surgical technical assessments. American Journal of Surgery, 2002. 184(1): p. 70-73. 34. Emam, T.A., G.B. Hanna, and A. Cuschieri, Ergonomic principles of task alignment, visual display, and direction of execution of laparoscopic bowel suturing. Surgical Endoscopy, 2002. 16(2): p. 267-271. 35. Lee, G., et al., Postural instability does not necessarily correlate to poor performance: case in point. Surgical Endoscopy, 2007. 21(3): p. 471-474. 36. Corlett, E.N. and R.P. Bishop, A technique for assessing postural discomfort. Ergonomics, 1976. 19(2): p. 175-82. 37. van der Schatte Olivier, R.H., et al., Ergonomics, user comfort, and performance in standard and robot-assisted laparoscopic surgery. Surgical Endoscopy, 2009. 23(6): p. 1365-71. 38. Matern, U., et al., MIS instruments. An experimental comparison of various ergonomic handles and their design. Surgical Endoscopy, 1999. 13(8): p. 756-762. 39. Chrousos, G.P. and P.W. Gold, The concepts of stress and stress system disorders. Overview of physical and behavioural homostasis. Journal of the American Medical Association, 1992. 267(9): p. 1244-52. 40. Arora, S., et al., The impact of stress on surgical performance: a systematic review of the literature. Surgery, 2010. 147(3): p. 318-30. 41. Demirtas, Y., et al., Plastic surgeon's life: marvelous for mind, exhausting for body. Plastic and Reconstructive Surgery, 2004. 114(4): p. 923-31. 42. Berguer, R. and W. Smith, An ergonomic comparison of robotic and laparoscopic technique: the influence of surgeon experience and task complexity. Journal of Surgical Research, 2006. 134(1): p. 87-92. 43. Taskforce, Heart rate variability. Standards of measurement, physiological interpretation, and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology Eur Heart J, 1996. 17(3): p. 354-81. 44. Arora, S., et al., The Imperial Stress Assessment Tool (ISAT): a feasible, reliable and valid approach to measuring stress in the operating room. World Journal of Surgery, 2010. 34(8): p. 1756-63. 45. Yang, H.K., et al., Application for the wearable heart activity monitoring system: analysis of the autonomic function of HRV. Conf Proc IEEE Eng Med Biol Soc, 2008. 2008: p. 1258-61. 46. Czyzewska, E., et al., The surgeon's mental load during decision making at various stages of operations. European Journal of Applied Physiology and Occupational Physiology, 1983. 51(3): p. 441-6. 47. Tendulkar, A.P., et al., Quantification of surgical resident stress "on call". J Am Coll Surg, 2005. 201(4): p. 560-4.

196 48. Chrousos, G.P., Stressors, Stress, and neuroendocrine integration of the adaptive response. The 1997 Hans Selye Memorial Lecture. Annals of the New York Acadamy of Sciences, 1998. 851: p. 311-35. 49. de Geus E.J., et al., Ambulatory measurement of respiratory sinus arrhythmia and respiration rate. Biological Psychology, 1995. 41(3): p. 205-227. 50. Takai, N., et al., Effect of psychological stress on the salivary cortisol and amylase levels in healthy young adults. Archives of Oral Biology, 2004. 49(12): p. 963-968. 51. Storm, H., Changes in skin conductance as a tool to monitor nociceptive stimulation and pain. Current Opinion in Anaesthesiology, 2008. 21(6): p. 796-804. 52. Critchley, H.D., Electrodermal responses: what happens in the brain. Neuroscientist, 2002. 8(2): p. 132-42. 53. Veltman, J.A. and A.W. Gaillard, Physiological workload reactions to increasing levels of task difficulty. Ergonomics, 1998. 41(5): p. 656-69. 54. Arora, S., et al., Managing intraoperative stress: what do surgeons want from a crisis training program? American Journal of Surgery, 2009. 197(4): p. 537-43. 55. Wetzel, C.M., et al., The effects of stress on surgical performance. American Journal of Surgery, 2006. 191(1): p. 5-10. 56. Marteau, T.M. and H. Bekker, The development of a six-item short-form of the state scale of the Spielberger State-Trait Anxiety Inventory (STAI). Br J Clin Psychol, 1992. 31 ( Pt 3): p. 301-6. 57. Parasuraman, R. and M. Rizzo, Neuroergonomics - the Brain at Work. 2008: Oxford University Press. 58. Sarter, N. and M. Sarter, Neuroergonomics: opportunities and challenges of merging cognitive neuroscience with cognitive ergonomics. Theoretical Issues in Ergonomics Science, 2003. 4(1-2): p. 142-50. 59. Scerbo, M.W., F.G. Freeman, and P.J. Mikulka, A brain-based system for adaptive automation. Theoretical Issues in Ergonomics Science, 2003. 4(1-2): p. 200-219. 60. Just, M.A., P.A. Carpenter, and A. Miyake, Neuroindices of cognitive workload: neuroimaging, pupillometric and event-related potential studies of brain work. Theoretical Issues in Ergonomics Science, 2003. 4(1-2): p. 56-88. 61. Baldwin, C.L., Neuroergonomics of mental workload: new insights from the convergence of brain and behaviour in ergonomics research Theoretical Issues in Ergonomics Science, 2003. 4(1-2): p. 132-41. 62. Hart, S.G. and L.E. Staveland, Human mental workload, in Human mental Workoad, P.A. Hancock and N. Meshkati, Editors. 1988, North Holland Press: Amsterdam. p. 139-183. 63. Marshall, S.P., The index of cognitive activity: measuring cognitive workload. Proceedings of the IEEE 7th Conference on Human Factors and Power Plants, 2002: p. 7- 5-7-9. 64. Gevins, A. and M.E. Smith, Neurophysiological measures of cognitive workload during human-computer interaction. Theoretical Issues in Ergonomics Science, 2003. 4(1): p. 113-131. 65. Gevins, A. and M.E. Smith, Neurophysiological measures of working memory and individual differences in cognitive ability and cognitive style. Cereb Cortex, 2000. 10(9): p. 829-39. 66. Fairclough, S.H., L. Venables, and A. Tattersall, The influence of task demand and learning on the psychophysiological response. International Journal of Psychophysiology, 2005. 56(2): p. 171-84. 67. Sassaroli, A., et al., Discrimination of mental workload levels in human subjects with functional near-infrared spectroscopy. Journal of Innovative Optical Health Sciences, 2008. 1(2): p. 227-237. 68. Izzetoglu, M., et al., Functional brain imaging using near-infrared technology - assessing cognitive activity in real-life situations. IEEE Engineering in Medicine and Biology Magazine, 2007. 26(4): p. 38-46. 69. Hitchcock, E.M., et al., Automation cueing modulates cerebral blood flow and vigilance in a simulated air traffic control task. Theoretical Issues in Ergonomics Science, 2003. 4(1-2): p. 89-112. 70. Coull, J.T., R.S. Frackowiak, and C.D. Frith, Monitoring for target objects: activation of right frontal and parietal cortices with increasing task time. Neuropsychologia, 1998. 35(12): p. 1325-34.

197 71. Leff, D.R., et al., "Circadian cortical compensation": a longitudinal study of brain function during technical and cognitive skills in acutely sleep-deprived surgical residents. Annals of Surgery, 2010. 252(6): p. 1082-90. 72. Leff, D.R., et al., Laparoscopic skills suffer on the first shift of sequential night shifts: program directors beware and residents prepare. Annals of Surgery, 2008. 247(3): p. 530-9. 73. Taffinder, N.J., et al., Effect of sleep deprivation on surgeons' dexterity on laparoscopy simulator. Lancet, 1998. 352(9151): p. 1191. 74. Calhoun, V.D., Functional Magnetic Resonance Imaging (fMRI) Advanced Methods and Applications to Driving, in Neuroergonomics the Brain at Work, R. Parasuraman and M. Rizzo, Editors. 2008, Oxford University Press. p. 51-64. 75. Spiers, H.J. and E.A. Maguire, Neural substrates of driving behaviour. Neuroimage, 2007. 36(1): p. 245-55. 76. Tsunashima, H. and K. Yanagisawa, Measurement of brain function of car driver using functional near-infrared spectroscopy (fNIRS). Computational Intelligence and Neuroscience, 2009. [Epub ahead of print](164958). 77. Rizzo, M. and I.L. Kellison, Eyes, brains and autos. Archives of Ophthalmology, 2004. 122(4): p. 641-7. 78. Rizzo, M., S. McEvoy, and J. Lee, Medical Safety and Neuroergonomics, in Neuroergonomics the Brain at Work, R. Parasuraman and M. Rizzo, Editors. 2008, Oxford University Press. p. 360-78. 79. Charabati, S., et al., Comparison of four different display designs of a novel monitoring system, the 'integrated monitor of anaesthesia (IMATM). British Journal of Anaesthesia, 2009. 103(5): p. 670-7. 80. Kojima, T., et al., Measurement of train driver's brain activity by functional near-infrared spectroscopy (fNIRS). Optical and Quantum Electronics, 2005. 37(13-15): p. 1319-38. 81. Ayaz, H., et al., Optical brain monitoring for operator training and mental workload assessment. Neuroimage, 2011. Epub ahead of print. 82. Izzetoglu, K., et al., Functional optical brain imaging using near-infrared during cognitive tasks. International Journal of Human-Computer Interaction, 2004. 17(2): p. 211-27. 83. Harada, H., et al., A comparison of cerebral activity in the prefrontal region between young adults and the elderly while driving. Journal of Physiological Anthrapology, 2009. 26(3): p. 409-14. 84. Miyai, I., et al., Cortical mapping of gait in humans: a near-infrared spectroscopic topography study. Neuroimage, 2001. 14(5): p. 1186-92. 85. Harada, T., et al., Gait capacity affects cortical activation patterns related to speed control in the elderly. Experimental Brain Research, 2009. 193(3): p. 445-54. 86. Suzuki, M., et al., Prefrontal and premotor cortices are involved in adapting walking and running speed on the treadmill: an optical imaging study. Neuroimage, 2004. 23(3): p. 1020-6. 87. Leff, D.R., et al., Modelling dynamic fronto-parietal behaviour during minimally invasive surgery--a Markovian trip distribution approach. Med Image Comput Comput Assist Interv Int Conf Med Image Comput Comput Assist Interv, 2008. 11(Pt 2): p. 595-602. 88. Ohuchida, K., et al., The frontal cortex is activated during learning of endoscopic procedures. Surgical Endoscopy, 2009. 23(10): p. 2296-2301. 89. Shin, C., Neurophysiologic basis of functional neuroimaging: animal studies. Journal of Clinical Neurophysiology, 2000. 17(1): p. 2-9. 90. Schurr, A., C.A. West, and B.M. Rigor, Lactate-supported synaptic function in the rat hippocampal slice preparation. Science, 1988. 240(4857): p. 1326-1328. 91. Dirnagl, U., Metabolic aspects of neurovascular coupling. Adv Exp Med Biol, 1997. 413: p. 155-159. 92. Roy, C.S. and C.S. Sherrington, On the regulation of the blood-supply of the brain. Journal of Physiology, 1890. 11: p. 85-108. 93. Obrig, H. and A. Villringer, Beyond the visible - imaging the human brain with light. Journal of Cerebral Blood Flow and Metabolism, 2003. 23(1): p. 1-18. 94. Strangman, G., D.A. Boas, and J.P. Sutton, Non-invasive neuroimaging with near- infrared light. Biological Psychology, 2002. 52(7): p. 679-693.

198 95. Leff, D.R., An assessment of cortical function in novice and expert surgeons, in Department of Biosurgery and Surgical Technology, Imperial College. 2009, University of London: London. p. 298. 96. Shibasaki, H., Human brain mapping: hemodynamic response and electrophysiology. Clinical Neurophysiology, 2008. 119(4): p. 731-43. 97. Jasper, H.H., The ten-twenty electrode system of the international federation. Electroencaphalogr Clin Neurophysiol, 1958. 10: p. 367-380. 98. Ogawa, S., et al., Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proceedings of the National Acadamy of Sciences of the United States of America, 1990. 87(24): p. 9868-9872. 99. Villringer, A. and B. Chance, Non-invasive optical spectroscopy and imaging of human brain function. Trends Neurosci, 1997. 20(10): p. 435-42. 100. Jobsis, F.F., Noninvasive, infrared monitoring of cerebral and myocardial oxygen sufficiency and circulatory parameters. Science, 1977. 198(4323): p. 1264-1267. 101. Delpy, D.T., et al., Estimation of optical pathlength through tissue from direct time of flight measurement. Physics in Medicine and Biology, 1988. 33(12): p. 1433-1442. 102. Okada, E., et al., Theoretical and experimental investigation of near-infrared light propagation in a model of the adult head. Applied Optics, 1997. 36(1): p. 21-31. 103. Hoshi, Y., Functional near-infrared spectroscopy: current status and future prospects. Journal of Biomedical Optics, 2007. 12(6): p. 062106. 104. Zeff, B.W., et al., Retinotopic mapping of adult human visual cortex with high-density diffuse optical tomography. Proceedings of the National Acadamy of Sciences of the United States of America, 2007. 104(29): p. 12169-12174. 105. Okamoto, M., et al., Three-dimensional probabilistic anatomical cranio-cerebral correlation via the international 10-20 system oriented for transcranial functional brain mapping. Neuroimage, 2004. 21(1): p. 99-111. 106. Leff, D.R., et al., Changes in prefrontal cortical behaviour depend upon familiarity on a bimanual co-ordination task: an fNIRS study. Neuroimage, 2008. 39(2): p. 805-13. 107. Zhu, F.F., et al., Implicit motor learning promotes neural efficiency during laparoscopy. Surgical Endoscopy, 2011. [Epub ahead of print]. 108. Wanzel, K.R., et al., Visual-spatial ability and fMRI cortical activation in surgery residents. Am J Surg, 2007. 193(4): p. 507-10. 109. Kelly, A.M. and H. Garavan, Human functional neuroimaging of brain changes associated with practice. Cereb Cortex, 2005. 15(8): p. 1089-102. 110. Petersen, S.E., et al., The effects of practice on the functional anatomy of task performance. Proc Natl Acad Sci U S A, 1998. 95(3): p. 853-60. 111. Leff, D.R., et al., Functional near infrared spectroscopy in novice and expert surgeons--a manifold embedding approach. Med Image Comput Comput Assist Interv Int Conf Med Image Comput Comput Assist Interv, 2007. 10(Pt 2): p. 270-7. 112. Leff, D.R., et al., Functional prefrontal reorganization accompanies learning-associated refinements in surgery: a manifold embedding approach. Comput Aided Surg, 2008. 13(6): p. 325-39. 113. Debaere, F., et al., Changes in brain activation during the acquisition of a new bimanual coordination task. Neuropsychologia, 2004. 42(7): p. 855-867. 114. Wanzel, K.R., et al., Effect of visual-spatial ability on learning of spatially-complex surgical skills. Lancet, 2002. 359(9302): p. 230-231. 115. Wanzel, K.R., et al., Visual-spatial ability correlates with efficiency of hand motion and successful surgical performance. Surgery, 2003. 134(5): p. 750-757. 116. Leff, D.R., et al., Visuo-Spatial ability and fMRI cortical activation in surgery residents. American Journal of Surgery, 2008. 195(1): p. 138. 117. Halsband, U. and R.K. Lange, Motor learning in man: a review of functional and clinical studies. J Physiol Paris, 2006. 99(4-6): p. 414-24. 118. Curtis, C.E., Prefrontal and parietal contributions to spatial working memory. Neuroscience, 2006. 139(1): p. 173-180. 119. Poldrack, R.A., et al., The neural correlates of motor skill automaticity. Journal of Neuroscience, 2005. 25(22): p. 5356-64. 120. Poldrack, R.A., Imaging brain plasticity: conceptual and methodological issues -- a theoretical review. Neuroimage, 2000. 12(1): p. 1-13. 121. Shadmehr, R. and H.H. Holcomb, Neural correlates of motor memory consolidation. Science, 1997. 277(5327): p. 821-825.

199 122. Sidhu, R.S., et al., Interpretation of three-dimensional structure from two-dimensional endovascular images: implications for educators in vascular surgery. Journal of Vascular Surgery, 2004. 39(6): p. 1305-1311. 123. Hedman, L., et al., High-level visual-spatial abililty for novices correlates with performance in a visual-spatial complex surgical simulator. Surgical Endoscopy, 2006. 20(8): p. 1275-1280. 124. Rattner, D. and A. Kalloo, ASGE/SAGES Working Group on Natural Orifice Translumenal Endoscopic Surgery. October 2005. Surg Endosc, 2006. 20(2): p. 329-33. 125. Shepard, R.N. and J. Metzler, Mental rotation of three-dimensional objects. Science, 1971. 171(972): p. 701-3. 126. Baddeley, A., Working memory: looking back and looking forward. Nat Rev Neurosci, 2003. 4(10): p. 829-39. 127. McAfoose, J. and B.T. Baune, Exploring visual-spatial working memory: a critical review of concepts and models. Neuropsychology review, 2009. 19(1): p. 130-142. 128. Curtis, C.E. and M. D'Esposito, The effects of prefrontal lesions on working memory performance and theory. Cognitive, affective and behavioural neuroscience, 2004. 4(4): p. 528-539. 129. Ciaramelli, E., The role of ventromedial prefrontal cortex in navigation: a case of impaired wayfinding and rehabilitation. Neuropsychologia, 2008. 46(7): p. 2099-105. 130. Smith, E.E. and J. Jonides, Storage and executive processes in the frontal lobes. Science, 1999. 283(5408): p. 1657-61. 131. Spiers, H.J., Keeping the goal in mind: prefrontal contributions to spatial navigation. Neuropsychologia, 2008. 46(7): p. 2106-8. 132. Badre, D. and A.D. Wagner, Selection, integration, and conflict monitoring; assessing the nature and generality of prefrontal cognitive control mechanisms. Neuron, 2004. 41(3): p. 473-87. 133. Miller, E.K. and J.D. Cohen, An integrative theory of prefrontal cortex function. Annu Rev Neurosci, 2001. 24: p. 167-202. 134. Ridderinkhof, K.R., et al., The role of the medial frontal cortex in cognitive control. Science, 2004. 306(5695): p. 443-7. 135. Kouneiher, F., S. Charron, and F. Koechlin, Motivation and cognitive control in the human prefrontal cortex. Nature Neuroscience, 2009. 12(7): p. 939-945. 136. Battaglia-Mayer, A., M. Mascaro, and R. Caminiti, The cortical network for eye-hand coordination and its relevance to understanding motor disorders of parietal patients. Neuropsychologia, 2006. 44(13): p. 2607-20. 137. Buneo, C.A. and R.A. Andersen, The posterior parietal cortex: sensorimotor interface for the planning and online control of visually guided movements. Neuropsychologia, 2006. 44(13): p. 2594-606. 138. van der Stigchel, S., et al., The limits of top-down control of visual attention. Acta Psychologica, 2009. 132(3): p. 201-212. 139. Theeuwes, J., Top-down and bottom-up control of visual selection. Acta Psychologica 2010. 135(277-99). 140. Kojima, H. and T. Suzuki, Hemodynamic change in occipital lobe during visual search: visual attention allocation measured with NIRS. Neuropsychologia, 2010. 48(1): p. 349- 352. 141. Sporns, O., G. Tononi, and G.M. Edelman, Connectivity and complexity: the relationship between neuroanatomy and brain dynamics. Neural Networks, 2000. 13(8-9): p. 909-22. 142. Tononi, G., G.M. Edelman, and O. Sporns, Complexity and coherency: integrating information in the brain. Trends in Cogntive Sciences, 1998. 2(12): p. 474-84. 143. Friston, K.J., Functional Integration, in Human Brain Function, R.S.J. Frackowiak, et al., Editors. 2004, Elselvier. p. 971-8. 144. Nucifora, P.G., et al., Diffusion-tensor MR imaging and tractography: exploring brain microstructure and connectivity. Radiology, 2007. 245(2): p. 367-84. 145. Friston, K.J., et al., Functional connectivity: the principal-component analysis of large (PET) data sets. Journal of Cerebral Blood Flow and Metabolism, 1993. 13(1): p. 5-14. 146. Friston, K.J., C.D. Frith, and R.S.J. Frackowiak, Time-dependent changes in effective connectivity measured with PET. Human Brain Mapping, 1993. 1(1): p. 69-79. 147. Friston, K.J., Functional and effective connectivity in neuroimaging: A synthesis. Human Brain Mapping, 1994. 2(1-2): p. 56-78.

200 148. Sun, F.T., et al., Functional connectivity of cortical networks involved in bimanual motor sequence learning. Cereb Cortex, 2007. 17(5): p. 1227-34. 149. White, B.R., et al., Resting-state functional connectivity in the human brain revealed with diffuse optical tomography. Neuroimage, 2009. 47(1): p. 148-56. 150. Greicius, M.D., et al., Resting-state functional connectivity reflects structural connectivity in the default mode network. Cereb Cortex, 2009. 19(1): p. 72-78. 151. Greicius, M.D., et al., Functional connectivity in the resting brain: a network analysis of the default mode hypothesis. Proceedings of the National Acadamy of Sciences of the United States of America, 2003. 100(1): p. 253-8. 152. Homae, F., et al., Development of global cortical networks in early infancy. Journal of Neuroscience, 2010. 30(14): p. 4877-4882. 153. Lu, C.-M., et al., Use of fNIRS to assess resting state functional connectivity. Journal of Neuroscience Methods, 2010. 186(2): p. 242-9. 154. Zhang, H., et al., Functional connectivity as revealed by independent component analysis of resting-state fNIRS measurements. Neuroimage, 2010. 51(3): p. 1150-61. 155. Biswal, B.B., et al., Task-dependent individual differences in prefrontal activity. Cereb Cortex, 2010. 20(9): p. 2188-2197. 156. Ma, L., et al., Changes in regional activity are accompanied with changes in inter- regional connectivity during 4 weeks motor learning. Brain Research, 2010. 1318: p. 64- 76. 157. Brazdil, M., et al., Effective connectivity in target stimulus processing: a dynamic causal modeling study of visual oddball task. Neuroimage, 2007. 35(2): p. 827-35. 158. Im, C.H., et al., Estimation of directional coupling between cortical areas using Near- Infrared Spectroscopy (NIRS). Optics Express, 2010. 18(6): p. 5730-9. 159. Katura, T., et al., Quantitative evaluation of interrelations between spontaneous low- frequency oscillations in cerebral hemodynamics and systemic cardiovascular dynamics. Neuroimage, 2006. 31(4): p. 1592-1600. 160. Euler, L., Solutio problematis ad geometriam situs perinentis. Commentarii Academiae Scientiarum Imperialis Petropolitanae, 1736. 8: p. 128-40. 161. Bullmore, E. and O. Sporns, Complex brain networks: graph theoretical analysis of structural and functional systems. Nature Reviews. Neuroscience, 2009. 10(3): p. 186-98. 162. Sporns, O., G. Tononi, and R. Kotter, The human connectome: A structural description of the human brain. PLoS Computational Biology, 2005. 1(4): p. e42. 163. Bassett, D.S. and E. Bullmore, Small-world brain networks. Neuroscientist, 2006. 12(6): p. 512-23. 164. Watts, D.J. and S.H. Strogatz, Collective dynamics of 'small-world' networks. Nature, 1998. 393(6684): p. 440-2. 165. Achard, S., et al., A resilient, low-frequency, small-world human brain functional network with highly connected association cortical hubs. Journal of Neuroscience, 2006. 26(1): p. 63-72. 166. Milgram, S., The small wall problem. Psychol Today, 1967. 1: p. 61-67. 167. Stam, C.J. and J.C. Reijineveld, Graph theoretical analysis of complex networks in the brain. Nonlinear Biomedical Physics, 2007. 1(1): p. 3. 168. Stephen, K.E., et al., Computational analysis of functional connectivity between areas of primate cerebral cortex. Philosophical Transactions of the Royal Society. Series B, Biological Sciences, 2000. 355(1393): p. 111-26. 169. He, Y., Z.J. Chen, and A.C. Evans, Small-world anatomical networks in the human brain revealed by cortical thickness from fMRI. Cereb Cortex, 2007. 17(10): p. 2407-19. 170. Sporns, O., et al., Organization, development and function of complex brain networks. Trends in Cogntive Sciences, 2004. 8(9): p. 418-25. 171. Strogatz, S.H., Exploring complex networks. Nature, 2001. 410(6825): p. 268-76. 172. Stam, C.J., et al., Small-world networks and functional connectivity in Alzheimer's disease. Cereb Cortex, 2007. 17(1): p. 92-9. 173. Bassett, D.S., et al., Cogntive fitness of cost-efficient functional networks. Proceedings of the National Acadamy of Sciences of the United States of America, 2009. 106(28): p. 11747-52. 174. Bullmore, E.T. and D.S. Bassett, Brain graphs: graphical models of the human brain connectome. Annual Review of Clinical Psychology, 2011. 7: p. 113-140. 175. Wang, J., et al., Parcellation-dependent small-world brain functional networks: a resting- state fMRI study. Hum Brain Mapp, 2009. 30(5): p. 1511-23.

201 176. Bernhardt, B.C., et al., Graph-theoretical analysis reveals disrupted small-world organization of cortical thickness correlation networks in temporal lobe epilipsy. Cereb Cortex, 2011. [Epub ahead of print]. 177. van den Heuvel, M.P., et al., Small-world and scale-free organization of voxel-based resting-state functional connectivity in the human brain. Neuroimage, 2008. 43(3): p. 528-39. 178. Achard, S. and E. Bullmore, Efficiency and cost of economical brain functional networks. PLoS Computational Biology, 2007. 3(2): p. 174-183. 179. Bassett, D.S., et al., Dynamic reconfiguration of human brain networks during learning. Proc Natl Acad Sci U S A, 2011. 108(18): p. 7641-6. 180. Bassett, D.S., et al., Hierarchical organization of human cortical networks in health and schizophrenia. Journal of Neuroscience, 2008. 28(37): p. 9239-48. 181. Palva, J.M., et al., Neuronal synchrony reveals working memory networks and predicts individual memory capacity. Proc Natl Acad Sci U S A, 2010. 107(16): p. 7580-5. 182. Stam, C.J., Functional connectivity patterns of human magnetoencephalographic recordings: a 'small-world' network? Neuroscience Letters, 2004. 355(1-2): p. 25-8. 183. Eguiluz, V.M., et al., Scale-free brain functional networks. Physical Review Letters, 2005. 94(1): p. 018102. 184. Fornito, A., A. Zalesky, and E. Bullmore, Network scaling effects in graph analytic studies of human resting-state fMRI data. Front Syst Neurosci, 2010. 4(22). 185. Lynall, M.-E., et al., Functional connectivity and brain networks in schizophrenia. Journal of Neuroscience, 2010. 30(28): p. 9477-87. 186. Meunier, D., et al., Age-related changes in modular organization of human brain functional networks. Neuroimage, 2009. 44(3): p. 715-23. 187. Salvador, R., et al., Neurophysiological architecture of functional magnetic resonance images of human brain. Cereb Cortex, 2005. 15(9): p. 1332-42. 188. van den Heuvel, M.P., et al., Efficiency of functional brain networks and intellectual performance. Journal of Neuroscience, 2009. 29(23): p. 7619-24. 189. Whitlow, C.T., R. Casanova, and J.A. Maldjian, Effect of resting-state functional MR imaging duration on stability of graph theory. Radiology, 2011. 259(2): p. 516-24. 190. Chen, Z.J., et al., Revealing a modular architecture of human brain structural newtorks by using cortical thickness from MRI. Cereb Cortex, 2008. 18(10): p. 2374-81. 191. Zalesky, A., et al., Whole-brain anatomical networks: does the choice of nodes matter? Neuroimage, 2010. 50(3): p. 970-83. 192. Deuker, L., et al., Reproducibility of graph metrics of human brain functional networks. Neuroimage, 2009. 47(4): p. 1460-8. 193. Langer, N., et al., Functional brain network efficiency predicts intelligence. Human Brain Mapping, 2011. [Epub ahead of print]. 194. Micheloyannis, S., et al., The influence of ageing on complex brain networks: a graph theoretical analysis. Human Brain Mapping, 2009. 30(1): p. 200-8. 195. Reijineveld, J.C., et al., The application of graph theoretical analysis to complex networks in the brain. Clinical Neurophysiology, 2007. 118(11): p. 2317-31. 196. Rubinov, M. and O. Sporns, Complex network measures of brain connectivity: uses and interpretations. Neuroimage, 2010. 52(3): p. 1059-69. 197. Latora, V. and M. Marchiori, Efficient behaviour of small-world networks. Physical Review Letters, 2001. 87(19). 198. Latora, V. and M. Marchiori, Economic small-world behavior in weighted networks. European Physical Journal B, 2003. 32(2): p. 249-63. 199. Smith, S.M., et al., Network modelling methods for fMRI. Neuroimage, 2011. 54(2): p. 875-91. 200. van Wijk, B.C., C.J. Stam, and A. Daffertshofer, Comparing brain networks of different size and connectivity density using graph theory. PLoS ONE, 2010. 5(10): p. e13701. 201. De Vico Fallani, F., et al., A graph-theoretical approach in brain functional networks. Possible implications in EEG studies. Nonlinear Biomedical Physics, 2010. 4(Suppl 1): p. S8. 202. Tzourio-Mazoyer, N., et al., Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage, 2002. 15(1): p. 273-89. 203. Wylie, G.R., et al., Using co-variations in the Hb signal to detect visual activation: a near infrared spectroscopic imaging study. Neuroimage, 2009. 47(2): p. 473-81.

202 204. Ricciardi, E., et al., Neural correlates of spatial working memory in humans: a functional magnetic resonance imaging study comparing visual and tactile processes. Neuroscience, 2006. 139(1): p. 339-49. 205. Clark, J., et al., The Natural Orifice Simulated Surgical Environment (NOSsE): Exploring the Challenges of NOTES Without the Animal Model. J Laparoendosc Adv Surg Tech A, 2009. 19(2): p. 211-4. 206. Oldfield, R.C., The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia, 1971. 9(1): p. 97-113. 207. Dager, S.R. and S.D. Friedman, Brain imaging and the effects of caffeine and nicotine. Annals of Medicine, 2000. 32(9): p. 592-99. 208. Higashi, T., et al., Changes in regional cerebral blood volume in frontal cortex during mental work with and without caffeine intake: functional monitoring using near-infrared spectroscopy. Journal of Biomedical Optics, 2004. 9(4): p. 788-93. 209. Obrig, H., et al., Length of resting period between stimulation cycles modulates hemodynamic response to a motor stimulus. Adv Exp Med Biol, 1997. 411: p. 471-80. 210. Jurcak, V., D. Tsuzuki, and I. Dan, 10/20, 10/10, and 10/5 systems revisited: their validity as relative head-surface-based positioning systems. Neuroimage, 2007. 34(4): p. 1600- 11. 211. Orihuela-Espina, F., et al., ICNA: A software tool for manifold embedded based analysis of functional near infrared spectroscopy. Neuroimage, 2009. 47(Supplement 1): p. S59. 212. Leff, D.R., et al., Assessment of the cerebral cortex during motor task behaviours in adults: a systematic review of functional near infrared spectroscopy (fNIRS) studies. Neuroimage, 2011. 54(4): p. 2922-36. 213. Tenenbaum, J.B., V. de Silva, and J.C. Langford, A global geometric framework for nonlinear dimensionality reduction. Science, 2000. 290(5500): p. 2319-23. 214. Kruskal, J.B., Multidimensional scaling by optimizing goodness for fit to a nonmetric hypothesis. Psychometrika, 1964. 29(1): p. 1-26. 215. Floyd, R.W., Algorithm 97: Shortest path. Communications of the ACM, 1962. 5(6): p. 345. 216. Rubner, Y., C. Tomasi, and L.J. Guibas, A metric for distributions with applications to image databases. Sixth International Conference on Computer Vision, 1998: p. 59-66. 217. Rubner, Y., C. Tomasi, and L.J. Guibas, The earth mover's distance as a metric for image retrieval. International Journal of Computer Vision, 2000. 40(2): p. 99-121. 218. Jain, A.K., M.N. Murty, and P.J. Flynn, Data clustering: A review. ACM Computing Surveys, 1999. 31(3): p. 264-323. 219. Funahashi, S., Prefrontal cortex and working memory processes. Neuroscience, 2006. 139(1): p. 251-261. 220. Kennerley, S.W. and J.D. Wallis, Reward-dependent modulation of working memory in lateral prefrontal cortex. Journal of Neuroscience, 2009. 29(10): p. 3259-3270. 221. Badre, D., Cognitive control, hierarchy, and the rostro-caudal organization of the frontal lobes. Trends in Cogntive Sciences, 2008. 12(5): p. 193-200. 222. Koechlin, F., C. Ody, and F. Kouneiher, The architecture of cognitve control in the human prefrontal cortex. Science, 2003. 302(5648): p. 1181-1185. 223. Carter, C.S. and V. van Veen, Anterior cingulate cortex and conflict detection: an update of theory and data. Cognitive, affective and behavioural neuroscience, 2007. 7(4): p. 367- 379. 224. Puttemans, V., N. Wenderoth, and S.P. Swinnen, Changes in brain activation during the acquisition of a multifrequency bimanual coordination task: from the cognitive stage to advanced levels of automacity. Journal of Neuroscience, 2005. 25(17): p. 4270-4278. 225. Arnsten, A.F., Stress signalling pathways that impair prefrontal cortex structure and function. Nature Reviews. Neuroscience, 2009. 10(6): p. 410-422. 226. Pruessner, J.C., et al., Deactivation of the limbic system during acute psychosocial stress: evidence from positron emission tomography and functional magnetic resonance imaging studies. Biological Psychiatry, 2008. 63(2): p. 234-240. 227. LLoyd-Fox, S., A. Blasi, and C.E. Elwell, Illuminating the developing brain: the past, present and future of functional near infrared spectroscopy. Neuroscience and Biobehavioral Reviews, 2010. 34(3): p. 269-84. 228. Mylonas, G.P., et al., Gaze-contingent motor channelling and haptic constraints for minimally invasive robotic surgery. Med Image Comput Comput Assist Interv Int Conf Med Image Comput Comput Assist Interv, 2008. 11(Pt 2): p. 676-83.

203 229. Yang, G.Z., et al., Perceptual docking for robotic control. Medical Imaging and Augmented Reality, Proceedings, 2008. 5128: p. 21-30. 230. Land, M.F., Eye-hand coordination: learning a new trick. Current Biology, 2005. 15(23): p. 955-956. 231. Fafrowicz, M. and T. Marek, Quo vadis, neuroergonomics? Ergonomics, 2007. 50(11): p. 1941-1949. 232. Rowe, J., et al., Attention to action: specific modulation of corticocortical interactions in humans. Neuroimage, 2002. 17(2): p. 988-98. 233. Hahn, B., T.J. Ross, and E.A. Stein, Neuroanatomical dissociation between bottom-up and top-down processes of visuospatial selective attention. Neuroimage, 2006. 32(2): p. 842-53. 234. Leff, D.R., et al., Could variations in technical skills acquisition in surgery be explained by differences in cortical plasticity? Ann Surg, 2008. 247(3): p. 540-3. 235. Koh, P.H., et al., Functional optical signal analysis: a software tool for near-infrared spectroscopy data processing incorporating statistical parametric mapping. Journal of Biomedical Optics, 2007. 12(6): p. 064010. 236. Liu, J., S.C. Cramer, and D.J. Reinkensmeyer, Learning to perform a new movement with robotic assistance: comparison of haptic guidance and visual demonstration. Journal of NeuroEngineering and Rehabilitation, 2006. 3(20). 237. Floyer-Lea, A. and P.M. Matthews, Distinguishable brain activation networks for short- and long-term motor skill learning. Journal of Neurophysiology, 2005. 94(1): p. 5r12-8. 238. Coynel, D., et al., Dynamics of motor-related functional integration during motor sequence learning. Neuroimage, 2010. 49(1): p. 759-66. 239. Steele, C.J. and V.B. Penhune, Specific increases within global decreases: a functional magnetic resonance imaging investigation of five days of motor sequence learning. Journal of Neuroscience, 2010. 30(24): p. 8332-41. 240. Orihuela-Espina F., et al., Quality control and assurance in functional near infrared spectroscopy (fNIRS) experimentation. Physics in Medicine and Biology, 2010. 55: p. 1- 24. 241. Mulliken, G.H., S. Musallam, and R.A. Anderson, Forward estimation of movement state in posterior parietal cortex. Proc Natl Acad Sci U S A, 2008. 105(24): p. 8170-8177. 242. Buneo, C.A., et al., Direct visuomotor transformations for reaching. Nature, 2002. 416(6881): p. 632-6. 243. Ferguson, S. and B.C. Stack, Addition of a telestrator for teaching during video-based procedures. Otolaryngology-head and neck surgery, 2010. 143(1): p. 159-160. 244. Schlachta, C.M., et al., Mentoring and telementoring leads to effective incorporation of laparoscopic colon surgery. Surgical Endoscopy, 2010. 24(4): p. 841-844. 245. Micali, S., et al., Feasibility of telementoring between Baltimore (USA) and Rome (Italy): the first five cases. Journal of Endourology, 2000. 14(6): p. 493-496. 246. Brennan, S.E., et al., Coordinating cognition: the costs and benefits of shared gaze during collaborative search. Cognition, 2008. 106(3): p. 1465-77. 247. Avraham, T., Y. Yeshurun, and M. Lindenbaum, Predicting visual search performance by quantifying stimulus similarities. Journal of Vision, 2008. 8(4:9): p. 1-22. 248. Tachtsidis, I., et al., Functional optical topography analysis using statistical parametric mapping (SPM) methodology with and without physiological confounds. Adv Exp Med Biol, 2010. 662: p. 237-43. 249. Bann, S.D., M.S. Khan, and D. A.W., Measurement of surgical dexterity using motion analysis of simple bench tasks. World Journal of Surgery, 2003. 27(4): p. 390-4. 250. Xeroulis, G., A. Dubrowski, and K. Leslie, Simulation in laparoscopic surgery: a concurrent validity study for FLS. Surgical Endoscopy, 2009. 23(1): p. 161-5. 251. Kanwisher, N. and E. Wojciulik, Visual attention: insights from brain imaging. Nature Reviews. Neuroscience, 2000. 1(2): p. 91-100. 252. Eichele, T., et al., Prediction of human errors by maladaptive changes in event-related brain networks. Proceedings of the National Acadamy of Sciences of the United States of America, 2008. 105(16): p. 6173-8. 253. Coyle, S.M., T.E. Ward, and C.M. Markham, Brain-computer interface using a simplified functional near-infrared spectroscopy system. Journal of Neural Engineering, 2007. 4(3): p. 219-26.

204 254. Sitaram, R., et al., Temporal classification of mutlichannel near-infrared spectroscopy signals of motor imagery for developing a brain-computer interface. Neuroimage, 2007. 34(4): p. 1416-27. 255. Caproni, M., et al. An Analysis Framework for Near Infrared Spectroscopy Based Brain Computer Interface and Prospective Application to Robotic Surgery. in IEEE/RSJ International Conference on Intelligent Robots and Systems. 2009. St. Louis.

205