CALIFORNIA STATE UNIVERSITY, NORTHRIDGE Nonlinear Dynamics on a Novel Neural Topology a Thesis Submitted in Partial Fulfilment O

CALIFORNIA STATE UNIVERSITY, NORTHRIDGE

Nonlinear Dynamics on a Novel Neural Topology

A thesis submitted in partial fulfilment of the requirements For the degree of Master of Science in Physics

Alan Perstin

May 2019

The thesis of Alan Perstin is approved:

______Dr. Eric Collins Date

______Dr. Tyler Luchko Date

______Dr. Yohannes Shiferaw, Chair Date

California State University, Northridge

Table of Contents

Signature Page ii

List of Figures v

Abstract vii

Chapter 1: Introduction 1

Chapter 2: Preliminary Biomedical Physics 5 2.1: Preliminaries Concerning Neurophysiology 5 2.2: Preliminaries Concerning Neural Diagnostics 11 2.3: Motivating the Island Network 15

Chapter 3: An Introduction to Graph Theory 20 3.1: Preliminaries Concerning Graphs 20 3.2: Graph Topology 21 3.2.1: The Adjacency Matrix 21 3.2.2: Degree Distribution and Edge Density 23 3.2.3: Average Shortest Path Length 25 3.2.4: Clustering Coefficient 27 3.2.5: Small-Worldness 29 3.2.6: Measures of Centrality 30 3.2.7: Hubs 34 3.2.8: Communities 37 3.2.9: Modularity 39 3.3: Topologies of Common Graphs 43 3.3.1: Random Networks 44 3.3.2: Small-World Networks 46 3.3.3: Scale-Free Networks 49 3.3.4: Closing Remarks 50

Chapter 4: Topology of the Island Network 52 4.1: Generating the Island Network 52 4.1.1: Properties of the Island Network 53 4.2: General Metrics 60 4.2.1 Degree Distribution and Edge Density 61 4.2.2 Small-Worldness 61 4.2.3 Hubs 63 4.2.4 Modular Communities 65 4.3 Discussion of Topology 66

Chapter 5: Dynamics of the Island Network 68 5.1: Preliminaries Concerning Connectome Dynamics 68 5.2: Nonlinear Stochastic Signal Spreading 69 5.3: Monte-Carlo Analysis 72 5.3.1: Dynamics as a Function of Community Structure 74 5.3.2: Dynamics as a Function of Hubs 80 5.4: Dynamics as a Function of Topology 82

iii

Chapter 6: Conclusions 86 6.1: Discussion 86 6.2: Preliminaries Concerning Future Investigations 87

References 90

Appendix 1: List of Network Properties 95

Appendix 2: Algorithms 96

List of Figures

2.1: “Morphology of a Typical Neuron” 6

3.1: “A Human Connectome” 20

3.2: “A Graph-Theoretical Model of the Internet” 22

3.3: “A Graph and its Corresponding Adjacency Matrix” 23

3.4: “A Graph and its Corresponding Degree Distribution” 24

3.5: “Two Different Graphs with Identical Degree Distributions” 25

3.6: “Shortest Path Between Two Nodes” 27

3.7: “Clustering Coefficient: An Illustration” 28

3.8: “Centrality Measures: An Illustration” 32

3.9: “Connector Hubs and Provincial Hubs” 36

3.10: “Modularity: An Illustration” 39

3.11: “Modularity in a High School Social Network” 40

3.12: “An Erdos Reyni Graph” 45

3.13: “Clustering Coefficient against Average Path Length” 47

3.14: “Watts Strogatz Graph with β Modulated” 49

4.1: “Watts-Strogatz Compared to the Island Network” 53

4.2: “Degree Distributions: Watts-Strogatz Compared to the Island Network (1)” 56

4.3: “Island Network with Modulated” 59

4.4: “The Island Network and휅 an Equivalent Random Network” 62

4.5: “The Island Network Modulated Across Classes 1 and 2” 62

4.6: “Degree Distributions: Watts-Strogatz Compared to the Island Network (2)” 65

5.1: “Monte Carlo Plots” 72

5.2: “An Illustration of Individual Activation” 75

5.3: “Pearson Test of Individual Activations” 76

5.4: “An Illustration of Activation in Series” 77

5.5: “Pearson Test of Activations in Series” 78

5.6: “An Illustration of Activation in Parallel” 79

5.7: “Pearson Test of Activations in Parallel” 80

6.1: “A Graph with a Rich Club” 89

Abstract

Nonlinear Dynamics on a Novel Neural Topology

By Alan Perstin

Master of Science in Physics

In this study, we seek to develop a simple model of the human brain that reproduces certain characteristic structural and dynamical features. The structural features of interest, which are cast in terms of graph theory, are small-worldness, the presence of hub structures, and highly modular communities. The presence of these features in our model is motivated by the wealth of empirical research in diagnostic biomedical physics that reports these characteristics in real human brains. The dynamical feature of interest, which we acquire via a Monte Carlo analysis of the nonlinear stochastic signals that we spread across our model, is metastability as a function of community structures. The presence of this feature in our model is motivated by the metastability theory of the brain, which itself is a product of decades of research into the biophysics of neural oscillations.

vii

Chapter 1: Introduction

The typical human brain contains approximately 85 billion neurons that interact with one another via electrochemical signals that pass through the tens of trillions of junctions where the neurons intersect[1]. Despite this extraordinarily large number of complexly- interacting components, it has nevertheless been a central aim of neuroscientists to develop a complete map (connectome) of the neural wirings of the human brain in order to better understand its structure and dynamics. The Human Connectome Project, analogously to the

Human Genome Project, seeks to develop a better understanding of the underlying neural architecture of the human brain in order to facilitate advances in both the clinical and theoretical realms, with the former making use of this knowledge to diagnose and treat brain- related illnesses like Alzheimer’s and schizophrenia[2][3][4], and the latter making use of this knowledge to develop increasingly sophisticated artificial neural networks[5].

In both cases, a clearer picture of the structure and dynamics of the human brain is sorely needed, and though much progress has been made in identifying general structural and dynamical features of connectomes, no mathematical or algorithmic “recipe” for a generalized model of the human brain has been forthcoming. More specifically, the connectonomic literature returns no results for a generalized model of the human brain that captures its characteristic structural and dynamical features at every scale, and this study represents an attempt at beginning to remedy this deficit.

Current applications of the widely used Watts-Strogatz[6] model to the clinical side of the connectome literature are generally restricted to a descriptive context; that is to say, imaging studies of the human brain consistently report on its small-world properties as well as the manner in which those properties are degraded in patients who present with neurophysiological syndromes[4][7][8][9], thus offering a description of the manner in which small-worldness contributes to the structure of the human brain. It has long been known, however, that small-world models alone are insufficient to adequately capture the structural features of the brain[10][11], which is why they are often used in tandem with other graph- 1 theoretical structures when appraising certain features of human brains[12][13]. That said, modifications to Watts-Strogatz graphs are typically restricted to mathematical and computational research adjacent to the connectonomic literature[14][15], and if the type of modification being introduced in this study has already been introduced in one of these adjacent fields, the current status of said model is that it has not captured the attention of the wider connectonomic research community. As far as may be reasonably ascertained via thorough engagement with the literature, modified Watts-Strogatz models of the sort being introduced in this study have not been employed in service of describing and explaining the structure of human brains, or the dynamics that follow from that structure.

Our approach to the problem of brain structure is distinct in that we introduce a modification to a commonly used structural model that results in an improved graph- theoretical model that exhibits nontrivial structural properties that are not shared by its predecessor. In this study, we introduce a modified version of the Watts-Strogatz “small- world” network[6], which we have named the “Island Network,” that is sufficiently different from its precursor to warrant its characterization as a novel network structure. We aim to establish this distinction via several topological metrics, before proceeding to spread a nonlinear stochastic signal across the Island Network in order to simulate the flow of electrochemical signal across neuronal networks. As we shall see, under the metastability paradigm[16][17][18], the Island Network’s structure is particularly well-disposed to facilitating the kind of signal behavior that explains how large-scale brain dynamics can emerge from small-scale neuronal interactions. As an added bonus, this model’s structure and dynamics are consistently concordant with experimental results from diagnostic studies involving tomography[7][19][20], functional magnetic resonance imaging (fMRI)[11][8][9], magnetoencephalography (MEG)[22][23], and electroencephalography (EEG)[24][25].

The existing connectome literature on the topic of dynamics employs a plurality of approaches to modelling the biophysics of the human brain, including treatments of neuronal dynamics in terms of nonlinear coupled oscillators across graph-theoretical structures, as in

2 the case of Hodgkin-Huxley and Kuromoto models of neural interaction[26][27][28]; nonlinear

Markov processes (without reference to graph-theoretical structures), as in the case of hybrid models that employ Kolmogorov relations[29]; and more simple linear threshold models across graph-theoretical structures[30]. As far as could be reasonably ascertained via thorough engagement with the literature, there appear to be no studies in which the elements of graph theory, stochasticity, and nonlinearity all simultaneously present in descriptions of neuronal dynamics. Our analysis of the dynamics, which does precisely this, is to the best of our knowledge the first of its kind within the context of human connectomes, and certainly with respect to the simple model being introduced. To briefly summarize the situation: some dynamical models make use of graph-theoretic models of the human brain; some make use of nonlinear dynamics in describing interactions between the basic constituents of brains; some make use of stochastic processes in describing the spread of signal through a connectome; and some involve a conjunction of some of the above. None make use of all of the above.

None involve the spread of nonlinear stochastic signal across a graph-theoretical structure.

In this study, we aim to demonstrate the uniqueness of the model of the human brain being introduced, as well as the predictive capabilities that are afforded by its structural features. We will attempt to explain how our results may be understood in terms generally accepted by the connectonomic research community, before finally proceeding to offer recommendations for future expansions of our model into a less simple, more mature version that more fruitfully captures those structural and dynamical features which the Human

Connectome Project endeavors to comprehensively understand.

What follows is a brief outline of what may be expected of this thesis: first, a basic primer on the neurophysiology and biomedical diagnostics used to appraise the human brain’s structure and dynamics. We will motivate the development of a model with particular graph-theoretic features via a brief overview of where the literature stands with regard to the brain’s emergent structural features.

This will be followed up with an introduction to the mathematical framework that supplies the mathematical paradigm employed by this thesis to describe and make sense of the topics under discussion; that framework includes descriptions and explanations of the adjacency matrix, degree distribution, edge density, average shortest path length, clustering coefficient, small-worldness, measures of centrality, hubs, communities, and modularity.

These concepts will be used to appraise the large-scale structures of commonly analyzed graphs, including random networks, small-world networks, and scale-free networks.

This will lead into the introduction of our model, the Island Network, and a characterization of its properties in terms of the paradigm introduced in the previous chapter.

Understanding both those terms and the types of properties that they describe will arm the reader with the conceptual framework needed to recognize that our model is successful in generating the structural properties of the human brain that our model seeks to capture. A summary discussing the particulars of our model and its implications will conclude the chapter.

We will then motivate and develop the dynamical model used to understand the time- dependent behavior of neural interactions, exploring the basics of connectome dynamics, nonlinear stochastic signal spreading, and the means by which our analysis is carried out. We will then explore how that behavior may ultimately be understood in terms of the structural model, and we will explain our results in terms of the dominant theoretical paradigm that pertains to dynamics in the connectonomic literature. Finally, we will close with a summary of our results and our recommendations for future improvements to our simple model of the human brain, the Island Network.

Chapter 2: Preliminary Biomedical Physics

2.1: Preliminaries Concerning Neurophysiology

The specific mathematical features of the Island Network, as well as the physical features of the simulated signals passing through it, were motivated by certain characteristics of the physiology of the central nervous system (CNS). As a result, we will open with a brief exposition of those neurophysiological attributes which motivate the mathematical formalism and physical theory that supply this study’s paradigm. Unless indicated otherwise, all of the material in this section comes from an introductory text on the subject of neurophysiology[31].

The human brain is composed of approximately 85 billion neurons and glia, with the former serving as the substrate for the spreading, processing, and integration of information, and the latter principally providing support and maintenance via, among other things, the tactical redistribution of ions for the purpose of regulating the external chemical environment, the production of myelin for the purpose of providing insulation, and the performance of phagocytic function for the purpose of clearing cellular debris. For the purposes of this study, the roles played by glial cells will be mostly disregarded since our primary interest is in the physics that underlie the flow of electrochemical signals through the human brain, and it is neurons that facilitate this process. Figure 2.1 illustrates a typical neuron.

Neurons, in addition to containing cellular nuclei and all the cytosolic constituents that are typical of animal cells, possess unique morphological features in the form of neurites, which are subdivided into two categories called axons and dendrites. Axons are long, slender protrusions that originate from a region in neuronal soma called the axon hillock, of which every healthy neuron contains no more than one. As the protrusion, the axon proper, extends away from the rest of the soma, it frequently branches off into a multitude of perpendicular channels called axon collaterals; these in turn terminate at regions called axon terminals, which store chemicals called neurotransmitters inside of vesicles. The axon terminal ends at a bulb-shaped region which connects to the soma and dendrites of other neurons.

Figure 2.1: morphology of a typical neuron. “Multipolar Neuron” licensed under Creative Commons by Blausen Medical Communications, Inc.

The locations of these connections, synapses, are where the neurotransmitters are expelled from the axon and received by receptors on the dendrites or soma. Thus synapses are effectively the junctions in neuronal circuits, whose basic elements are the neuron and the neurites. Within a neural circuit, the cell which transmits the chemical signal is referred to as the presynaptic neuron, and the cell which receives the chemical signal is called the postsynaptic neuron. The gap between the axon terminal and the postsynaptic neuron is called the synaptic cleft, and is saturated with and ions, which as we shall see play a 2+ + central role in the transmission of chemicalCa signalsNa between neurons.

Neurotransmitters, though highly varied in their chemical structures and functions, may nevertheless be simply categorized in terms of their electrochemical effects on a postsynaptic neuron; they are either excitatory, meaning that their contribution to the membrane voltage of postsynaptic neurons is positive, or they are inhibitory, meaning that their contribution the membrane voltage of postsynaptic neurons is negative. To understand what exactly this means, some discussion concerning the biophysics of action potentials is licensed.

The membrane of a neuron at rest (that is, a neuron that isn’t currently releasing neurotransmitters) establishes a potential difference between the cell’s negatively charged interior and positively charged exterior. The negatively charged fluid within the cytoplasm is electrostatically attracted to the positively charged extracellular fluid outside of the cell membrane, causing their attendant charges to line the different sides of the phospholipid bilayer, producing a potential difference across it. The neuronal membrane is thus effectively a capacitor, with the interior serving as the anode and the exterior serving as the cathode.

Because a plurality of ions, most notably , , , and , line the interior and + − + 2+ exterior of the neuron, there exist multipleK concentrationCl Na gradientsCa across the neuronal membrane, which are negotiated via the tactical opening and closing of gated ion channels and the selective initiation of ion pumps. These channels and pumps work in concert to produce a state of electrochemical equilibrium between the cell’s interior and exterior, supplying a net potential difference of approximately in order to counteract the forces associated with the ions’ concentration gradients.−70 Thus mV the voltage across a neuron’s membrane is determined by the varying concentrations of ions within and without the cell, which in turn are manipulated by ion channels and ion pumps.

The flow of a signal through a neural circuit is initiated by an electrochemical impulse called the action potential, which is typically stimulated within a postsynaptic neuron by the interaction between its neuroreceptors and the neurotransmitters delivered to them. Excitatory neurotransmitters will typically be delivered to dendrites, the neurites which project from the neural soma and branch out to the axons of presynaptic neurons, while inhibitory neurotransmitters will typically be delivered directly to the soma. Once the neurotransmitters bind with their corresponding receptors on the postsynaptic neuron, they will have the effect of opening various ion channels across the membrane, which changes the voltage across it.

Excitatory neurotransmitters will have the net effect of decreasing the potential difference while inhibitory neurotransmitters will have the net effect of increasing it; the former depolarizes the neural membrane while the latter hyperpolarizes it.

Once a threshold voltage (typically ) is reached at the membrane surrounding the axon hillock, the action potential is initiated−35 viamV the opening of voltage-gated ion channels that admit positively charged ions from the extracellular fluid into the cytoplasm, changing the concentration gradient and initiating a chain reaction that results in the opening of all of the membrane’s ion channels. This exponentially raises the neuron’s electric potential such that the voltage across its membrane reverses polarity; indeed, the voltage typically peaks at around before the ion channels close and the positively charged ions are actively pumped40 out mV of the neuron in order to restore its resting potential. In the CNS, the ion principally involved in the initiation of the action potential is . + Once the excitatory neurotransmitters’ contributions toNa the neuron’s potential outweighs those of the inhibitory neurotransmitters sufficiently to make the voltage across the membrane at the axon hillock reach the threshold, the aforementioned process generates an impulse that travels down the axon proper and into its terminals. To minimize the leakage of the attendant current, the axon is insulated by myelin, which is wrapped around the majority of its length; regularly spaced regions along the axon where such insulation is absent are called nodes of Ranvier, and are present in many neurons in order to maximize the speed with which the current travels to the axon terminals. These exposed regions are highly enriched in voltage-dependant ion channels, allowing the action potential to be regenerated at each node at regular intervals such that the current travels across the axon in discrete steps rather than continuously.

Upon reaching the axon terminal, the action potential triggers voltage-dependent calcium channels, which admit into the terminal. These calcium ions trigger the release 2+ of neurotransmitter-laden vesicles,Ca which fuse with the membrane at the end of the axon terminal and expel their contents into the synaptic cleft. Now free to travel across the synapse, these neurotransmitters arrive at receptors waiting for them at the soma and dendrites of a postsynaptic neuron. The process then repeats itself, with each action potential generally firing over the course of about .

2 ms 8

The dominant neurophysiological paradigm pertaining to the brain’s structural organization is that its higher-order properties, like neural oscillations and its regional divisions, emerge from lower-order phenomena, like the electrochemical interactions between neurons and the neural architecture of individual circuits. Since our task is to produce a simple model of the human brain that allows large-scale structural and dynamical features to emerge from small-scale structural and dynamical features, it pays to have some heuristic for organizing the different scales of the brain.

One difficulty with this, however, is that at every scale, there exists an exhaustingly diverse set of phenomena that cannot be easily compartmentalized within clean and discrete categories. Recognizing that modelling such an extraordinarily complex system as the human brain necessitates simplifications as a matter of course, we have elected to briefly detail the various nuances that our model will be omitting in accordance with the scope of this study.

We maintain that the simplification of these phenomena is both warranted by precedents set in the literature[2][3][4][19][8][25][12][13][32][33][34] and, at least at this stage, necessary for the development of a model that is sufficiently malleable as to be able to incorporate these details in future studies, which we anticipate will build upon our simple model in ways that will successfully apply those nuances that we have discarded. Nevertheless, for the sake of completion and full disclosure, we will be outlining what some of those details are.

First of all, the most fundamental components of human connectomes, neurons, are not uniform in either their structure or their function. The preceding exposition on the physiology and biophysics of neuronal circuits will be sufficient for the purposes of this study, but we note here that our implicit portrayal of neurons as approximately interchangeable is stretching the truth.

To start with, not all neurons possess both axons and dendrites; some have only one or the other, and some synapses involve only one type of neurite (axon-to-axon and dendrite-to- dendrite.) Additionally, neurons come in a number of shapes and serve a plurality of different functions, exhibit a variety of electrophysiological properties in terms of their action

9 potentials, and may even differ in terms of how they transmit signals; for example, some synapses are purely electrical, involving only the exchange of ions between neurons without the involvement of any neurotransmitters. Moreover, evidence has emerged of the existence of neural backpropagation, in which the action potential can trigger the release of sodium ions from the branches of a post-synaptic neuron’s dendrites into their corresponding synapses and pre-synaptic neurons, though the strength of the resulting signal is quite weak compared to that of the standard propagation mechanism detailed above. These details are important because the structural and dynamical models that we introduce are simplifications that treat all neuronal ensembles as though they transmit signals of the same type, and uniformly with respect to direction.

At the meso-scale, neurons accumulate into structures that in the CNS are typically organized into two categories: nuclei, which are composed of a multitude of neural circuits underneath the cortices, and layers, which characterize the architecture of cerebellar and cerebral cortices. Nuclei are clusters of neurons located deep within the cerebral hemispheres and brainstem, and are composed of uniquely identifiable groups of interconnected neural circuits whose constituent neurons exhibit roughly the same structures and functions; within the vertebrate brain, there exist hundreds of identifiable nuclei (From this point forward, unless otherwise specified, “nucleus” shall refer to these bundles of neurons rather than the organelle of the same name.) Layers are found exclusively in the cortical regions of the brain, and are composed of unique distributions of neurons that are folded on top of each other into distinct sections. Layers may be further subdivided into modules that are functionally indistinguishable from nuclei, and likewise number in at least the hundreds. Both nuclei and the modular subdivisions of layers shall hereafter be collectively referred to as neuronal ensembles, and will be treated identically.

Neuronal ensembles are connected to one another via tracts, which are bundles of axons. Tracts which connect nuclei may be classified as different types of fibers depending on the regions that are connected; intra-hemispheric connections are facilitated by association

10 fibers, inter-hemispheric connections are facilitated by commissural fibers, and connections between cortical and sub-cortical regions are facilitated by projection fibers. Meanwhile, connections between the modular subdivisions of layers are facilitated by inter-cortical tracts.

As with neurons and neuronal ensembles, this study will not distinguish between different types of tracts, whose various categories are mentioned only for the sake of completion.

Finally, at the macro-scale, neuronal ensembles accumulate into brain regions, which are sections of the brain that serve unique functions. The definition of “brain region” will vary from context to context and can range from a single neuronal ensemble to an entire hemisphere, with individual studies identifying “regions of interest” (ROIs) from the outset in order to establish the parameters of investigation. Most generally, however, the major regions of the brain are the cerebrum, the cerebellum, and the brainstem. None of these by themselves qualify as neuronal ensembles, though some of their subdivisions, and certainly many of their sub-subdivisions, do.

2.2: Preliminaries Concerning Neural Diagnostics

The diagnostic equipment typically employed to acquire the data informing our understanding of neural architecture includes electroencephalography (EEG), magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI), and tomographic modalities like positron emission tomography (PET) and computed tomography

(CT). In this section, we will briefly introduce these modalities, some of their properties, and some of their limitations for the purpose of clarifying the means by which the data was collected by many of the empirical studies referenced throughout this text. That data was subsequently reorganized via computational techniques, most notably among them nonmetric multidimensional scaling[35] (which will be briefly summarized in the next section), into what we would recognize as the network structures that played a role in motivating this thesis.

Unless otherwise specified, the information contained within this section was supplied by an introductory text to medical physics[36].

EEG involves the insertion of non-invasive electrodes into regions of the scalp that lie above ROIs on the outermost layer of the brain, the cortex. As the cortical neurons experience incremental changes in their membrane potentials due to the synaptic outputs of their neighbors, the ions that diffuse across the membranes produce electrical currents which, aggregated over many neurons, may synchronize to produce a wave in a process called volume conduction. The electrodes on the subject’s scalp detect the resultant changes in voltage, transmitting the signals down individual wires where they are ultimately digitized.

Despite having excellent time resolution, EEG suffers from poor spatial resolution and an attendant inability to accurately measure sub-cortical voltage changes. Consequently,

EEG can only detect the synchronous activity of many neural circuits, and is unable to reliably map the unique activity of individual regions.

MEG, analogously to EEG, measures the magnetic fields induced by synchronous neuronal activity. The magnetic fields generated by neurons are significantly weaker than ambient magnetic noise, but what signal can be detected has greater spatial resolution than that detected by EEG due to the fact that magnetic fields are less distorted by the skull and cap than are electric fields. Another important difference between the two methods is that

EEG can detect both radial and tangential sources of cortical activity, while MEG can only capture tangential sources. In summary, EEG can detect more cortical activity than MEG can,

MEG can detect cortical activity with greater accuracy within the spatial intervals that it is capable of tracking, both capture only the aggregate effects of neuronal activity, and neither can reliably detect the activity of any but the most superficial regions of the brain.

Other methods of brain signal detection, like fMRI and tomographic modalities, have lower time resolution due to the comparatively slow physiological processes that supply the signals being captured, but have far better spatial resolution and can capture and isolate activity in any region of the brain, irrespective of depth. CT and PET respectively utilize x- rays and gamma rays. CT scans fire x-rays from many angles and measure the extent to which the radiation is absorbed by brain tissues at different locations, which is facilitated by

12 x-ray-absorbing radiocontrast agents injected into the bloodstream. PET scans detect the gamma rays that emerge from different regions of the brain as a consequence of electron- positron annihilation, which is facilitated by positron-emitting radioactive tracers injected into the bloodstream. In both instances, the distribution of ionizing radiation yields data concerning the flow of chemicals through the brain, offering information about the extent to which various regions undergo certain processes.

In addition to being able to render images of the brain in three dimensions and distinguish between individual regions and layers, tomography has the advantage of distinguishing between the types of functions being performed based on the types of compounds that interact with the radiation, be it glucose or oxygen. Another tomographic modality, called single-photon emission computed tomography (SPECT), is similar to PET in that it employs gamma rays, but instead of indirectly provoking gamma emissions via electron-positron annihilation, a gamma-emitting radionuclide is injected into the bloodstream, resulting in direct emissions from the introduced substance.

Because tomographic modalities employ ionizing radiation as their diagnostic, their principle disadvantage (aside from the health hazards associated with exposure to ionizing radiation) is the brevity of the data-collecting time window, which is necessarily constrained by the brief absorption period of the radiocontrast agents in CT and the brief half-lives of the isotopes in PET and SPECT. Aside from this, the physiological functions that are associated with the flow of blood and the metabolism of glucose introduce delays that compromise the time resolution of these methods. If a diagnostic tool indirectly measures brain activity via the distribution of physiological processes like metabolism and oxygenation, as opposed to directly measuring the electric/magnetic fields generated by neural activity (as with EEG and

MEG), there will necessarily be a time delay between the physiological function being measured and the dynamics of the corresponding neural signals.

Another limitation of the diagnostic tools involved in imaging techniques, including tomography and fMRI, is that they do not directly capture the structure of the brain, but rather

13 the processes that take place within it. Nevertheless, imaging modalities offer important insights into structural organization because the manner in which signals spread through the brain is contingent upon the architecture of its constituent neuronal ensembles, and the manner in which those signals spread can be inferred from the distribution of metabolic and respiratory activity.

The final brain diagnostic modality that shall be discussed is fMRI, which detects changes to the magnetic field of blood as oxygen-rich blood displaces oxygen-poor blood in the brain in order to supply oxygen to whichever neurons are active at the time. Oxygenated hemoglobin is less paramagnetic than deoxygenated hemoglobin, and the replacement of the latter with the former at a particular location produces measurable changes to the external magnetic fields supplied by an fMRI machine. The result is a completely non-invasive neuroimaging technique which does not require the subject to ingest or be injected with any substances, nor be exposed to ionizing radiation. This non-invasiveness, which is otherwise characteristic only of EEG and MEG, combined with outstanding spatial resolution, has made fMRI the default diagnostic neuroimaging technique of choice by researchers, though it is often used in concert with the other aforementioned techniques in order to capture the most complete representation of a functioning human brain.

As with other functional imaging techniques, the fMRI’s diagnostic is contingent upon secondary physiological processes rather than direct signals of neural activity, resulting in sub-optimal time resolution. Moreover, statistical noise from sources other than the fMRI can corrupt readings, requiring additional care to be taken when drawing inferences from apparent activity. The lack of such noise in tomographic techniques, due to the comparative rarity of ambient sources of x-rays and gamma rays, serves as their principle advantage over fMRI.

With a basic understanding of diagnostic neuroimaging techniques in hand, we are in a position to discuss the results of experimental and clinical studies with respect to the physiology of the human brain. Those results are of interest to us because they motivate the

14 model being introduced in this study. However, our survey of the connectome literature’s discussion of the human brain’s topological features will have to wait until after we develop the theoretical framework used to describe those results. Once we do that, we may then return to the experimental and clinical work whose application of the modalities introduced in this section yielded the results that motivated the model that serves as the topic of this thesis, before finally turning our attention to the question of whether that model does in fact exhibit those features. Before discussing the results of that literature, as well as the model that we have created in response to that literature, we must first acquaint ourselves with the theoretical paradigm used to discuss these things.

Formalizing the structural and dynamical properties of 85 billion neurons, and attempting to discern how these properties aggregate into those that characterize thousands of neuronal ensembles, and how those in turn aggregate into the structural and dynamical characteristics of entire brain regions, is an extraordinarily ambitious task that necessitates the development of specialized mathematical tools that are well disposed toward the formalization of such complex systems. In the following chapter, then, we shall develop the mathematical formalism that will allow us to express our results in a surprisingly elegant and coherent fashion, introducing those mathematical properties that are of greatest interest to us.

The framework that we shall be employing throughout this thesis is graph theory.

2.3: Motivating the Island Network

In the next chapter, we will spend a considerable amount of time developing the metrics that are used to appraise and characterize graphs in order to be able to describe the features particular to the model that we are introducing, the Island Network, but we will first elaborate in some detail upon the motivations behind the development of a graph with the set of structural characteristics that we are pursuing in this study. We will therefore review some of the relevant empirical studies that detail the evidence for the presence of those attributes in real human brains.

We seek a model of the human brain whose topology is characterized by high small- worldness (which entails the conjunction of short average path lengths with high clustering coefficients), hubs (which are identified and characterized in terms of various centrality measures), and highly modular community structures. We shall now explore the rationale behind the decision to focus on these topological metrics[37].

The human brain’s structural organization is such that it facilitates the integration and segregation of the signals that pass through its ROIs[17], whose separation from one another is sufficiently sparse that they can be properly designated as distinct regions. The presence of structurally and functionally distinguishable ROIs is supplied by highly modular communities of neural ensembles; these ensembles manage to connect to one another principally via connector hubs[33], which employ tracts to weave them together; and the integration and segregation of their signals is facilitated by small-world architecture[38].

Small-world organization is inherently well suited to modelling systems with both modular and distributed organization, with its high clustering coefficient allowing for the coalescence of groups of neuronal ensembles (and thus the facilitation of segregated processing), and its short path lengths allowing for the efficient traversal of signals between them (and thus the facilitation of the integration of information)[34]. It should therefore come as no surprise that small-world structure is pervasive throughout living systems’ central nervous systems, from macaques and cats to nematodes and humans[39].

Tract-tracing studies, which involve the injection of a tracer molecule into a particular brain region, euthanasia of the animal, and the subsequent dissection of its brain in order to identify the tract-laden paths taken by the tracer prior to uptake, served as the basis for the first graph-theoretic studies of the brain[40]. Cats and macaques, being the initial test subjects of these studies in the early 1990s, have since had entire databases dedicated solely to housing data pertaining to the connections found in their brains, with the largest of them,

Collation of Connectivity on the Macaque Brain (COCOMAC), supplying more than 200 different mapping schemes and over 30,000 different anatomical connections[41].

Employing the data from such freely available resources allowed researchers to discern the spatial and connective relationships between different regions of animal brains in order to visualize their topologies via a multivariate method called nonmetric multidimensional scaling (NMDS)[35]. NMDS techniques plot strongly connected areas in close proximity to one another and maximize the graphical distance between disconnected areas. The resulting graphs were found to exhibit high clustering coefficients and low average path lengths, which taken together serve as the definition of small-worldness. Ultimately, tract-tracing and NMDS allowed for the development of the first fully formalized connectome, characterizing the nervous system of the nematode Caenorhabditis Elegans in terms of its 2,462 synaptic connections and constituent neurons[42]. The corresponding adjacency matrix unequivocally demonstrated that its nervous system strongly exhibited small-worldness, which was cited by Watts and Strogatz in their seminal paper[6] as a principle motivator for the development of the network topology that bears their names.

Subsequent studies making use of EEG[25], MEG[23], and fMRI[43] have substantiated the hypothesis of small-world organization in the human brain by recording a time series from each of several brain regions, determining the extent to which activity in those regions correlate with one another, and assigning binarized connectivity values with respect to some threshold. The resulting adjacency matrices consistently suggest that small-worldness is a critical feature of human brain structure. Moreover, fMRI studies of schizophrenic test subjects indicate that the disruption of small-worldness in a brain’s organization coincides

(and perhaps may even become implicated in) certain brain pathologies, suggesting that early diagnoses might become possible if anomalous deviations from normal values of small- worldness in patients’ brain topologies are detected early[4].

As previously discussed, however, the symmetric degree distributions of small-world networks cause them to fail to exhibit hub structures, which presents a problem for small- world models of human brains because hubs have unequivocally been detected by encephalography[23], tomography[7], and fMRI[23]. By characterizing hubs in terms of various

17 measures of centrality, researchers[44] investigating brain topology at the meso-scale have consistently identified the precuneus, anterior and posterior cingulate cortex, insular cortex, superior frontal cortex, temporal cortex, and lateral parietal cortex as densely anatomically connected regions with a central position in the overall connectome. At the macro-scale, central network positions have been confirmed for the medial parietal, frontal, and insular regions of the brain, with these findings shown to be consistent across different cortical and subcortical divisions.

Hubs are among the most metabolically active regions in the brain, displaying complex cellular and microcircuit properties[45]. In aggregate, these distinctive characteristics of network hubs may suggest differences in their local physiology, energy metabolism, and neural processing that distinguish them from other, less-central network components. Hubs have been identified not only in the human brain, but also in several other mammalian species. The location of network hubs within the cerebral cortex has been remarkably consistent, with graph analyses of human, macaque, and cat brain networks converging on a set of high-centrality regions in the parietal, frontal, and insular cortices[44]. High-degree ‘hub neurons’ have also been shown to be present in the neuronal network of C. elegans, suggesting that the existence of network hubs may be a universal feature of connectome organization across many, if not all, species with a central nervous system. This universality has been suggested to be related to obligatory trade-offs between optimization of network performance, wiring cost, and spatial and metabolic constraints.

As with small-worldness, atypical hub structures have been found to correlate with neurological disorders, and in particular have been implicated in the onset of neurodegenerative disorders like Alzheimer’s disease[2] and frontotemporal dementia[46].

Preliminary connectome analyses have indicated the involvement of regions that have high spatial overlap in the etiology of these disorders, suggesting that hub structures might, along with small-worldness, someday serve a medical diagnostic function.

The final structural feature of interest in brain topologies is the presence of highly modular communities of neuronal ensembles, which share unusually dense connections within their groups and relatively sparse connections between them. Highly connected nodes within these modules that play little role in connecting to nodes outside of their communities are typically identified as provincial hubs, while lower degree nodes that monopolize their connections to multiple modules are typically identified as connector hubs. Thus modular communities of neuronal ensembles effectively serve as the substrate for negotiating small- worldness (which is characterized by the modules themselves) and hub structures (which are characterized by the connections between the modules.) Additionally, modularity in the human brain is thought to be hierarchical[47], meaning that modules may be understood in terms of their neural substrates at multiple levels of organization: modules of neuronal ensembles constitute brain regions, and modules of brain regions constitute the brain as a whole.

As might be expected, disruptions to modularity in the human brain have been implicated in a number of neural pathologies, particularly during early childhood development. Developmental studies report modified intramodular and intermodular connectivity of densely connected limbic, temporal, and frontal regions in children with autism[48], and childhood-onset schizophrenia has been associated with a disrupted modular architecture together with disturbed connectivity of network connector hubs in multimodal association cortices[3].

The upshot of these findings, as well as those associated with small-worldness and hubs, is that these topological features are not only typical of the brains of humans (among other animals), but are critical for their normal functioning. Thus with these organizational metrics in mind- small-worldness, hubs, and modular communities- we may proceed to demonstrate that the Island Network, our candidate simple model of a human brain’s topology, exhibits each of them.

Chapter 3: An Introduction to Graph Theory

3.1: Preliminaries Concerning Graphs

Unless otherwise specified, all of the material in this chapter was supplied by an introductory text on the subject of graph theory[37]. A graph is composed of nodes

(vertices) and edges (links), with the nodes representing some퐺(푁, set 퐸) of elements and the푁 edges representing the퐸 connections between those elements. For example, a graph representing a social network will be composed of nodes that represent people and edges that represent the relationships between them; a graph representing a metropolitan transit system will be composed of nodes that represent train and/or bus stations, and edges that represent the routes taken between them; and a graph representing a neuronal network will be composed of nodes that represent neurons or nuclei, and edges that represent neurites or tracts. Figure 3.1 illustrates a human connectome organized in the terms laid out here, while Figure 3.2 illustrates an example of a simple graph in a non-biological application; its inclusion is intended to aid the reader in understanding that graphs serve a plurality of functions, not all of which are necessarily biophysical.

Figure 3.1: A human connectome; nodes represent neuronal ensembles and edges represent neurites/tracts. Darker edges signify denser connections, and green nodes represent regions of interest. Image courtesy of CHUV-UNIL.

3.2: Graph Topology

3.2.1: The Adjacency Matrix

The emergent structural and organizational properties of a network are collectively referred to as its topology, and may be formally understood via a variety of metrics. The overall structure of graph may be formalized via the adjacency matrix , which has dimension and is composed퐺(푁, 퐸) of elements whose values sum to . The푨 columns of

푖푗 the adjacency푁× matrix 푁 represent each individual node,퐴 and the rows represent퐸 their relation to every other node in the graph. The value of element represents the number of edges

푖푗 between nodes and . 퐴

When 푖 푗 , nodes and don’t share an edge; when , nodes and share

푖푗 푖푗 exactly one edge,퐴 and= 0 they are 푖said to푗 be neighbors; and when 퐴 =, 1 nodes and푖 are푗 said

푖푗 to be multi-edged. If a diagonal element of has a value greater퐴 than> 1 , then the푖 node푗 that it corresponds to is said to contain at least one퐴 self-edge. And if 0 while , the

푖푗 푗푖 edge connecting nodes and is said to be directed from to 퐴. A =graph 1 which퐴 contains= 0 no directed edges thus has 푖a symmetric푗 adjacency matrix , and푖 is푗 said to be an undirected graph. And an undirected graph that contains no multi-edges푨 or self-edges has an adjacency matrix that is both symmetric and composed exclusively of elements with values of or ; such graphs퐴 are said to be simple graphs, and are the type that will be used exclusively0 1 throughout this study. Figure 3.3 illustrates an example of a graph and its corresponding adjacency matrix.

Figure 3.2: a graph representing a portion of the internet as of 15 January 2005, with each node representing an IP address and each edge representing transport layers that are color coded according to their corresponding domains. Image licensed under Creative Commons by the Opte Project.

Figure 3.3: a graph and its corresponding adjacency matrix. Beginning from 0, each column corresponds to the index of a particular node , with each row indicating the indices of the nodes that either is ( ) or is not ( ) connected to. 푖 푗 푖 1 0

3.2.2: Degree Distribution and Edge Density

The degree of a node is the number of edges that connect it to other nodes, and is given by 푘푖 푖

. (3.1) 푁 푖 푗=1 푖푗 푘 = ∑ 퐴

The degree distribution of a graph is represented by a function which expresses the probability that a randomly selected퐺(푁, node 퐸) will be characterized by degree푓(푘) . For sufficiently large networks, degree distributions푖 have a mean value such that푘

〈푘〉

. (3.2) 1 푁 푖=1 푖 〈푘〉 = 푁 ∑ 푘

A plot of a graph and its accompanying degree distribution is showcased in Figure 3.4.

Figure 3.4: A graph and its corresponding degree distribution.

퐺(푁, 퐸)

The mean degree is closely related to the edge density (also known as the connectivity) of a sufficiently large,〈푘 simple〉 graph :

퐺

(3.3) 〈푘〉 휌 = 푁 , where is the fraction of the maximum possible number of edges. Taken together, and can be 휌used to produce characteristic degree distributions and edge densities which 〈can푘〉 be 휌 used to compare different types of graphs. The advantage of this heuristic is that given two different graphs and , underlying similarities and differences in their topologies may be퐺1 inferred(푁1, 퐸1) with 퐺reference2(푁2, 퐸2 )to the relationships between the elements that constitute them. and may have identical mean degree values and degree distributions, and yet exhibit wildly퐺1 different퐺2 characteristics in their overall large-scale structures, as demonstrated by Figure 3.5. This feature is a testament to the fact that the emergent systems explored by graph theory are more than just the sums of their parts, and must be understood in terms of the interactions between their constituents.

Figure 3.5: The two graphs displayed here have the same number of nodes and edges ( ) with identical degree distributions as given by the histogram above, which plots the frequency that a particular degree appears against the value of that degree. Consequently, both of the graphs have푁 the = 18same, 퐸 =mean43 degree and edge density . Despite the fact that both graphs contain identical numbers of the same components with identical distributions of connections, the resulting large-scale structures differ 〈dramatically.푘〉 = 5.444 This illustrates how휌 = networks 0.32 must be understood not merely as the sums of their parts, but as emergent structures whose features are comprehensible only in terms of the combinatory manner in which their components are oriented with respect to one another, as given by their differing adjacency matrices . Complex systems cannot be understood merely in terms of their components’ attributes; they must be understood in terms of the patterns that are obeyed by the relationships between those components in 푨 aggregate.

3.2.3: Average Shortest Path Length

A path in a graph is any route that runs across the graph from node to node along the edges of the network. 퐺In a connected graph, every node has at least one path that connects it to every other node . And for every node pair 푖and within a connected graph, there exists at least one path which푗 is shorter than all of the푖 others;푗 that is to say, there exists at least one path where the number of edges that lie along it is lesser than that of all alternative paths 푑(푖,between 푗) and .

The average shortest푖 path푗 length within , or alternatively the average path length for every node pair and , is given by 퐺 푙퐺

푖 푗

. (3.4) 푑(푖,푗) 퐺 ∑푖,푗 푁(푁−1) 푙 =

For disconnected graphs, where at least one node pair and do not lie on any common path, the above equation is restricted to those node pairs that푖 do lie푗 on a common path.

The shortest path between nodes and is typically found via a search algorithm. This study incorporates푑(푖, 푗) Dijkstra’s algorithm푖 푗 [49], a graph-traversal search algorithm developed in 1959. The Dijkstra algorithm attempts to assign initial distance values and improves upon them step by step in the following manner:

1. All nodes are assigned to the unvisited set.

2. A tentative distance value is assigned to every node; the initial node is given value

, and every other node is given tentative value . 푖

3. 0Beginning with node , the current node is identified.∞ The distances from the current

node to each of its unvisited푖 neighbors are calculated. Generally, the distance between

neighbors may be formalized in terms of edge weights, which in an undirected graph

may range from unity to infinity.

4. The sum of the value of the current node and its distance to a neighbor will be the

tentative value of that neighbor, if and only if that value is less than or equal to the

neighbor’s previously calculated tentative value.

5. Once all of the neighbors of the current node have been assigned tentative values, the

current node is removed from the unvisited set.

6. If has the smallest tentative distance among the members of the unvisited set, or has

been푗 removed from it altogether, stop.

7. Otherwise, assign the member of the unvisited set with the smallest value as the

current node and return to step 3.

Though this algorithm was designed for graphs with non-uniform edge weights, the graphs explored in this study shall be exclusively of the unweighted variety. Nevertheless, we shall be employing Dijkstra’s algorithm so that this aspect of our analysis may be generalized to weighted systems. An example of the shortest path between two nodes is highlighted in

Figure 3.6.

Figure 3.6: The shortest path between nodes and is highlighted in red, and has a length of . The average shortest path length of the graph is . 퐴 퐵 푙퐴퐵 = 3 푙퐺 = 2.083

3.2.4: Clustering Coefficient

An immensely useful topological feature is a measure of the tendency of nodes to form tightly knit groups characterized by a relatively high density of ties. The parameter which formalizes this characteristic for an individual node is its clustering coefficient , which can be understood in terms of neighborhoods. 푖 퐶푖

The neighborhood that node is a member of contains all immediate neighbors of

, the edges that connect to푁푖 its neighbors,푖 and any edges which connect these neighbors to one푖 another. 푖 27

The set of nodes that are linked to are, along with any edges that link 's neighbors to one another, members of the neighborhood푖 . In order for to exhibit any clustering,푖 there must be at least two other nodes linked to ;푁 the푖 number of such푁푖 triplets within neighborhood

may be denoted by . If every node within푖 a triplet is linked to one another, then the

푁triplet푖 is called a triangle,휏푖 and the number of such triangles within may be denoted by .

The clustering coefficient of node is just the fraction of the푁 푖triplets in its 훾푖 neighborhood that also happen to be triangles:푖

. (3.5) 훾푖 퐶푖 = 휏푖

It is easy to see, then, that the clustering coefficient of is effectively the probability that the neighbors of are also neighbors of one other. 푖

푖

Figure 3.7: The nodes with red highlights, along with the edges between them, are members of the neighborhood of node . forms triplets with node pairs, of which node pairs form triangles with . Thus ’s clustering coefficient has the value 퐴 퐴 6 3 퐴 퐴 퐶퐴 = 0.5.

The average clustering coefficient of a graph , which aggregates this property over the entire network, is given by 퐺

. (3.6) 1 푁 ̅ 푁 푖=1 푖 퐶 = ∑ 퐶

An example of a neighborhood, as well as a demonstration of the means by which clustering coefficient may be appraised, is given by Figure 3.7.

3.2.5: Small-Worldness

Small-worldness is the tendency for a system to contain tightly knit groups that nevertheless have short path lengths between them, and is formalized as

, (3.7) 퐶̅ 푙퐺 푆 = where is the average clustering coefficient and is the average path length. More broadly, small-worldness퐶̅ is defined as the property of having푙퐺 average path lengths comparable to that of an equivalent random graph, whilst simultaneously having a significantly greater average clustering coefficient. Small-worldness is particularly important to modeling problems involving complex systems, since many real world networks (among them human connectomes) exhibit this property.

The small-worldness of a graph may be determined in relation to an equivalent random graph , where is a graph퐺 which, despite being comprised of edges that are randomly assigned퐺푟푎푛푑 via a uniform퐺푟푎푛푑 probability distribution to a set of nodes, 퐸nevertheless produces a network with equivalent degree distribution and edge density푁 . Equivalent random graphs are useful because they may serve as a baseline for the comparison휌 of certain topological metrics between and for the purpose of identifying unique emergent

푟푎푛푑 퐺 퐺 29 features in . In the case of small-worldness, is said to exhibit small-world characteristics if its small-world퐺 coefficient is greater than퐺 that of its equivalent random graph; equivalently, small-worldness푆 퐺is established if

. (3.8) 푆퐺 푆푟푎푛푑 > 1

Equivalent random graphs are developed via the Configuration Technique, which will be briefly elaborated upon in Section 3.2.9.

3.2.6: Measures of Centrality

One topological metric of considerable interest is centrality, which gives some indication of the relative importance of certain nodes in a network. Moreover, the distribution of nodes’ centralities can yield insights about a graph’s structure. The centrality of a node may be assessed via a number of different measuring schemes, and comparisons between the distributions of different centrality measures often yields important insights about the structure of a particular graph.

The first and most straightforward centrality measure is degree centrality, which is simply the degree of a node. Though by itself degree centrality may not be very informative, the comparison푘 of a node’s degree relative to the graph’s degree distribution to other centrality measures may important offer hints about its role in connecting different parts of the network, and so it is often a useful starting point when considering centrality.

Closeness centrality is the measure of the average path length between a node and every other node in the graph. The node with the shortest value of is 푙therefore “closest푖” to every other node in the network. In an undirected graph, the equation푙 for a particular node’s closeness centrality is

, (3.9) 1 퐶(푖) = ∑푗 푑(푖,푗)

where is the shortest path length between nodes and . The reciprocal of the sum of every shortest푑(푖, 푗) path length is computed such that푗 the푖 node with the greatest value of

푗 is the node with the lowest∑ 푑(푖, net 푗) average path length, thus giving it the greatest closeness centrality.퐶(푖)

Betweenness is another measure of centrality concerned with average path lengths, this one establishing the number of times that a particular node lies along the shortest path between every other pair of nodes. The full set of node pairs and will have a number

푥푦 of shortest paths running between them, whose lengths are given푥 by푦 . Of these휎 shortest paths, a certain subset of them will contain a particular node . The ratio푑(푥, of 푦) the number

푥푖푦 of such paths to the total number of shortest path lengths,푖 aggregated over all node 휎pairs,

푥푦 yields the betweenness centrality 휎of :

푖

, (3.10) 휎푥푖푦 퐵(푖) = ∑푥≠푖≠푦 휎푥푦

Thus betweenness is effectively a measure of a particular node’s contribution to a graph’s average path length, such that nodes with higher values of betweenness will have a more significant impact on the network’s connectivity.

Yet another centrality measure of considerable import is the eigenvector centrality, which ranks the importance of a node in terms of the relative importance of its neighbors to the rest of the network; this is achieved via a power series approximation.

We begin this approximation by initially setting the eigenvector centrality of each node equal to unity. We then improve our initial guess of by defining in terms푥푖 of the ′ centralities푖 of 's neighbors: 푥푖 푥푖

푖 31

. (3.11) ′ 푖 푗 푖푗 푗 푥 = ∑ 퐴 푥

Figure 3.8: Here we have four different centrality measures, represented by these heat maps upon the same graph. (A) Betweenness Centrality (B) Closeness Centrality (C) Eigenvector Centrality (D) Degree Centrality. Image licensed under Creative Commons by Tapiocozzo[50].

We may express this in matrix notation as , with vector being composed of ′ elements . 풙 = 푨풙 풙

The푥푖 initial iteration of this process is related to the subsequent iteration via the recurrence relation 풙(푡0)

, (3.12) 푨풙(푡0) ( 0 ) ‖푨풙(푡0)‖ 풙 푡 + 1 = which produces a subsequence that converges to an eigenvector that is associated with the dominant eigenvalue, provided풙(푡 )that 1) contains an eigenvalue that is strictly greater than its other eigenvalues, and 2) has푨 a non-zero component in the direction of an

풙(푡0) 32 eigenvector associated with the dominant eigenvalue. Provision 1 is met by the fact that only the identity matrix produces a homogeneous spectrum of eigenvalues, and in the graphs considered in this study, none will have an adjacency matrix equivalent to the identity matrix since such a graph would be composed exclusively of nodes with self-edges. Provision 2 is met by the fact that there exist components of are defined from the outset as being non- zero, so long as even a single edge exists to make풙(푡 0) .

푖푗 Iterating equation (3.11) over the course of퐴 t steps ≠ 0 yields

, (3.13) 푡 where is a linear combination of풙 the(푡) eigenvectors= 푨 풙(0) corresponding to the adjacency matrix 풙(0): 풗푖

푨

. (3.14)

풙(0) = ∑푖 푐푖풗푖

Plugging this expression back into the equation above gives the power series

, (3.15) 푡 푡 푡 푡 휆푖 ( ) ∑푖 푖 푖 ∑푖 푖 푖 푖 1 ∑푖 푖 휆1 푖 풙 푡 = 푨 푐 풗 = 푐 휆 풗 = 휆 푐 ( ) 풗 which converges such that is the greatest among the eigenvalues of the adjacency

1 푖 matrix . Consequently, all휆 , and so 휆 휆푖 휆1 푨 ≤ 1

. (3.16) 푡 lim푡→∞ 풙(푡) = 푐1휆1풗1

The limiting centrality vector is thus proportional to the leading eigenvector of the adjacency matrix. Equivalently,풙 satisfies

풙

, (3.17)

푨풙 = 휆1풙 where every element is the eigenvector centrality of node :

푥푖 푖

. (3.18) 1 푖 휆1 ∑푗 푖푗 푗 푥 = 퐴 푥

Eigenvector centrality is effectively a measure of the extent to which nodes are connected to

“influential” nodes; that is, to nodes with high degrees, or nodes connected to nodes of high degrees, with the value of eigenvector centrality being contingent upon both the degrees and the distances involved.

Centrality measures in general are important to our analyses of network topology because they allow us to determine where the most essential elements of a graph are located.

As previously suggested, differences in the distributions of different types of centrality are reflected in differences in the topologies of different graphs; examples of the different aforementioned centrality metrics, imposed over the same graph, are showcased by Figure

3.8. Considered more locally, the rankings of individual nodes along different centrality measures can offer insights into the roles played by those nodes both in structural and functional terms.

3.2.7: Hubs

When considering the functional aspect of certain nodes, we are analyzing the effect that those nodes have on the manner in which information flows through the network structure. The information referred to here may represent any number of situations, depending on the application of the network in question; for example, a graph that models traffic in Los Angeles might consider information flow in terms of the flux of vehicles across a network of highways; a graph that models the spread of electric current through a person’s body during defibrillation might consider information flow in terms of the electric currents 34 spreading through a network of blood vessels; and a graph that models the human brain might consider information flow in terms of the diffusion of electrochemical signals through a connectome. In every instance, the functional aspect of a node’s role in the spread of information through a network is intimately tied to the structural aspects of its contributions to the graph’s topology, and centrality measures are the principle means of identifying the nodes of greatest importance with respect to both structure and function.

Such nodes will hereafter be referred to as hubs, which by definition play an essential role in both the connectivity of a network and the spread of information through it. Though no formal set of criteria exists to allow for the unambiguous identification of hubs, the existing literature on the subject tends to emphasize two general features: first, hubs will only emerge in networks whose degree distributions are positively skewed. What this in effect implies is that hubs may only arise in networks where the probability of randomly selecting nodes of high degree is substantially lower than the probability of randomly selecting nodes of any other degree. As such, hubs do not emerge in graphs with bell-shaped degree distributions.

It is essential to note that though hubs can be (and often are) the aforementioned nodes of relatively high degree, the possession of relatively high degree is neither a necessary nor sufficient condition for being a hub[32]. Figure 3.9 illustrates a network and its accompanying distribution where red nodes and are identified as hubs. Take note that node , despite having comparatively low degree,퐴 still퐵 meets the general criteria for hubs, while퐴 node , despite having the highest degree in the graph, plays little role in its structure

(and consequently,퐵 the flow of information through it.) Recognizing this, we apply a more general heuristic for hubs that simultaneously explains this state of affairs, and makes use of centrality measures for the purpose of accurate hub detection. The specifics of this heuristic will be elaborated upon in subsequent chapters as the relevant circumstances calling for its application are encountered.

Figure 3.9: Node has degree while node has degree . Despite this major difference in degree centrality, contributes more to the structure of the graph than any other node because it holds the 퐴 퐵 highest values of 퐴betweenness centrality,푘 = 5 closeness퐵 centrality, and푘 eige= 14nvector centrality. Nodes like are known as “connector퐴 hubs” while nodes like are known as “provincial hubs.” 퐴 퐵

Ranking nodes according to different measures of centrality allows for the identification of potential candidates for hubs, yielding insights into the structural and functional contributions particular to those nodes. For example, a node with high degree centrality, average closeness centrality, high betweenness centrality, and low eigenvector centrality, will tend to have many links that are redundant to the overall structure of the network due to the presence of multiple alternate paths that more efficiently connect the graph. The table on the next page tabulates the patterns that tend to emerge from different combinations of centrality ranking[44]. It goes without saying that hubs with high rankings in each category are unambiguously the most important nodes in the network.

For the purposes of this study, we will distinguish between two different types of hubs: provincial hubs, which are defined as nodes that play little role in the structure of the graph and the spread of information through it in spite of their high degrees, and connector hubs, which are defined as nodes that play an enormous role in the structure of the graph and the spread of information through it in spite of potentially low degrees. Most hubs are an ambiguous mixture of the two types, but the distinction is nevertheless useful.

Low Degree Low Closeness Low Betweenness

High Degree Embedded in a The node’s many neighborhood that is connections are far away from the rest redundant; other paths of the network bypass it High Closeness Key node tied to The graph is so highly nodes of high degree; connected that the node often exhibits high has no major advantage eigenvector centrality over others High Betweenness The few ties held by Monopolizes its few this node are crucial to central connections to the structure of the connect key nodes to graph rest of the graph

3.2.8: Communities

Analogously to hubs, there exist groups of nodes that may be of particular importance to a graph’s topology. These groups are generally understood to exhibit dense connections between members and sparse connections with the rest of the graph; the manner in which this is quantified is given by the modularity, which will be discussed in detail in the following section, while the groups themselves are called communities.

The manner in which community structure shall be detected in this study is given by the Girvan-Newman algorithm[51], which identifies a network’s divisions by building a dendrogram based on betweenness centrality. Unlike the previously discussed version of betweenness, however, the one employed by the Girvan-Newman algorithm evaluates the betweenness of edges via

, (3.19) 휎푥퐴푖푗푦 퐵(퐴푖푗) = ∑푥≠푦 휎푥푦

where is the edge that connects nodes and , is the number of shortest paths that lie

푖푗 푥푦 between퐴 the total set of node pairs and 푖, and 푗 휎 is the number of such paths that

푥퐴푖푗푦 contain the edge . 푥 푦 휎

퐴푖푗 37

The function of this version of betweenness, hereafter referred to as edge betweenness, is to identify those connections which are most responsible for bridging otherwise poorly connected regions of a graph. If the removal of a particular edge would result in the partitioning of a graph into multiple disconnected regions, then those regions contain nodes that are more densely connected with one another than with members of other regions. Consequently, these regions are different communities, and the edges which connect them will tend to have higher values of edge betweenness. The Girvan-Newman algorithm takes advantage of this relationship between edge betweenness and community structure in the following manner:

1. Calculate the edge betweenness of all edges .

푖푗 2. Remove the edge with the highest value of edge퐴 betweenness .

푖푗 3. Repeat until no edges remain. 퐵(퐴 )

The resulting dendrogram reveals the underlying community structure, with individual leaves representing the nodes of the graph. With a method for identifying community structure in hand, we may now turn our attention to the question of how to determine the extent to which communities are discreet from one another so that we may quantify a graph’s “lumpiness.”

Figure 3.10: (a) A highly modular graph containing six communities, with Q = 0.91

3.2.9: Modularity

In order to be able to quantify the degree to which community structures are really present in a given graph, we shall employ the topological metric of modularity, a value which ranges from to such that values of describe graphs with community structures푄 characteristic of those−1 found1 in connected random0 graphs. Thus a graph with a positive value for modularity exhibits more community structure than can generally be expected from a connected random graph, while a graph with a negative value for modularity exhibits less.

This is because modularity is broadly defined as the fraction of edges that belong to distinct community structures, minus the fraction that would have been expected had those edges been distributed randomly. An example of modularity in a graph is illustrated by Figure 3.11.

Figure 3.11: A graph representing friendships between 470 students of black, white, or other self-identified racial backgrounds at a US high school. Data from the National Longitudinal Study of Adolescent Health, graph compiled by M.E.J Newman. Reproduction protected by Section 107 of the Copyright Act of 1976.

As a general rule, graphs with smaller edge density rely more on their edges to maintain their structure than their more dense counterparts; 휌such graphs are said to be less resilient. Consequently, graphs with smaller values for will tend to have less homogeneous distributions of edge betweenness values, making their 휌partitioning into different communities via the Girvan-Newman algorithm more pronounced. Thus graphs with smaller edge densities tend to have greater modularities than do graphs with greater edge densities.

To formalize the means by which modularity푄 is to be acquired, we will begin with its definition as the difference between the fraction of edges that belong to distinct communities, and the fraction of edges that would be expected to belong to distinct communities if the edges had been randomly distributed. We will illustrate what this means by way of an example: the figure below depicts a graph that was compiled from data taken from a US high school in which students were surveyed about their friendships, and were asked to name other students who they considered their friends[37]. Each node represents a student, and each edge represents a friendship. Each node is color-coded according to the self-identified races of the

40 students. The graph reveals that the school is sharply divided into two groups: one composed principally of white students, and one composed principally of black students.

This is not what we would expect if friendships between students were organized randomly; thus we begin our analysis of the situation by calculating the fraction of edges that emerge between the same members of a self-identified race in a scenario where the friendships represented by these edges are assigned randomly. We want the hypothetical random case to be equivalent to that of the real non-random case in terms of degree distribution and the number of nodes and edges, and , in order to compare like scenarios: the case given by Figure 3.12 illustrates a situation푁 in which퐸 470 students have their friendships organized according to a particular degree distribution. Each student has a particular number of friends, and we wish to preserve this. All that we seek to 푖change, for the hypothetical푘푖 randomized scenario, is the manner in which those friends are selected. We must preserve all other facts of the case in order to make a valid comparison. What follows is a brief exposition of how we are able to produce an equivalent degree distribution while still assigning connections randomly; this is called the Configuration Technique, and it will be important in later discussions concerning the production of equivalent random graphs.

Consider a particular edge connected to node , which has degree ; that is to say, student has a number of friends. Similarly, student푖 has a number 푘of푖 friends. Each

푖 푗 edge connected푖 푘 to has two ends: one which connects to푗 , and푘 one which connects to another node. Because푖 there are such edge ends throughout푖 the entire network, the probability that a particular edge2퐸 end that connects to is also connected to one of the edge

푖 푘푗 ends attached to is just , assuming that connections are made purely at random. The 푘푗 reconfiguration of푗 these edge2퐸 ends in this manner, which preserves the degree distribution of the original graph, is what is referred to as the Configuration Technique. is the probability 푘푗 that a particular edge end connected to will also connect to ; the expected2퐸 total number of edges shared by and is 푖 푗

푖 푗 41

. (3.20) 푘푖푘푗 푟푎푛푑 < 퐸 > = 2퐸

We shall return to this result from the hypothetical randomized scenario momentarily.

In the real-world example under consideration, each student belongs to one of three self- identified racial categories such that they are either a member푖 of a particular racial category

, or they are not a member of racial category .

(푆푖푟 = 1)The product will be equal to unity if students푟 (푆 푖푟 and= 0) are both members of self-

푖푟 푗푟 identified race , and푆 zero푆 if not. Summing over all such categories,푖 푗 we define

푟

(3.21)

푖 푗 푟 푖푟 푗푟 훿(푐 , 푐 ) = ∑ 푆 푆 and sum it over all node pairs

, (3.22) 1 푖푗 푖 푗 푖푗 푟 푖푗 푖푟 푗푟 ∑ 훿(푐 , 푐 ) = 2 ∑ ∑ 퐴 푆 푆 which represents the total number of edges that run between nodes that belong to the same category. The leading factor of ensures that we do not double count the node pairs when 1 performing the second sum. Equation2 3.22, in the context of the real-world example, represents the total number of friendships that exist between students of the same self- identified race.

We also wish to identify the expected total number of friendships between students of the same self-identified race for the hypothetical, randomized scenario, which we do by taking the product of Equations 3.21 and 3.22, and summing them over all node pairs:

. (3.23) 1 푘푖푘푗 푖푗 푟푎푛푑 푖 푗 푖푗 푟 푖푟 푗푟 ∑ < 퐸 > 훿(푐 , 푐 ) = 2 ∑ ∑ 2퐸 푆 푆 42

Taking the difference between Equations 3.22 and 3.23 gives us the difference between the actual total number of friendships between students of the same self-identified race, and the expected total number of friendships between students of the same self-identified race:

. (3.24) 1 1 푘푖푘푗 1 푘푖푘푗 푖푗 푟 푖푗 푖푟 푗푟 푖푗 푟 푖푟 푗푟 푖푗 푟 푖푗 푖푟 푗푟 2 ∑ ∑ 퐴 푆 푆 − 2 ∑ ∑ 2퐸 푆 푆 = 2 ∑ ∑ [퐴 − 2퐸 ] 푆 푆

Finally, we divide this result by the total number of edges in order to arrive at the fraction of such edges, which gives us the modularity of the graph:

. (3.25) 1 푘푖푘푗 푖푗 푟 푖푗 푖푟 푗푟 푄 = 2퐸 ∑ ∑ [퐴 − 2퐸 ] 푆 푆

In the example that we considered, the categories under which nodes were partitioned were defined on the basis of self-identified race in order푟 to determine the extent to which the surveyed students self-segregate. In this study, the categories are defined in terms of a node’s membership in a particular community, as defined by the푟 Girvan-Newman algorithm, in order to determine the extent to which a graph’s community structures partition their constituent nodes.

3.3: Topologies of Common Graphs

We have thus far prepared the various ingredients that characterize a network’s topology. The manner in which those ingredients are arranged is reflected in different classes of network; in fact, it is their defining feature, for though a variety of algorithms may be employed to generate a certain class of network, the recipe of topological features that define it must remain immune to differences in the manner in which their adjacency matrices are computed. Indeed, as we shall see with the Island Network, even an apparently modest modification of an extant network’s algorithm can produce non-trivial novel topological

43 features. A general exposition of the topologies of common networks is therefore warranted, as it will illustrate how the global structures of graphs are emergent from local patterns among nodes. Our discussion will be limited to the three most commonly discussed graphs in relation to models of brain structure: random networks, small-world networks, and scale-free networks.

3.3.1: Random Networks

Within the present context, a random network is defined as the set {G} of graphs

that can be generated from a given number of nodes and edges via an algorithm

퐺(푁,that could 퐸) produce any member of set {G} with equal probability.푁 Random퐸 graphs typically have binomial degree distributions that in the limit of large become Poisson distributions, and are characterized both by their short average path lengths푁 and their miniscule average clustering coefficients , respectively given by 푙퐺

퐶̅

(3.26) log(푁) 퐺 log(퐾) 푙 =

and . (3.27) 〈푘〉 ̅ 푁−1 퐶 =

The most commonly used random network was developed after the mathematicians

Paul Erdos and Alfred Reyni[52], after whom it was named; the algorithm that produces their graph is provided in Appendix 2.1. The process by which the Erdos-Reyni network is generated simply amounts to creating an adjacency matrix of dimension , where

퐸푅 once again is the predetermined number of nodes in the graph.푨 An element 푁 × is 푁 then

푖푗 selected via푁 a uniform random probability; if is already equal to , or if 퐴 , then a

푖푗 different element is randomly selected. Otherwise,퐴 is set equal1 to unity푖 = along 푗 with

푖푗 푖푗 element . This 퐴process is iterated a total of times, thus퐴 producing an adjacency matrix

퐴푗푖 퐸44 where elements have a value of , which in an undirected graph corresponds to a network with 퐸 edges. An example of an Erdos1 -Reyni graph is given in Figure 3.13.

퐸

Figure 3.12: A typical Erdos-Reyni graph with nodes and edges.

100 1,000

Because connected Erdos-Reyni graphs do not have positively skewed degree distributions, hubs do not emerge. Moreover, the value of the elements of an Erdos-Reyni’s modularity matrix would, by definition, amount to approximately zero, making its modularity as a whole approximately zero. This is to be expected, given that random distributions of edges will not spontaneously form community structures. Additionally, and most relevantly to this study, the average clustering coefficient of the Erdos-Reyni network is extremely low

퐶̅

45 because, as with modularity, distributions of edges will not spontaneously form structures that are dominated by triangles.

The low average clustering coefficient of the Erdos-Reyni network proved to be its greatest shortcoming because most real-world networks exhibit clustering coefficients that are orders of magnitude greater. To address this weakness, the sociologist Duncan Watts and mathematician Steven Strogatz developed a graph[6] that exhibited the low average path length of the Erdos-Reyni model while also featuring a significantly greater average

퐺 clustering푙 coefficient.

3.3.2: Small-World Networks

Watts and Strogatz generated their graph via the incremental alteration of a ring lattice network. A ring lattice network is composed퐺(푁, 퐾, 훽)of nodes that each possess edges, half of which are individually shared with their closest푁 neighbors in the clockwise2퐾 direction. This makes for a total of edges, with node sharing individual links

푊푆 with . By design,퐸 a =ring푁퐾 lattice network has 푖an extremely high average clustering푖±1,푖±2,…,푖±퐾 coefficient, but a comparatively large average shortest path length.

The Watts-Strogatz network, the prototype for small-world networks, introduces a parameter that ranges from to . A random probability is uniformly assigned to each unity-valued훽 element 0of the1 adjacency matrix . The probabilities are assigned in

푖푗 푊푆 such a manner that 퐴 has= the 1 same probability as ;푨 thus every edge in the network

푖푗 푗푖 becomes associated퐴 with a random probability . If퐴 , then both and are set to ;

푖푗 푗푖 the edge is removed. Otherwise, the edge remains푝 in푝 its > original 훽 place.퐴 퐴 0

The row corresponding to the node whose edge has just been removed is then assigned a new set of random uniform probabilities푖 for every whose value is . The

푖푗 element with the greatest probability assigned to it is given퐴 a value of unity, along0 with its

푖푗 corresponding퐴 element . What this effectively translates to is the rewiring of one of 's

퐴푗푖 푖 46 edges to a new destination; to prevent from connecting to itself or to the node that it had previously been connected to, the elements푖 on row are assigned probabilities of if

푖푗 corresponds to the previously connected node퐴 or if 푖 . This procedure is reiterated 0for all푗

rows. 푗 = 푖

푁 Put simply, what the Watts-Strogatz network does is to take a ring lattice network and rewire its edges with probability in order to drastically decrease the average path length

퐺 whilst maintaining a high average훽 clustering coefficient . Figure 3.14 plots and against푙

퐺 , showing that the former drops rapidly even at low values퐶̅ of while the latter푙 falls퐶̅ only at relatively훽 high values of ; thus the Watts-Strogatz network achieves훽 its goal of producing low average path lengths훽 and high average clustering coefficients for a wide range of .

Different regimes of will of course yield different topological features, but as훽 with any other class of network, Watts훽 -Strogatz graphs’ topologies obey certain characteristic patterns. The degree distribution, as with the random graph, may be approximated as a

Poisson relation, and peaks at , which also happens to be equal to the mean

푚푎푥 degree . 푘 = 2퐾

⟨푘⟩

Figure 3.13: A plot of clustering coefficient and average path length from Watts’ and Strogatz’s original paper, where the convention defined the rewiring parameter as . 퐶 퐿 훽 푝 47

For the ring lattice, the average clustering coefficient is given by

, (3.28) 3(2퐾−2) 퐶푟푖푛푔̅ = 4(2퐾−1)

which may be used to determine at higher values of :

퐶̅ 훽

. (3.29) 3 푟푖푛푔 퐶̅(훽) = 퐶̅ (1 − 훽)

At , when every edge is randomly rewired, the clustering coefficient may be approximated훽 = equivalently 1 to that of the Erdos-Reyni network:

. (3.30) ⟨푘⟩ 2퐾 ̅ 푁−1 푁−1 퐶 = =

At very low values of , the average path length is approximately

훽

, (3.31) 푁 퐺푟푖푛푔 푙 = 4퐾 ≫ 1 but rapidly approaches the average path length of a random graph as approaches unity:

훽

. (3.32) log(푁) 푙퐺(훽) = log(2퐾)

Figure 3.15 illustrates a small-world network as its rewiring parameter is modulated to higher values.

Because of the lack of a positively skewed degree distribution, the Watts-Strogatz network does not produce hubs. Its modularity is substantially greater than that of the Erdos-

Reyni network, and its small-world coefficient is unsurprisingly also much greater because the small-world network attempts to maximize 푆small-worldness by design.

Figure 3.15: The Watts-Strogatz network, with nodes and edges, with various values of rewiring parameter . (a) (b) (c) .15 (d) 100 1,000 훽 훽 = 0 훽 = 0.015 훽 = 0 훽 = 0.5

3.3.3: Scale-Free Networks

The final network structure that is of relevance to us is the scale-free network, which we will only briefly touch upon. Just as the fundamental shortcoming of Erdos-Reyni graphs, insofar as they attempt to model real-world phenomena, is their extremely small clustering coefficient, so too does the Watts-Strogatz network suffer from a topological defect. In real world networks, hubs feature prominently as the principle elements through which information flows, which makes the Watts-Strogatz graph, which lacks hubs, an inoperable model for most real-world systems.

The Barabasi-Albert Network[53], named after the physicists who introduced it, is explicitly devoted to producing hubs by engineering a degree distribution that obeys a power law. Such graphs, called scale-free networks, will necessarily have extremely positively skewed degree distributions, thus meeting the condition under which hubs may emerge.

Scale-free networks are of interest here only to the extent that their topologies may be contrasted with those of small-world networks with respect to three topological features: small-worldness, hubs, and modularity. Small-world networks, having bell-shaped degree distributions, cannot produce hubs, while scale-free networks, having strongly positively skewed degree distributions, do. However, in scale-free networks, the average clustering coefficient also obeys a power law as a function of degree, with its decay governed by

. The consequence of this is that unlike small-world networks, scale-free −1 퐶networks̅(푘) = 푘 have low average clustering coefficients, and consequently lack small-worldness.

Thus for systems like the brain, which exhibit both small-worldness and hubs, neither scale- free nor small-world networks are adequate connectonomic models.

It is also worth noting that scale-free networks are even more lacking in community structure than are random graphs, with Barabasi-Albert networks often taking on negative values for modularity[54]. Because modularity is practically a defining feature of brain networks, scale-free graphs are almost completely inoperable as simple models for the human brain.

3.3.4: Closing Remarks

In the next chapter, we will motivate the creation of a network in terms of the topological properties that have been discussed thus far with reference to both the aforementioned common graphs and those empirical studies which represent the most comprehensive attempts at developing models of connectonomic structure. A thorough account of the motivations which underlie the development of a model with both hubs and small-world structure will be given, as will a complete overview of the properties of the

50 graph-theoretical structure being proposed in this study. We shall then demonstrate the manner in which we have verified that the proposed neural network structure, the Island

Network, does in fact exhibit a novel connectome topology. Our model and its attendant dynamics were developed via C++, and all images and analyses of it and the other graphs presented in this study were generated via NetworkX, an open source Python library.

Chapter 4: Topology of the Island Network

4.1: Generating the Island Network

The creation of a graph which possesses both hubs and small-worldness will require the development of a network with a comparatively high average clustering coefficient, comparatively short average path lengths, and a positively skewed degree distribution. In pursuing a novel connectome topology with these features, we have settled upon a somewhat unconventional solution: we have decided to produce a disconnected graph.

We begin with a ring lattice, just as Watts and Strogatz did, and even employ the rewiring parameter in the exact same manner. Thus we produce a graph that meets the condition of high clustering훽 coefficient, but because of its symmetrical degree distribution, is unable to produce hubs. Our task now is to shift the degree distribution to the left so as to produce a positively skewed degree distribution whilst still retaining the comparatively high clustering coefficient that allows for small-worldness. We achieve this via the introduction of a deletion parameter , whose value ranges from to . Once the adjacency matrix for the

Watts-Strogatz network휅 is generated, every element0 1 with a value of unity is assigned a

푖푗 random value from a uniform probability distribution.퐴 If , and its associated

푖푗 transpose are푝 set to zero, thus deleting the corresponding푝 edge. > 휅 퐴Otherwise, and are

푗푖 푖푗 푗푖 left alone and퐴 the corresponding edge survives. Consequently, a value of 퐴 results 퐴in the deletion of every edge, while a value of results in no deletions whatsoever;휅 = 0 a value of

results in the survival of approximately휅 = 1 20% of the initial number of edges, while the remaining휅 = 0.2 80% are deleted. Thus the Island Network contains edges. Figure 4.1 illustrates a small-world network and an equivalent Island Network퐸퐼 = side 푁퐾휅 by side, where

“equivalence” is understood to mean the equivalence of all parameters other than .

휅

Figure 4.1: A graph with nodes, edges ( ), and . (a) The deletion parameter is set to unity, meaning that of the edges are preserved and of them are deleted. This is identical to a Watts-Strogatz푁 = graph.500 (b) The퐸 deletion = 5,000 parameter퐾 = 10is set to 훽 =, meaning 0.02 that of the edges are preserved휅 and of them are deleted.100% Naturally, some proportion of nodes0% will be left edgeless as a consequence; these contribute nothing to the Island Network, whose휅 feature0.1s of interest are to10% be found on the connected components.90% The unconnected components shall hereafter be referred to as “residue.”

4.1.1: Properties of the Island Network

The Island Network is a modification of the Watts-Strogatz network that introduces a deletion parameter for the purpose of shifting the graph’s degree distribution to the left.

This produces a positively휅 skewed degree distribution, which allows for the presence of hubs.

To understand the properties of the Island Network, we shall begin by deriving its degree distribution. We make the approximation that , such that prior to deletion, the network approximately assumes the topology of a circular훽 ≪ 1lattice composed of nodes and

edges, with every node being attached to edges. With the introduction푁 of deletion퐸 = 푁퐾 parameter , each node has some probability2퐾 of undergoing a deletion event , in which the휅 node’s degree is decreased by 푝amount = 1 − 휅. In other words, the number of connections푑 lost by a node as a consequence푘 of edge deletion푑 is , which means that the degree of that node is given by the difference between its initial number푑 of connections and the number of connections that are lost upon being acted upon by the deletion parameter:

. (4.1)

푘 = 2퐾 − 푑

For each node, the probability of there being exactly deletion events, given edges per node, is given by the binomial relation 푑 2퐾

. (4.2) 2퐾 푑 2퐾−푑 푑 퐵 = ( 푑 )푝 (1 − 푝)

Substituting in and , we are left with

푝 = 1 − 휅 푑 = 2퐾 − 푘

, (4.3) 2퐾 푘 2퐾−푘 퐵 = (2퐾−푘)휅 (1 − 휅) where due to the fact that every node’s degree obeys the relation . 2퐾 2퐾 Thus we(2퐾−푘 arrive) = at ( the푘 ) degree distribution 푘 ≤ 2퐾

(4.4) 2퐾 푘 2퐾−푘 푘 퐵 = ( 푘 )휅 (1 − 휅) .

Being a binomial distribution, the degree distribution has average value

, (4.5)

〈푘〉 = 2퐾휅 as well as mode

(4.6) ⌊(2퐾 + 1)휅⌋ 푖푓 (2퐾 + 1)휅 = 0 표푟 ∉ ℤ 푘푚푎푥 = {(2퐾 + 1)휅 푎푛푑 (2퐾 + 1)휅 − 1 푖푓 (2퐾 + 1)휅 휖 {1, … ,2퐾} 2퐾 푖푓 (2퐾 + 1)휅 = 2퐾 + 1

The degree distribution of small world graphs is such that the mean degree is related to the total number of edges by ⟨푘⟩푆푊

퐸푊푆 = 푁퐾

(4.7) 2퐸푊푆 푆푊 ⟨푘⟩ = 2퐾 = 푁 , where is the initial ratio of edges to nodes. In Watts-Strogatz graphs, the mean degree

is퐾 equivalent to the value of the degree distribution’s peak .

⟨푘⟩ 푆푊 With the introduction of deletion parameter , however, the푘푚푎푥 total number of edges becomes and the above equation changes휅 to

퐸퐼 = 푁퐾휅

, (4.8) 2퐸퐼 〈푘〉 = 2퐾휅 = 푁 which must be less than or equal to the average degree of an equivalent small-world graph by virtue of the fact that . This represents⟨푘⟩ 푆푊a leftward shift in the degree distribution, causing it to become0 ≤ positively 휅 ≤ 1 skewed. Figure 4.2 illustrates this very shift between the degree distribution of a small-world network, and that of an equivalent Island

Network.

For the purposes of this study, we have designated two distinct regimes of Island

Network as Class 1 and Class 2 graphs. These Classes of Island Network are distinguished by certain qualities of their degree distributions, and are defined in relation to the limits of the

Island Network as the deletion parameter gets modulated. The limits of the Island Network are defined as 휅

. (4.9) −1 0 ≤ 휅 < 퐾

Figure 4.2: (a) a Watts-Strogatz graph’s degree distribution . (b) an Island Network’s degree distribution . Both graphs have identical parameters save for the deletion coefficient, which evidently has the effect of shifting degree distributions to the(휅 left. = 1)At sufficiently low values, has the effect of making an otherwise(휅 = 0.075 symmetric) degree distribution positively skewed. 휅

At , all edges are deleted, resulting in disconnected nodes. At , −1 Equation 4.8휅 =indicates 0 that , and that that the 푁ratio of edges to nodes is less휅 = than 퐾 unity 퐸퐼 at values of . It is only푁 = at 1 greater values of that it becomes possible for graphs to be −1 connected, with휅 < the 퐾 proportion of connected graphs 휅to disconnected graphs within a set

of all possible graphs increasing as the deletion parameter increases from to −1 {퐺(퐸,the maximum 푁)} possible value, . We take to be the upper limit휅 of the Island Network퐾 −1 because all possible graphs with1 values 휅 = 퐾 are disconnected. −1 The degree distributions of graphs휅 seen 퐾 corresponds to Island Networks at values〈푘〉

. According to Equation퐾휅 4.8, such values for will generate mean degree .

퐾휅Otherwise,< 1 at values , the corresponding mean퐾휅 degree becomes . Thus〈푘 〉 < 2 represents the upper limit퐾휅 > of 1 mean degree values that may characterize an〈푘 〉Island< 2 Network.〈푘〉 = 2

With the limits of the Island Network in hand, in which the deletion parameter and the average degree value respectively range and −1 0 ≤ 휅 < 퐾

, (4.10)

0 ≤ 〈푘〉 < 2 we may now formally define the two Classes of Island Network. Class 1 Island Networks have deletion parameters and average degree values that are respectively characterized by the range and , while Class 2 Island Networks have −1 2퐾 deletion0 ≤parameters 휅 < (2퐾 and + 1) average 0degree ≤ 〈푘〉 values≤ 2퐾+1 that are respectively characterized by the range and . The motivation behind −1 −1 2퐾 partitioning(2퐾 + the 1) Island≤ 휅 Network < 2(2퐾 into + 1) these two2퐾+1 Classes≤ 〈푘 〉lies< with 2 the mode of the degree distribution, which gives the peak .

First, we consider the peak 푘of푚푎푥 Class 1 graphs, whose average degree values range

. Equation 4.6 indicates that for average degree value , in 2퐾 0which ≤ 〈푘 〉 ≤must2퐾+1 be equal to , the peak of the degree distribution is at 〈푘〉 = 2퐾휅; for= average 0

푚푎푥 degree 휅value 0 , in which must be equal to 푘 ,= the 0 degree 2퐾 −1 distribution has〈푘 〉two= 2peaks:퐾휅 = one2퐾+1 at 휅, and the other at (2퐾 + 1). Thus Class 1 graphs are the subset of Island Networks whose푘푚푎푥 = peaks 1 are characterized푘푚푎푥 by =the 0 range .

Next, we shall consider the peak of Class 2 Island graphs, whose average0 ≤ degree 푘푚푎푥 ≤ 1 values are characterized by the range . Referring again to Equation 4.6, we 2퐾 4퐾 2퐾+1 ≤ 〈푘〉 ≤ 2퐾+1 find that for average degree value , in which must be equal to 4퐾 , the degree distribution〈푘〉 again= 2퐾휅 has= two2퐾+1 peaks: one at휅 , and the other at −1 2(2퐾 + 1). Thus Class 2 graphs are the subset of Island Networks 푘whose푚푎푥 = peaks 2 are characterized푘푚푎푥 = 1 by the range .

The remaining range1 that ≤ 푘 푚푎푥constitutes≤ 2 the Island Network, in which the average degree values are characterized by the range , the corresponding deletion parameter 4퐾 is characterized by the range 2퐾+1 ≤ 〈푘〉 ≤ 2 , and the resulting peak is always −1 −1 , shall be hereafter be2(2퐾 referred + 1) to as≤ the 휅 ≤ “ 퐾border,” since it effectively serves as the

푚푎푥 푘 = 2 57 demarcation between the Island Network and the Watts-Strogatz Network. Along with the

Class 1 graphs, the Border graphs are of limited interest to us; in this study, the bulk of our attention shall be focused upon the Class 2 Island Network, which exhibits the structural and dynamical features that are of greatest interest to us. Figure 4.3 illustrates the Island Network as its deletion parameter gets modulated to higher values; notice that the average values and peaks of the corresponding degree distributions obey the relations outlines in this section.

In summary, we have modified the Watts-Strogatz Network by introducing a deletion parameter such that the fraction of edges that are randomly deleted from the network is given by 휅 , leaving behind edges. The Island Network is that subset of graphs that(1 are − 휅generated)퐸 by setting 퐸the = deletion 푁퐾휅 parameter to values within the range

. Correspondingly, the Island Network’s degree휅 distributions have mean values −1 0within ≤ 휅

and , and are respectively characterized by −1 −1 −1 degree0 ≤ 휅

Strogatz graph so as to shift its degree distribution to the left and make it positively skewed, the Island Network possesses properties that are notably lacking in Watts-Strogatz graphs, and we shall demonstrate this by comparing the two.

Figure 4.3: Island Networks with N = 1000, = 0.02, and modulated such that each Class’ typical network structure and accompanying degree distribution is displayed. Notice how the degree distribution shifts further to the right as more and more edges are restored훽 to the graph.휅 Also take note of how the mean degree and the peak of the degree distribution obey the relations outlined in this section with respect to the corresponding Classes. 푚푎푥 (a) Class 1 (b) Class 1 transitioning into푘 Class 2 (c) Class 2 (d) Upper limit of the Island Network

We shall see over the course of this chapter and the next that the status of the Island Network as its own unique graph is warranted by the presence of topological properties and dynamical behaviors that set it apart from the Watts-Strogatz network that inspired it, just as the Watts-

Strogatz network exhibits features that set it apart from the lattice network that preceded it. 59

4.2: General Metrics

Establishing the presence of small-worldness and modular communities will require comparisons with equivalent random graphs where appropriate; to demonstrate small- worldness, we shall compare the average path length and clustering coefficient of a particular

Island Network to those of an equivalent random network. To demonstrate the presence of hubs, no comparison shall be required since their presence is determined by positively skewed degree distributions, but a heuristic cast in terms of centrality measures shall nevertheless be provided for the purpose of identifying them within the Island Network. And to demonstrate the presence of highly modular communities, we shall again compare our graph to an equivalent random network after applying the Girvan-Newman algorithm to both of them.

Each Island Network with nodes will be generated along with its accompanying degree distribution, in which degree푁 appears in the degree distribution with frequency .

The weighted sum over all degree frequencies,푘 , is equal to , the total number of푓(푘) edges in the Island Network. ∑ 푘푓(푘) 퐸

An equivalent random network is generated in such a manner that edges are randomly assigned to disconnected nodes, such that there appear an 퐸 number of nodes with degree . This shall푁 be achieved via the Configurational Technique.푓(푘) Their equivalent degree distributions푘 (and consequently, equivalent connectivity ) to those of corresponding

Island Networks will serve as the justification for their designation휌 as equivalent random graphs. For every applicable topological metric that is evaluated, there will be 20 trials in which Island Networks and their equivalent random graphs are generated. The metrics under consideration will be averaged over each of the 20 trials in order to arrive at a mean value for each metric in relation to both the Island Network and its equivalent random graphs. It is these mean values that shall serve as the basis for comparison between the two, so as to establish those features of the Island Network which are intrinsic to its topology and are not the result of random chance. 60

4.2.1: Degree Distribution and Edge Density

The parameters of our characteristic Island Network will place it within the Class 2 regime , , which is where we shall restrict the bulk our 2퐾 4퐾 푚푎푥 analysis( 2퐾+1for the≤ purposes ⟨푘⟩ ≤ 2퐾+1 of this푘 study.= 1) We have selected the number of nodes to be

, the initial ratio of edges to nodes to be , the rewiring parameter to be 푁 = , and1000 the deletion parameter to be . This퐾 = translates10 to initial edges, 훽of =which 0.02

are rewired and are deleted,휅 = 0. 075leaving behind edges.10,000 Figure 4.4 illustrates the

200types of graphs that were9,250 generated for the comparisons750 to come, as well as their corresponding degree distributions.

Correspondingly, the average degree is , with the degree distribution peaking at ; consequently, the〈 distribution푘〉 = 2퐾휅 = of 1.5 this Class 2 Island Network is unequivocally푘푚푎푥 positively= 1 skewed. The connectivity is ; this, along with the −3 average degree value , must be made equal to that of휌 =our 1.5 equivalent × 10 random graph if we are to use it as a basis 〈for푘〉 comparison. Once again, each metric is computed 20 times for each graph pair.

4.2.2: Small-Worldness

Small-Worldness, which is defined as the ratio of the average clustering coefficient to the shortest average path length , was computed for 20 pairs of Island Networks and 퐶̅

퐺 equivalent random networks. Small푙-worldness in the Island Network was confirmed by computing the ratio of to for each of the 20 pairs of Island Networks and their

퐼 푟푎푛푑 equivalent random networks.푆 푆

Figure 4.4: An Island Network and its Equivalent Random Network, both accompanied by their degree distributions. Both contain nodes, initial edges, deletion parameter , and , with for the Island Network and for the Equivalent Random Network; consequently, the connectivity푁 = 1,000 is equal퐸 for = 10 both.,000 As we shall see, despite appearing 휅identical = 0.075 in nearly 푘푚푎푥 = 1 = 1.47 = 1.46 every respect (down even to the profiles<푘> of their degree distributions), the Island Network differs from its 푁 random equivalent sufficiently 휌to =allow for the unambiguous identification of certain topological properties.

Figure 4.5: Portions of Island Networks with values of nodes, initial edges (with ), rewiring parameter , and a varying deletion parameter that gradually merges the “islands” together. (a) , designating this graph as a lower-limit푁 = 100 member of Class 2. 퐸(b) = 1,000 퐾 = 10 , designating훽 = this 0.02 graph as an unambiguous member of Class 2. (c) , designating this휅 graph = 0.05 as ⇒functionally〈푘〉 = 1 indistinguishable from a Watts-Strogatz[6] network. 휅 = 0.075 ⇒ 〈푘〉 = 1.5 휅 = 0.15 ⇒ 〈푘〉 = 3

The average small-worldness coefficient for the Island Network was found to be

( , and the average small-worldness coefficient for the −2 −3 푆equivalent퐼 = 1.147 random× 10 network휎 = 4. 880was ×found10 to) be ( . −3 −3 푟푎푛푑 Because , the Island Network exhibits푆 small-worldness.= 7.401 × 10 휎 = 1.022 × 10 ) 푆퐼 푆푟푎푛푑 > 1

4.2.3: Hubs

Before we demonstrate that hubs do in fact manifest within the Island Network, it is first necessary to define what exactly qualifies as a hub for the purposes of this study. We formally identify individual hubs in terms of nodes with the greatest values of betweenness, closeness, or eigenvector centrality. Each graph has three distributions of normalized centrality measures (one for each node), such that the frequency of normalized푁 values for betweenness centrality , closeness centrality , and eigenvector centrality , are plotted against those values. Those퐵 normalized centrality퐶 measures whose values serve푉 as the peaks of their respective distributions correspond to nodes that shall provisionally be referred to as

“hub candidates.” For the purposes of this study, the criteria for a node to be considered a hub are as follows:

1) The node must be a hub candidate; that is, at least one of the distributions of

normalized centrality measures must peak at the index corresponding to that node.

2) The hub candidate must be the member of a community which contains at least

one other hub candidate; this requirement is immediately satisfied if a hub

candidate qualifies as a candidate in multiple centrality measure distributions

simultaneously. In other words, a hub candidate qualifies as a hub if it belongs to

the same community as another hub candidate, and/or if its value is greater than

those of all other nodes in the graph for more than one centrality metric.

As we shall see in the next chapter, this heuristic carries predictive capabilities with respect to the dynamics of nonlinear signals that get spread across the network, but for our purposes

63 here, the criteria for the identification of individual hubs will serve to establish a working definition for hubs within the context of its structure.

More granular distinctions between different types of hubs must be made on a case- by-case basis with the guidance of the table reproduced in the previous chapter. However, a detailed discussion and analysis of the manner in which particular combinations of different centrality metrics affect the structure of the resulting graph lies beyond the scope of this study. For the purposes of this study, the distinction between provincial hubs and connector hubs, as outlined under Chapter 1, shall be purely qualitative; we offer here no formal analysis of these different hub types beyond broad descriptions of their most general features.

Our objective in this chapter is to establish an operational definition of hubs so that they may be individually identified, and to establish that the sole criterion for the presence of hubs within a network (a positively skewed degree distribution) is in fact met by the Island

Network.

To demonstrate that the Island Network does possess hubs, and thus differs sufficiently from the Watts-Strogatz Network to warrant its distinction as a novel graph topology, we present the average degree distribution over 20 Watts-Strogatz graphs with identical parameters to that of the Island Network’s (save, of course, for the deletion parameter , which is set equal to unity), compared against the average degree distribution over 20 Island휅 Networks. The result is unambiguous; the Island Network is capable of producing hubs, and the Watts-Strogatz network is not. The average degree distributions of small-world networks and equivalent Island networks, aggregated over 20 sets of distributions for each bin, are displayed in Figure 4.6 below.

Figure 4.6: The average degree distributions, aggregated over 20 sets of distributions for each bin. (a) Watts-Strogatz Network (b) Island Network

4.2.4: Modular Communities

The manner in which modularity is assessed is akin to that of small-worldness, in that we compare the modularity of an Island Network to that of an equivalent random graph.

Modularity is computed via the application of the Girvan-Newman algorithm, which supplies the communities used to compute the membership matrix . Together with the modularity matrix , the modularity equation may be applied to find 푺, which ranges from the negative values characteristic푩 of scale-free graphs, to the near-zero 푄values characteristic of random graphs, to the positive values characteristic of small-world graphs.

We compute the modularities of 20 different Island Networks and their 20 equivalent random graphs. The Island Networks exhibit average modularity (

), and the equivalent random networks exhibit average modularity푄퐼 = 0.9478 휎 = 1.847 × −2 푟푎푛푑 10 ( ). As with small-worldness and hubs, the data unambiguously푄 = 1.033 × −5 −5 10indicates휎 =that 2. 729the Island× 10 Network is extremely modular.

4.3: Discussion of Topology

As we can see, the Class 2 Island Network exhibits small-worldness, hub structures, and highly modular communities, as established by measurements of its clustering coefficient, average shortest path length, degree distribution, and the application of the

Girvan-Newman algorithm in concert with the modularity coefficient. The results of the comparisons between these metrics and those of equivalent random or small-world graphs consistently indicate that the Island Network exhibits a unique set of properties that are not shared by Watts-Strogatz or Erdos-Reyni Networks, thus warranting the claim that the Island

Network stands as a unique graph theoretical structure in its own right. Having established this, it is worth clarifying the qualitative features concerning the structure of the Class 2

Island Network that are of greatest interest to us, as the subsequent discussion on dynamics draws heavily from our appraisal of graph topology.

As suggested by Figure 4.5, modulating the deletion parameter such that the Island

Network transitions from Class 1 to Class 2, results in the coalescence of휅 “islands” of node structures into larger connected components. One might intuit from this picture that the nodes which serve as “bridges” between “islands” of nodes serve as the focal points of the network, such that their deletion would impact the structure of the graph to a far greater extent than would a node picked at random. Such nodes would have unusually high values of centrality, and thus would serve as principle candidates for hubs. This is effectively the suggestion that the Island Network’s structure may fundamentally be understood in terms of highly modular communities of nodes that are connected to one another by hubs, and that the deletion parameter facilitates the extent to which these communities are interconnected.

At regimes휅 of that produce Class 1 graphs , the “islands” are too 2퐾 disconnected from one휅 another to integrate into any large(0 ≤ connected〈푘〉 ≤ 2퐾+1 components.) At regimes of that produce networks that are effectively indistinguishable from Watts-Strogatz graphs

휅 , the “islands” are so closely interwoven into a large connected component that

(there〈푘〉 ≥ are 2) no nodes whose relative positions confer any structural advantages in terms of 66 centrality. In other words, such networks, in keeping with their resemblance to Watts-

Strogatz graphs, have no hubs; the density of connections between the “islands” is comparable to those within them, rendering “bridges” obsolete. In a network where the boundaries between communities are poorly defined, there are no nodes whose deletion would impact the structure of the graph to a significantly greater extent than that of a node selected at random. But at regimes of that produce Class 2 Island Networks

, the “islands” are휅 sufficiently distinct from one another to allow for the 2퐾 4퐾 2퐾+1 2퐾+1 (emergence≤ 〈푘 〉of≤ highly) central “bridges” that connect them together, but are not so distinct so as to be unable to coalesce into large connected components.

These qualitative observations about the structure of the Island Network matter because they illustrate those features that are most desirable in neural network topologies: they allow for the integration and segregation of specialized brain regions (communities, or

“islands”) that are sparsely connected via highly central pathways (hubs, or “bridges”). For reasons that will be explored in the next chapter, this kind of structure is particularly important where the dynamics of brain signals are concerned. Here, we simply take note of the fact that the dynamics will be downstream of the topology, and conclude our discussion of topology with the promise that the manner in which the Island Network’s structure facilitates the behavior of nonlinear stochastic signals is simultaneously intuitive and highly illuminating.

Chapter 5: Dynamics of the Island Network

Having carefully established the structural properties of the Island Network, the motivations underlying the development of a topology that exhibits those properties, and the means by which one may verify that the Island Network’s topology is in fact characterized by those properties, we now turn our attention to the final objective of this thesis: a detailed description of the dynamics of a nonlinear stochastic signal that spreads across the Island

Network. As indicated earlier, the dynamics of a signal that travels through a network is contingent upon the structure of that network; more compactly, dynamics are downstream of topology. In this chapter, we shall motivate the dynamical model[55] that will be used to simulate the spread of electrochemical signals across a network of tracts and neuronal ensembles.

5.1: Preliminaries Concerning Connectome Dynamics

The discharge of an action potential down an axon is contingent upon the voltage at the axon hillock, which must reach a certain threshold value in order to depolarize the cell.

This voltage across the neuronal membrane constantly undergoes minute changes in both the positive and negative directions principally as a consequence of the neurotransmitters that arrive at its somatic and dendritic receptors, which activate chemical cascades in the cytoplasm that open ion channels in the membrane. The resulting flux of ions either incrementally increases or decreases the voltage, with the former effect being initiated by excitatory neurotransmitters and the latter effect being initiated by inhibitory neurotransmitters. From this sequence of events, it is clear that a postsynaptic neuron’s acquisition of threshold voltage depends nonlinearly on the transmission of neurotransmitters from its presynaptic neuron(s), as the accumulation of excitatory neurotransmitters at one set of receptors may have their effects nullified by the accumulation of inhibitory neurotransmitters at another set of receptors, which has the effect of delaying or blocking the signal altogether. This is a consequence of postsynaptic neurons often having multiple 68 presynaptic neurons synapse with it. Equivalently, presynaptic neurons may synapse with multiple postsynaptic neurons, as one axon collateral may terminate at a junction with a different cell from that of another axon collateral. The upshot of all of this is that the dynamics of the signal depends on both the manner in which the neurons are connected, and the extent to which those neurons are supplied with excitatory versus inhibitory neurotransmitters.

It is worth noting that none of this implies that the extent to which a neuron is excited will have no impact on its resulting action potentials; after all, neurons that have been stimulated by stronger depolarizing currents will produce spike trains with greater frequency than neurons that have been stimulated by weaker depolarizing currents. We merely mean to emphasize that the extent to which a postsynaptic signal spreads from a neuron depends nonlinearly on the signal that it receives.

All of this motivates a model of stochastic nonlinear signal-spreading that aggregates the chaotic waveforms produced by isolated neural circuits into predictable waveforms characteristic of neural oscillations via the integration of multiple brain regions. Put simply, the brain as a whole exhibits characteristic dynamical behavior; this global behavior is emergent from interactions between differentiated ROIs, whose local behaviors are stochastic and nonlinear; and thus a model of connectome dynamics that explains how the confluence of otherwise unpredictable processes among local, disparate brain regions manages to produce global, coherent brain signals, is warranted. The introduction of a simplified version of such a model, whose topology and dynamics live up to this order, is one of the primary objectives of this study.

5.2: Nonlinear Stochastic Signal Spreading

The action potentials of individual neurons have been modelled by coupled nonlinear differential equations since the genesis of quantitative neurophysiology[56]; however, existing models of neural computation, which attempt to discern the dynamics of signal spreading

69 through networks of aggregated neurons, typically employ either graph-theoretic structures in conjunction with signals which exhibit purely linear dynamics[30], combine stochasticity with nonlinearly coupled oscillator dynamics without reference to graph-theoretical structures[29], or make use of nonlinear dynamical equations in the absence of stochasticity[26][27][28], while omitting graph-theoretic structures altogether. Our own investigation of the dynamics that underlie neural processes is unique in that the graph-theoretic structures that we use to model the human brain are employed in conjunction with simulations of stochastic nonlinear signals; as stated in the introduction, we aim to develop a model of neuronal dynamics that makes use of all three in order to more efficiently capture the properties of real brains.

In this study, we represent the dynamics of brain networks as a function of the activation times of neuroreceptors in postsynaptic neuronal ensembles, represented by a node

, upon having made contact with the neurotransmitters and ions expelled from presynaptic neuronal푗 ensembles, represented by a node . The activity of a neuronal ensemble is given by

if it is inactive (the threshold voltages푖 of its constituent neurons have not been reached),휂푖 = 0 and if it is active (spike trains initiated.) The nodes are subject to a two-state reaction scheme:휂푖 = 1

, (5.1) 훼 0 ⇌ 1 훽 where the rate of the reverse reaction for a node is held at , and the rate for ’s forward reaction is given by 푖 훽 = 1 푖

, (5.2) 훾 푖 푖 훼 = 푔 푐푖 with being a constant proportional to the excitability of node , representing the relative concentration푔푖 of excitatory neurotransmitters at the neurons which푖 푐 compose푖 node , and

70 푖 훾 defined as an exponent controlling the degree of sensitivity of that node to changes in the concentration of excitatory neurotransmitters. We set the exponent in order to model the nonlinearity of the system at hand. 훾 = 2

The local signalling concentration over time is given by

, (5.3)

푖 0 푗≠푖 푖푗 푗 푐 (푡) = 푐 + 푟 ∑ 퐴 휂 (푡) where the constant sets the background activity of each site in the absence of coupling, parameter sets the푐 0strength of the connection between nodes, and is an element of

푖푗 adjacency matrix푟 , with the latter representing the configuration of퐴 neural circuits in a brain network, and the former푨 representing whether or not a presynaptic neuronal ensemble is coupled to postsynaptic neuronal ensemble . 푖

As with the rate of the reverse reaction,푗 , the excitability constant will be held constant at unity for every connection in the network.훽 Moreover, we will define푔 the dimensionless excitability parameter

, (5.4) 푔푐0훾 훽 휒 = as well as the dimensionless coupling parameter

, (5.5) 푟 푐0 휉 = in order to simplify the equation for the rate of the forward reaction:

. (5.6) 훾 푖 푗≠푖 푖푗 푗 훼 = 휒[1 + 휉 ∑ 퐴 휂 (푡)]

5.3: Monte-Carlo Analysis

Due to the novelty of our dynamical approach within the context of connectomes, our numerical appraisal of those dynamics is likewise somewhat unconventional; however, its fruitful application within the context of cardiac arrhythmia[55], in conjunction with the dynamical model upon from which our own was adapted, warrants its reproduction in this context. We will now detail the Monte-Carlo technique that was employed to make sense of our data. As with the previous chapter, the contributions of residue (small, unconnected components of the Island Network) are not captured by our analysis.

In order to track the dynamics of the signal being spread across the Island Network, we generate a plot of the fraction of activated nodes,

, (5.7) 1 푗 푗 < 휀 > = 푁 ∑ 휂 against time in seconds, with each time step spanning an interval of one millisecond. We respectively set the excitability parameter and coupling parameter at and .

The resulting dynamics differ as the Island Network gets modulated from휒 = 0.Class025 1 휉 = 100

to Class 2 ( ) and beyond, as indicated by Figure 5.1. −1 4퐾 0 ≤ 휅 < (2퐾 + 1) 1 ≤ 〈푘〉 ≤ 2퐾+1

Figure 5.1: Monte Carlo plots of the dynamics of graphs with varying values for , with the resulting Class of graph changing the attendant dynamics. Class 1 Island Networks ( ) exhibit slow, weakly coupled dynamical behavior; Class 2 Island Networks ( ) exhibit metastable dynamical휅 behavior; Class 3 Island Networks ( ) exhibit rapid on/off dynamical behavior. 휅 = 0.025 휅 = 0.025 휅 = 0.125 72

When comparing these results to those of Figure 4.3, which showcases how the structure of the Island Network changes with the modulation of , it is easy to see that a complementary phenomenon occurs with respect to the dynamics:휅 graphs with more strongly connected components will spread the signal almost immediately, as the activation of a single node is sufficient to activate the rest of the network (with the exception of any residue that happens to be disconnected from the main connected components.)

Graphs which exhibit the structure of Class 2 Island Networks are composed of several connected components (“islands,”) the largest of which are composed of multiple connected communities. Upon being activated, a node will spread the signal to every member of its community, as well as to every adjacent community, until all members of the island have been activated. This is represented by the transitions between different attractors, which as we shall see can be explained entirely in terms of the community structures which populate the Class 2 Island Network.

Finally, graphs which exhibit the structure of Class 1 Island Networks exhibit very slow and gradual activation in contrast to the other networks, which is owed to its extreme sparseness. The absence of any nontrivial connected components makes the propagation of signal throughout the network impossible; individual nodes (“residue”) have no edges

( ,) and thus have their dynamics governed by the same activation probability

푗≠푖 푖푗 ∑ 퐴 =: 0

푝푖 = 훼푖 Δ푡

(5.8) 훾 −5 푖 푗≠푖 푖푗 푗 푝 = 휒[1 + 휉 ∑ 퐴 휂 (푡)] Δ푡 = 휒Δ푡 = 2.5 × 10 for all nodes that lack neighbors. Because the rate of the reverse reaction is , the deactivation probability for all nodes that lack neighbors is 훽. = 1

푖 It is worth noting that the “jumps” in the dynamics of푝 Class= Δ푡 2 = Island 0.001 Networks are contingent upon the nonlinearity of the system. Figure 3.1 exhibits, from top to bottom, the dynamics of a Class 1 Island Network, the dynamics of a Class 2 Island Network, and the 73 dynamics of a Watts-Strogatz network. For each of the Monte Carlo plots exhibited below in

Figure 3.1, has been set equal to unity.

We 훾would like to draw particular attention to Class 2 graphs and the periods of relative stability around a certain value of , punctuated by sudden transitions to higher values of , because a principle aim of< our 휀 > exploration of the dynamics of the Island

Network is< to 휀 >explain how these attractor transitions take place. Specifically, we aim to understand how the community structure of the Island Network and the presence of hubs are related to the profiles of these transitions.

5.3.1: Dynamics as a Function of Community Structure

As with our exploration of the topology, we shall restrict the bulk of our analysis of the dynamics to Class 2 Island Networks with node count , initial ratio of edges to nodes , rewiring parameter , and deletion푁 parameter = 1000 . We claim that the퐾 community = 10 structures are entirely훽 = 0. 02predictive of the dynamics of휅 =the 0. Island075 Network within this regime, and that attractor transitions are governed by three types of dynamical processes: Individual Activations, Activations in Series, and Activations in Parallel. These dynamical processes are understood in terms of transitions between different attractor states, which we define as the activation of at least 1% of the graph’s nodes over the course of a single time step. In our graphs, which contain a total of nodes, time steps which precede and succeed the activation of at least nodes mark1000 the boundaries of the intervening transition. 10

“Individual Activations” refers to scenarios where the interval between the beginning and end of a transition in the Monte Carlo plot is characterized almost exclusively by the activation of nodes that are the members of a single community; a visual aid is given by

Figure 5.2. In such scenarios, it is sufficient to know the index of a single node that became active during this interval; the members of its community can subsequently be identified via the application of the Girvan-Newman algorithm. 74

Figure 5.2: An illustration of the manner in which signal spreads during Individual Activation.

We construct a vector, , representing the dynamical status of every node in the

푏푒푓표푟푒 network at a time step immediately푙 preceding the interval during which the transition occurs.

The vector has components, with values of at indices corresponding to nodes that are inactive during 푁that particular time step and values0 of at indices corresponding to nodes that are active. A second vector, , is constructed in a 1near-identical manner, except this one

푎푓푡푒푟 corresponds to a time step immediately푙 after the transition has taken place. The difference between these two nodes, , represents the nodes that became activated

푑푖푓푓 푎푓푡푒푟 푏푒푓표푟푒 during the interval that the푙 transition= 푙 occurred− 푙 within, with values of representing nodes that were activated during the interval of interest and values of representing1 the opposite.

We then construct another vector, , which is also composed0 of components. From

, we select a single node with random푉 uniformI probability, and then푁 identify the

푑푖푓푓 members푙 of its community via the Girvan-Newman algorithm. The components of that correspond to the indices of both the randomly selected node and its community members푉I are set to unity; all other components of are set to . A Pearson correlation test may then be executed, comparing the nodes that became푉I active0 during the interval in which a transition took place to the nodes that we predict to have transitioned during that interval based on their community membership in relation to the randomly selected node from . A Pearson test

푑푖푓푓 of and which yields a high correlation coefficient indicates that the푙 transition was

푑푖푓푓 I indeed푙 governed푉 by the activation of a single community.

As indicated by Figure 5.3, this technique is extremely reliable for describing transitions characterized by the activation of up to 10% of the graph’s nodes, meaning that transitions of that scale are dominated by the dynamical process of Individual Activation.

However, the rapidly decreasing reliability of Individual Activation to account for transitions greater that 10% of the graph’s nodes motivates the development of techniques which can capture the full range of dynamical processes that Class 2 Island Networks exhibit. These techniques describe Activation in Series and Activation in Parallel.

1 0.9 0.8 0.7 0.6 0.5 Individual Activation 0.4 0.3 0.2 0.1 0 <5 5 to 10 10 to 15 15 to 20 >20

Figure 5.3: The average values of correlation coefficients for Individual Activation ( ) over n = 20 trials for each interval (n = 100 in total), plotted against the percentage of the network that was 퐼 퐼 activated during transition. 푥̅ = 0.6742 , 휎 = 0.09956

“Activation in Series” refers to scenarios where the interval between the beginning and end of a transition in the Monte Carlo plot is characterized almost exclusively by the activation of nodes that are members of both an individual community, and all adjacent communities; a visual aid is given by Figure 5.4. As previously discussed, it is possible for a node to have a neighbor that isn’t a member of the same community; such nodes effectively act as bridges between modules of nodes that have denser connections among themselves than between each other. These communities, as well as all other communities which connect to them in this manner, are adjacent to each other. 76

Figure 5.4: An illustration of the manner in which signal spreads during Activation in Series.

We represent them via the construction of a vector ; as before, one node corresponding to the active components of is selected푉 with푆 a random uniform

푑푖푓푓 probability, and the members of its community푙 are identified via the Girvan-Newman algorithm. Then, each node has its neighbors identified; if the neighbors of each node are all members of the same community, then there are no adjacent communities. However, if there are neighbors which do not belong to the community, then the communities which they do belong to are identified. The process of searching for adjacent communities then repeats itself until all of them are found. The members of each adjacent community are subsequently encoded by , which then undergoes a correlation test with .

푆 푑푖푓푓 As indicated푉 by Figure 5.5, this technique is extremely푙 reliable for describing transitions characterized by the activation of up to 15% of the graph’s nodes. Because

Individual Activation is sufficient to account for activations of up to 10%, transitions that result in activations of anywhere between 10% and 15% of the graph’s nodes are dominated

77 by the dynamical process of Activation in Series. For transitions greater than 15%, we claim that the dynamical process of Activation in Parallel dominates.

1 0.9 0.8 0.7 0.6 0.5 Individual Activation 0.4 Activation in Series 0.3 0.2 0.1 0 <5 5 to 10 10 to 15 15 to 20 >20

Figure 5.5: The average values of correlation coefficients for Individual Activation ( ) over n = 20 trials for each interval (n = 100 in total), plotted against the percentage of the network 푆 푆 that was activated during transition. 푥̅ = 0.8230 , 휎 = 0.09392

“Activation in Parallel” refers to scenarios where the interval between the beginning and end of a transition in the Monte Carlo plot is characterized almost exclusively by the activation of nodes that are the members non-adjacent communities; a visual aid is given by

Figure 5.6. It is possible, especially where particularly large transitions are concerned, for multiple disparate communities to simultaneously contribute to a transition. If a node becomes active while a community that it neither belongs to nor is adjacent to is undergoing a transition, then that node’s community, as well as any adjacent community, will also transition, and the Monte-Carlo plot will register the combined activations of all communities that are involved.

Figure 5.6: An illustration of the manner in which signal spreads during Activation in Parallel.

Once again, we test our hypothesis via the construction of a vector, . The components of this vector with values of are found in the following manner:푉푃 first, a node that became activate during the interval of1 interest is selected with random uniform probability from . Then, every community is identified via the Girvan-Newman

푑푖푓푓 algorithm, and each푙 one is provisionally assigned a value , which is the correlation coefficient between the vector of that community and 푃푆 ; essentially, the technique for

푆 푑푖푓푓 identifying Activation in Series푉 is imposed on every community.푙 Those with negligible values can be surmised to have not undergone a transition within the interval of interest.

However, those with values of are taken to be communities which did indeed activate during the transition being푃푆 ≥ analyzed, 0.2 but are nevertheless insufficient to by themselves fully account for it. The components of every such community are encoded in the

Activation in Parallel vector, , which is finally tested against .

푃 푑푖푓푓 As indicated by Figure푉 5.7, this technique is extremely reliable푙 for describing transitions characterized by the activation of anything over 15% of the graph’s nodes, meaning that transitions of that scale are dominated by the dynamical process of Activation in

Parallel. Because Individual Activation and Activation in Series are sufficient to account for activations of up to 15%, transitions involving more than 15% of the graph’s nodes are dominated by the dynamical process of Activation in Parallel.

1 0.9 0.8 0.7 0.6 Individual Activation 0.5 Activation in Series 0.4 Activation in Parallel 0.3 0.2 0.1 0 <5 5 to 10 10 to 15 15 to 20 >20

Figure 5.7: The average values of correlation coefficients for Individual Activation ( over n = 20 trials for each interval (n = 100 in total), plotted against the percentage of the 푃 network that was activated during transition. 푥̅ = 0.9586, 휎푃 = 0.03484)

With the complete histogram, it is clear that the transitions characteristic of Class 2

Island Networks may be understood entirely in terms of the dynamical processes of its constituent communities. Moreover, one is able to discriminate between these different types of processes simply by comparing the correlation coefficients of each individual case.

Activation in Parallel will have high correlation coefficients when is employed, and much lower correlation coefficients when employing and . Activation푉푃 in Series will have high correlation coefficients when and are employed,푉푆 푉and퐼 much lower correlation coefficients when employing 푉푃. And푉 Individual푆 Activations will always have high correlation coefficients, regardless푉퐼 of which type of vector is employed.

5.3.2: Dynamics as a Function of Hubs

We shall now turn our attention to the role played by hubs in the diffusion of nonlinear stochastic signals across the Island Network. First, we shall review the criteria that ought to be met by nodes in order to qualify as hubs, before moving on to our expectations of how hub structures facilitate signal spreading, and finally discussing the methodology and results. 80

As indicated in Section 4.3.3, the criteria for a node to be considered a hub are as follows: