METHODS FOR COMPUTATION AND ANALYSIS OF MARKOVIAN DYNAMICS ON COMPLEX NETWORKS by W. Garrett Jenkinson A dissertation submitted to The Johns Hopkins University in conformity with the requirements for the degree of Doctor of Philosophy. Baltimore, Maryland October, 2013 c W. Garrett Jenkinson 2013 All rights reserved Abstract A problem central to many scientific and engineering disciplines is how to deal with noisy dynamic processes that take place on networks. Examples include the ebb and flow of biochemical concentrations within cells, the firing patterns of neurons in the brain, and the spread of disease on social networks. In this thesis, we present a general formalism capable of representing many such problems by means of a master equation. Our study begins by synthesizing the literature to provide a toolkit of known mathematical and computational analysis techniques for dealing with this equation. Subsequently a novel exact numerical solution technique is de- veloped, which can be orders of magnitude faster than the state-of-the-art numerical solver. However, numerical solutions are only applicable to a small subset of processes on networks. Thus, many approximate solution techniques exist in the literature to deal with this problem. Unfortunately, no practical tools exist to quantitatively eval- uate the quality of an approximate solution in a given system. Therefore, a statistical tool that is capable of evaluating any analytical or Monte Carlo based approximation to the master equation is developed herein. Finally, we note that larger networks ii ABSTRACT with more complex dynamical phenomena suffer from the same curse of dimension- ality as the classical mechanics of a gas. We therefore propose that thermodynamic analysis techniques, adapted from statistical mechanics, may provide a new way for- ward in analyzing such systems. The investigation focuses on a behavior known as avalanching|complex bursting patterns with fractal properties. By developing ther- modynamic analysis techniques along with a potential energy landscape perspective, we are able to demonstrate that increasing intrinsic noise causes a phase transition that results in avalanching. This novel result is utilized to characterize avalanching in an epidemiological model for the first time and to explain avalanching in biologi- cal neural networks, in which the cause has been falsely attributed to specific neural architectures. This thesis contributes to the existing literature by providing a novel solution technique, enhances existing and future literature by providing a general method for statistical evaluation of approximative solution techniques, and paves the way towards a promising approach to the thermodynamic analysis of large complex processes on networks. Primary Reader: John Goutsias Secondary Reader: Andrew Feinberg iii Acknowledgments Ostensibly, this doctoral thesis is the product of six years of education, ex- cogitation, and effort; however the reality is that these six years of work at Johns Hopkins University were propped up by the preceding twenty-two years of learning. Likewise, although this thesis bears my name, none of it would have been possible without an amazing complex network of support. Complete enumeration of this net- work is beyond my space limitations, so I will attempt to discuss its hubs, which have played the most critical role. The most supportive member of my network is my fianc´ee,Dr. Lara Walkoff, who has been by my side for the past decade to see me grow from a high school math geek to the absent-minded academic of the present. She makes my journey through life enjoyable and exciting, while her brilliance and work ethic set the precedent that I continually strive to achieve. By far, the most important professional node in my network is my advisor, Prof. John Goutsias. He considered me a peer from day one, and our often contentious and heated debates about science (which were never taken personally) served to pu- iv ACKNOWLEDGMENTS rify our scientific claims like gold from fire. Despite our enormous mutual respect, neither of us has trusted a scientific claim from the other until we have understood it individually. Peer review started long before we submitted our papers|and this, in my opinion, is how science should always be conducted. I will be forever grateful that John has fostered my mathematical thinking and skills to a level that I never imagined possible. Graduate school is intended to be a modern apprenticeship, and I certainly feel that John has succeeded in bringing our minds to a convergent state. More often our thoughts are in lockstep, and thus our conversations can occur in rapid shorthand that would surely be incomprehensible to an outside observer. His men- torship has been invaluable, and I cherish the fact that we will undoubtedly remain life-long friends. I am indebted to my second reader, Prof. Andy Feinberg, whose ebullient enthusiasm demonstrates the true meaning of life-long learning, even after a lifetime of astonishing accomplishments. I look forward to many years of collaboration where our productivity will only be matched by the enjoyment of the voyage. I would also like to express gratitude to my committee members, Prof. Howard Wienert and Prof. Pablo Iglesias, for taking the time to improve and evaluate this work. Graduate school has been made possible by the following funding sources: the Wolman family for an Abel Wolman Fellowship, the Institute for Nanobiotech- nology for a National Science Foundation (NSF) IGERT Fellowship, the Department v ACKNOWLEDGMENTS of Defense for an NDSEG Fellowship, Prof. Goutsias for funding me through two NSF Grants, CCF-0849907 and CCF-1217213, and finally the Siebel Foundation for their prestigious Siebel Scholarship. I would like to extend special thanks to some of the most influential mentors who helped me crystalize a love for teaching. I am grateful to Prof. Carey Priebe at Johns Hopkins, the most talented teacher I have encountered in 23 years of classes, for providing me with a standard to emulate. Prof. Jelena Kovaˇcevi´cand Prof. Jos´eMoura at Carnegie Mellon University instilled in me the joy of research as an undergrad. Then, there were those teachers who inspired me early on: Sue Frennesson showed me the joys of mathematics in middle school, Dr. Michael Greene gave me the love of physics in high school, and Marcia Mett brought chemistry to life in the high school laboratory. Most importantly, my friends and family have made the journey entertaining. The support of my grandparents Ashford and Carolyn Jenkinson has been invaluable, and I always look forward to calling them to share the joy of my latest achievement. Special thanks to my brother and best friend, Gavin, for relocating to Baltimore and keeping the sometimes mundane life of a graduate student amusing. vi Dedication This thesis is dedicated to my parents who, through their emphasis on quality education, infected me with an incurable intellectual bug that prepared me for this career choice. They have taught by example the meaning of hard work and taking pride in one's vocation. This lesson is the main reason I was able to persevere and become the first Dr. Jenkinson in our family. Undoubtedly, the fact that I had an outstanding childhood has also contributed to my sanity, which most people would have lost from spending so many hours toiling upwards into the night. Their con- tinuing support, encouragement and interest in my arcane pursuits make me joyous beyond words and are causal features of my perpetual smile. Alan and Debbie, I love you both, and hope you enjoy reading my monolithic tribute to the greatest parents in the world|although I will understand if you choose to skim some of the details! vii Contents Abstract ii Acknowledgments iv List of Tables xii List of Figures xiii 1 Introduction 1 1.1 Motivation . 1 1.2 Scope and organization of thesis . 3 2 Markovian Reaction Networks: A Coherent Framework 9 2.1 Reaction networks . 10 2.1.1 Chemical systems and reaction networks . 10 2.1.2 Stochastic dynamics on reaction networks . 13 2.2 Examples . 16 2.2.1 Biochemical networks . 16 2.2.2 Epidemiological networks . 17 2.2.3 Neural networks . 20 2.3 Solving the master equation . 21 viii CONTENTS 2.3.1 Exact analytical methods . 23 2.3.2 Numerical methods . 23 2.3.3 Computational methods . 26 Exact sampling . 27 Poisson leaping . 28 Gaussian leaping . 30 2.3.4 Linear noise approximation . 31 2.3.5 Macroscopic approximation . 34 2.4 Mesoscopic (probabilistic) behavior . 35 2.5 Potential energy landscape . 41 2.6 Macroscopic (thermodynamic) behavior . 46 2.6.1 Balance equations . 48 2.6.2 Thermodynamic equilibrium . 51 3 Numerically Solving the Master Equation: Implicit Euler Method 54 3.1 Motiviation . 55 3.2 Methods . 57 3.2.1 Disease dynamics . 57 3.2.2 Exploiting structure . 58 3.2.3 Numerical solver . 60 3.2.4 Practical considerations . 61 3.3 Results . 64 3.4 Discussion . 69 4 Statistical Testing of Master Equation Approximations 73 4.1 Motivation . 74 4.2 LNA for the population process . 75 ix CONTENTS 4.3 Testing the validity of analytical approximations . 77 4.3.1 The one-dimensional case . 78 Hypothesis testing . 78 Choosing the significance level and sample size . 82 4.3.2 Extension to multiple dimensions . 85 4.4 Testing the validity of approximative sampling . 87 4.4.1 The one-dimensional case . 88 Hypothesis testing . 88 Choosing the sample size and significance level . 90 4.4.2 Extension to multiple dimensions . 91 4.5 Results . 91 4.6 Discussion . 105 5 Thermodynamic Analysis of Leaky Markovian Networks 109 5.1 Motivation . 110 5.2 LMN theory and analysis . 114 5.2.1 Leaky Markovian networks . 114 5.2.2 LMNs, Markovian reaction networks, and Boolean networks . 116 5.2.3 Coarse graining .