University of Pennsylvania ScholarlyCommons Publicly Accessible Penn Dissertations 2015 Optimal Control of Epidemics in the Presence of Heterogeneity Soheil Eshghi University of Pennsylvania, [email protected] Follow this and additional works at: https://repository.upenn.edu/edissertations Part of the Applied Mathematics Commons, Electrical and Electronics Commons, and the Epidemiology Commons Recommended Citation Eshghi, Soheil, "Optimal Control of Epidemics in the Presence of Heterogeneity" (2015). Publicly Accessible Penn Dissertations. 1704. https://repository.upenn.edu/edissertations/1704 This paper is posted at ScholarlyCommons. https://repository.upenn.edu/edissertations/1704 For more information, please contact [email protected]. Optimal Control of Epidemics in the Presence of Heterogeneity Abstract We seek to identify and address how different types of heterogeneity affect the optimal control of epidemic processes in social, biological, and computer networks. Epidemic processes encompass a variety of models of propagation that are based on contact between agents. Assumptions of homogeneity of communication rates, resources, and epidemics themselves in prior literature gloss over the heterogeneities inherent to such networks and lead to the design of sub-optimal control policies. However, the added complexity that comes with a more nuanced view of such networks complicates the generalizing of most prior work and necessitates the use of new analytical methods. We first create a taxonomy of heterogeneity in the spread of epidemics. We then model the evolution of heterogeneous epidemics in the realms of biology and sociology, as well as those arising from practice in the fields of communication networks (e.g., DTN message routing) and security (e.g., malware spread and patching). In each case, we obtain computational frameworks using Pontryagin’s Maximum Principle that will lead to the derivation of dynamic controls that optimize general, context-specific objectives. We then prove structures for each of these vectors of optimal controls that can simplify the derivation, storage, and implementation of optimal policies. Finally, using simulations and real-world traces, we examine the benefits achieved by including heterogeneity in the control decision, as well as the sensitivity of the models and the controls to model parameters in each case. Degree Type Dissertation Degree Name Doctor of Philosophy (PhD) Graduate Group Electrical & Systems Engineering First Advisor Saswati Sarkar Keywords Epidemic Routing, Epidemics, Heterogeneity, Optimal Control, Patching, Visibility Subject Categories Applied Mathematics | Electrical and Electronics | Epidemiology This dissertation is available at ScholarlyCommons: https://repository.upenn.edu/edissertations/1704 OPTIMAL CONTROL OF EPIDEMICS IN THE PRESENCE OF HETEROGENEITY Soheil Eshghi A DISSERTATION in Electrical and Systems Engineering Presented to the Faculties of the University of Pennsylvania in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy 2015 Supervisor of Dissertation Saswati Sarkar, Professor of Electrical and Systems Engineering Graduate Group Chairperson Alejandro Ribeiro, Rosenbluth Associate Professor of Electrical and Systems Engineering Dissertation Committee: Santosh S. Venkatesh, Associate Professor of Electrical and Systems Engineering George J. Pappas, Joseph Moore Professor and Chair of Electrical and Systems Engineering Victor M. Preciado, Raj and Neera Singh Assistant Professor of Electrical and Systems Engineering Olgica Milenkovic, Associate Professor of Electrical and Computer Engineering (University of Illinois at Urbana-Champaign) *Dedication This thesis is dedicated first and foremost to my parents, Mahin and Sassan, who have been my bedrock, my guides, and my friends. I am exceptionally grateful to all who have helped me learn, question, and grow. ii Acknowledgments This thesis owes its existence to my advisor, Professor Saswati Sarkar. Her ex- cellent advice and integrity have guided me through my years at Penn, and have shaped my academic approach. I am forever grateful for her dedication to my academic training and rigor. My co-advisor, Professor Santosh S. Venkatesh, has had an equally immense role in this undertaking. His critical eye and simultane- ous attention to the big picture and minute detail has been critical to this project, and to my graduate education. I will forever cherish his wonderful lectures. Thanks are also due to my academic collaborators – first and foremost among them, Dr. M.H.R. (Arman) Khouzani, who mentored me in my first few years at Penn, and introduced me to the topic of epidemic control. I have also had the pleasure of working with, and learning from, Professor Ness Shroff, Dr. Rakesh M. Patil, and Professor Victor M. Preciado. I am also very grateful to the members of my committee, Professor George J. Pappas and Professor Olgica Milenkovic, for their support and insight, which opened up new lines of academic inquiry. Finally, I am exceptionally grateful for my friends and colleagues here in Philadelphia, who have made my stay here some of the greatest years of my life. iii ABSTRACT OPTIMAL CONTROL OF EPIDEMICS IN THE PRESENCE OF HETEROGENEITY Soheil Eshghi Saswati Sarkar We seek to identify and address how different types of heterogeneity affect the optimal control of epidemic processes in social, biological, and computer net- works. Epidemic processes encompass a variety of models of propagation that are based on contact between agents. Assumptions of homogeneity of communica- tion rates, resources, and epidemics themselves in prior literature gloss over the heterogeneities inherent to such networks and lead to the design of sub-optimal control policies. However, the added complexity that comes with a more nu- anced view of such networks complicates the generalizing of most prior work and necessitates the use of new analytical methods. We first create a taxonomy of heterogeneity in the spread of epidemics. We then model the evolution of hetero- geneous epidemics in the realms of biology and sociology, as well as those arising from practice in the fields of communication networks (e.g., DTN message rout- ing) and security (e.g., malware spread and patching). In each case, we obtain computational frameworks using Pontryagin’s Maximum Principle that will lead to the derivation of dynamic controls that optimize general, context-specific ob- jectives. We then prove structures for each of these vectors of optimal controls that can simplify the derivation, storage, and implementation of optimal policies. iv Finally, using simulations and real-world traces, we examine the benefits achieved by including heterogeneity in the control decision, as well as the sensitivity of the models and the controls to model parameters in each case. v Contents 1 Overview 1 1.1 Epidemics & Epidemic Modeling . .1 1.2 Motivating Heterogeneity in Epidemic Models . .4 1.2.1 Rate-Heterogeneity . .4 1.2.2 Resource-Heterogeneity . .6 1.2.3 Heterogeneity of Epidemics . .7 1.3 Motivation of this work . .8 1.4 Summary of Contributions . .9 1.5 Rate-Heterogeneity: Optimal Control of Clustered Malware Epi- demics . 11 1.6 Resource-Heterogeneity: Optimal Energy-Aware Epidemic Routing in DTNs . 16 1.7 Epidemic-Heterogeneity: Visibility-Aware Malware Epidemics . 21 1.8 Literature review and positioning of our contributions . 28 1.8.1 Rate Heterogeneity . 28 vi 1.8.2 Resource Heterogeneity . 30 1.8.3 Epidemic Heterogeneity . 31 2 Rate-Heterogeneity: Optimal Patching in Clustered Malware Epidemics1 34 2.1 Introduction . 34 2.2 System Model and Objective Formulation . 39 2.2.1 Dynamics of non-replicative patching . 40 2.2.2 Dynamics of replicative patching . 45 2.2.3 Motivation of the models and instantiations . 46 2.2.4 Key observations . 50 2.2.5 The optimality objective . 55 2.3 Optimal Non-Replicative Patching . 59 2.3.1 Numerical framework for computing optimal controls . 59 2.3.2 Structure of optimal non-replicative patching . 62 2.3.3 Proof of Lemma 4 . 66 2.4 Optimal Replicative Patching . 67 2.4.1 Numerical framework for computing the optimal controls . 67 2.4.2 Structure of optimal replicative dispatch . 71 2.4.3 Proof of Lemma 6 . 74 2.4.4 Proof of Lemma 8 . 75 1Presented in the Information Theory and Applications Workshop (ITA) 2012 and published in the IEEE Transactions on Networking, November 2015 [22]. vii 2.5 An Alternative Cost Functional . 77 2.6 Numerical Investigations . 79 2.7 Conclusion . 92 3 Resource-Heterogeneity: Optimal Energy-Aware Epidemic Routing in DTNs 2 94 3.1 Introduction . 94 3.2 System Model . 98 3.2.1 System Dynamics . 99 3.2.2 Throughput constraint and objective functions . 107 3.3 Optimal Forwarding Policies . 112 3.3.1 Structure of the optimal controls . 112 3.3.2 Proof of Theorem 5 . 117 3.3.3 Proof of Theorem 6 . 136 3.4 Numerical Investigations . 140 3.4.1 Validation of the mean-field deterministic model . 142 3.4.2 Performance advantage of optimal control over heuristics . 143 3.4.3 Sensitivity of the optimal control to synchronization and residual energy determination errors . 147 3.4.4 Multiple Message Transmission . 148 2Presented in IEEE MobiHoc 2012 [44] and published in the IEEE Transactions on Automatic Control, June 2015 [?]. viii 3.5 Conclusion . 150 4 Epidemic-Heterogeneity: Visibility-Aware Optimal Contagion of Mal- ware Epidemics3 161 4.1 Introduction . 161 4.1.1 Problem Description . 164 4.1.2 Results . 169 4.2 Literature Review . 171 4.3 System Model and Objective Formulation . 174 4.3.1 SGZP Model with no halting . 175 4.3.2 SGZP Model with halting . 177 4.3.3 SGZP Model with no halting and adaptive defense . 178 4.3.4 Key observations . 179 4.3.5 Utility Function . 181 4.3.6 Problem statement . 182 4.4 Structural Results . 182 4.4.1 Results for the no halting model (proved in §4.4.2) . 183 4.4.2 Proof of Theorem 10 for the no halting model . 184 4.4.3 Results for the halting model (proved in §4.4.4) . 192 4.4.4 Proof of Theorem 11 . 193 3Presented at Information Theory and Application Workshop (ITA) 2015 and submitted to IEEE Transactions on Automatic Control, July 2015 [23].