MENG INDIVIDUAL PROJECT IMPERIAL COLLEGE LONDON DEPARTMENT OF COMPUTING Communication-free and Parallel Simulation of Neutral Biodiversity Models Supervisor: Dr. Anthony Field Second Marker: Author: Prof. Paul Kelly Momo Langenstein [they/them] External Advisors: Dr. James Rosindell Prof. Ryan A. Chisholm arXiv:2108.05815v1 [cs.DC] 12 Aug 2021 14th June 2021 Abstract Biodiversity simulations are used in ecology and conservation to predict the effect of habitat de- struction on biodiversity. We present a novel communication-free algorithm for individual-based probabilistic neutral biodiversity simulations. The algorithm transforms a neutral Moran ecosys- tem model into an embarrassingly parallel problem by trading off inter-process communication at the cost of some redundant computation. Specifically, by careful design of the random number generator that drives the simulation, we ar- range for evolutionary parent-child interactions to be modelled without requiring knowledge of the interaction, its participants, or which processor is performing the computation. Critically, this means that every individual can be simulated entirely independently. The simulation is thus fully reproducible irrespective of the number of processors it is distributed over. With our novel algorithm, a simulation can be (1) split up into independent batch jobs and (2) simulated across any number of heterogeneous machines – all without affecting the simulation result. We use the Rust programming language to build the extensible and statically checked simulation package necsim-rust. We evaluate our parallelisation approach by comparing three traditional simulation algorithms against a CPU and GPU implementation of our Independent algorithm. These experiments show that as long as some local state is maintained to cull redundant indi- viduals, our Independent algorithm is as efficient as existing sequential solutions. The GPU imple- mentation further outperforms all algorithms on the CPU by a factor ranging from ∼ 2 to ∼ 80, depending on the model parameterisation and the analysis that is performed. Amongst the parallel algorithms we have investigated, our Independent algorithm provides the only non-approximate parallelisation strategy that can scale to large simulation domains. For example, while Thompson’s 2019 necsim simulation takes more than 48 hours to simulate 108 individuals, we have been able to simulate 7:1 · 1010 individuals on an HPC batch system within a few hours. Acknowledgements This project started on July 27th 2020, when I reached out to James, asking if we could collaborate on my individual project. I have known James since my amazing 2019 UROP with him at Silwood Campus. Last year, James welcomed me back with open arms and five suggestions for an MEng project, out of the first two of which this project was born. Two days later, James introduced me to Ryan, and we all had our first meeting together on August 3rd. The same day, I also first contacted Tony, who later agreed to supervise this project. On August 6th, I talked with my friend and Maths student Philipp about a very early attempt at solving what would later become a core part of this project: jumping around inside homogeneous Poisson point processes. Therefore, my first round of thanks goes to my three supervisors. From the beginning, their ideas and experience have inspired and streamlined this project whilst also allowing me to slowly direct it exactly where I wanted it to go. I could not be more grateful for Tony, James and Ryan, who have met with me (almost) every week throughout this project and have brought their diverse science backgrounds to push this project from all sides. Thank you for always encouraging me to think further, plan ahead, not go for the most outlandish approach first, and keep track of my mental health throughout this process. Besides my supervisors, I also want to express my gratitude to my many mentors at ICL. Thank you to Francesco, Philippa, Holger and Paul, who have all given me invaluable advice and feed- back on this project and life in general. Paul’s Advanced Computer Architecture course still remains my favourite class to this day. My special thanks go to Chris and Elizabeth, who have helped and formed me in so many more ways than they could imagine. Finally, this project would have crashed and burned without the help of CSG, in particular Lloyd, who have always supported me. Next, I would like to thank all Biodiversity Lab group members who have supported me both personally and academically throughout this challenging year. In particular, I would like to thank Francis for his critical thinking, Pokman and Olivia for their feedback, and Rach for just being an inspiration. Most importantly, though, I am enormously grateful for Sam, who wrote the original necsim simulation that this project is based on, and Lucas, who has always been a saviour and source of encouragement (and Maths help). It was such an honour to work with you during my UROP and now this master’s project. Last but not least, I want to thank my family and friends. After living through the first lockdown in London, I am so grateful that I could spend this past year back home. At the start of the second year, ICL gifted us rubber ducks to listen to our computing problems. Without any doubt, though, my parents have been the best ducks I could have asked for. After one year of swamping them with my thoughts and fears on countless walks, I can only apologise for talking so much about this project. I want to further thank my second family back in the UK, Freddie, George, Jack, Jessica and Steve, who opened their arms and hearts to me many years ago and who have since been my rock abroad. I would also like to express my gratitude to Alexander, Declan, Isabella, Iurii, Philipp and Tiger, who have been great pillars of emotional support. Finally, my eternal gratit- ude goes to my friend Jamie, without whom I would not have gotten through this year’s dark times. This project is a labour of much love, sweat and tears. I would not have been able to do it without my amazing support system around me, and I thank all of them from the bottom of my heart. Table of Contents 1 Introduction 7 2 Review of Scientific Background9 2.1 Biodiversity Loss and Conservation . .9 2.2 The Neutral Theory of Biodiversity . .9 2.3 A Neutral Coalescence-based Simulation . 10 2.4 Three Neutral scenarios . 12 2.4.1 Non-Spatial . 12 2.4.2 Spatially Implicit . 12 2.4.3 Spatially Explicit . 13 2.5 Homogeneous Poisson Point Processes . 13 2.5.1 Properties of Homogeneous Poisson Point Processes . 13 2.5.2 Properties of the Exponential and Poisson distribution . 13 2.6 Random Number Generation . 14 2.6.1 Hash functions and the Avalanche Effect . 14 2.6.2 Pseudo-Random Number Generation . 15 2.7 The Gillespie Algorithm . 15 2.7.1 The “Direct” Method . 16 2.7.2 The “First-reaction” and “Next-reaction” Methods . 16 2.7.3 Tau Leaping . 16 3 Review of Technical Background 17 3.1 The Rust Programming Language . 17 3.1.1 The Rust Type System . 17 3.1.2 Verification . 18 3.2 Different Types of Parallelisation . 18 3.2.1 Task vs Data Parallelism . 18 3.2.2 Data Sharing and Communication . 19 3.2.3 Shared Memory Parallelism: CUDA . 19 3.2.4 Message Passing Parallelism: MPI . 20 4 Review of Related Work 22 4.1 The necsim library . 22 4.2 The msprime library . 22 4 Table of Contents Table of Contents 4.3 Parallel Random Number Generation . 23 4.3.1 Splitting Random Number Generators . 23 4.3.2 Counter-based Pseudo-Random Number Generators . 23 4.3.3 Reproducible Random Number Generation . 24 4.4 Random Number Scrambling . 24 4.5 Parallelising the Gillespie Algorithm . 24 4.5.1 Parallelisation on a GPU . 25 4.5.2 Parallelisation in a High-Performance Computing environment . 25 4.6 Parallelising Spatial Simulations . 25 5 The Declaration of Independence 26 5.1 Coalescence à la Gillespie . 26 5.2 The Independent Algorithm . 28 5.2.1 Environmental RNG Priming . 28 5.2.2 Aligning the RNGs of Colliding Individuals . 29 5.2.3 Exponential inter-event Times . 30 5.2.4 Summary of the Independent Algorithm . 31 6 The Simulation Architecture Design 32 6.1 The Statically Checked Component System . 32 6.2 The Reporter Analysis System . 35 6.3 The Parallelisation Architecture . 37 7 Implementation and Parallelisation 38 7.1 Implementing the Independent Algorithm . 38 7.1.1 Deduplicating Redundant Individuals . 38 7.1.2 Sorting Events using the Water-Level Algorithm . 39 7.1.3 Reporting Coalescence Events Independently . 40 7.2 The MPI partitioning backend . 40 7.3 The Simulation Parallelisation Strategies . 41 7.3.1 The Monolithic Algorithms . 41 7.3.2 The Independent Algorithm . 43 7.4 The Independent Algorithm on the GPU . 44 7.5 Command-Line Interface . 45 8 Evaluation 46 8.1 Implementation Correctness . 46 8.2 Statistical Correctness . 46 8.2.1 Randomness in the Independent Algorithm . 46 8.2.2 Event Statistics . 48 8.2.3 Neutral Scenarios . 49 8.2.4 Convergence . 50 8.3 Event Generation Performance . 50 5 5 Table of Contents Table of Contents 8.3.1 Independent Exponential Inter-Event Time Sampling . 51 8.3.2 Event Reporting Performance Baseline . 51 8.3.3 Event Throughput Baseline . 52 8.4 Configuration Sweetspot Analysis . 53 8.4.1 The Parallelised Monolithic Algorithms . 54 8.4.2 The Independent Algorithm on the CPU . 54 8.4.3 The Probabilistically Communicating Independent Algorithm . 55 8.4.4 The Independent Algorithm on a CUDA GPU .