Orbital Resonances and GPU Acceleration of Binary Black Hole Inspiral Simulations

Orbital resonances and GPU acceleration of binary black hole inspiral simulations by Adam Gabriel Marcel Lewis A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Department of Physics University of Toronto c Copyright 2018 by Adam Gabriel Marcel Lewis Abstract Orbital resonances and GPU acceleration of binary black hole inspiral simulations Adam Gabriel Marcel Lewis Doctor of Philosophy Department of Physics University of Toronto 2018 Numerical relativity, the direct numerical integration of the Einstein field equations, is now a mature subfield of computational physics, playing a critical role in the generation of signal templates for comparison with data from ground-based gravitational wave de- tectors. The application of numerical relativity techniques to new problems is at present complicated by long wallclock times and intricate code. In this thesis we lay groundwork to improve this situation by presenting a GPU port of the numerical relativity code SpEC. Our port keeps code maintenance feasible by relying on various layers of automation, and achieves high performance across a variety of GPUs. We secondly introduce a C++ soft- ware package, TLoops, which allows numerical manipulation of tensors using single-line C++ source-code expressions resembling familiar tensor calculus notation. These expressions may be compiled and executed immediately, but also can be used to automatically generate equivalent GPU or low-level CPU code, which then executes in their place. The GPU code in particular achieves near-peak performance. Finally, we present simulations of eccentric binary black holes. We develop new methods to extract the fundamental frequencies of these systems. Using these frequencies we identify when these binaries pass through coordinate resonances, at which points high mass-ratio inspirals can experience short-timescale phase-dependent deviations from smooth inspiral called \kicks". We find no evidence for such kicks at comparable mass-ratio. ii For my mother, my late father, and my sister. iii Acknowledgements It is impossible to adequately thank the numerous colleages, confidantes, and collabora- tors who made this work possible. First and foremost, I am indebted to my supervisor Harald Pfeiffer for his unwavering support. He has provided me with personal and profes- sional skills I will use throughout my career. I am also thankful for the warm mentorship of Charles Dyer, along with the support of my collaborator Aaron Zimmerman. Finally, I would like to thank my mother Gail Coulas, my father David Lewis, and my sister Anne Lewis, for supporting their relative despite his odd career choices. iv Table of Contents Dedication iii Acknowledgements iv Table of Contentsv 1 Overview1 2 Numerical Relativity Orientation6 2.1 Introduction..................................6 2.2 3+1 Formulation...............................7 2.2.1 From an Initial Value Formulation to a Numerical Relativity Code 10 2.3 Applications of NR.............................. 15 3 GPU-Accelerated Simulations of Isolated Black Holes 18 3.1 Introduction.................................. 18 3.2 Technological Overview............................ 23 3.2.1 The Spectral Einstein Code..................... 23 3.2.2 Programming for NVIDIA GPUs.................. 24 3.2.3 GPU Benchmarking......................... 27 3.3 Details of our port.............................. 32 3.3.1 Overview............................... 32 3.3.2 Memory Management......................... 33 3.3.3 Jacobian multiplication........................ 35 3.3.4 Spectral Operations: Differentiation and Filtering......... 41 3.3.5 DataMesh Operations, Apply BCs, GHEqns............ 54 3.4 Benchmarks of Overall Code......................... 57 3.5 Conclusions.................................. 58 3.6 Acknowledgments............................... 61 v 4 Tensor Loops 62 4.1 Introduction.................................. 62 4.2 Direct evaluation of tensor loops using expression templates....... 65 4.2.1 Spectral Einstein Code........................ 65 4.2.2 Tensors and SpEC 's Tensor class.................. 66 4.2.3 Capturing a TLoops-expression as a type.............. 68 4.2.4 Evaluation of TLoops-template................... 76 4.2.5 Sum-operations............................ 79 4.3 Automatic code generation.......................... 81 4.4 Design Considerations of Automatically Generated CUDA Code..... 86 4.5 Benchmarks.................................. 91 4.5.1 Methodology............................. 91 4.5.2 Benchmarks of simple expressions.................. 94 4.5.3 Benchmarks of more complex expressions.............. 97 4.5.4 Impact of CPU compiler....................... 100 4.5.5 Impact of templating over Tensor-indices.............. 100 4.6 Conclusion................................... 103 5 Frequencies and Resonances of Generic BBH Inspirals 104 5.1 Abstract.................................... 104 5.2 Introduction.................................. 104 5.3 Motion in Kerr spacetimes.......................... 108 5.3.1 Geodesic orbits in Kerr........................ 108 5.3.2 Fundamental frequencies of Kerr.................. 110 5.3.3 Orbital resonances.......................... 111 5.4 Numerical relativity simulations....................... 113 5.4.1 Simulations of eccentric, precessing black hole binaries...... 113 5.4.2 Dynamics of simulated binaries................... 115 5.4.3 Plots/Eccentricity decay....................... 121 5.5 Equatorial binaries.............................. 121 5.5.1 Frequencies for equatorial orbits................... 122 5.5.2 Results................................. 123 5.6 Inclined binaries................................ 125 5.6.1 Definition of frequencies....................... 125 5.6.2 Precession rates............................ 128 5.6.3 Resonances.............................. 131 vi 5.7 Conclusions.................................. 136 5.8 Appendices.................................. 138 5.8.1 The Kerr metric............................ 138 5.8.2 Envelope subtraction method in the slow time approximation.. 139 5.8.3 Evolution of eccentricity and semi-major axis........... 141 5.8.4 Validation of frequency extraction methods............. 142 5.8.5 Estimation of resonant accretion................... 146 6 Conclusions 148 Bibliography 151 Bibliography 151 vii Chapter 1 Overview The Einstein equations for the geometry of the relativistic gravitational field are intricate. To derive exact solutions, one must impose stringent symmetry assumptions. Despite these assumptions, several of the solutions obtained in this way are physically useful. Notable among these are the asymptotically flat, vacuum black holes: the Schwarzschild [319] and the Kerr [174] families of spacetimes. There appear to exist [82,147] real objects, \astrophysical black holes", which form naturally from the gravitational collapse of large stars [181,278,279], possessing many properties of these solutions [277]. At times, stars large enough to become astrophysical black holes form close enough together to be gravitationally bound, but far enough apart to avoid direct exchange of matter [112, 244]. The two bound stars in this case evolve independently, potentially collapsing into a pair of astrophysical black holes called a black hole binary (BBH) [83, 244]. This motivates the so-called \two-body problem in general relativity" [101], of finding exact solutions with two black holes. Except in extremely contrived situations (e.g. [134]), such systems admit no spacetime symmetries at all [83]. While exact BBH solutions are therefore not available, exact general relativity (GR) nevertheless gives some insight. Having a time-varying quadrupole moment, the binary will radiate gravitationally [216,335], causing its constituents to draw closer and eventually to merge [68, 83]. To leading order the radiated power will be quintic in the inverse binary separation [335]. This is a much sharper scaling than that from other astrophysical effects such as dynamical friction or gas drag, and so the BBH should eventually decouple from its environment and be well-described by vacuum GR [274]. Further detail requires quantitative access to the evolution. There are essentially three approaches. In the Post-Newtonian expansion [56], relativistic corrections are added successively to the Newtonian theory. The resulting equations are most accurate when motion is slow. They are thus suitable for the description of well-separated BBHs during 1 Chapter 1. Overview 2 early inspiral. Meanwhile, in the self-force formalism [108] the trajectory of one hole is computed perturbatively from the timelike geodesics of a Kerr spacetime appropriate to its counterpart. This is most accurate when the actual trajectories in fact resemble Kerr geodesics. Since the Kerr spacetime is a fully relativistic solution, speed and field strength are not directly relevant, but the size of the perturbation, and therefore the mass-ratio of the binary, is. Black hole binaries with very small mass-ratios are thought to occur naturally during, for example, mergers between supermassive and stellar black holes in galactic cores [52,274], so this technique does have astrophysical motivation. Finally, in numerical relativity (NR), the chief subject of this thesis, the Einstein field equations are solved directly by numerical integration. In principle any BBH solution could be obtained in this way, but in practice computational expense restricts attention to comparable-mass binaries with low-to-moderate spins

Load more