From Theoretical Promise to Observational Reality: Calibration, Foreground Subtraction, and Signal Extraction in Hydrogen Cosmology by Adrian Chi-Yan Liu A.B., Princeton University (2006) Submitted to the Department of Physics in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY June 2012 c Massachusetts Institute of Technology 2012. All rights reserved. Author.............................................................. Department of Physics May 25, 2012 Certified by. Max Erik Tegmark Professor of Physics Thesis Supervisor Accepted by . Krishna Rajagopal Professor of Physics Associate Department Head for Education 2 From Theoretical Promise to Observational Reality: Calibration, Foreground Subtraction, and Signal Extraction in Hydrogen Cosmology by Adrian Chi-Yan Liu Submitted to the Department of Physics on May 25, 2012, in partial fulfillment of the requirements for the degree of Doctor of Philosophy Abstract By using the hyperfine 21 cm transition to map out the distribution of neutral hy- drogen at high redshifts, hydrogen cosmology has the potential to place exquisite constraints on fundamental cosmological parameters, as well as to provide direct ob- servations of our Universe prior to the formation of the first luminous objects. How- ever, this theoretical promise has yet to become observational reality. Chief amongst the observational obstacles are a need for extremely well-calibrated instruments and methods for dealing with foreground contaminants such as Galactic synchrotron ra- diation. In this thesis we explore a number of these challenges by proposing and testing a variety of techniques for calibration, foreground subtraction, and signal extraction in hydrogen cosmology. For tomographic hydrogen cosmology experiments, we explore a calibration algorithm known as redundant baseline calibration, extending treatments found in the existing literature to include rigorous calculations of uncertainties and extensions to not-quite-redundant baselines. We use a principal component analysis to model foregrounds, and take advantage of the resulting sparseness of foreground spectra to propose various foreground subtraction algorithms. These include fitting low-order polynomials to spectra (either in image space or Fourier space) and in- verse variance weighting. The latter method is described in a unified mathematical framework that includes power spectrum estimation. Foreground subtraction is also explored in the context of global signal experiments, and data analysis methods that incorporate angular information are presented. Finally, we apply many of the afore- mentioned methods to data from the Murchison Widefield Array, placing an upper limit on the Epoch of Reionization power spectrum at redshift z = 9:1. Thesis Supervisor: Max Erik Tegmark Title: Professor of Physics 3 4 Acknowledgments What a thrilling ride this has been! To be in a position to experience all this has been a wonderful privilege, and the delightful company along the way has made everything even better. It goes without saying, then, that I have a lot of people to thank1. I'd like to start by thanking my advisor, Max Tegmark. Max was one of the main reasons I came to MIT, and I could not have made a better choice in deciding to work for him. Working for a world-class scientific mind has been eye-opening to say the least. But in addition, Max has been|to a crude approximation|an infinitely wise advisor on pretty much anything. Our conversations on topics as wide-ranging as economics, politics, careers, and the state of the human species have been fascinating, and I am forever grateful to him for the experience. I'd also like to thank my other committee members, Jacqueline Hewitt and Scott Hughes. While she has probably forgotten about this by now, Jackie was the first person to explain radio interferometry to me in a way that I was able to understand. Since then we've collaborated on a number of other projects, and I have also benefitted from her thoughtful advice. Scott has been a great person to go to for advice, and it has been a real privilege taking his classes, teaching with him, and doing service work with him. All that we have left to do is to write a paper together, and I look forward to the prospect! (Primordial gravitational wave background signatures in hydrogen cosmology, perhaps?) One of the wonderful things about the astrophysics division at MIT is the oppor- tunity to interact with other faculty members on an informal basis. I thank Deepto Chakrabarty for his wise advice on navigating graduate school and beyond, Saul Rap- paport for his \simple" questions, Rob Simcoe for our fun, informal interactions, and both Paul Schechter and Ed Bertschinger for pushing all of us to do the best work possible. Research in astrophysics and cosmology is naturally a collaborative venture, and I thank my collaborators. Judd Bowman, Matias Zaldarriaga, Jonathan Pritchard, and Avi Loeb had a particularly strong influence on me. I would also like to thank my \academic siblings"|the older ones (Molly Swanson, Yi Mao, Mark Hertzberg, and Courtney Peterson) for acting as role models that I could look up to, a \twin" (Andy Lutomirski) for using his razor-sharp intellect to make me a better physicist, and the younger ones (Josh Dillon and Jeff Zheng) for forcing me to \grow up" and 1This is not meant to be an exhaustive list. Naturally there are many people who should be here as well, but did not fit in the page and a half. You know who you are, and I hope you realize what a difference you made. 5 be more of a leader. It has also been an enormous honor to work with our bright UROPs, Michael Valdez, Nevada Sanchez, Jon Losh, Scott Morrison, Katelin Schutz, Hrant Gharibyan, Ioana Zelko, and Victor Buza. Thank you, also, to all my friends. Phil Zukin for experiencing the ups and downs of being an aspiring cosmologist since we met during the various grad school open houses. Chris Williams for being a fantastic apartment-mate as well as a brilliant research collaborator. Leo Stein for his greetings and phonetic hiatuses. Leslie Rogers for showing us all how a truly nice person behaves. Sarah Vigeland for geeking out with me over the West Wing. Mike Matejek for the Princeton pride. Nicolas Smith for his humor. Beyond the astrograd crowd, I would like to additionally thank Jan Hsi Lui for the filet o' fishes over the years, Weston Powell for his hahas, and Angela Wu for her wisdom. I would also like to thank Emily Berger for believing in me (and for providing strawberry ice cream at the appropriate critical juncture), Diann Benti for her incredible support and her making me more aware of the world beyond the ivory tower, and Kaitlin Reilly for completely understanding me. Finally, I would like to express my deep, deep gratitude for my family. My mother taught me my compassion, my father inspired my interest in science, and my sister has been a wonderful partner in crime. This paragraph is disproportionately short because there is only one thing that I have to thank them for|everything. 6 \Don't promise too much, Mr. Theorist, don't promise too much!" snapped the detec- tive. \You may find it a harder matter than you think." |Detective Athelney Jones, in \The Sign of the Four" by Sir Arthur Conan Doyle \Technology of that kinda can be worth an untold fortune. Imagine being able to control a radio device simply by sending a command via radio waves" |Sherlock Holmes, in the 2009 movie \Sherlock Holmes". 7 8 Contents 1 Introduction 27 1.1 The Theoretical Promise of Hydrogen Cosmology . 27 1.1.1 High redshifts (z ∼ 200 to z ∼ 40) . 30 1.1.2 Low redshifts (z ∼ 6 to z ∼ 2) . 32 1.1.3 Intermediate redshifts (z ∼ 40 to z ∼ 6) . 33 1.2 The Experimental Reality of Hydrogen Cosmology . 35 1.3 Current and Near-Future Hydrogen Cosmology Experiments . 39 1.3.1 Global Signal Experiments . 40 1.3.2 Tomographic Experiments . 41 1.4 Roadmap . 42 2 Precision Calibration of Radio Interferometers Using Redundant Baselines 45 2.1 Introduction . 45 2.2 Calibration Using Redundant Baselines . 47 2.2.1 Basic Point Source Calibration . 49 2.2.2 Redundant Baseline Calibration . 50 2.2.3 The Logarithmic Implementation . 51 2.2.4 The Noise Covariance Matrix . 54 2.2.5 Simulation Results and Error Properties . 56 2.2.6 Problems with the logarithmic implementation . 64 2.2.7 The Linearized Approach . 68 2.2.8 Numerical issues for the linearized approach . 71 2.3 A Generalized Calibration Formalism: Calibration as a Perturbative Expansion . 74 2.3.1 Zeroth Order: Perfect Point Sources or Perfect Redundancies . 77 2.3.2 First Order: Unknown Sources and Near Perfect Redundancies 77 2.3.3 First Order: Non-coplanar Arrays . 81 9 2.4 Conclusions . 83 3 How well can we measure and understand foregrounds with 21 cm experiments? 85 3.1 Introduction . 85 3.2 Foreground and Noise Model . 87 3.2.1 Definition of the mean and the foreground covariance . 87 3.2.2 Extragalactic point sources . 88 3.2.3 Galactic Synchrotron Radiation . 90 3.2.4 Free-free Emission . 91 3.2.5 Total Foreground Contribution . 91 3.2.6 Noise model . 92 3.3 The Ease of Characterizing Foregrounds . 93 3.3.1 Eigenforeground Modes . 93 3.3.2 Features of the Eigenforegrounds . 96 3.3.3 Understanding the Eigenforegrounds . 99 3.3.4 Eigenforeground Measurements . 101 3.4 The difficulty in measuring physical parameters . 111 3.5 Conclusions . 118 4 Will point sources spoil 21 cm tomography? 119 4.1 Introduction . 119 4.2 Methodology . 123 4.2.1 Step I: Simulation of Foregrounds . 123 4.2.2 Step II: Simulation of Radio Interferometer .