RICE UNIVERSITY Forecasting Geomagnetic Activity Indices using the Boyle Index through Artificial Neural Networks by Ramkumar Balasubramanian A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE Doctor of Philosophy APPROVED, THESIS COMMITTEE: Patricia H. Reiff, Trofessor Physics & Astronomy Thesis Advisor David Ale^&fider, Professor Physics & Astronomy Erzsebet Merenyi, Research Professor Electrical & Computer Engineering Houston, Texas April, 2010 UMI Number: 3421158 All rights reserved INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if material had to be removed, a note will indicate the deletion. Dissertation Publishing UMI 3421158 Copyright 2010 by ProQuest LLC. All rights reserved. This edition of the work is protected against unauthorized copying under Title 17, United States Code. ProQuest LLC 789 East Eisenhower Parkway P.O. Box 1346 Ann Arbor, Ml 48106-1346 ABSTRACT Forecasting Geomagnetic Activity Indices using the Boyle Index through Artificial Neural Networks by Ramkumar Balasubramanian Adverse space weather conditions affect various sectors making both human lives and technologies highly susceptible. This dissertation introduces a new set of algo- rithms suitable for short term space weather forecasts with an enhanced lead-time and better accuracy in predicting Kp, Dst and the AE index over some leading mod- els. Kp is a 3-hour averaged global geomagnetic activity index good for midlatitude regions. The Dst index, an hourly index calculated using four ground based magnetic field measurements near the equator, measures the energy of the Earth's ring current. The Auroral Electro jet indices or AE indices are hourly indices used to characterize the global geomagnetic activity in the auroral zone. Our algorithms can predict these indices purely from the solar wind data with lead times up to 6 hours. We have trained and tested an ANN (Artificial Neural Network) over a complete solar cycle to serve this purpose. Over the last couple of decades, ANNs have been successful for temporal prediction problems amongst other advanced non-linear tech- niques. Our ANN-based algorithms receive near-real-time inputs either from ACE (Advanced Composition Explorer), located at LI, and a handful of ground-based magnetometers or only from ACE. The Boyle potential, $ = —)2 + U-7^ sin3 (0/2) kV, or the Boyle Index (BI) is an empirically-derived formula that approximates the Earth's polar cap po- tential and is easily derivable in real time using the solar wind data from ACE. The iii logarithms of both 3-hour and 1-hour averages of the Boyle Index correlate well with the subsequent Kp, Dst and AE: Kp = 8.93 log1Q<BI> - 12.55, Dst = 0.355<BI> - 6.48, and AE = 5.87<BI> - 83.46. Inputs to our ANN models have greatly benefitted from the BI and its proven record as a forecasting parameter since its initiation in October, 2003. A preconditioning event tunes the magnetosphere to a specific state before an impending geomagnetic storm. The neural net not only improves the predictions but also helps the prediction by capturing the influence of preconditioning. Two of our models have been running in near-real-time forecast mode already, and the BI and Kp predictions can be obtained from http://space.rice.edu/ISTP/wind.html. Acknowledgments Dr. Reiff, it was in my eighth grade when I first told my math teacher, with a lot of shyness, that I wanted a doctorate in physics. Here I am. Thank you for all the education, financial support and motivation. Part of my success comes from the freedom you offered and uninhibited approach I was allowed to take during the course of my PhD. Dr. David Alexander, I thank you for your time, tips and advice. It is truly humbling to have known you. Dr. Erzsebet Merenyi, I thank you for your formal education and insights on artificial neural networks. All the neuro computing I know today is all what you taught me. Dr. Will Burgett, I thank you for your support and constant encouragement. Rice University would have just been a dream for me without you. Yes, you are right, it is a tremendous institution! Dr. Chris Johns-Krull, I thank for your financial support and introduction to Astrophysics. I wish to thank the CISM and the William and Elva Gordon Fund for partially supporting my research. I thank my parents, sisters, friends and relatives for their constant support and motivation. Finally, to Srini and Srimathy, I dedicate this thesis to you both. Contents Acknowledgments iii List of Illustrations ix List of Tables xxii Symbols and Notations xxiv 1 Introduction 1 1.1 Geomagnetic Activity Indices 3 1.1.1 The Kp Index 5 1.1.2 The AE index 8 1.1.3 The Dst index 12 1.2 Motivation 14 1.2.1 Solar Wind Coupling Functions 17 1.3 Linear Predictor 20 1.4 Persistence forecasting 23 1.5 ANN and forecasting 24 1.6 Related Work 25 1.6.1 Takahashi Kp Nowcast Model 26 1.6.2 Costello NN Kp Model 27 1.6.3 Boberg NN Kp Model 27 1.6.4 APL Kp Models 28 1.6.5 Wu and Lundstedt NN Dst model 29 V 1.6.6 Temerin and Li Dst Model 31 1.6.7 Gleisner and Lundstedt NN AE model 31 1.7 Scientific Objective 31 1.8 Thesis Organization 33 2 Artificial Neural Networks 35 2.1 Artificial Neural Networks 35 2.1.1 Biological Neurons 35 2.1.2 Artificial Neurons 36 2.2 A Non-linear Neuron Model 38 2.3 The Backpropagation Algorithm 41 2.3.1 Learning through Gradient-descent Technique 44 2.3.2 Backpropagation algorithm for a two-layer network 49 2.3.3 Notes on Backprop 50 2.4 Conjugate Gradient Algorithm 51 2.4.1 Newton's Method 52 2.4.2 Conjugate Directions 53 2.4.3 The CG Algorithm 54 2.4.4 Levenberg-Marquardt Method 55 3 Research Methodology 58 3.1 Data and Instruments 58 3.1.1 WIND 59 3.1.2 IMP-8 59 3.1.3 ACE 59 3.1.4 Kp 60 VI 3.1.5 Dst and AE 60 3.1.6 Real-time data 61 3.2 Problem Definition 61 3.3 Hypotheses to Test 63 3.4 Proposed Models 63 3.4.1 Training Parameters 64 3.4.2 Model 1: Kp prediction with 1-hour lead time using only the BI 66 3.4.3 Model 2: Kp prediction with 3-hour lead time using only the BI 67 3.4.4 Model 3: Kp Prediction with 1-hour Lead-Time using the BI and Kp history 72 3.4.5 Model 4: Kp Prediction with 2-hour Lead-Time using the BI and Kp history 78 3.4.6 Model 5: 1-hour lead time Dst predictions from BI 82 3.4.7 Model 6: 3-hour lead time Dst predictions from BI 84 3.4.8 Model 7: 1-hour lead time AE predictions from BI 85 3.4.9 Model 8: 3-hour lead time AE predictions from BI 88 3.5 Model Validation 88 3.5.1 Skill Scores 93 3.5.2 Tests of Significance 95 3.5.3 Autocorrelation Function 97 4 Solar Wind-Magnetosphere Coupling 99 4.1 Solar wind-Magnetospheric Interactions 99 4.2 Magnetospheric Convection 102 4.2.1 Open and Closed Models 102 4.3 Role of the Ionosphere 107 vii 4.4 Polar Cap Potential 109 4.5 The Boyle Index: A Solar wind-Magnetosphere Coupling Function . 113 4.6 Polar Cap Saturation 115 4.7 Newell functions: another coupling formula 117 4.8 Boyle Index: Effect of Preconditioning Events 118 4.9 The BI and space weather 119 5 The Algorithms 122 5.1 Linear Correlations 123 5.2 Cross-correlation Analysis 127 5.3 Prediction Algorithms 130 5.3.1 Model 1: 1-hour lead time Kp predictions from BI 131 5.3.2 Model 2: 3-hour lead time Kp predictions using only the BI . 136 5.3.3 Model 3: 1-hour lead time Kp predictions using the BI and Kp history 140 5.3.4 Model 4: 3-hour lead time Kp predictions using the BI and Kp history 150 5.3.5 Kp models: Understanding accuracy 156 5.3.6 "Persistence" vs. "true" forecasting 162 5.3.7 Post-test Correlation Analysis 163 5.3.8 Kp models compared 166 5.3.9 "Spacalrt" real-time warning system 168 5.3.10 Model 5: 1-hour lead time Dst predictions from BI 170 5.3.11 Model 6: 3-hour lead time Dst predictions from BI 173 5.3.12 Model 7: 1-hour lead time AE predictions from BI 176 5.3.13 Model 8: 3-hour lead time AE predictions from BI 178 viii 5.3.14 Green's function test 183 5.4 Pressure term inclusion 187 5.4.1 Network Training 187 5.4.2 The "Ram" functions 189 5.5 Longer range predictions 195 6 Discussion and Conclusion 199 6.1 Summary 201 6.2 Kp models: Possible applications 203 6.3 Future Possibilities 204 6.3.1 Borovsky Function 205 6.3.2 Improving long term forecasts 206 References 209 Illustrations 1.1 Kp Network Stations 5 1.2 An illustration of the typical diurnal variation of the horizontal component of the Earth's magnetic field as seen at a Kp station [McPherron, 1997] 6 1.3 AE Network Stations (Figure courtesy: World Data Center C2 for Geomagnetism, Kyoto, Japan) 9 1.4 A severe geomagnetic activity is shown here as an example indicating the values of various solar wind parameters and the geomagnetic indices.