Air Force Institute of Technology AFIT Scholar Theses and Dissertations Student Graduate Works 3-21-2019 Lightning Prediction Using Recurrent Neural Networks Dominick V. Speranza Follow this and additional works at: https://scholar.afit.edu/etd Part of the Meteorology Commons Recommended Citation Speranza, Dominick V., "Lightning Prediction Using Recurrent Neural Networks" (2019). Theses and Dissertations. 2317. https://scholar.afit.edu/etd/2317 This Thesis is brought to you for free and open access by the Student Graduate Works at AFIT Scholar. It has been accepted for inclusion in Theses and Dissertations by an authorized administrator of AFIT Scholar. For more information, please contact [email protected]. LIGHTNING PREDICTION USING RECURRENT NEURAL NETWORKS THESIS Dominick V. Speranza, 2nd Lt, USAF AFIT-ENS-MS-19-M-150 DEPARTMENT OF THE AIR FORCE AIR UNIVERSITY AIR FORCE INSTITUTE OF TECHNOLOGY Wright-Patterson Air Force Base, Ohio DISTRIBUTION STATEMENT A APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED. The views expressed in this document are those of the author and do not reflect the official policy or position of the United States Air Force, the United States Department of Defense or the United States Government. This material is declared a work of the U.S. Government and is not subject to copyright protection in the United States. AFIT-ENS-MS-19-M-150 LIGHTNING PREDICTION USING RECURRENT NEURAL NETWORKS THESIS Presented to the Faculty Department of Operational Sciences Graduate School of Engineering and Management Air Force Institute of Technology Air University Air Education and Training Command in Partial Fulfillment of the Requirements for the Degree of Master of Science in Operations Research Dominick V. Speranza, 2nd Lt, USAF March 21, 2019 DISTRIBUTION STATEMENT A APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED. AFIT-ENS-MS-19-M-150 LIGHTNING PREDICTION USING RECURRENT NEURAL NETWORKS THESIS Dominick V. Speranza, 2nd Lt, USAF Committee Membership: Lt Col A. Geyer, Ph.D. Advisor R. Hill, Ph.D. Reader AFIT-ENS-MS-19-M-150 Abstract Cape Canaveral Air Force Station (CCAFS), Kennedy Space Center (KSC), and Patrick Air Force Base (PAFB) all reside in the thunderstorm capital of the United States. According to the Florida Climate Center, these installations experience more thunderstorms per year than any other place in the United States. It is the mission of the 45th Weather Squadron to provide timely and accurate warnings of weather conditions such as lightning that pose a risk to assets and personnel CCAFS, KSC and PAFB. To aid 45th Weather Squadron forecasters, a network of 30 Electric Field Mills (EFM) was installed in the area in and around CCAFS, KSC, and PAFB. EFMs record the electrification of the local atmosphere. Several efforts have been made over the years to find an optimal way to utilize the EFM network data to improve lightning prediction. These efforts approached the problem using atmospheric science as well as traditional statistical regression techniques with mixed results. In this paper, hourly statistics were generated from the raw EFMs data set used in Hill [1]. Input variables were generated from surface observations from every station within 50 miles of CCAFS and then combined with the EFM statistics for the same time periods. This combined data set was used to create Long Short-term Memory (LSTM) Neural Networks designed to capture trends within the data for each obser- vation. A variety of different LSTM model structures were created and trained to see which model structure performed best when predicting lightning around CCAFS, KSC, and PAFB. By utilizing design of experiments techniques, optimal parameters for the LSTM model structures are narrowed down providing a solid baseline for future endeavors in predicting lightning. iv Acknowledgements I would like to express my deep appreciation for the guidance and mentorship pro- vided by my advisor Lt. Col. Andrew Geyer. Without his help, this project would not have been possible. I would also like to thank the 45th Weather Squadron for the continued support and funding throughout the project and follow on research. With- out the hardware purchased for the experiment, this project would have never been completed. Additionally, I would like to thank Dr. Raymond Hill for his guidance and support as a mentor and reader of this thesis. Finally, I would like to thank my parents, for the undying love and support throughout my life, and my siblings for setting high expectations which I strive to live up to! Dominick V. Speranza v List of Tables Table Page 1. Output for Risk Areas in Colombia. [2] . .6 2. Possible Parameter Values . 20 3. Fractional Factorial Parameter Test . 21 4. Results for All 5-Layer LSTM Runs . 28 5. Results for the RareEvents Model Structure . 47 6. Results for the 3 Layer LSTM Model Structure . 48 vi List of Figures Figure Page 1. Basics of an Artificial Neural Network [3] . .8 2. Output from the ANN and comparisons [4] . 10 3. Comparing the Neural Network models with Other Models [5] . 11 4. 5 Layer LSTM Model Structure. 14 5. 3 Layer LSTM Model Structure. 15 6. 5-Layer LSTM Result for Delay = 18, Lookback = 36 . 24 7. Rare Event LSTM Result for Delay = 18, Lookback = 36 . 25 8. 3-Layer LSTM Result for Delay = 18, Lookback = 36 . 26 9. Parameter Estimates for Validation Accuracy for the FinalEpoch ................................................... 30 10. Residual Plot for Validation Accuracy for the Final Epoch......................................................... 30 11. Residual Plot for Validation Loss for the Final Epoch . 31 12. Residual Plot for Validation Accuracy for the Final Epoch in the Quadratic Model . 32 13. Residual Plot for Validation Accuracy for the Final Epoch in the Quadratic Model . 32 14. Residual Plot for Validation Loss for the Final Epoch in the Quadratic Model . 33 15. Residual Plot for Validation Loss for the Final Epoch in the Quadratic Model . 33 16. Ordinal Parameter Estimates for Accuracy . 34 17. Lightning Count by the Hour . 36 18. Single Layer LSTM . 42 vii Figure Page 19. 5 Layer No Dropout LSTM . 42 20. Large Single Layer LSTM . 42 21. Small Convolutional LSTM . 43 22. Larger Convolutional LSTM . 43 23. Leverage Plot for Lookback in Validation Accuracy . 44 24. Leverage Plot for Lookback in Validation Accuracy . 44 25. Leverage Plot for Lookback Squared in Validation Accuracy . 45 26. Leverage Plot for Delay in Validation Loss . 45 27. Leverage Plot for Lookback in Validation Loss . 46 28. Leverage Plot for Lookback Squared in Validation Loss . 46 viii List of Abbreviations Abbreviation Page Cape Canaveral Air Force Station (CCAFS) . .1 Patrick Air Force Base (PAFB) . .1 Kennedy Space Center (KSC). .2 Artificial Neural Networks (ANNs) . .3 ground flash density (GFD) . .5 Electric Field Mills (EFM) . .6 Meteorological Aerodrome Reports (METARs) . .7 Recurrent Neural Networks (RNN) . .8 Predictive Mean Matching (PMM) . .9 Long Short-term Memory Neural Network (LSTM) . 12 Design of Experiments (DOE) . 13 Visual Basic for Applications (VBA) . 17 lightning detection and range (LDAR) . 18 ix Table of Contents Page Abstract . iv Acknowledgements . .v List of Tables . vi List of Figures . vii List of Abbreviations . ix I. Introduction . .1 1.1 Background . .2 1.2 Problem Statement . .3 1.3 Research Question . .3 1.4 Organization of the Thesis . .3 II. Literature Review . .5 2.1 Introduction . .5 2.2 Multivariate Techniques for Lightning Prediction . .5 2.3 Current Model/Data Used . .6 2.4 Neural Networks . .7 2.5 Conclusion . 16 III. Methodology . 17 3.1 Introduction . 17 3.2 Software and Data Pre-processing . 17 3.3 Building the Neural Networks . ..