Deep Learning Models for Visibility Forecasting by Luz Carolina Ortega Gomez Master of Science in Systems Engineering Department of Computer Engineering and Sciences Florida Institute of Technology Melbourne, Florida 2016 Master of Science in Engineering Management Department of Computer Engineering and Sciences Florida Institute of Technology Melbourne, Florida 2013 Bachelor of Science Electrical Engineering Universidad Metropolitana Caracas, Venezuela 2012 A dissertation submitted to the College of Engineering and Science at Florida Institute of Technology in partial fulfillment of the requirements for the degree of Doctorate of Philosophy in Systems Engineering Melbourne, Florida December, 2019 © Copyright 2019 Luz Carolina Ortega Gomez All Rights Reserved ___________________________________________________________________ The author grants permission to make single copies We the undersigned committee hereby approve the attached dissertation Deep Learning Models for Visibility Forecasting by Luz Carolina Ortega Gomez _________________________________________________ Luis Daniel Otero, Ph.D. Associate Professor Department of Computer Engineering and Sciences Committee Chair _________________________________________________ Munevver Subasi, Ph.D. Associate Professor Department of Mathematical Sciences Outside Committee Member _________________________________________________ Aldo Fabregas, Ph.D. Assistant Professor Department of Computer Engineering and Sciences Committee Member _________________________________________________ Ersoy Subasi, Ph.D. Assistant Professor Department of Computer Engineering and Sciences Committee Member _________________________________________________ Phillip J. Bernhard, Ph.D. Associate Professor and Department Head Department of Computer Engineering and Sciences Abstract Title: Deep Learning Models for Visibility Forecasting Author: Luz Carolina Ortega Gomez Advisor: Luis Daniel Otero, Ph. D. This dissertation addresses the task of visibility forecasting via deep learning models using data from weather stations. Visibility is one of the most critical weather impacts on transportation systems. Low visibility conditions can seriously impact safety and traffic operations, leading to adverse scenarios, causing accidents, and jeopardizing transportation systems. Accurate visibility forecasting plays a key role in decision-making and management of transportation systems. However, due to the complexity and variability of weather variables, visibility forecasting remains a highly challenging task and a matter of significant interest for transportation agencies nationwide. This dissertation explores the use of deep learning models for the task of single-step visibility forecasting (i.e., estimation of visibility distance for the next hour) using time series data from ground weather stations. The aforementioned task has not been fully addressed in the literature, thus, this work represents a baseline for further research. The author explores five neural network architectures: multilayer perceptron (MLP), traditional convolutional neural network (TCNN), fully convolutional neural network (FCNN), multi-input convolutional neural network (MICNN), and long short-term memory (LSTM) network. Models were evaluated using two datasets from Florida (one of the top states across the US dealing with visibility problems). Three cases of lag observations were considered: three, six, and nine hours input data. iii Table of Contents Abstract .................................................................................................................... iii Table of Contents ..................................................................................................... iv List of Figures ......................................................................................................... vii List of Tables............................................................................................................ xi List of Abbreviations............................................................................................... xii Acknowledgments .................................................................................................. xiv Dedication .............................................................................................................. xvi Chapter 1 Introduction ............................................................................................... 1 1.1. Motivation .................................................................................................. 3 1.2. Research Objectives ................................................................................... 4 1.3. Solution Approach and Contributions ........................................................ 4 1.3.1. Solution Approach ............................................................................... 4 1.3.2. Contributions ........................................................................................ 6 1.4. Organization of Dissertation ...................................................................... 7 Chapter 2 Literature Review ...................................................................................... 8 Chapter 3 Background.............................................................................................. 17 3.1. Machine Learning .................................................................................... 17 iv 3.1.1. Supervised Learning........................................................................... 19 3.2. Deep Learning .......................................................................................... 19 3.2.1. ANN Architecture .............................................................................. 21 3.2.2. Activation Function ............................................................................ 26 3.2.3. Training .............................................................................................. 28 3.2.4. Multilayer Perceptron......................................................................... 29 3.2.5. Convolutional Neural Network .......................................................... 29 3.2.6. Long Short-Term Memory Network .................................................. 33 3.3. Time Series............................................................................................... 34 3.4. Time Series and Machine Learning ......................................................... 37 Chapter 4 Methodology............................................................................................ 39 Chapter 5 Data.......................................................................................................... 44 5.1. Data Understanding .................................................................................. 44 5.1.1. Data Collection and Description ........................................................ 44 5.1.2. Data Exploration ................................................................................ 50 5.2. Data Preparation ....................................................................................... 68 5.2.1. Data Imputation .................................................................................. 68 5.2.2. Feature Engineering ........................................................................... 69 5.2.3. Data Scaling and Normalization ........................................................ 72 5.2.4. Data Transformation to a Supervised Learning Problem ................... 73 v Chapter 6 Experimental Settings and Results .......................................................... 75 6.1. Experimental Settings .............................................................................. 75 6.2. Network Architectures ............................................................................. 76 6.2.1. Multilayer Perceptron......................................................................... 76 6.2.2. Traditional Convolutional Neural Network ....................................... 77 6.2.4. Multi-Input Convolutional Neural Network ...................................... 79 6.2.5. Long Short-Term Memory Network .................................................. 80 6.3. Model Evaluation ..................................................................................... 81 6.1.1. Performance Metrics .......................................................................... 85 6.1.2. Ranking Comparison .......................................................................... 95 6.1.3. Other Proposed Methods .................................................................... 96 Chapter 7 Conclusion and Future Research ............................................................. 98 References .............................................................................................................. 101 vi List of Figures Figure 1 — General solution approach for a machine learning problem .................. 5 Figure 2 — Example of a visibility estimation model defined as a regression problem ............................................................................................................ 10 Figure 3 — Example of a visibility estimation model defined as a classification problem with three classes ............................................................................... 11 Figure 4 — ANN single node .................................................................................. 22 Figure 5 — Example