UNIVERSITY OF CALIFORNIA, SAN DIEGO Forecasting for power grids with high solar penetration A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Engineering Sciences (Mechanical Engineering) by Amanpreet Kaur Committee in charge: Professor Carlos F.M. Coimbra, Chair Professor Farhat Beg Professor Jan Kleissl Professor Tajana S. Rosing Professor Daniel L. Rudnick 2015 c Amanpreet Kaur, 2015 All rights reserved. The dissertation of Amanpreet Kaur is approved, and it is acceptable in quality and form for publication on microfilm and electronically: Chair University of California, San Diego 2015 iii DEDICATION To my parents and brother for their love and support. iv EPIGRAPH If you want to find the secrets of the universe, think in terms of energy, frequency and vibration. | Nikola Tesla v TABLE OF CONTENTS Signature Page . iii Dedication . iv Epigraph . v Table of Contents . vi List of Figures . ix List of Tables . xi Acknowledgements . xiii Vita........................................ xv Abstract of the Dissertation . xvi Chapter 1 Introduction . 1 1.1 Notation for forecasting . 2 1.2 Organization of the thesis . 2 Chapter 2 Load forecasting . 4 2.1 Related research . 5 2.2 Proposed methodology . 9 2.2.1 HAM re-forecast . 9 2.2.2 DAM re-forecast . 9 2.2.3 Preprocessing . 11 2.2.4 Detrending . 11 2.2.5 Re-forecast models . 12 2.2.6 Ensemble re-forecast . 13 2.2.7 Recursive least squares model ensemble (RLS-ME) 16 2.3 Data . 17 2.3.1 California Independent System Operator (CAISO) 17 2.3.2 Electric Reliability Council of Texas (ERCOT) . 19 2.4 Results and discussion . 19 2.4.1 Error statistics . 19 2.4.2 Residual analysis . 22 2.5 Summary . 24 vi Chapter 3 Impact of increasing solar penetration on load forecasting . 26 3.1 Data . 29 3.1.1 Preprocessing data . 29 3.1.2 Training and testing datasets . 30 3.2 Load forecast methods . 30 3.2.1 Persistence-Load (PL) . 31 3.2.2 Smart Persistence-Load (SPL) . 31 3.2.3 Autoregressive (AR) . 31 3.2.4 Autoregressive Moving Average (ARMA) . 32 3.2.5 k-th Nearest Neighbor (kNN) . 32 3.2.6 Error metrics . 34 3.3 Results and discussion . 34 3.3.1 1-hour forecasts . 35 3.3.2 15-minute forecasts . 36 3.3.3 Error distribution . 37 3.3.4 Effect of solar penetration on the forecast error . 40 3.4 Summary . 44 Chapter 4 Solar resource forecasting . 46 4.1 Data sets . 47 4.1.1 NWP data . 47 4.1.2 Ground data . 47 4.1.3 Solar power modeling . 49 4.2 Solar forecast methods . 50 4.2.1 Persistence (P) . 50 4.2.2 Smart Persistence (SP) . 50 4.2.3 Support Vector Regression (SVR) . 50 4.2.4 Reforecast . 51 4.3 Forecast results . 52 4.3.1 Intra-week forecast (day-ahead) . 52 4.3.2 Intra-hour forecast . 53 4.4 Summary . 58 Chapter 5 Net load forecasting for power grids with high solar penetration 60 5.1 Background . 61 5.1.1 Solar power generation forecasting . 62 5.1.2 Load forecasting . 63 5.1.3 Net load forecasting . 63 5.2 Microgrids as testbeds . 66 5.2.1 Testbed data . 66 5.3 Proposed methodology . 67 5.3.1 Solar power forecast . 68 5.3.2 Net load forecast . 72 vii 5.4 Model implementation . 74 5.5 Results and discussion . 75 5.5.1 Clear sky identification . 75 5.5.2 Solar power forecast . 79 5.5.3 Net load forecast . 79 5.5.4 Assessment of forecast uncertainty . 82 5.6 Solar and net load forecast errors . 83 5.6.1 Cointegration of solar and net load forecast errors 85 5.7 Summary . 86 Chapter 6 Benefits of forecasting . 89 6.1 Energy imbalance markets . 90 6.1.1 Goals . 90 6.1.2 Previous work . 91 6.1.3 Market design variables . 92 6.1.4 EIM in Western Interconnection . 93 6.1.5 Forecasting for EIMs . 95 6.2 Implications on EIMs . 96 6.2.1 Reserve scheduling . 96 6.2.2 EIMs with GCT greater than 75 minutes . 97 6.2.3 EIMs with GCT lower than 75 minutes . 97 6.2.4 Implications for the Western Interconnection EIM 100 6.3 Summary . 103 Chapter 7 Conclusions . 105 Bibliography . 109 viii LIST OF FIGURES Figure 2.1: Schematic of the proposed methodology for re-forecasting. 10 Figure 2.2: The CAISO (top) and ERCOT (bottom) time-series. 18 Figure 2.3: Time-series of actual CAISO load and HAM load prediction for several days in 2013 by CAISO and NARX re-forecast model (top) and the instantaneous absolute error (bottom). 23 Figure 2.4: Autocorrelation and partial autocorrelation for the residuals from CAISO and NARX re-forecast models for HAM load pre- diction. 23 Figure 2.5: Temporal distribution of the mean error (actual load-forecast HAM load) over the day for CAISO (left) and ERCOT (right). 24 Figure 3.1: UC Merced load profile for 06-Oct-2010 through 09-Oct-2010. 28 Figure 3.2: Daily demand load profile for the UC Merced campus with no onsite power generation (top) and onsite solar power power gen- eration (bottom). 29 Figure 3.3: Error distribution for 15-minute load forecasting with solar pen- etration. 37 Figure 3.4: Error distribution for 1-hour and 15-minute load forecasting with and without solar penetration. 39 Figure 3.5: Cumulative Distribution Function of absolute value of step change in load demand with (VLg) and without (VLng) onsite solar gen- eration. 40 Figure 3.6: Error distribution as a function of penetration with different levels of variability. 41 Figure 3.7: Time-series of load and error profiles for a clear, cloudy and overcast day for 1 hour forecast case. 43 Figure 4.1: Sample day of GHI forecast by NAM and reforecast model. 53 Figure 4.2: Mean and standard deviation of day-ahead forecast errors for GHI as a function of hour of the day. 54 Figure 4.3: RMSE and standard deviation for short-term GHI forecasts for the 15 minute forecast interval with forecast horizon ranging from 15 to 75 minutes. 56 Figure 4.4: RMSE and standard deviation for short-term solar forecasts with horizons of 5 to 75 minutes in 5 minute intervals. 57 Figure 4.5: Cumulative frequency distribution (CDF) of the solar forecast errors in five minute intervals for the Persistence (P), Smart persistence (SP) and forecast model (SVR-GA). 58 Figure 5.1: Block diagram for net load for UC Merced system. 67 ix Figure 5.2: Time series for the solar power generated for the 10 consecutive days from the year 2011. 77 Figure 5.3: Time series for the actual solar power and forecast for 1-h fore- cast horizon (top) and absolute error, AE (bottom) with night values removed. 78 Figure 5.4: Time series for the net load and solar power forecast for six consecutive days (11/06/2011 to 11/12/2011) from the testset. 80 Figure 5.5: Error correlation for net load forecast models. 82 Figure 5.6: Comparison of net load forecast errors for both additive and integrated models w.r.t. the solar forecast errors from SVRα,h model during daytime. 83 Figure 5.7: Net load and solar forecast uncertainty assessment by applying inverse CDF during daytime. 84 Figure 5.8: Cointegrating relation between the hourly integrated net load and solar forecast errors during the daytime for the first con- secutive 50 hours. 86 Figure 6.1: Timeline for the real-time Energy Imbalance Market in the Western Interconnection for the operating hour beginning at time T (Market 1). 95 Figure 6.2: Standard deviation in forecast error for forecast horizons rang- ing from 1 day to 5 minutes ahead forecast horizons in 1 hour, 15 minute and 5 minute resolution. 98 Figure 6.3: Benefits of 5 minute versus 15 minute reserve allocation for real- time markets considering 5 minute data interval. 99 Figure 6.4: Flexibility reserve required for 1 MW solar plant for 5 minute intervals. 100 Figure 6.5: Energy imbalances (probability for the error to exceed 7.5%) for 1 MW solar plant for 5-75 minutes and 15-75 minutes forecast horizon with 5 minute and 15 minute resolution. 102 Figure 6.6: Inverse cumulative frequency distribution of the expected fore- cast errors that market operator has to optimize for in 5-minute market after scheduling the awards for 15 minute market. 103 x LIST OF TABLES Table 2.1: Ensemble models applied for short term load forecasting. 8 Table 2.2: Polynomials used for various model structures . 13 Table 2.3: Details for the dataset used for HAM and DAM forecast. 17 Table 2.4: Number of model parameters used for CAISO and ERCOT re- forecast time-series models. 20 Table 2.5: Statistical error metrics for HAM load forecast using various models for CAISO (3-18-2013 to 06-11-2013) and ERCOT (05- 01-2013 to 06-20-2013). 20 Table 2.6: Statistical error metrics for the DAM load forecast using various models for CAISO (3-18-2013 to 06-11-2013) and ERCOT (05- 01-2013 to 06-20-2013). 21 Table 3.1: The coefficients of the polynomial.