Efficient load profiling and forecasting in large electric power systems Imre Lendák1, Tomáš Horváth1 Data Science and Engineering Department, Faculty of Informatics, Eötvös Loránd University, Budapest, Hungary, [email protected], WWW home page: http://t-labs.elte.hu Abstract: The goal of this paper is to present an efficient algorithm, consisting of a daily load profile clustering and load forecasting algorithm for large electric power sys- a load forecasting phase. tems. It uses a combination of nearest neighbor-based load The following sections of this document contain more profile clustering and rule-based load forecasting. The detailed description of each of the above steps. load data was sliced into daily load curves, which were K-Means-clustered, thereby compressing data and simpli- fying the solution. K-Means was chosen in the proof of 2 State-of-the-art concept phase and will be substituted with more precise solutions later. In the forecasting phase the daily load The body of electric load forecasting knowledge is very profile is predicted based on the forecast date, day type large, with numerous papers published in all major (e.g. weekday or weekend) and historical consumption domain-specific journals and conferences. As an exten- data for similar days in the past. The solution was tested sive review of all relevant solutions would not be feasible on a large dataset consisting of one year-long, 5-minute due to the page limits, we will only refer to those research measurement data in a 1900-power-line system. The so- results, which specifically focus on time series clustering, lution showed excellent performance in both the training smart meter big data management and the combination of and forecast phases. It produced meaningful forecasts solutions from these domains used in load forecasting. even when the input data contained significant amounts of anomalies. An additional advantage of the presented 2.1 Time series analysis solution is that it can be used for medium and long-term forecasting with limited and/or missing input data. Reference [19] presents two novel time series cluster- ing methods, namely k-shape and k-MultiShapes (k-MS), which rely on scalable iterative refinement procedures 1 Introduction based on shape-based distances (SBD). The authors claim that their solution(s) achieve similar results to dynamic The challenge to accurately predict the power flows in to- time warping, which at a lower computational cost. k- day’s large electric power systems receives ample atten- Shape is quoted as a suitable and novel solution for creat- tion. Numerous papers are published in specialized smart ing homogeneous and well-separated clusters of time se- grid journals with the promise of being able to predict ries data. The positive characteristics of k-Shape are do- the electricity consumption of single households or their main independence, accuracy and efficiency [20]. groups. Others develop solutions which predict the power Reference [33] describes a convolutional neural flows in electric power transmission systems, which span network-based time series classification solution, in which over large geographic regions, e.g. sizable parts of conti- the time series features are automatically learned instead nental Europe or the USA. Yet another group of scientists of handpicking. The authors describe the process of works on data compression algorithms, with the intention data preparation, filtering, and the structure of the used to lower the communication and storage costs incurred in network. The authors of reference claim that semi- modern smart grids. supervision can boost time series clustering performance Within this setting, we start from the idea that the flows [7]. on the power lines in electric power transmission systems have some form of periodicity. More specifically, we will theorize that the configurations of these large systems does 2.2 Data compression not change frequently, and under the same load condi- Reference [29] contains an application-oriented review of tions the flows will be similar on the power lines for sim- smart meter data analysis solutions. Three main applica- ilar days, e.g. for Wednesdays in July the load will most tion areas are identified, namely load analysis, load fore- probably be very similar under the same loading condi- casting, and load management. This is a rare reference tions. Therefore we propose a 2-phase load forecasting which addresses the data privacy and security aspect of the analyzed solutions as well. The most important mo- Copyright c 2019 for this paper by its authors. Use permitted un- der Creative Commons License Attribution 4.0 International (CC BY tivation behind data compression in smart metering are 4.0). reduced congestion of communication channels used for data transmission, storage overhead, as well as improved use the stationarity property of the estimated models to data mining efficiency. Reference [30] presents a com- identify daily customer profiles. prehensive study on smart meter big data compression so- The authors of reference [13] analyze annual load lutions. The authors of reference [25] present a feature- curves of households and create annual and weekly load based, load data compression method for smart metering profiles. They also show how additional features of house- infrastructures. The solution is not lossless. The authors hold affect annual consumption and random variation in claim it is efficient, with little reconstruction error. The household energy consumption. Reference [32] presents solution was validated on the Irish Smart Metering Trial an analysis of the daily consumption data of 300 residen- Data. The authors of reference [22] present lossless com- tial customers in China. The authors identify four types pression algorithms for power system operational data. of monthly usage patterns and 9 abnormal users, with sig- The authors of reference [28] use K-SVD sparse repre- nificantly different electricity use patterns. They prove that sentation technique. In the dictionary learning phase, they more than 80% of households have a similar monthly elec- decompose load profiles into linear combinations of sev- tricity usage pattern. eral partial usage patterns (PUPs). In the sparse coding The authors of references [31] used k-Shape for build- phase, a linear support vector machine (SVM) is used to ing energy usage pattern analysis and tested their solution classify load profiles as residential or small and medium- on real-life data measured in ten institutional buildings. sized enterprises (SMEs). The authors claim that their Reference [14] goes even further, by using ML techniques solution outperforms k-means, the discrete wavelet trans- to guess the lifestyles of energy consumers based on their form (DWT), principal component analysis (PCA), as well consumption patterns. as piecewise aggregate approximation (PAA). The solution presented in reference [12] utilizes deep- stacked auto-encoders in electric load data compression 3 Problem definition and classification. It is necessary to develop a load forecasting solution which is capable to predict loads in extremely large, Europe-wide 2.3 Load classification and forecasting electric power transmission systems consisting of thou- sands of power lines. The input data will consist of histori- A more general review of smart meter data intelligence cal measured loads with a sampling rate of 5 minutes avail- is provided in references [1] and [15]. The authors of able for at least the last 1-year period. This data will be re- references [10][21] and [24] explore state-of-the-art ma- ferred to as dynamic data, due to its frequency of change. chine learning approaches in load forecasting. They re- Due to data privacy limitation, the input data will not con- view more than 50 research papers and group their contri- tain the complete static data model of the system under butions into single and hybrid computational intelligence- consideration. This means that there will be no data pro- based approaches. They perform a qualitative analysis vided to build a mathematical graph consisting of the bus- based on accuracy and prove the superiority of hybrid so- bars (vertices) and power lines (edges) connecting them. lutions. Various short-term load forecasting techniques It is expected that the prediction horizon will be 5 min- were compared as early as 1989 in reference [17]. Var- utes ahead. The solution should be extensible and be ca- ious machine learning-based short-term load forecasting pable to provide acceptable mid- and long-term forecasts techniques ranging from moving averages to deep neu- (1 day or 1 week ahead) as well. Additionally, it is neces- ral networks are addressed in references [2][5][6][8][11] sary for the solution to handle temporary unavailability of [23][24]. Smart meter forecasting from one minute to one- significant amounts of measurements, when those will not year horizons is presented in reference [16]. Electricity be provided in a timely manner by one or more countries price and demand forecasting is tackled by the authors in and/or companies in the geographical area under consider- [18]. Bus load forecasting is addressed in reference [3]. ation. Optionally, the solution should be able to incorpo- Reference [27] presents a smart meter data characteriza- rate weather forecast and other freely available 3rd party tion method based on the Gaussian mixture (GM) model. data and thereby increase the accuracy of its outputs. The The authors claim that compared to other state-of-the-art relevance of such data might vary, as the extent of data solutions, theirs offers significantly better fitting for me- anonymization required will not allow the forecasting tool ter data. Reference [26] describes a hybrid clustering and access to the geographical location of system resources classification technique in short-term energy consumption (i.e. power lines). forecasting. Reference [4] proposes to use clustering in bottom- up, short-term load forecasting. The authors cluster load 4 Solution curves by using wavelets to measure similarity and thereby create super-consumer profiles.