DATA MINING AND GRAPH THEORY FOCUSED SOLUTIONS TO SMART GRID CHALLENGES BY SUDIPTA DUTTA DISSERTATION Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Electrical and Computer Engineering in the Graduate College of the University of Illinois at Urbana-Champaign, 2012 Urbana, Illinois Doctoral Committee: Professor Thomas J. Overbye, Chair Professor Peter W. Sauer Assistant Professor Alejandro Domínguez-García Professor David Nicol Dr. James Weber, PowerWorld Corporation ABSTRACT The Smart Grid represents a transition of the power and energy industry into a new era of improved efficiency, reliability, availability, and security, while contributing to economic and environmental health. However, several challenges must be addressed for real-life implementation of Smart Grids. Demonstrating the effectiveness of data mining and graph theory in solving some of these problems is the motivation of this dissertation. One of the key challenges in taking advantage of what the Smart Grid offers is to extract information from volumes of power system data accumulated by a suite of new sensors and measurement devices. Data presents unprecedented potential of developing better understanding of the underlying system. Handling “data explosion” in power systems and mining it for information is hence a critical challenge, necessitating the development of sophisticated algorithms. To address this need, a particular instance of power system data, namely transient stability data is studied. Generator frequencies in a large power system are analyzed with data mining techniques to extract information such as groups of coherent generators. An effective visualization method based on “spark-lines” is also presented answering a long-time question of how to best display time-varying power system data. Spark-lines are automatically placed on a geographical map of the system employing methods of graph drawing. Developed methods detected abnormal behavior in two generators of the system which was caused by errors in the generators’ simulation models that were previously undetected and subsequently corrected. This brings out the power of the developed methodology. Another important aspect of the Smart Grid is to enable integration of large quantities of renewables such as wind power. This requires installation of large wind farms and in turn ii availability of advanced methods for designing wind farms. The electrical collector system is the single most important element of a wind farm after the wind turbines, and its optimal design is necessary for optimal wind farm operation. However, there is a need for algorithms to automatically design optimal wind farm collector systems. This represents the second problem addressed in this dissertation. A graph-theoretic approach has been applied to design an optimal wind farm collector system with minimum total trenching length. Clustering techniques have also been found extremely useful in handling specific design constraints. Application of the developed methods generated designs with significantly lower costs compared to an actual real- world wind farm. The third and final challenge addressed is reliably integrating large quantities of wind power into the system. Inherent problems of variability of wind power can be overcome by developing better wind power forecasting methods and incorporating energy storage units such as batteries. A least squares estimation based short-term wind power forecasting method has been presented. Additionally, methods have been developed to determine optimal storage capacity required and optimal generation commitment for a wind farm with on-site energy storage. Both methods have been found to be extremely sensitive to the statistical properties of wind and load forecast data. In summary, this work applies tools, techniques, and concepts from the areas of graph theory and data mining to address three critical challenges of real-life implementation of Smart Grids. It is anticipated that the work presented in this dissertation will encourage future research in application of graph theory and data mining to other Smart Grid challenges. iii ACKNOWLEDGMENTS I would like to extend my heartfelt gratitude first and foremost to my Ph.D. advisor, Professor Thomas Overbye, for his continual guidance and support throughout my time at the University of Illinois. I consider myself extremely fortunate to have had the opportunity of working with him and learning from him. I would also like to thank all of my committee members for their interest, involvement, and encouragement of my work over the past years. Additionally, several people I have worked with both at the University of Illinois and in other places have inspired me, and who I want to thank. Specifically, I would like to extend my sincere gratitude to Professor Peter Sauer, who besides being on my thesis committee has been a mentor to me and with whom I have had the opportunity of teaching ECE 330 at University of Illinois. Professor Anjan Bose of Washington State University, Dr. Ratnesh Sharma at NEC Labs America, Dr. Jinjun Xiong, and Brian Gaucher at IBM Watson Research Center, Professor S. P. Singh, my M.S. advisor at the Institute of Technology, BHU, India are some of the other people who have trained me at different stages of my academic career. I would like to thank Dr. Lalit Bahl for the Joan and Lalit Bahl fellowship, and financial support from Global Climate and Energy Project (GCEP), Electric Power Research Institute (EPRI) and General Electric (GE) Energy. Thanks to additional financial support from the University of Illinois Ernest A. Reid Fellowship, the Grainger Foundation Power Engineering Award, and Washington State University Summer Doctoral Fellowship (NSF ADVANCE program). Also thanks to my friends and colleagues at University of Illinois and in particular Kate Rogers Davis, Ray Klump, Angel Aquino-Lugo, Komal Shetye, Saurav Mohapatra, Soobae Kim, iv Siming Guo, Chris Recio, Trevor Huchins, Robin Smith, Joyce Mast, and my ECE 333 and 330 students. Finally, I would like to thank all of my family for their love and support. My parents Jharna and Debabrata Dutta, and my sister Dr. Sudeshna (Dutta) Tapadar have always been tremendous sources of inspiration to me. I am especially grateful to my parents-in-law Swapna and Pradip Basu. Last but not least, thanks to my husband Dr. Anirban Basu for the constant love, support, encouragement, and mentoring at every step of my Ph.D. right from taking my GRE exam to my defense, and without whom this dissertation would not have seen the light of the day. v TABLE OF CONTENTS 1. INTRODUCTION .................................................................................................................... 1 1.1 Motivation ........................................................................................................................ 1 1.2 Background ...................................................................................................................... 3 Graph theory .............................................................................................................. 3 Data mining .............................................................................................................. 13 1.3 Dissertation overview .................................................................................................... 21 2. INFORMATION PROCESSING AND VISUALIZATION OF POWER SYSTEM TIME-VARYING DATA ....................................................................................................... 23 2.1 Motivation ...................................................................................................................... 23 Information extraction from time-varying data ....................................................... 23 Visualization of power system time-varying data ................................................... 25 2.2 Prior art .......................................................................................................................... 32 2.3 Methodology .................................................................................................................. 35 Data volume reduction ............................................................................................. 35 Pattern identification ................................................................................................ 40 Spark-line display .................................................................................................... 46 2.4 Case study and discussions ............................................................................................ 52 2.5 Conclusions .................................................................................................................... 59 3. APPLICATION OF GRAPH THEORY AND CLUSTERING ALGORITHMS TO WIND FARM COLLECTOR SYSTEM DESIGN ............................................................. 61 3.1 Motivation ...................................................................................................................... 62 3.2 Prior art .......................................................................................................................... 64 3.3 Clustering-based design ................................................................................................. 65 3.4 Collector system design with minimum total trenching length and application of spanning trees................................................................................................................