By Rodrigo Henriquez-Auba, Jose Daniel Lara, Duncan S. Callaway, and Clayton Barrows Transient Simulations With a Large Penetration of Converter- Interfaced Generation ©SHUTTERSTOCK.COM/PLUIE_R Scientific computing challenges and opportunities. URRENT TRENDS IN ENERGY SYSTEMS generation, which reduces system inertia. It results in point to renewable energy sources (RESs) higher rates of change of frequency, a lower-frequency and battery energy storage systems nadir following disturbances, and complex interactions C (BESSs) becoming prevalent in power sys- between the dynamics of power electronics converters and tem operations. As of writing this article, other power systems components. the United States has more than 37 GW of utility-scale Over many decades, the power system community has solar capacity and an additional 112 GW under develop- developed standards to model and study synchronous ment. With the rapidly declining capital costs of many of machines and controls. These standards include IEEE 1110 these technologies, we can expect significant deploy- and IEEE 421, which provide guidelines about the model- ments in the coming years. ing practices required to adequately assess system stabili- The change in the primary energy supply mix isn’t the ty and performance. But the introduction of RESs and only large-scale shift happening in power systems. Inte- BESSs render the study of these relationships more chal- grating massive numbers of generators interfaced through lenging because many of the assumptions upon which power electronics is also updating our basic understanding existing standards are based may no longer be valid (see of system stability and control. One such example of an Milano et al. 2018 and Markovic et al. 2021). advancement is the displacement of traditional rotating Due to the complexity and scale of power system analysis, researchers rely on computational tools and simulations to explore and understand the new phe- Digital Object Identifier 10.1109/MELE.2021.3070939 Date of current version: 8 June 2021 nomena that might emerge in the future systems 72 IEEE Electrification Magazine / JUNE 2021 2325-5897/21©2021IEEE characterized by widespread RESs and BESSs. In 2021, Scientific Computing Practices the U.S. National Academy of Sciences recognized the There is no single definition of scientific computing as importance of understanding how the grid of the the field has evolved from purely computational issues future will behave and how operators and policy mak- to the general use of computers and software to answer ers can ensure its continued reliability. The capacity to scientific questions. However, all scientific computing do this rests on improving the simulation capabilities definitions share the objective of enabling 1) the repro- necessary to build and test the integration of new ducibility of the experiments and 2) the mechanisms devices and components. used to verify the validity of the conclusions derived. Fortunately, the array of software tools available for Lara et al. 2020 provides a detailed discussion of these researchers to conduct large-scale studies has also definitions for power system operational studies. Fig- grown significantly. Progress in numerical solver algo- ure 1 summarizes the three steps required to perform a rithms, model-order-reduction techniques, automatic simulation experiment, which follow best practices in differentiation (AD), and symbolic and numerical com- scientific computing: data, computational modeling, and putation, among other areas, are advances essential to results subprocesses. the development of the new analysis techniques In practical terms, achieving reproducibility and valida- demanded by the realities of a modernizing grid. The tion requires that two major components, the environ- power systems field is ripe with current and future ment and the experimental workflow, are considered. opportunities to exploit these techniques for novel mod- xxEnvironment: The environment comprises the hard- eling and control methods. However, critical obstacles ware components and the configuration of all the remain and among these, researchers must develop the software used to implement a computational experi- capacity to surmount the dramatic increases in model- ment. The environment may include elements such ing complexity. as cloud services, third-party software, file manage- This article focuses on the importance of scientific ment scripts, and external tests. computing principles and showcases the modern xxWorkflow: The workflow is the full process of data software developments that enable large-scale power intake, computation, and results. The development system studies. We stress that computer-aided simu- and careful documentation of a workflow is a require- lation research has to be replicable, the results must ment for validation and reproducibility. be validated, and systems should be scalable such In this respect, open source tools provide the required that they are of realistic proportions and make a sig- capabilities to facilitate scientific computing for power nificant contribution to our understanding. We dis- systems research. Despite challenges, several open cuss how the Julia programming language can be source efforts have grown to be successful in the com- used to tackle these challenges and outline the ways munity and reduce the barriers to entry for those seek- that The Scalable Integrated Infrastructure Planning ing to perform high-quality research and reliable (SIIP) (https://www.nrel.gov/analysis/siip.html) initia- analysis. MATPOWER (https://matpower.org) is a MAT- tive at the National Renewable Energy Laboratory LAB-based tool that is widely used to perform steady- (NREL) is capturing these opportunities in power sys- state analyses such as power flow, continuation tems analyses. power flow, and optimal power flow. OpenDSS (https:// Test Set Simulation 1 Output 1 Simulation Metrics Experimental Select Trial Test Set Simulation for the Data Data 2 Output 2 Trial . Decision . Model . + . Test Set Emulator Simulation n Output n Data Process Computing Process Results and Reporting Process Figure 1. The computing steps required for conducting a single trial in a simulation experiment. These steps follow the best practices in scien- tific computing to achieve reproducibility and validation: data, computational modeling, and results subprocesses. (Adapted from Lara et al. 2020; used with permission.) IEEE Electrification Magazine / JUNE 2021 73 smartgrid.epri.com/Simulation The environment money for license acquisition. This Tool.aspx) is used for multiple distri- common practice reduces the capac- bution systems analysis. In the field comprises the ity for innovation in the power sys- of transient simulations, the MAT- tems field for researchers who must, LAB-based Power System Analysis hardware rather than use industrial tools, Toolbox (PSAT) (http://faraday1 components and the reimplement these well-established .ucd.ie/psat.html) provides elec- models, develop their own data sets, tromechanical simulations with configuration of all and handle the integration libraries. multiple models. More recently, the This reality translates to the fact that Python-based tool ANDES (https:// the software used to scientific reproducibility is often docs.andes.app/en/stable/) has implement a either not achieved or limited to become available for power system “code sharing.” simulation. It uses a hybrid symbol- computational Simulation reproducibility requires ic-numeric framework for numeri- the separation of models and algo- cal analysis. experiment. rithms when developing modeling However, there is a high setup libraries. Figure 2 depicts the software cost associated with developing stack of a dynamic modeling applica- dynamic simulation applications tion as follows: that can run large-scale experiments. For this rea- xxModeling layer: This layer contains all of the code repre- son, researchers resort to using industrial tools once sentations of the system behavior in differential and/or their systems of interest grow larger than hundreds of algebraic equations; for instance, charging and dis- buses. However, these proprietary models and algo- charging capacitors or proportional–integral–derivative rithms are not openly available; this means that both (PID) controllers. using them and replicating the results obtained by xxIntegration algorithms: The libraries that implement other researchers who use them require large sums of an integration scheme and are used to obtain the Electromagnetic Generator Model Inverter Model Classical Generator Models Model Implicit Multistep One-Step Methods Methods (BDF) Integration (Euler, RK) Algorithm Explicit Libraries Multistep Methods (AB2) SuperLU MT Linear Algebra LAPACK Libraries BLAS Figure 2. A description of the common software layers in a scientific research software program. A modeling layer is shown with the code repre- sentation of the equations and algorithms used to solve the models, along with the numerical linear algebra libraries utilized to perform the cal- culations. RK: Runge–Kutta; AB2: Adams–Bashforth-2; BDF: backward differentiation formula. 74 IEEE Electrification Magazine / JUNE 2021 numerical solution of the mod- From an analytical the singular perturbation theory pop- els’ differential equations. For ularized in the 1980s. The seminal instance, Euler, backwards differ- perspective, RMS papers showed that a simplified entiation formulas, and Runge- model is still a valid representation of Kutta,