2020 MRO Regional Risk Assessment
January 28, 2020
Table of Contents
1. PREFACE...... 4 2. INTRODUCTION ...... 5 3. 2019 MRO REGIONAL RISK IDENTIFICATION ...... 6 3.1 2019 RISC Report ...... 6 3.1.1 Risks Associated with Grid Transformation ...... 7 MRO Region’s Exposure to Grid Transformation ...... 8 3.1.2 Risks Associated with Extreme Natural Events ...... 12 MRO Exposure to Extreme Weather Events ...... 12 MRO Exposure to Geo-Magnetic Disturbance Events...... 14 3.1.3 Physical and Cyber Security Risks ...... 15 Physical and Cyber Security Threats Identified in SOR Report ...... 15 MRO Region Physical, Cyber Security and Control System Threats ...... 16 Adequate Security Staffing ...... 16 Asset Inventory ...... 17 Cloud Storage ...... 17 Connectivity ...... 17 Copper Theft and Vandalism ...... 18 Insider Threat ...... 18 Mitigating System Vulnerabilities ...... 18 Saboteurs...... 18 Security Visibility, Monitoring, and Incident Response ...... 19 Spear Phishing...... 19 Supply Chain...... 19 UASs ...... 19 3.1.4 Critical Infrastructure Interdependencies: ...... 20 3.1.5 MRO Exposure to Critical Infrastructure Interdependencies ...... 20 Impact of Natural Gas Disruption on Bulk Power System...... 20 Critical Communication Circuit Sunset...... 22 3.2 2019 State of Reliability Report ...... 22 Trending Misoperations Associated with Event Analysis Reports ...... 23 3.3 2019 NERC Long-Term Reliability Assessment Report ...... 25 3.4 Additional Operational and Planning Risks Identified for the MRO Region ...... 27 3.5 2020 ERO CMEP Implementation Plan ...... 29
2
3.6 Requirements with High Risk Violations ...... 30 3.7 Reliability Standards and Requirements ...... 31 FAC, TOP, and IRO Operating Limits Reliability Standards ...... 31 GMD Reliability Standards EOP-010-1 and TPL-007-1 ...... 32 Model Data Reliability Standards ...... 32 Planning Standard TPL-001-4 ...... 33 New Reliability Standards Enforceable in 2020 ...... 34 Oversight of “Risks in Aggregate” ...... 34 4. CONCLUSION ...... 36 References ...... 37
3 2020 MRO Regional Risk Assessment PREFACE
1. PREFACE
Midwest Reliability Organization (MRO) is dedicated to its vision of a highly reliable and secure North American bulk power system. To ensure reliability of the bulk power system in the United States, Congress passed the Energy Policy Act of 2005, creating a new regulatory organization called the Electric Reliability Organization (ERO) to establish mandatory Reliability Standards and monitor and enforce compliance with those standards on those who own, operate or use the interconnected power grid.
In 2006, the Federal Energy Regulatory Commission (FERC) approved the North American Electric Reliability Corporation (NERC) as the ERO under section 215(e)(4) of the Federal Power Act. NERC delegates its authority to monitor and enforce compliance to six Regional Entities established across North America, including MRO. Recognizing the international nature of the grid, NERC as the ERO, along with MRO, established similar arrangements with provincial authorities in Canada.
The MRO region spans the provinces of Saskatchewan and Manitoba, and all or parts of the states of Arkansas, Illinois, Iowa, Kansas, Louisiana, Michigan, Minnesota, Missouri, Montana, Nebraska, New Mexico, North Dakota, Oklahoma, South Dakota, Texas, and Wisconsin. The region includes approximately 200 organizations that are involved in the production and delivery of electric power, including municipal utilities, cooperatives, investor- owned utilities, transmission system operators, federal power marketing agencies, Canadian Crown Corporations, and independent power producers.
MRO's primary responsibilities are to: ensure compliance with mandatory Reliability Standards by entities who own, operate, or use the North American bulk power system; conduct assessments of the grid's ability to meet electric power demand in the region; and analyze regional system events. Additionally, MRO creates an open forum for stakeholder experts in the region to discuss important topics related to addressing risk and improving reliable operations of the bulk power system.
4 2020 MRO Regional Risk Assessment INTRODUCTION
2. INTRODUCTION
The MRO Regional Risk Assessment (RRA or assessment) is a document that MRO staff, with input from industry subject matter experts, prepares annually to identify risks to the reliable and secure operations of the bulk power system within MRO’s regional footprint. Several continent-wide risks identified by the ERO are summarized in the assessment. The sources for this information include:
2019 ERO Reliability Issues Steering Committee (RISC) Priorities Report (RISC report) 2020 ERO Compliance Monitoring and Enforcement Program Implementation Plan (CMEP IP) 2019 NERC State of Reliability Report (SOR report) 2019 NERC Long-Term Reliability Assessment (LTRA report)
Some risks identified at the North American level can broadly present themselves in any of the ERO regions, such as Human Performance and Skilled Workforce. However, other risks may be more geographic, regional, or registered entity-specific, that is, certain areas, regions or entities may have a higher exposure to, or are more susceptible to, a specific risk. Severe weather conditions or high concentrations of renewable generation are examples of these. Therefore, this assessment identifies which of the risks in the NERC-wide reports may have a higher (or lower) probability of occurrence within the MRO Region, and seeks to identify any additional risks to bulk power system reliability and security that may be regionally unique.
In addition to assessing which ERO-wide risks may be more (or less) applicable to the MRO Region, MRO staff leveraged the expertise of industry subject matter experts comprising the three MRO advisory councils (Reliability, Compliance Monitoring and Enforcement Program (CMEP), and Security) to identify any additional risks that the region may be susceptible to experiencing.
This RRA also identifies which risks highlighted in the ERO-wide CMEP Implementation Plan should be a focus of MRO’s compliance monitoring and enforcement activities.
To the extent possible, recommendations and suggestions are presented in this assessment to help registered entities become more aware of and reduce risk to their individual systems. However, not all risk topics identified in the RRA include discussion on ways to reduce the risk.
5 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
3. 2019 MRO REGIONAL RISK IDENTIFICATION
The following sections summarize the risks identified by the various ERO reports, and how these risks correlate to the MRO Region. Additionally, regional risks identified by MRO staff and/or the MRO advisory councils are provided. Note that some risks identified in this assessment may apply to the Midwest portion of the continent (which is largely congruent with the MRO footprint) or, in some cases, to a functional entity such as an Independent System Operator/Regional Transmission Operator (ISO/RTO), Planning Coordinator (PC), Reliability Coordinator (RC), Generator Owner/Operator (GO/GOP), or Transmission Owner/Operator (TO/TOP) that might span multiple regions.
The 2019 RISC Report1 highlights forward-looking risks to the bulk power system that merit attention and then recommends actions that align with those risks. The report consolidates the identified risks into four high level categories: 1) Grid Transformation, 2) Extreme Natural Events, 3) Security Vulnerabilities, and 4) Critical Infrastructure Interdependencies:
1 The charts and figures in this section were pulled directly from the 2019 RISC Report and are shared with permission.
6 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
The risks were then classified as to whether they should be actively managed or monitored:
Following this classification, the risks were prioritized based on an industry-wide survey of bulk power system experts across the ERO footprint. The heat maps shown below summarize the results of the survey, with each individual risk mapped with likelihood of the risk occurring on the X-axis and the impact if the risk occurred on the Y-axis. Therefore, the upper right quadrant of the graph indicates higher risk and the lower left quadrant indicates lower risk.
The risks associated with Grid Transformation are highlighted below in orange in Figure 1:
GRID TRANSFORMATION BASE LIKELIHOOD [X AXIS] 4.5 BASE IMPACT [Y AXIS]
Cyber Security Risk 4
Extreme Natural Events Critical Infrastructure Interdependencies Loss of Situational Awareness Changing Resource Mix 3.5 Bulk Power System Planning Resource Adequacy and Performance
Physical Security Risks Human Performance and Skilled Workforce 3 Increasing Complexity in Protection and Control Systems
2.5
2 2 2.5 3 3.5 4 4.5
Figure 1: Risks Associated with Grid Transformation
7 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
MRO Region’s Exposure to Grid Transformation Three of the risks associated with Grid Transformation would impact the MRO Region comparably to the other ERO regions; these are: Human Performance and Skilled Workforce, Bulk Power System Planning, and Increasing Complexity in Protection and Control Systems. More detail on these risks is provided in the 2019 RISC Report. However, the risks identified as Loss of Situational Awareness, Changing Resource Mix, and Resource Adequacy & Performance tend to be different for the MRO Region as compared to the industry-wide assessment of risk. The conditions within MRO that influence these risks are discussed below.
Loss of Situational Awareness
Situational Awareness (SA) can be a very broad topic, depending on how it is defined. In the 2019 RISC Report, it is defined as the awareness of system operating personnel as to the current state of bulk power system configuration, flows, and voltage. The construct of Energy Management System (EMS) software and its various applications within the MRO footprint have resulted in a robust and reliable system for operational situational awareness. The TOPs and BAs within MRO have very high availability of their EMS systems and advanced applications such as Real-Time Contingency Analysis (RTCA). In addition, these entities’ EMS models often include a large segment of neighboring systems, which allows for increased visibility and thus awareness of bulk electric system (BES) conditions beyond defined borders. MRO registered entities also have back-up/secondary EMS facilities, and many are running in parallel at a backup facility to cover the potential temporary loss of a primary facility.
In addition to Transmission Operators’ own primary and backup EMS systems, the MRO Region is unique in that it has two large ISO/RTOs (Mid-Continent Independent System Operator (MISO) and Southwest Power Pool (SPP)) that serve as Reliability Coordinators, Balancing Authorities, and Planning Coordinators for most of the region and provide a fully redundant set of EMS models and advanced applications for their members. With the exception of one small utility in North Dakota and Saskatchewan Power Corporation, all TOPs and GOPs are members of either MISO or SPP. MISO and SPP also have a significant overlap of each other’s EMS models and a comprehensive seams agreement to manage the lengthy border that they share. MISO and SPP also have overlap with SaskPower’s EMS data to provide backup SA for SaskPower including RTCA.
Since the advent of NERC’s Event Analysis program in 2011, no temporary EMS outage within MRO has resulted in a loss of firm load or emergency system conditions that required operator action. The multiple layers of EMS models and advanced applications systems within MRO have provided very high availability rates, and therefore the risk associated with Loss of Situational Awareness is determined to be lower in MRO than the NERC-wide survey average shown in Figure 1 (indicated by the downward arrow).
Changing Resource Mix / Resource Adequacy and Performance
There are several factors and conditions occurring within the MRO Region that can cause higher than average risk due to changing resource mix and resource adequacy. These factors are discussed below.
8 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
Wind Generation Availability In September 2016, a storm in South Australia tested the voltage ride-through capabilities of multiple wind plants when five faults occurred on the extra-high-voltage (EHV) transmission system in under 90 seconds (South Australia Blackout Report). The Australian regulator determined that the wind plants did not meet Australian generator performance standard requirements (similar to NERC Standard PRC-024, voltage and frequency ride-through requirements). The loss of these wind plants contributed to the South Australian blackout that occurred that day, impacting about 850,000 homes. Within the US, as detailed in a NERC Lessons Learned, five similar events have occurred on the ERCOT system, where sequential transmission line faults or bus faults also resulted in the loss of wind generation due to “blocking” (cessation of current injection into the bulk power grid).
In the MRO footprint, there are about 34,000 MW of BES connected wind generation, which is more wind-connected generation than in any other ERO region. Although there is no event history of BES wind resources in the MRO Region not meeting the voltage ride-thru requirements of NERC Standard PRC-024, verification of voltage ride-through capabilities required by PRC-024 is essential for BES wind plant owners within the MRO Region. This is similar to, and consistent with, the NERC Alert titled “Loss of Solar Resources during Transmission Disturbances due to Inverter Settings” that was issued to industry on June 20, 2017, after multiple BES solar plants in the western U.S. blocked in response to multiple 500 kV line trips and recloses that occurred during the Blue Cut California fires.
Wind plants in the MRO Region are also susceptible to shutting down during severe cold weather if the ambient temperature at the wind plant is less than the cold weather protection package of the wind turbines. This is discussed further in the next section on Winter Planning Reserve Margins and in section 3.1.2, Extreme Natural Events.
Winter Planning Reserve Margins In the MRO footprint, severe cold weather can jeopardize generation availability in multiple ways, thereby causing actual winter planning reserve margins to be below forecasted levels. Additionally, severe cold weather can result in record winter peak loads. The following are four factors that can influence and result in actual winter reserve margins being lower than expected.
Wind facilities are typically assumed to be available during winter peak demand when calculating reserve margins. However, on January 29 and 30, 2019, a significant portion of the wind resources operating in the upper Midwest during high output levels, hit their cold weather cutout temperatures, causing the plants to shut down. Approximately 6,500 MW within MISO shut down over the course of a few hours, as shown in the Figure 2:
9 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
Figure 2: January 30, 2019 Wind Turbine Cold Weather Cutouts
A second risk to generation availability, and thus reserve margins during severe cold weather, is non-firm gas supply and transportation contracts for natural gas generators, which are common contracting practices. If gas supply becomes insufficient and needs to be rationed, which can occur during severe cold conditions in the upper Midwest, non-firm customers (including bulk power system gas generation plants) will be curtailed first.
A third risk to generation availability, and thus reserve margins during severe cold weather, is lack of winterization of gas generation facilities, particularly in the southern portion of the Midwestern U.S. The severe cold weather event that hit the southern portion of the MRO footprint on January 17, 2018 resulted in the generation outages reflected in Figure 3, most of which were due to lack of winterization.
10 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
Figure 3: Early January 17, 2018 Generation Outages and Derates
A fourth risk to reserve margins during severe cold weather involves the increasing uncertainty in load forecasting. In recent years, load forecasting has been subject to increasing error due to net metering with distributed generation (e.g., rooftop solar panels) and the continuously changing makeup of load (high efficiency products, inverter connected products, increasing reactive component of new loads, etc.). However, during severe cold weather, particularly in the southern US, load can spike due to resistive heating. An example of the impact that resistive heating can have on load was witnessed on January 17, 2018, when one large entity indicated that a 5 degree F drop in ambient temperature (from 18 degrees F to 14 degrees F) would cause a 15 percent increase in winter peak demand on its system. It was also recognized on January 17, 2018, that throughout the southern portion of the Eastern Interconnection, winter peak loads were at or near 100 percent of forecast summer peak loads.
Therefore, PCs should consider incorporating these various factors when determining their winter reserve margins by:
Identifying potential wind plant cutouts and excluding their availability during winter peak conditions when appropriate. Potentially reducing gas generation in studies by an appropriate factor to account for non- firm gas supply/transportation and the risk of gas curtailment.
11 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
Identifying ambient temperature operating capabilities of gas generators, particularly in the southern U.S., to determine which plants will be unavailable during cold weather so as to secure replacement generation well in advance. Accurately forecasting the potential spike in peak demand due to resistive heating and using that higher load as appropriate in winter reserve margin calculations.
Based on the above discussion, the risk associated with Changing Resource Mix and Resource Adequacy and Performance is determined to be higher in MRO than the NERC-wide survey average shown in Figure 1 (indicated by the upward arrows).
The risk associated with extreme natural events is highlighted (in orange) in the Figure 4 heat map:
EXTREME NATURAL EVENTS BASE LIKELIHOOD [X AXIS] 4.5 BASE IMPACT [Y AXIS]
GMD Cyber Security Risk 4
Extreme Natural Events Critical Infrastructure Interdependencies Loss of Situational Awareness Changing Resource Mix 3.5 Bulk Power System Planning Resource Adequacy and Performance
Physical Security Risks Human Performance and Skilled Workforce 3 Increasing Complexity in Protection and Control Systems
2.5
2 2 2.5 3 3.5 4 4.5
Figure 4: Risks Associated with Extreme Natural Events
MRO Exposure to Extreme Weather Events MRO has significant exposure to extreme weather events, and in multiple ways. In the past two years, MRO has experienced two very severe cold weather events that would normally have the
12 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
probability of a once in 10- or 20-year type of event. The January 17, 2018 event brought record low temperatures to the southern portion of the MRO and SERC footprints as shown in Figure 5:
Figure 5: Severe Low Temperatures of January 17, 2018
This event led to simultaneous record winter peak loads throughout the southern US, coupled with load forecast errors, and over 30,000 MW of generation outages, most of which were attributed to lack of winterization. Transmission constraints prohibited available generation in the upper Midwest from reaching the southern Midwest. Multiple next contingency scenarios would have resulted in firm load shed. The January 17, 2018 Inquiry Report for this event, which was authored by staff from FERC, NERC, MRO, ReliabilityFirst and SERC, has provided multiple recommendations to industry to better prepare for these types of events, which may occur more frequently.
On January 29-30, 2019, a severe cold weather event struck the upper Midwest as shown in Figure 6 on page 14. Temperatures in the southern half of the Midwest were moderate during this event as compared to January 17, 2018. This allowed generation located in MISO-South to be exported to MISO-North since load demands in the southern Midwest were well below forecast winter peak levels.
13 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
Figure 6: Severe Low Temperatures of January 29-30, 2019
Wind generation in both SPP and MISO was at high output levels when the wind turbines started to shut down due to their cold weather temperature cutout limits being reached. The lessons learned from this event was that Reliability Coordinators will need to forecast, in advance, when these wind plant cutout temps may be reached, such that sufficient replacement energy can be secured well in advance of the loss of wind output. The two ISO/RTOs within MRO have the capability to forecast when wind plants will shut down due to severe low ambient temperatures.
Based on winter weather patterns in recent years, the risks associated with severe cold weather events may impact the Midwestern portion of the country more frequently than other regions. This is illustrated with an upward direction arrow as shown in Figure 4 on page 12.
MRO Exposure to Geo-Magnetic Disturbance Events The northern portion of the MRO Region can be significantly impacted by a large-scale geomagnetic disturbance (GMD) event due to its proximity to the magnetic north pole. NERC and the industry have been assessing this risk in detail for the past decade and significant progress has been made. A ground resistance model is being developed and will be used in conjunction with traditional bulk power system models to further study GMD events and their impact on the system. Analysis has been performed by several TOs within MRO to identify any EHV bulk power transformers that may be at risk of damage due to excessive Geo-magnetically Induced Currents (GIC). Some TOs have installed protection schemes on certain EHV auto-transformers to protect them from potentially damaging GIC during a large scale GMD event. The risk associated with a 1:100 year very large scale GMD event is considered potentially high impact, but low frequency, and therefore would be in the upper left quadrant of the map as shown individually on the risk heat map as shown in Figure 4 on page 12.
14 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
The risks associated with Physical and Cyber Security are highlighted (in orange) in the Figure 7 heat map:
SECURITY RISKS 4.5 BASE LIKELIHOOD [X AXIS] BASE IMPACT [Y AXIS]
Cyber Security Risk 4
Extreme Natural Events Critical Infrastructure Interdependencies Loss of Situational Awareness Changing Resource Mix 3.5 Bulk Power System Planning Resource Adequacy and Performance
Physical Security Risks Human Performance and Skilled Workforce 3 Increasing Complexity in Protection and Control Systems
2.5
2 2 2.5 3 3.5 4 4.5
Figure 7: Risks Associated with Physical and Cyber Security
Physical and Cyber Security Threats Identified in SOR Report In 2019, as in previous years, there were no reported cyber or physical security incidents on the bulk power system that resulted in a loss of load. This is the single most important security measure because it shows that the combined efforts of industry, the ERO Enterprise, the Electricity Information Sharing and Analysis Center (E-ISAC), and government partners have so far been successful in protecting bulk power system reliability. Nonetheless, grid security (particularly cyber security) is an area where the ERO Enterprise and industry must continually improve defenses as threats and technologies rapidly evolve.
The most prominent physical and cyber security threats affecting industry from January to December 2019 include gunfire, theft, crypto-jacking, phishing, and malware. The cyber security threat landscape constantly changes, and industry must stay informed about adversaries’ latest tactics, techniques, and procedures. While many physical security threats and impacts remain similar from year-to-year, the threat from saboteurs continues to evolve as the adversary becomes more capable.
Continuing physical threats include: Gunfire Theft Saboteurs Unmanned Aerial Systems (UASs)
15 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
Continuing cyber threats include: Supply chain Credential harvesting Exploiting trust relationship between targeted organizations and their business partners Network device targeting Use of native tools
MRO Region Physical, Cyber Security and Control System Threats To support registered entities in addressing cyber, physical, and control system threats, MRO established the Security Advisory Council (SAC) in 2017. The SAC holds an annual regional security risk assessment meeting attended by SAC members, MRO staff, E-ISAC staff and registered entity representatives from the MRO Region. The key outcome from the meeting is the identification of security risks most likely to affect the MRO Region. The compiled, non-prioritized list of security risks from the MRO 2019 Regional Security Risk Assessment is:
Adequate security staffing Asset inventory Cloud storage Connectivity Copper theft and vandalism Insider threat Mitigating system vulnerabilities Saboteurs Security visibility, monitoring, and incident response Spear phishing Supply chain Unmanned Aerial Systems (UAS)
Adequate Security Staffing Adequate security staffing is a high risk identified by registered entities of all sizes. Small, medium, and large entities all identified this as one of the more likely risks to impact reliable operations as people are the most important component of effective security.
There are a number of reasons that adequate staffing is a risk. Inadequate numbers of trained security staff to identify, assess, and mitigate evolving security threats can result in delays implementing security measures, inadequate security hygiene (e.g. patching), delayed threat detection and response, inaction on threat intelligence, lack of active defense (e.g. threat hunting), and inability to gain return on investment from security tools. Available staff may be allocated to security only on a part-time basis (particularly at smaller utilities with fewer resources), or may be prioritizing compliance risk over security risk (e.g., focusing more on compliance evidence gathering and documentation and less on the security of non-CIP scope assets). Inadequate security staffing can exacerbate the remaining risks identified in this assessment.
The level of security risk rises with the workload related to increasingly high volumes of threat intelligence, vulnerabilities, and patches to address. Further exacerbating this is retirements of experienced staff, reductions in new hires, and a lack of security training programs and effective
16 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
knowledge transfer between personnel in the highly specialized areas of security, technology, and operational environments.
Asset Inventory Without people, process, and technology to develop and maintain accurate digital asset inventories, assets cannot be managed and monitored at the pace required by the evolving security threat. Patch monitoring and application may not be completed for assets that are not inventoried, rogue devices may be harder to detect, and asset connectivity may not be known, resulting in exposed remote access points. Legacy equipment that may lack vendor support and have vulnerabilities that cannot be patched, transient cyber assets (.i.e., maintenance laptops or other support systems that temporarily connect into operational networks or systems), and removable media may not be known and may not be securely managed as a result.
Risk is increased with high quantities and varieties of assets, high volumes of new or removed assets, and lack of robust people, process, and technology to prevent asset inventories from becoming stale.
Cloud Storage There are a number of MRO registered entities concerned with cloud technologies and ensuring an adequate control environment exists when using the cloud. While cloud storage can save costs and reduce system complexities through economies of scale, organizations must manage the adjusted risks that moving to the cloud may introduce. Cloud data storage and computing may alter an organization’s attack surface. For example, attackers or insider threats may target cloud providers to gain access to sensitive information that is not directly controlled by the data owners, therefore, accidental leakage may also occur with (or without) the data owner’s knowledge.
Risk is increased if there is a loss of provider diversity and multiple registered entities leverage a single provider, if cloud providers’ security controls and hygiene are inadequate or unknown, and if hosting locations are unknown.
Connectivity Communication networks and connectivity may be targeted by attackers for various reasons, including to disrupt operations, gain remote control capability, restrict or redirect communications, and gain access to registered entities and sensitive information.
Risk is increased with a lack of path diversity and redundancy for critical communications, unknown/unacceptable third party communication network providers, reliability/uptime standards and security controls, legacy communications protocols that may have limited security features, lack of segmentation and access controls that may allow threats to pass between networks, and inadequately designed and shared infrastructure for both operations and enterprise networks.
Risk is increased with remote connectivity for operational staff or vendors who may have unknown or inadequate security controls and hygiene. Risk is further increased without an inventory of all connectivity (i.e., connections with enterprise networks, external connections to vendors and other organizations) and awareness of any related vulnerabilities. Without knowledge of all connections, it is more difficult to adequately respond to security events if isolation from external connectivity is required, and difficult to alert connected partners who may also be impacted. Lack of plans to
17 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
address operational consequences in losing connectivity (e.g., data exchange capabilities) may increase impacts.
Copper Theft and Vandalism The MRO Region covers a large amount of remote area with some facilities hours away from the nearest law enforcement agency. Copper theft results in safety hazards to utility personnel (e.g., if grounds are removed), financial loss, equipment outages, and project delays while waiting for replacement parts. Copper theft is the most common type of physical security incident shared with the E-ISAC. Substations are the most common target for theft, placing critical assets at increased risk for vandalism and sabotage. Vandals may also physically attack power system equipment such as lines, insulators and transformers, resulting in equipment outages.
Risk is increased with shared facilities, insider threats, limited security visibility, monitoring, and access controls, lack of inventory management, tracking, and storage, and remote locations with longer response times. Risk is also influenced by the price of copper and the season.
Insider Threat Insiders can pose a substantial threat to organizations with their knowledge and access to proprietary systems that allow them to bypass security measures through legitimate means.
Risk is increased with a lack of training to recognize potential signs of an insider threat, lack of robust detection methods for unauthorized access and privilege escalation, lack of segregation of duties for potentially high risk system actions, lack of thresholds for immediate access revocation and procedures for removing access to terminated employees, lack of least privilege access to critical systems, and lack of policies and procedures to establish how an organization will address insider threats.
Mitigating System Vulnerabilities Failure to mitigate security vulnerabilities on a timely basis leaves openings that cyber attackers can exploit to cause disruption to operations, damage to assets, financial loss, and loss of sensitive information.
Risk is increased with rising volumes of patches, and high quantities and varieties of assets. Remotely located assets can present challenges, particularly with a lack of remote management solutions to safely and efficiently apply patches or inadequate staffing to manually apply patches broadly and visit difficult to reach locations. Time to address patches can be increased with software support contracts that require vendor certification of third-party patches. As well, a lack of test systems to adequately validate patches for specialized equipment that is in operation can result in potential operational risk when applying patches or delaying applying patches until planned equipment outages are available.
Saboteurs The MRO footprint has a vast amount of remote locations that may not have the same level of situational awareness. Risk is increased with insufficient situational awareness of saboteur groups and corresponding response plans and training, inadequate physical access controls, remote locations with long incident response times, and when undertaking new facility construction or building new transmission lines.
18 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
Blockades, occupations, protests, and sabotage have the potential to cause disruption to operations, damage to assets and safety risks, and adverse financial and reputational impacts.
Security Visibility, Monitoring, and Incident Response Without people, process, and technology to provide visibility and monitoring (e.g. cameras / video analytics, access controls and monitoring, security monitoring/operations centers, security information and event management systems), threats may not be detected resulting in a false sense of security, and timely incident response may not occur.
Risk is increased with a lack of coordinated and practiced incident response plans across the organization, longer response times to remote sites, and high volumes of threat intelligence and indicators of compromise without mature processes to prioritize information analysis and application.
Spear Phishing Spear phishing may lead to information leaks, credential theft, unauthorized access, malware installation including ransomware, operational impact including downtime, reputational damage, fraud and other financial loss. Spear phishing remains a common tool for attackers to gain footholds in critical environments. Social engineering and the use of compromised or spoofed email addresses and look-a-like domains contribute to the believability of phishing emails. Phishing through voice/phone spoofing, and other newer variants are emerging concerns.
Risk is increased with a lack of corporate-wide security awareness, self-phishing tests, dedicated email gateway filtering and containment controls, network segmentation, cellular phone security controls, full network visibility and monitoring, and qualified periodic internal and independent security assessments.
Supply Chain MRO registered entities highlighted that the supply chain risk is more likely to affect cyber and operational systems and less likely to impact physical systems. The risk stems from the lack of transparency from vendors about internet connected network components (i.e., Internet of Things, cameras, network devices, etc.). The supply chain risk is exacerbated due to the embedded software and individual components within a network connected device that may contain intentional or unintentional vulnerabilities including hard-coded credentials, remote access backdoors, malicious code, surveillance, and data exfiltration capability.
Risk is increased if project timelines and operational needs reduce allotted time for device vulnerability assessments, and if procurement practices do not adequately consider and weight security factors and vetting.
UASs UASs are increasingly being used for operational inspections and monitoring of remote assets. UASs may have security vulnerabilities or network configurations that could be exploited to steal sensitive information. Threat actors may use UASs to conduct surveillance of substations to gain sensitive information, and there is a potential to use UASs as weapons or to carry payloads with the potential intent to harm people and industry assets. Non-malicious users may also cause inadvertent damage. Detecting and disabling UASs is challenging and there may be little recourse when UASs are detected other than reporting to law enforcement.
19 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
Risk is increased with a lack of drone detection, and policies on reporting UASs, addressing downed UASs, and corresponding engagement with law enforcement.
The risks associated with Critical Infrastructure Dependencies are highlighted (in orange) in the Figure 8 heat map:
CRITICAL INFRASTRUCTURE INTERDEPENDENCIES 4.5 BASE LIKELIHOOD [X AXIS] BASE IMPACT [Y AXIS]
Cyber Security Risk 4
Extreme Natural Events Critical Infrastructure Interdependencies Loss of Situational Awareness Changing Resource Mix 3.5 Bulk Power System Planning Resource Adequacy and Performance
Physical Security Risks Human Performance and Skilled Workforce 3 Increasing Complexity in Protection and Control Systems
2.5
2 2 2.5 3 3.5 4 4.5
Figure 8: Critical Infrastructure Interdependencies
Impact of Natural Gas Disruption on Bulk Power System In November of 2017, NERC released a Special Assessment titled “Potential Bulk Power System Impacts Due to Severe Disruptions on the Natural Gas System.” The assessment provides an analysis of the potential impacts to bulk power system reliability as a result of a large disruption on the natural gas system. This report identified major clusters of natural gas generation and conducted a screening analysis to determine at a high level where risks may occur. One of the key findings states that:
“Natural gas facility disruptions can have varying impacts depending on geographical location and overall infrastructure dynamics. Disruptions to natural gas facilities that impact BPS reliability are highly dependent on a variety of area-specific issues, including the amount and distance from natural gas supply sources, the amount of natural gas-fired generation commonly connected to the pipeline system, resilience and preparation measures, and market and regulatory requirements.”
20 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
For example, the pipeline system in areas such as Texas–Oklahoma–Louisiana is highly interconnected, resembles more of a grid structure, is close in proximity to many supply sources, and is less vulnerable to transportation disruptions as compared to New England and portions of California and Arizona, where an outage of nearly any major natural gas facility (e.g., one interstate pipeline, key compressor station, or LNG terminal) during a summer or winter peak condition could likely lead to some level of electric generation outages. Figure 9 from NERC’s special assessment reflects the natural gas generation clusters within the U.S. and Canada:
Figure 9: Natural Gas Generation Clusters
In Figure 10 on page 22, the assessment narrowed down the clusters to perform screening in a steady state power flow. The screening process examined if the bulk power system could support the import of replacement power equal to the loss of generation that resulted from a natural gas supply disruption. An extreme scenario of a large natural gas supply disruption combined with peak load conditions was assumed. The results of the screening indicated that 18 groups of generation facilities could experience voltage and stability issues if 2 or more GW of natural gas-fired generation were to be disrupted. None of the 18 generation clusters were within the MRO Region. These results, combined with the fact that natural gas generation (nameplate MW) within MISO and SPP are projected to be the same or lower over the next 10 years (reference Table 2 in section 3.4, LTRA Report), illustrate that the risk to the bulk power system from a natural gas disruption in the MRO Region is less than in the other ERO regions. Therefore, the risk associated with Critical Infrastructure Interdependencies is determined to be lower in MRO than the NERC-wide survey average shown in Figure 8 (indicated by the downward arrow).
21 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
Figure 10: Clusters Where Bulk Power System Issues Were Identified
Critical Communication Circuit Sunset Telecommunications providers are starting to discontinue support of aging leased facilities, such as VG-36 copper four wire circuits used widely by the bulk power system industry. Specifically, these telecomm facilities are often used for SCADA, metering, or Frequency Shift Key Tone circuits used for transmission protection in high-speed pilot schemes or direct transfer trip remote clearing of transformer or breaker failure schemes. Telecommunications companies are moving towards discontinuing support of telecomm repairs, parts, and personnel related to these aging facilities, posing a risk to continued reliable operations of the bulk power system. Present protection systems require latency in the msec timeframe, while telecomm alternatives often focus on available bandwidth over latency. Further, replacement network based telecom solutions are in some cases early in development and adoption. Telecomm facility sunset schedules can be ambiguous, with the full scope of impacted equipment uncertain. As the likelihood of telecomm facility sunsets appear to be an eventuality, risk mitigation will likely include the execution of a complex communication facility replacement project over a multi-year timeframe.
Analyses of past bulk power system performance serves to document system adequacy and to identify performance trends. The NERC 2019 SOR Report2 provides these detailed analyses of past performance while providing technical support for those interested in the underlying data and detailed analytics.
2 The charts and figures in this section are pulled directly from the State of Reliability Report and used here with permission.
22 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
The SOR Report also provides objective and concise information to policymakers, industry leaders, and the NERC Board of Trustees on issues affecting the reliability, security and resilience of the North American bulk power system. Specifically, the report does the following:
Identifies system performance trends and emerging reliability risks Reports on the relative health of the interconnected system Measures the success of mitigation activities deployed
For the purposes of the RRA, the below portions are included to illustrate how the entities within the MRO Region have performed in aggregate as compared to other regions. Discussion is also provided to identify the cause for the difference, if available. In Figure 11 below, several of the data sources primarily used for the SOR Report are shown:
Figure 11: Data Sources used in the SOR Report
The SOR also contains analysis and trending of primary frequency response as measured by actual frequency excursion events on the various interconnections in North America.
Trending Misoperations Associated with Event Analysis Reports Starting in 2015, the correlation between misoperations and event analysis reports was quantified. In 2015, there were 50 transmission-related system disturbances of Category 1 or above. Of those 50 events, a total of 34 events, or 68 percent, had associated misoperations. It became clear to the ERO that if misoperations, specifically high impact misoperations such as breaker failure or bus differential misoperations, could be reduced, the number of events could also be reduced.
23 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
Since 2015, the percentage of events with associated misoperations has decreased ERO-wide. High impact misoperations such as breaker failure or bus differential misoperations have also decreased as shown in Figure 10 on page 25. However, in 2019, there were 16 transmission-related events submitted within the MRO Region, and four of these events (25 percent) had a breaker failure scheme misoperation associated with them.
In June 2017, the MRO Protective Relay Subgroup (PRS) authored a technical paper on how to improve the security of breaker failure schemes and bus differential schemes. This paper was circulated throughout the industry and presented at several reliability conferences. This is an example of how the ERO can leverage the collective knowledge and expertise of its staff and registered entities, and then with broad outreach, help to improve reliability throughout the industry. MRO is planning outreach in 2020 to revisit this technical paper in order to help entities reduce the occurrences of breaker failure scheme misoperations.
Figure 12: Trending ERO-wide Misoperations Associated with Event Analysis Reports
One of the risk elements identified by the CMEP IP was that misoperations must be thoroughly reviewed, investigated or mitigated. This holds true for not only the registered entities that experience misoperations, but also for Regional Entity staff that have an obligation to review the MIDAS submittals within their region and assure that they are properly investigated and mitigated to prevent reoccurrence. In the MRO Region, MRO staff in conjunction with the MRO PRS members, review each misoperation that occurs in the region to verify that it has been effectively studied and mitigated. Often times, MRO staff will contact the submitting entity with questions regarding the misoperation. The MRO members also actively share their misoperation experiences at in-person PRS meetings, particularly Lessons Learned or to seek out technical help or alternative mitigation
24 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
methods. This collaboration of MRO staff and members has been very successful in improving the accuracy of MIDAS submittals and has helped reduce misoperation rates steadily for four years, as shown in Figure 13:
Figure 13: MRO Misoperations Rate per Year
The LTRA Report3 is a forward looking report that essentially assesses planning reserve margins and therefore resource adequacy risk 10 years out. The report is not a prediction of future conditions, but an assessment of what changes are occurring with respect to generation and load and its availability as a capacity or load modifying resource.
As identified in section 3.1.1, planning reserve margins are highly sensitive to the assumptions that go into the calculations. Overly optimistic assumptions can introduce risk. Retirements of conventional synchronous generation and the rapid addition of variable resources, more so in some areas than others, are altering the operating characteristics of the various interconnections.
Figure 14 on page 27 shows the NERC-wide conventional generation changes from 2012 to present and projected retirements through 2029. The magnitude of coal generation retirements and the large build out of gas generation (to date) is shown. It should be noted that the projected retirements shown may be understated since generation retirements can often be announced shortly before the effective date.
3 Some of the charts and figures in this section are pulled directly from the Long-Term Reliability Assessment and are used here with permission.
25 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
Figure 14: Capacity Changes since 2012 and Retirements Projected through 2029
The MRO Region is highly impacted by resource changes. For example, the 10-year projection for natural gas generation for 2029 is less than the existing installed capacity. With other forms of conventional generation (nuclear/coal/oil) being retired over the next 10 years, it is clear that renewable/variable generation will likely make up the difference in future years.
MISO Conventional Generation Capacity Resources (MW) 2020 2029 % Diff: Coal 56,795 51,839 -8.7% Petroleum 2,982 2,668 -10.5% Natural Gas 61,526 61,564 0.1% Nuclear 12,433 11,620 -6.5% Total 133,736 127,692 -4.5% Table 1: MISO Conventional Generation Capacity Projections 2020 through 2029
SPP Conventional Generation Capacity Resources (MW) 2020 2029 % Diff: Coal 23,985 22,840 -4.8% Petroleum 1,446 1,325 -8.4% Natural Gas and Other Gases 29,583 27,776 -6.1% Nuclear 1,944 1,944 0.0% Total 56,958 53,885 -5.4% Table 2: SPP Conventional Generation Capacity Projections 2020 through 2029
26 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
The interconnection queues of SPP and MISO are used for analysis of future planning reserve margins. Resources within these interconnection queues are categorized into three tires: Tier 1 resources are typically in progress, funded, and are highly likely to occur. Tier 2 resources are typically under study for feasibility and a fair portion of Tier 2 generation projects can materialize. Tier 3 projects are typically still in the investigative stage.
Table 3 below shows the magnitude of solar and wind generation nameplate capacity in the MISO and SPP interconnection queues (Manitoba and Saskatchewan have minimal additions). It can be clearly seen with Tables 1-3, future generation in the Midwest interconnection queues almost entirely consists of renewables. To provide comparison with a region similar to MRO in terms of renewable generation prospects, ERCOT is also shown below. However, ERCOT expects to maintain the majority of their 70 GW of conventional synchronous generation through 2029.
Nameplate Capacity of Solar (MW) Nameplate Capacity of Wind (MW) Assessment Area Existing Tier 1 Tier 2 Tier 3 Total Existing Tier 1 Tier 2 Tier 3 Total MISO 280 2,040 60,125 640 63,084 19,172 7,598 27,468 5,714 59,953 Manitoba Hydro 0 0 0 0 0 259 0 0 0 259 SaskPower 0 10 20 50 80 242 0 0 400 642 SPP 276 0 650 25,307 26,233 20,486 300 2,500 31,905 55,191 ERCOT 1,857 7,699 27,376 26,155 63,087 22,090 14,457 15,191 5,864 57,602 Table 3: Solar and Wind Installed Capacity, Existing and Planned Additions through 2029
The above tables provide additional evidence that the risk identified as Changing Resource Mix in the RISC Report has significant implications within the MRO Region. The interconnection queues of MISO and SPP are predominantly filled with wind and solar interconnection requests. Any gas generation or other conventional synchronous generation in the queue is outweighed by the retirements of conventional generation. It will be imperative for the two ISO/RTOs within MRO to manage future generation installations with rigorous planning studies to continue to reliably serve and follow load within their BA and to meet performance criteria.
Overhead Transmission Line Ratings
Industry-wide, there is wide disparity regarding how static thermal ratings are calculated and used by TOs of overhead conductors. Often times, TOs do not use static winter seasonal ratings that reflect ambient winter conditions. Some TOs do not use emergency ratings and instead use normal ratings for post-contingency scenarios4. This can cause undue risk to the bulk power system as identified in the FERC January 17, 2018 Inquiry Report. TOPs that take operator action prior to when it may be necessary (e.g., system reconfiguration, de-energization of facilities, etc.) can weaken the bulk power system and divert the MW flow to other heavily-loaded transmission facilities. Overly conservative ratings can also strand available generation. Ultimately, firm load shedding can result
4 This assumes the conductor is the most limiting element, e.g., terminal equipment and relay settings associated with the line are not limiting the facility below the conductor’s thermal capability.
27 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
prior to its necessity if transmission line ratings are not accurately reflecting appropriate seasonal conditions and emergency ratings.
On September 10-11, 2019, FERC held a two-day workshop to discuss industry practices regarding the use of ambient adjusted ratings and dynamic ratings (Managing Transmission Line Ratings Report). It is likely that there will be increased activity in the area of overhead transmission ratings. Several TOs and RCs are already working together to establish and implement ambient adjusted ratings and, when appropriate, dynamic ratings for use in the operating horizon to assure efficient and reliable use of these assets. TOs should evaluate their criteria for rating overhead conductors to assure it includes, at a minimum, summer normal, summer emergency, winter normal and winter emergency ratings.
Reactive Resource Management and Voltage Stability
During the severe cold weather in the southern half of the U.S. on January 17, 2018, large amounts of natural gas generation became abruptly unavailable due to its inability to operate in the cold temperatures that occurred that day. The only option to continue to serve firm load was to transfer large amounts of power from the upper Midwest to the southern U.S. Large imports from remote areas can create a voltage stability issue that is not typically studied in the planning arena and may not be readily identified in the operation horizon. During unusually high transfer conditions, and when bulk power system voltages are declining, RCs should compare actual voltage conditions to an on- line or off-line model for that day and perform voltage stability analysis to assure they have sufficient reactive margin to avert a voltage collapse for the next most severe single contingency. RCs should communicate with their TOPs and GOPs to use all available reactive resources proactively (early) to maintain and manage voltages levels at acceptable levels. Adjacent RCs and TOPs should jointly establish a voltage stability criteria that is based on credible events when voltage stability may be at risk.
Reactive Capability of Renewable Generation
Wind and solar plants connected to the bulk power system are included in bulk power system powerflow and stability models. Sufficient model information needs to be provided by these generator owners to accurately model the reactive capabilities of these plants. For example, detailed modeling information of leading and lagging power factor capability will be needed to assure that a given transmission bus voltage schedule can be maintained under various steady state conditions. Similarly, accurately modeling the plant’s dynamic reactive capability to help maintain voltage stability under various contingencies will be necessary to assure that the bulk power models are accurately reflecting the actual dynamic reactive capability of these renewable energy plants.
Accuracy of Bulk Power Models (Powerflow and Short Circuit)
The PCs and Transmission Planners presently build their portion of the Eastern Interconnection (EI) powerflow and stability models, per Reliability Standard TPL-001-5. An EI-wide model is then assembled by a third party, presently the Multi-Regional Modeling Working Group. There is discussion of the EI-wide model building responsibility being transitioned to a different independent entity. The integrity of the models must be maintained through all of these transitions. When combined with the difficulties of accurately modeling renewable generation, load, interchange assumptions between each PC model, etc., the effort will be challenging and may be subject to errors and inaccuracies. Similarly, short circuit models are subject to reduced short circuit strength,
28 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
lack of negative sequence current from inverter-connected generation, etc., presenting challenges for protection engineers to continue to provide dependable and secure protection to the bulk power system in the future.
As part of its 2020 CMEP IP (CMEP) Implementation Plan (IP), the ERO establishes Risk Elements that it uses to identify and prioritize interconnection and continent-wide risks to the reliability of the bulk power system. The ERO Risk Elements establish a continent-wide risk baseline, and the risks discussed in the RRA may further refine MRO’s compliance monitoring and oversight activities. When risks are identified in the MRO Region that may have an impact at the ERO level, the RRA is the conduit to provide this risk feedback through the annual CMEP Implementation Plan development process. While MRO will determine individual monitoring decisions for each registered entity based on its unique characteristics, registered entities should consider the Risk Elements identified in the 2020 CMEP IP and the associated areas of focus, as well as specific regional risks identified in the MRO RRA, as they evaluate opportunities to prioritize and enhance their internal controls and compliance operations focus.
The 2020 ERO Risk Elements are shown side by side with the 2019 ERO Risk Elements:
Comparison of 2019 Risk Elements and 2020 Risk Elements
2019 Risk Elements 2020 Risk Elements
Improper Management of Employee and Insider Management of Access and Access Controls Access
Insufficient Long-Term Planning Due to Inadequate Models Insufficient Long-Term and Operations Planning Due to Inadequate Models Insufficient Operational Planning Due to Inadequate Models
Spare Equipment with Extended Lead Time Loss of Major Transmission Equipment with Extended Lead Times
Inadequate Real-time Analysis During Tool and Inadequate Real-time Analysis During Tool and Data Data Outages Outages
Improper Determination of Misoperations Improper Determination of Misoperations
Gaps in Program Execution Gaps in Program Execution
Inhibited Ability to Ride Through Events (Texas RE: Resource Adequacy)
Table 4: Comparison of CMEP Risk Elements, 2019 vs. 2020
29 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
In order to evaluate progress toward a key reliability goal of less severe instances of noncompliance, MRO developed the Compliance Severity Index (CSI) to represent the total risk that all instances of noncompliance present to the reliability and security of the bulk power system in the MRO Region. MRO Risk Assessment and Mitigation staff undertake a rigorous process to evaluate each instance of noncompliance, based upon an analysis of the facts and circumstances, to determine the potential and actual risk to the reliability and security of the bulk power system. The product of this evaluation is a Risk Determination with an assigned risk level of Minimal, Moderate, or Serious. MRO uses the Risk Determination and the finding discovery method (Audit Finding, Self-Certification, Self-Report, etc.) to calculate the CSI.
MRO has mapped all historic instances of noncompliance into the current, equivalent Reliability Standards and requirements. This allows analysis of the same risk associated with varying instances of noncompliance, regardless of new associated Reliability Standards or requirements.
Figure 15 provides the fifteen highest risk requirements based on the Total CSI, which reflects noncompliance history in the entire MRO Region back to 2007. Figure 16 provides the fifteen highest risk requirements for the past three years, allowing MRO to shape oversight on more recent trends.
MRO uses the CSI to evaluate trends in instances of noncompliance of higher risk requirements. If a requirement indicates a year over year increase in total CSI, MRO may prioritize the oversight for that requirement through modification of an entity's Compliance Oversight Plan. 15 Highest Risk Requirements based on Total 450 Compliance Severity Index 400 (2007 - 2019) 350 Minimal Moderate Serious 300 250 200 150 100 50
Compliance Severity Severity IndexCompliance 0
NERC Reliability Standards and Requirements
Figure 15: Fifteen Highest Risk Requirements Based on Total CSI
30 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
Figure 16: Fifteen Highest Risk Requirements- CSI for Last Three Years
FAC, TOP, and IRO Operating Limits Reliability Standards As part of Project 2014-03, several FAC, TOP, and IRO-related Reliability Standards became subject to enforcement in 2017. The project was developed to address FERC technical concerns identified in the 2013 NOPR, including operating within System Operating Limits and Interconnection Reliability Operating Limits.
Revised Reliability Standards enforceable January 1, 2017:
IRO-001-4 Reliability Coordination Data Specifications and Collection; and TOP-003-3 Operational Reliability Data.
Revised Reliability Standards enforceable April 1, 2017:
FAC-010-3 System Operating Limits Methodology for the Planning Horizon; FAC-011-3 System Operating Limits Methodology for the Operating Horizon; IRO-001-4 Reliability Coordination – Responsibilities; IRO-002-4 Reliability Coordination - Monitoring and Analysis; IRO-008-2 Reliability Coordination Operational Analysis Real-time Assessments; IRO-014-3 Coordination Among Reliability Coordinators; IRO-017-1 Outage Coordination; TOP-001-3 Transmission Operations; and TOP-002-4 Operations Planning.
31 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
Included within these Reliability Standards is a new requirement that TOPs ensure a Real Time Assessment (RTA) is performed at least every 30 minutes. Many medium and small TOPs in the MRO footprint have historically relied on their RC to provide this technical support. These entities are at risk of having issues if their RC’s RTA or communication capabilities prevent the TOP system operators from ensuring the RTA is performed every 30 minutes. MRO has increased its oversight and outreach to educate entities on this new requirement. Possible remedies to manage risk include having a TOP run its own RTA, or contracting backup and primary RTA to a neighboring TOP or other third party.
GMD Reliability Standards EOP-010-1 and TPL-007-1 The enforceable standards EOP-010-1 and TPL-007-1 were developed to address the impact of potential geomagnetic disturbances to the bulk power system. Geomagnetic disturbances are a temporary disturbance of the Earth's magnetosphere caused by a solar wind shock wave or cloud of magnetic field, which interacts with the Earth's magnetic field. Geomagnetically induced currents (GICs) are electrical currents induced as a result of a GMD.
GMDs are considered a risk in the MRO Region because:
1. Geographic location in the northern hemisphere where the anticipated effects of GMD are significant; 2. The resistivity of the Earth is high in parts of the MRO Region based on its geology; and 3. The bulk power system in the MRO Region largely consists of high voltage and lengthy transmission lines, which are most susceptible to GMD.
Because GMDs are low frequency, high impact events, operating entities have little or no experience mitigating the effects. Therefore, studying and planning for operations during such events is critical.
GICs can have direct near-term impact on the bulk power system by potentially causing excessive VAR demand that can result in voltage collapse and possible damage to transformers and protection system misoperations.
The long-term impact of GMD is damage to large power transformers, which can be devastating given the long lead-time to manufacture BES transformers. It is critical that large transformers are protected against such an event since maintaining spare transformers in large quantities is not a cost effective option. Until permanent protection equipment and system design changes can be put in place per TPL-007-1 requirements, operators will be required to maintain temporary mitigation and actions per EOP-010-1, including implementation of a GMD Operating Plan.
Model Data Reliability Standards Modeling, Data, and Analysis (MOD) Reliability Standards, MOD-032-1 and MOD-033-1 provide a framework for model data requirements and reporting procedures to support the creation and validation of interconnection-wide, steady state, dynamic, and short-circuit models. The cases developed through these processes form the foundation of power system analysis used to plan and operate the bulk power system.
MOD-032-1, Data for Power System Modeling and Analysis, requires Planning Coordinators (PCs) and TPs to jointly develop model data requirements and reporting procedures. Other requirements mandate applicable entities to provide the data specified under R1 to their PC and TP. Additionally,
32 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
the PC is required to provide its models to the entity or entities designated by NERC in support of interconnection-wide case creation.
MOD-033-1, Steady State and Dynamic System Model Validation, complements MOD-032-1 and ensures that models used to plan and operate the bulk power system accurately reflect reality. The PC is required to develop and implement a process to validate its steady state and dynamic models, while the RC and TOP must provide data to the PC as needed to perform the validation. Of particular interest is the validation of model frequency response, as incorrect load models, governor models, or plant-level control models can have a significant effect on a case’s response to an event. When performing validation under MOD-033-1, the focus is on validating the PC’s portion of its existing system.
Several other NERC Reliability Standards are also closely related to the model validation work performed under MOD-033-1. PRC-002-2, Disturbance Monitoring and Reporting Requirements, ensures that PCs will have recorded data available to facilitate analysis of BES disturbances. TOs are required to determine where disturbance monitors will be located, and the TOs and GOs must provide captured data to their PCs. PRC-002-2 provides an avenue for the PC to collect data for model validation under MOD-033-1.
Reliability Standards, MOD-026-1 and MOD-027-1 are parallel generator model verification Reliability Standards that require the GOP to perform verification of its exciter and governor models. The GOP chooses its component models from those allowed by the TP (as detailed in MOD-032), and compares an individual unit’s simulated response against the actual response to either a system disturbance or a staged test. The verification performed under MOD-026-1 and MOD-027-1 covers the individual components of a generator model, while MOD-033-1 validates the system-wide generator and BES facility response. MOD-025-2 requires the verification and reporting of a unit’s Real and Reactive Power capabilities. The MOD-025-2, MOD-026-1, and MOD-027-1 Reliability Standards require the GOP to verify a percentage of its units or MVA over a phased-in implementation period.
MRO has observed an increase in the number of issues associated with data verification standards over the last three years, principally due to a lack of clarity specifying what is due and when. This is not necessarily unique to the MOD standards, as effective dates of requirements are not always clear across the suite of NERC standards, particularly when complex implementation plans are involved. In response to this observed trend across the ERO, a Compliance Practice Guide was developed to provide clarity on phased implementation plans.
Understanding the many relationships between the RC, PC, TP, TO, and GO will be a critical aspect of successful model creation and validation. Obtaining assurance that this coordination is taking place will be a specific focus area for compliance monitoring activities in the MRO Region.
Planning Standard TPL-001-4 Reliability Standard TPL-001-4 establishes the requirements needed to develop a BES that will operate reliably over a broad spectrum of system conditions following a wide range of probable events. The transmission planning and system analysis of the BES is noteworthy to the MRO Region primarily for the following reasons:
1. The current standard lacks a requirement for PCs to jointly conduct system analysis as a whole system within the Eastern Interconnection; and
33 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
2. Market-based dispatch is not required to be modeled, and analyzed.
The TPL-001-4 standard requires the distribution of planning assessment results to adjacent PCs and adjacent TPs, but does not require entities to coordinate analysis jointly within an interconnection. The coordination requirement that was previously included in the retired “fill-in-the- blank” TPL-005 standard was not mapped to the new standard. While there is a history of coordination between PCs within the MRO footprint, there is a potential lack of assurance that the planning events in TPL-001-4, Table 1 are being studied beyond the PCs’ borders.
The absence of a market based dispatch requirement in TPL-001-4 specific to the treatment of imports and exports should be reexamined in order to ensure consistency with the market flow- based systems utilized by ISO/RTOs. Closer alignment of flows experienced in real-time operations will ensure that power flow models utilized in transmission planning are reflective of actual system conditions. This risk could be addressed by considering such real-time flows as a dispatch scenario in TPL-001-4’s sensitivity studies. Imports and exports are critical components of the power transfer calculation because they have significant impacts on flow gates and loop flows. Real-time monitoring tools have limited capabilities to monitor stability impacts. By incorporating impactful reoccurring real-time dispatch constraints in the planning world, planners can confirm there are no unknown stability constraints in the real-time dispatch conditions. It is important to note that TPL-001-4 does not require that Corrective Action Plans(s) be developed to meet the performance requirements of a single sensitivity case, but provides an opportunity to coordinate between planning and operations to improve reliability. This concept is related to the proposed revisions to the FAC standards, where RCs (operations) receive input from PCs (planning) to determine System Operating Limits (SOLs).
New Reliability Standards Enforceable in 2020 CIP-013-1 – U.S. Enforceable Date July 1, 2020. CIP-013-1 is a new standard developed as part of Project 2016-03 Cyber Security Supply Chain Risk Management. The purpose of CIP-013-1 is to mitigate cyber security risks to the reliable operation of the BES by implementing security controls for supply chain risk management of BES Cyber Systems.
PRC-027-1 – U.S. Enforceable Date October 1, 2020. PRC-027-1 is a new standard developed as part of Project 2007-06 System Protection Coordination. The purpose of PRC-027-1 is to maintain the coordination of Protection Systems installed to detect and isolate Faults on BES Elements, such that those Protection Systems operate in the intended sequence during faults. PRC-027-1 also introduces a new term that will be included in the Glossary of Terms Used in NERC Reliability Standards:
Protection System Coordination Study: An analysis to determine whether Protection Systems operate in the intended sequence during Faults.
As part of Project 2007-06, and in conjunction with the approval and adoption of TOP-009-1, Reliability Standard PRC-001-1.1(jj) will be retired.
Oversight of “Risks in Aggregate” As part of the ongoing refinement of Risk Based Monitoring, MRO has developed a new initiative to address aggregated risks across multiple low inherent risk registered entities. The intent of the initiative is to identify any critical Reliability Standards that, when evaluated across multiple low inherent risk registered entities, rises to a level that supports active monitoring. This risk in aggregate concept would primarily affect those entities whose Compliance Oversight Plans (COPs)
34 2020 MRO Regional Risk Assessment 2019 MRO REGIONAL RISK IDENTIFICATION
do not presently include any standards or requirements for monitoring, typically either small GO/GOPs or small TO/TOPs. Some examples of the types of risk MRO has observed which are elevated when evaluated in aggregate may include; voltage control and ride through, protection system settings and maintenance, frequency response and ride through, and response to directives.
MRO’s initial evaluation of aggregated risk to be considered for oversight in 2020 includes four focused risks for this initiative:
1. Cyber Security for Low Impact BES Cyber Systems 2. Protection System Maintenance 3. Generator Frequency and Voltage Relay Settings 4. Generator Operation for Maintaining Network Voltage Schedules
As a pilot program to address this risk, MRO intends to conduct a series of Self-Certifications targeting those entities where individual inherent risk is low, but may be impactful to the bulk power system when evaluated in aggregate with other entities.
35 2020 MRO Regional Risk Assessment CONCLUSION
4. CONCLUSION
The 2020 MRO Regional Risk Assessment identifies key risks that are brought forth from analysis of ERO-wide assessments and reports to identify MRO regional differences. The ERO sources used for this year’s RRA include:
2019 ERO RISC Priorities Report 2020 ERO Compliance Monitoring and Enforcement Program Implementation Plan 2019 NERC State of Reliability Report 2019 NERC Long-Term Reliability Assessment
In addition to assessing which ERO-wide risks may be more (or less) applicable to the MRO Region, MRO staff leveraged the expertise of the three MRO advisory councils (Reliability, Compliance Monitoring and Enforcement Program (CMEP), and Security) to identify any additional risks that the MRO region may be susceptible to experiencing.
Each year, MRO staff will continue to conduct and publish a regional risk assessment to identify and monitor risks posed to the bulk power system within the MRO region, and ensure that efforts are properly focused in order to best monitor and mitigate those risks. MRO staff will communicate key findings and recommendations to registered entities and NERC in order to promote effective mitigation activities, future standards development, and other process improvements to improve the reliability of the bulk power system.
36 2020 MRO Regional Risk Assessment References
REFERENCES
1. 2019 ERO Reliability Risk Priorities Report https://www.nerc.com/comm/RISC/Related%20Files%20DL/RISC%20ERO%20Priorities%20 Report_Board_Accpeted_November_5_2019.pdf#search=ERO%20RISC 2. 2020 ERO Compliance Monitoring and Enforcement Program Implementation Plan https://www.nerc.com/pa/comp/Reliability%20Assurance%20Initiative/2020%20ERO%20CMEP% 20Implementation%20Plan%20V%201.0.pdf 3. 2019 NERC State of Reliability Report https://www.nerc.com/pa/RAPA/PA/Performance%20Analysis%20DL/NERC_SOR_2019.pdf 4. 2019 NERC Long-Term Reliability Assessment https://www.nerc.com/pa/RAPA/ra/Reliability%20Assessments%20DL/NERC_LTRA_2019.pdf 5. NERC Event Analysis Program https://www.nerc.com/pa/rrm/ea/Pages/EA-Program.aspx 6. 2016 South Australia Blackout Report https://www.aemo.com.au/- /media/Files/Electricity/NEM/Market_Notices_and_Events/Power_System_Incident_Reports/2017 /Integrated-Final-Report-SA-Black-System-28-September-2016.pdf 7. NERC Standard PRC-024-2, “Generator Frequency and Voltage Protective Relay Settings” https://www.nerc.com/pa/Stand/Reliability%20Standards%20Complete%20Set/RSCompleteSet.p df 8. NERC Lessons Learned: Loss of Wind Turbines due to Transient Voltage Disturbances on the Bulk Transmission System https://www.nerc.com/pa/rrm/ea/Lessons%20Learned%20Document%20Library/LL20170701_Lo ss_of_Wind_Turbines_due_to_Transient_Voltage_Disturbances.pdf 9. NERC Alert “Loss of Solar Resources during Transmission Disturbances due to Inverter Settings” https://www.nerc.com/pa/rrm/bpsa/Alerts%20DL/NERC%20Alert%20Loss%20of%20Solar%20Re sources%20during%20Transmission%20Disturbance.pdf 10. NERC/FERC January 17, 2018 Cold Weather Event Inquiry Report https://www.ferc.gov/legal/staff-reports/2019/07-18-19-ferc-nerc-report.pdf 11. January 17, 2018 Weather Map, Source of Data: NOAA weather data, prepared using ABB Ventyx Velocity Suite© software 12. NERC 2017 Special Assessment “Potential Bulk Power System Impacts Due to Severe Disruptions on the Natural Gas System” https://www.nerc.com/pa/RAPA/ra/Reliability%20Assessments%20DL/NERC_SPOD_11142017_ Final.pdf 13. NERC Standard TPL-001-5, “Transmission System Planning Performance Requirements” https://www.nerc.com/pa/Stand/Reliability%20Standards%20Complete%20Set/RSCompleteSet.p df 14. MRO Standard Application Guides https://www.mro.net/assurance/StandardsandRules/Pages/Standards-Guidance.aspx 15. Eastern Interconnection Planning Collaborative https://eipconline.com/
37 2020 MRO Regional Risk Assessment References
16. MRO Phase II Misoperations White paper https://www.mro.net/MRODocuments/PRS%20Phase%20II%20Misoperations%20White%20Pap er%206-26-17%20Final.docx.pdf 17. Managing Transmission Line Ratings https://elibrary.ferc.gov/idmws/common/downloadOpen.asp?downloadfile=20190823%2D4002% 2833750704%29%2Epdf&folder=16052723&fileid=15333745&trial=1
38