Communications and Networking for Smart Grid Systems Dusit Niyato Nanyang Technological University (NTU), Singapore Rose Qingyang Hu Utah State University Ekram Hossain University of Manitoba, MB, Canada Yi Qian University of Nebraska-Lincoln

IEEE GLOBECOM 2011, Houston, USA December 9, 2011

1 Tutorial Outline 1. Introduction, Background, and Overview of Smart Grid Systems 2. Data Communication Requirements in Smart Grid 3. Communication Architectures, Area Networks, and Components for Smart Grid 4. Data Communications and Networking in Smart Grid 5. Cyber Security and Privacy in Smart Grid Communications Infrastructure 6. Field Trials and Case Studies 7. Open Issues and Future Research Directions 8. Summary

IEEE GLOBECOM'11 2 Introduction • What is smart grid? – Smart grids – add communication capabilities and intelligence to traditional grids • What enables smart grids? – Intelligent sensors and actuators – Extended data management system – Expanded two way communications between power generation, distribution, and customers – Network security – etc.

IEEE GLOBECOM'11 3 Smart Grid: The “Energy Internet”

2-way flow of electricity and information

Standards Provide a Critical Foundation 4 Transition

• Transition from traditional power grid to smart grid

IEEE GLOBECOM'11 5 Motivations

Smart Grid Enables: • Higher Penetration of Renewables • Smart Charging of Electric Vehicles • Consumers to Control Energy Bills • Efficient Grid Operations & Reduced Losses • Reduced Distribution Outages • Improved System Reliability & Security

IEEE GLOBECOM'11 6 Primary objectives

• National integration

• Self healing and adaptive –Improve distribution and transmission system operation

• Allow customers freedom to purchase power based on dynamic pricing

• Improved quality of power-less wastage

• Integration of large variety of generation options

IEEE GLOBECOM'11 7 Economic and social benefits

• Provide Customer Benefits • Reduce Peak Demand • Increase Energy Conservation & Efficiency • Reduce Operating Expenses • Increase Utility Worker Safety • Improve Grid Resiliency and Reliability • Reduce Greenhouse Gas Emissions • Promote Energy Independence • Promote Economic Growth & Productivity

IEEE GLOBECOM'11 8 More Background on Smart Grid

IEEE GLOBECOM'11 9 Distributed Generation • Hybrid Energy Resource – Fossil-Fuel – Wind – Solar – Bio-Mass – Batteries – Capacitors – Flywheel – Etc.

IEEE GLOBECOM'11 10 Smart Metering • Automatic Metering – Automatic Meter Reading (AMR) – Automated Metering Management (AMM) – Advanced Metering Infrastructure (AMI) Example smart metering systems:

(a) Google PowerMeter (b) Microsoft Hohm IEEE GLOBECOM'11 11 Intelligent electronic devices (IEDs)

• Protection relay • Auxiliary relay • Cheap contractors • Remote terminal units • Circuit breaker monitor • Revenue meters • Solar flare detectors • Power quality monitors GE CFD Intel 4004 • Phasor measurement units • Communication processors • Communication alarm • Etc.

12 Monitoring and Controlling

– Supervisory Control And Data Acquisition (SCADA) – Energy management system (EMS) – Information and Communications Technology (ICT)

IEEE GLOBECOM'11 13 Telecontrol

• Different protocols for different operations – Proprietary protocols (more than 100) – Standards • SCADA • Modbus • DNP • IEC61850

IEEE GLOBECOM'11 14 SCADA Protocols

• Siemens quad 4 meter • CONITEL 2000 • CONITEL 2100 • CONITEL 3000 • CONITEL 300 • HARRIS 5000 • HARRIS 5600 • HARRIS 6000 • UCA 2.0 or MMS • PG & E 2179 • MODBUS • DNP3 • ICCP • IEC 61850

IEEE GLOBECOM'11 15 General Protocols

• MODBUS -Primitive without security and not very extensible • DNP3 –Advanced SCADA protocol • DNP1 and 2 are proprietary protocols • IEC 61850 the most used protocol for new implementations • ICCP

IEEE GLOBECOM'11 16 Groups working on smart grids

• UCA International user group  www.ucaiug.org • International electrochemical commission  www.iec.ch • Electric power research institute  www.epri.com • Intelligrid consortium and architecture  www.intelligrid.epri.com • IEEE smart grid  www.smartgrid.ieee.org • NIST  csrc.nist.gov

IEEE GLOBECOM'11 17 Smart Grid Communication Architecture

IEEE GLOBECOM'11 18 NIST Conceptual Reference Model for Interoperability

IEEE GLOBECOM'11 19 Communication Media • Urge for new FCC allocation for smart grids • PLC –Power line carriers • Ethernet • WLAN • ZigBee • Bluetooth • Optical fiber • Microwave • Etc.

IEEE GLOBECOM'11 20 Data Communications in Smart Grid

Optimal Network(s) • Broadcast data (Demand Response, price signals, emergency events, etc.) – Low volume, infrequent Broadband – Can use currently available communication infrastructure (cellular, broadband, WiFi, Pager) Cellular WiFi, etc. with standard internet security measures • Real-time Consumption Data (high volume, frequent) – Useful primarily for real-time control & usage information to consumer – We favor meter  premises where displays & controllers can locally act upon this data along Direct meter with pricing information to HAN • Minimizes risk (privacy & network stability) and maximizes benefit from real time info. • Raw Billing Data (reading when price changes) AMI – Utility operations • Aggregate Data – Comparison over time & among neighbors, best practices, consumption pattern recognition, suggest corrective actions, etc. • Utility or third party cloud-based applications operating on anonymous summary data • Little risk for privacy or network stability in case of breach of security Internet portal • Can use standard internet communication with standard security measures

• T&D: relatively few points (substations); mission-critical, but already connected Existing connectivity

IEEE GLOBECOM'11 21 Match Info To The Communications Medium

Information Smart Grid Signals Detailed Consumption Data category Examples ToU pricing, critical peak pricing, Periodic meter readings (e.g., once reliability, carbon content, etc. a minute) Location of Utility servers connected to Internet Embedded meter hardware information

Evolution potential High, as new applications arise (e.g., Very low PHEVs, micro-grids)

Optimal approach

AMI-centric approach

Communication General telecom infrastructure Specialized embedded hardware medium (Internet): broadband, cellular, (short-range radio, power-line municipal WiFi, etc. carrier, etc.)

IEEE GLOBECOM'11 22 Priority and types of information

Communication model , source: NIST Vol 1

IEEE GLOBECOM'11 23 Data Communication Requirements in Smart Grid

IEEE GLOBECOM'11 24 Requirements • Latency • Bandwidth • Interoperability • Scalability • Security • Standardization

IEEE GLOBECOM'11 25 Latency

• The real-time operational data communications in smart grid include online sensor/meter reading and power system control signals. • The communication is characterized by the fact that most of interactions must take place in real time, with hard time bound. • The communication requirements define the design of the technical solutions. • For real-time sensing/metering purposes, reading messages should be transmitted within a very short time frame. – For instance, the maximum allowed time is in the range of 12-20 ms, depending on the type of protection scheme which origins from the fact that the disconnection of fault current should within approximately 100 ms. • Power System Control signals mainly include supervisory control of the power process on secondary or higher levels. These systems are of the kind SCADA/EMS. – Measured values must not be older than 15 seconds, when arriving at the control center. Breaking information shall arrive no later than 2 seconds after the emergency event has occurred

IEEE GLOBECOM'11 26 Bandwidth

• As more and more interconnected intelligent elements are added to the electricity network with the evolution of the smart grid, the communication infrastructure should be able to transport more and more messages simultaneously without severe effect on latency. • The network bandwidth must increase faster than the demand of these interconnected intelligent elements in the network. • An Example: (A. Aggarwal, S. Kunta, P. K. Verma, “A proposed communications infrastructure for the smart grid,” in Innovative Smart Grid Technologies (ISGT), 2010, pp. 1-5.) – Model the communication bandwidth requirements for a moderate size electricity distribution system. In this model, a distribution substation is connected to 10,000 feeders and each feeder connects to 10 customers. – Assuming that every electric meter generates a message every second to the distribution substation, the total is 100,000 messages per second. The feeders themselves will generate messages to each other and to the distribution substation. – The authors in this paper modeled the messages in the smart grid arriving at servers located at the control center as M/M/1 traffic. Then, the transmission line bandwidth is evaluated over 100 Mbps through the M/M/1 queuing model. It can be observed that this situation results in a very poor bandwidth utilization of the transmission facilities as well. – Unfortunately, a higher level of utilization will not permit meeting the assumed latency constraint.

IEEE GLOBECOM'11 27 Interoperability

• The ability of 2 or more networks, systems, devices, applications, or components to communicate & operate together effectively, securely, & without significant user intervention – Communication requires agreement on a physical interface & communication protocols – Exchanging meaningful & actionable information requires common definitions of terms & agreed upon responses – Consistent performance requires standards for the reliability, integrity, and security of communications – Interoperability may include: • “Plug and play”: connect them & they work together • Interchangeability: Ability to readily substitute components

IEEE GLOBECOM'11 28 Interoperability (cont‟d)

IEEE GLOBECOM'11 29 Scalability • Phenomena in Smart Grid Communication

IEEE GLOBECOM'11 30 Standards • EISA 2007 Directs National Institute of Standards & Technology (NIST) to: – Coordinate the development of model standards for interoperability of smart grid devices and systems • Create flexible, uniform, and technology neutral standards • Enable traditional resources, distributed resources, renewables, storage, efficiency, and demand response to contribute to an efficient, reliable grid • EISA Directs FERC, when sufficient consensus, to: – Adopt standards necessary to insure smart-grid functionality and interoperability in the interstate transmission of electric power, and regional and wholesale electricity markets – EISA did not expand FERC‟s Federal Power Act authority to enforce standards • State Commissions: – May adopt standards by regulation, separately or in parallel with FERC – May consider standards when approving utility investments • Considerations for Regulators: – Ensuring interoperability & security, without impeding innovation – Consistent action will influence the vendor community – Vendors often will follow standards that are not legally mandated – SGIP standards reflect efforts to build broad stakeholder consensus IEEE GLOBECOM'11 31 Standardization (cont‟d)

• IEEE – IEEE P2030 • Power Engineering Technology • Information Technology • Communications Technology • IEC – IEC 61968 - Distribution Management – IEC 61970 - Common Information Model – IEC 60870 - Inter-control Center Communication Protocol – IEC 62210 - Data and Communication Security – IEC 62357 - Reference Architecture – IEC 61850 - Standard for Design of Substation Automation • IEC 61850-7-420 - Integration of Distributed Energy Resources • IEC 61850-7-410 - Integration of Hydro Resources – IEC 61400 - Integration of Wind Farms to Utility Communication Network – IEC 62056 - Communication

IEEE GLOBECOM'11 32 Security

• DISA Security Technical Implementation Guides (STIGs) • FIPS 201 • North American Electrical Reliability Corporation-Critical Infrastructure Protection (NERC CIP) • National Infrastructure Protection Plan (NIPP) • IEEE 1402 • International Society of Automation(ISA) • ISO 17799 • NIST GWAC – DEWGs • Home-to-Grid (H2G) • Building-to-Grid (B2G) • Industrial-to-Grid (I2G) • Transmission and Distribution (T&D) • Business and Policy (B&P)

IEEE GLOBECOM'11 33 Summary

CURRENT FUNCTIONAL REQUIREMENTS Application Security Bandwidth Reliability Coverge Latency Back-up Power Advanced Metering Infrastructure High 14-100 kbps per node 99.0-99.99% 20-100 % 2000 ms 0-4 hours AMI Network Management High 56-100 kbps 99.00% 20-100% 1000-2000 ms 0-4 hours Automated Feeder Switching High 9.6-56 kbps 99.0-99.99% 20-100% 300-2000 ms 8-24 hours Capacitor Bank Control Medium 9.6-100 kbps 96.0-99.00% 20-90% 500-2000 ms 0 hours Charging Plug-In Electric Vehicles Medium 9.6-56 kbps 99.0-99.90% 20-100% 2000 ms - 5 min. 0 hours Demand Response High 56 kbps 99.00% 100% 2000 ms 0 hours Direct Load Control High 14-100 kbps per node 99.0-99.99% 20-100 % 2000 ms 0-4 hours Distributed Generation High 9.6-56 kbps 99.0-99.99% 90-100% 300-2000 ms 0-1 hour Distribution Asset Management High 56 kbps 99.00% 100% 2000 ms 0 hours Emergency Response Medium 45-250 kbps 99.99% 95% 500 ms 72 hours Fault Current Indicator Medium 9.6 kbps 99.00-99.999% 20-90% 500-2000 ms 0 hours In-home Displays High 9.6-56 kbps 99.0-99.99% 20-100% 300 -2000 ms 0-1 hour Meter Data Management High 56 kbps 99.00% 100% 2000 ms 0 hours Network Protection Monitoring Medium - High 56-100 kbps 99.00-99.999% 100% 2000-5000 ms 0 hours Outage Management High 56 kbps 99.00% 100% 2000 ms 0 hours Price Signaling Medium 9.6-56 kbps 99.0-99.90% 20-100% 2000 ms - 5 min. 0 hours Real-time Pricing High 14-100 kbps per node 99.0-99.99% 20-100 % 2000 ms 0-4 hours Remote Connect/Disconnect High 56-100 kbps 99.00% 20-100 % 2000-5000 ms 0 hours Routine Dispatch Medium 9.6-64 kbps 99.99% 95% 500 ms 72 hours Transformer Monitoring Medium 56 kbps 99.00-99.999% 20-90% 500-2000 ms 0 hours Voltage and Current Monitoring Medium 56-100 kbps 99.00-99.999% 100% 2000-5000 ms 0 hours Workforce Automation Medium 256-300 kbps 99.90% 90% 500 ms 8 hours National Broadband Plan: RFI Communications Requirements Comments of Utilities Telecom Council, July 12, 2010

IEEE GLOBECOM'11 34 Challenges for Smart Grid Communication Infrastructure

• Complexity • Efficiency • Reliability • Security

IEEE GLOBECOM'11 35 Complexity

• Need to support multi-physics approach • Need to support multidisciplinary approach • Need to support dynamic and reconfigurable model level definition • Need to provide visualization to support system analysis • Need to provide support for uncertainty propagation

IEEE GLOBECOM'11 36 Efficiency

• Better Telemetry • Faster Controls • More Robust Controls • Embedded Intelligent Devices Communication • Integrated And Secure Communications • Enhanced Computing Capabilities • Internet Technology

37 Reliability • Renewable Resources • Demand Response • Load Management • Storage Devices

IEEE GLOBECOM'11 38 Security

• Information security domains – Public, supplier, maintainer domain – Power plant domain – Substation domain – Telecommunication domain – Real-time operation domain – Corporate IT domain • SCADA – De-coupling between operational SCADA/EMS and admin IT – Governmental coordination on SCADA security • Threats to – AMI (similar to WSN) – SCADA

IEEE GLOBECOM'11 39 Communication Architectures • Communication Architecture and Model for Distribution Network • Home-Area Networks (HANs) • Neighborhood-Area Networks (NANs) • Wide-Area Networks (WANs) • Sensor and Actuator Networks (SANETs)

IEEE GLOBECOM'11 40 Communication Architectures Communication Architecture and Model for Distribution

n e o r

Network i t u t a c c i u n r t u

Control Center Control Center Control Center s a m r f m n o I C Customer Networks Communication Core Network E.g., ZigBee, WiFi, E.g., TCP/IP Network, WiMax, Cellular (GSM or CDMA), Ethernet PLC

Smart Meter/ Sensor Network DAU/ HAN GW MDMS NAN GW

e

l r Transmission a u t Distribution c Substation i c r u Feeder t r HAN c t Transmission e s l Distribution a Feeder E r Substation f n Wind Solar I Turbines WAN Energy NAN Generation Tansmission and Distribution Customer Premises Last Mile Connection

Advanced Metering Infrastructure (AMI) Electric Flow Information Flow

Legends: DAU=Data Aggregator Unit, MDMS=Meter Data Management System, HAN=Home Area Network, NAN=Neighborhood Area Network, WAN=Wide Area Network , GW = Gateway IEEE GLOBECOM'11 41 Communication Architectures Communication Architecture and Model for Distribution Network • Smart grid follows the same electrical architecture • Electricity is delivered from the generation to consumers through transmission and distribution substations • Transmission substation delivers electricity from power generation plant over a high voltage transmission line (over 230kV) to the distribution substation • Distribution substation converts the electric power to medium voltage level • Distribution feeder then converts the medium voltage to lower level for distributing to the consumer‟s end

IEEE GLOBECOM'11 42 Communication Architectures Customer Premise and Customer Network

Control Control Center Center

MDMS MDMS

DAU/NAN GW DAU/NAN GW

Bluetooth/ZigBee/ WiFi HAN1 HAN3 HAN1 HAN3 HAN2 HAN2 NAN2 NAN1 Smart Meter BACnet, KNX, Display (HAN Gateway) PLC protocol

Smart Devices (e.g., AC) With Sensors HAN3

IEEE GLOBECOM'11 43 Communication Architectures Customer Premise and Customer Network • Data aggregator unit (DAU) also referred to as NAN GW acts as a data sink to collect and relay the information from the consumer side to meter data management system (MDMS) • MDMS will provide storage, management, and processing of meter data for proper usage by other power system applications and services

IEEE GLOBECOM'11 44 Communication Architectures Home-Area Networks (HANs) • HAN (sometimes referred to as Premise Area Network (PAN) or a Building Area Network (BAN)) is the smallest subsystem in the hierarchical chain of smart grid • HAN provides a dedicated demand side management (DSM), including energy efficiency management, and demand response by proactive involvement of power users and consumers • HAN consists of smart meter, smart devices with sensors and actuators, and in-home display for energy management system (EMS) – EMS will provide means of reducing energy consumption by monitoring and controlling different electrical appliances

IEEE GLOBECOM'11 45 Home-Area Networks (HANs) General Structure

Electric supply from Transmission

Smart Smart Devices Devices

Actuators Actuators

In-Home Wired/Wireless Display HAN Connection (e.g., Zigbee, DAU/NAN Gateway BACnet (Smart Meter or Light Actuators Dedicated in-home Sensors Smart Gateway) Devices Voltage Temperature

IEEE GLOBECOM'11 46 Home-Area Networks (HANs) Enabling Communications Technologies • Short Range Wireless Technologies – Wi-Fi, Bluetooth, ZigBee, Z-Wave • Z-Wave: – Proprietary wireless standard designed for home control automation, specifically to remote control applications in residential homes – Z-Wave was originally developed by Zensys A/S and is being marketed by Z-Wave Alliance – Z-Wave wireless protocol provides reliable and low-latency communication of small data packets within HANs – Z-Wave also uses a mesh networking approach with source routing

IEEE GLOBECOM'11 47 Home-Area Networks (HANs) Enabling Communications Technologies • Z-Wave: – Bandwidth: 9,600 bit/s or 40 kbit/s – Modulation: GFSK – Range: Approximately 100 feet (or 30 meters) – Frequency band: The Z-Wave Radio uses the 900 MHz ISM band • 908.42 MHz (United States) • 868.42 MHz (Europe) • 919.82 MHz (Hong Kong) • 921.42 MHz (Australia/New Zealand)

IEEE GLOBECOM'11 48 Home-Area Networks (HANs) Enabling Communications Technologies: Wireless Technologies

Key Criteria WiFi Bluetooth ZigBee Z-Wave

Feature - Designed for providing wireless -Designed for consumer electronics to - Designed specifically for industrial and - Designed for home automation,

connection for accessing Internet provide short-range wireless home automation for connecting specifically to remote control

and is direct replacement to communication sensors, applications in residential home such as

traditional Ethernet to connect a wide range of devices monitors and control devices light, entertainment systems, etc

network easily and quickly

Frequency Band - 2.4/5 GHz - 2.4 GHz - 2.4 GHz, 915MHz and 868MHz - 900 MHz

Standards - International Standard (IEEE - International Standard (IEEE 802.15.1) - International Standard (IEEE 802.15.4) - Proprietary Standard (Z-Wave

802.11 a/b/g/n) - Open Standard - Open Standard Alliance and Zensys)

- Open Standard - Closed standard

Speed - 54 Mbps ( 802.11. b/g) - 2.1 Mbps (V 2.0) - 250 Kbps - 9600 bits/s

- 150 Mbps (802.11 n) - 20 Mbps (V 3.0, recently released)

Range - 70m (indoor) to 250m (outdoor) - 10m - 70m (indoor) to 400m (outdoor) - 30m (indoor) to 100m (outdoor)

Power Consumption - High - Lower than WiFi - Lower than WiFi and Bluetooth - Almost same as ZigBee

Maximum Nodes - 2007 - 8 - > 64000 - 232

IEEE GLOBECOM'11 49 Home-Area Networks (HANs) Enabling Communications Technologies: Wireless Technologies

Key Criteria WiFi Bluetooth ZigBee Z-Wave

Security - WEP (Wired Equivalent privacy) - E0 stream cipher - 128 AES (Advanced - 3 DES(Triple Data Encryption

-WPA (Wi-Fi Protected access) - More Secure than WiFi Encryption Standard ) Standard)

- WPA2 keys

Strength - Easy to deploy, - Most popular protocol for transferring - Low power requirements and - Low power, low latency, and low cost

equipment costs data and wireless alternative to RS- implementation costs - Less interference due to use of sub-

dropping rapidly 232 data cables - Particularly designed for use in GHz frequency

- Supports mesh topology - Supports ring topology industrial and home automation or - Higher propagation range 2.5 times

security applications the 2.4 GHz signal

- Scalable and flexible - Supports mesh topology

- Supports mesh topology

Concern - High power consumption -Lack of proper installation in consumer - Limited range and low data rates - Low data rates

- Higher data latency portal context such as fire alarm, security - Interference due to overlapping with - Requires to add devices into network

- Additional security layer should be sensors, etc WiFi standard manually

implied to use WiFi within HAN - Does not support mesh networking - Slightly higher installation cost than

ZigBee

- Offers less flexibility due to close

nature IEEE GLOBECOM'11 50 Home-Area Networks (HANs) Enabling Communications Technologies: Wired Technologies

Key X10 HomePlug GP BACnet KNX

Criteria

Feature - Simple and popular protocol designed for -Designed specifically for smart grid to - A data communication protocol that - Global standard protocol designed

providing simple automation functionality such as provide lower power consumption attempts to unifies all the proprietary basically for home automation and

on and off. communication protocol into single control

communication language

Wireless - Yes - Yes - - Yes

Support - 310 MHz U.S. 433 MHz European - Recent ZigBee/HomePlug initiatives - KNX RF (868.3 MHz)

Standards - De facto Standard - International Standard (IEEE 1901) - ANSI/ASHRAE 135-2008 - CENELEC EN 50090 and CEN EN

- Open Standard - ISO 16484-5 13321-1

- Open Standard - ISO/IEC 14543-3

- GB/Z 20965, ANSI/ASHRAE 135

Speed - 20 bits/s - 4 to 10 Mbps - Depends on choice of LAN technology - wired 9.6 kbps

used - wireless 16.4 kbps

Maximum - 256 - 253 (theoretically) - No limit - 57600 network nodes for wired

Nodes - 10 (Practically) connection

IEEE GLOBECOM'11 51 Home-Area Networks (HANs) Enabling Communications Technologies: Wired Technologies

Key X10 HomePlug GP BACnet KNX

Criteria

Security - lack of encryption - AES pro 128 - Assume that all devices are sitting - EIBsec

Security (128-bit triple AES encryption behind a firewall

and a time lock)

Strength - Commonly used with variety of equipment -Ubiquitous reach throughout the home - Well established as an enabler for - Interoperable with other KNX

available in the market environment commercial building automation products

- No installation cost as uses power line -Interoperability with consumer home technologies - Hardware/software independent

networking - Already has the needed functionality - Well established promoter which

-Low-cost and low-power network for energy management and load control provide any application for home

interfaces - Independent of current LAN or WAN control

-Cross-compatibility between wired and technologies - Compatible with any buildings

wireless Smart Grid applications -Scalable

Concern - Extreme low data rate - Limited connection (10) when - client-server system might create - Low data rates

- Lack of standard and security transferring data simultaneously bottleneck when fully deployed to

- Limited functionality - Susceptible to power line interference consumer premise

- Prone to interference from neighbors using the and old wiring in home. - Security concerns

same X10 device addresses - Object model is limited to low-level IEEE GLOBECOM'11 types 52 Neighborhood-Area Networks (NANs) General Structure

Utility Network Back Bone HAN = Home Area Network

C e l lu B l P a L r ,I , P W n i e M t w A o X r k , DAU

HAN DAU Neighborhood Area HAN Network (NAN) MDMS

, rk Neighborhood Area o w et X Network (NAN) n A P I iM r, la , W lu L el P PLC, C B DAU ANSI C12 MDMS HAN DAU

HAN Neighborhood Area Network (NAN) Neighborhood Area Network (NAN)

Wide Area Network (WAN) IEEE GLOBECOM'11 53 Neighborhood-Area Networks (NANs) Neighborhood-Area Networks (NANs) • NAN connects multiple HANs together • Wired Technologies – Power Line Communication (PLC): • Ultra narrow band (UNB) operates in 0.3-3 kHz bands • Narrow band (NB) PLC operates in 3-500 KHz bands • Broadband (BB) PLC or BPL operates in 1.8-250 MHz bands – Internet Protocol (IP)-Based Networks – Internet Based Virtual Private Networks (Internet VPN) • Internet VPN technology can provide reliable, secure, and robust alternative to ensure security and QoS requirement • Wireless Technologies – 3G and LTE cellular Networks – WiMAX Technology

IEEE GLOBECOM'11 54 Wide-Area Networks (WANs) Core Communication Network and Last Mile Connectivity

Enabling Technolgies Scope Strength Concern 1. Power Line Communication - Communication Core - Complete control over the communication path - The power line are connected to various equipments such as

Network and Last Mile with extensive coverage that is solely controlled motor, power supplies, which can act as noise sources that

Connection by the utility industry eventually degrades the performance of PLC

- Provides low cost solution to overlay the - The load impedance fluctuation, and electromagnetic

communication network over already available interference causes signal attenuation and distortion, which can

power lines result to failure of communication link

- Provides direct route between controllers and - Lack of standard status and government regulation due to

other subsystem to ensure low latency industry fragmentation result in high interference from other

- Mature technology with many variants available PLC technology deployed at close range

commercially - Cost of PLC modem are still high

- Coexistence issue from many commercial technologies

2. Internet Protocol (IP)-Based - Communication Core - IP-based networks have rich convergence - In case of master/slave configuration, transmitting IP packets

Networks Network and Last Mile capabilities which can help to connect the overall from slave is not possible, which might increase the data

Connection systems and subsystems in smart grid latency for those applications which requires fast response as in

-Can provide QoS and reliable connection using case of smart grid

technologies such as DiffServ and MPLS - Unless private IP-based network (e.g., Internet VPN) is used,

Security can be enhanced using technologies security remains crucial issue

(IPSec) IEEE GLOBECOM'11 55 Wide-Area Networks (WANs) Core Communication Network and Last Mile Connectivity

Enabling Technolgies Scope Strength Concern 3. Wireless Communication - Communication Core Network and - Huge coverage area, potential for low cost - Utilities have to depend on these technologies

Last Mile Connection - Packet-Switched Cellular Data has lower cost and without any control over them

much higher data rates - Packet switch technologies are not available in all

- WiMAX can support mesh networks for higher deployed cellular structures

reliability - Requires connection to network before transmitting

the data and might be problem in case of outage and

emergency

4. Communication and Networking - Communication Core Network - Hybrid network can provide better needs to - Requires more research to combine technologies to

Middleware specific smart grid application form network middleware

- Improves Interoperability

IEEE GLOBECOM'11 56 Communication Architectures Standard Activities: Standard Developing Organization (SDO) • ANSI - American National Standards Institute (www.ansi.org) • IEC - International Electrotechnical Commission (www.iec.ch) • IEEE - Institute of Electrical and Electronics Engineers (www.ieee.org) • ISO - International Organization for Standardization (www.iso.org) • ITU - International Telecommunication Union (www.itu.int)

IEEE GLOBECOM'11 57 Communication Architectures Standard Activities (1) Standards Application Strength Concern ANSI C12 Suite - Defines utility industry end device data - Defines format of data for meter - Does not specify protocol to transport it

ANSI C12.19/IEEE 1377 tables for representing the data produced by - Provides transport independent application -Lacks full interoperability due to

ANSI C12.22 revenue meters. level protocol for data exchange with low specialized local profile

- Standard protocol for network overheads between nodes. - Requires complexity in implementation in

communication - Supports transport of C12.9 table data clients

- Provide authentication & encrypting the C12.9

data.

ANSI/ASHRAE 135/ISO 16484-5 - Defines information model & messages as - Open, mature standard with interoperability - Object model might be limited to low

BACnet objects for providing common language for testing developed and maintained by SDOs level protocols

different proprietary protocols - Serves as customer side communication - Requires structural view & specific profile

protocol with relevancy in price, DR/DER & to address consumer portals

energy usage

ANSI /EIA/CEA 709 & CEA 8521 - General purpose LAN protocol for - Widely used matured protocol - de facto standard controlled by Echelon

Protocol Suite LONworks providing communication over with home & - Specify as one of the data link & physical with limited support in power industry

ANSI/CEA 709.1-B building automation layer option for BACnet - Lack of complex object model to support

ANSI/CEA 709.2 - The Control Network function

ANSI/CEA 709.3 - Power Line Carrier Physical Layer

ANSI/CEA 709.4 - Twisted Pair Physical Layer IEEE GLOBECOM'11 58 - Fiber Optic Physical Layer Communication Architectures Standard Activities (2) Standards Application Strength Concern ZigBee/ HomePlug Smart Energy Profile Strategic alliance of ZigBee & HomePlug to - Interoperable between two distinct HAN technology

provide communication & information model - Technology independent

in HAN

IEC 62056 Device Language Message - Standard representation of metering data - Supports object modeling of application data as object

Specification (DLMS) & used for accessing and exchanging structured identification system (OBIS) and the Open Systems

Companion Specification for data models Interconnection (OSI) model

Energy Metering (COSEM) - Matured and internationally recognized standard

- Supports variety of media such as PSTN, GSM network,

PLC and recently ZigBee protocols

IEEE 1901 - Broadband communications over - High speed (>100 Mbps) communication for devices - Short range due to higher

Powerline medium access control using frequency below 100 MHz. attenuation of the medium as a

(MAC) and physical layer (PHY) - Uses inter-system protocol (ISP), which allow device to result of using broadcast channels

Protocols for HAN and also access coexist with devices based on ITU-T G.hn standard above 80 MHz

application - Initiate harmonization and coexistence of PLC with

other technologies

- Has backward compatibility with HomePlug standard

IEEE GLOBECOM'11 59 Communication Architectures Standard Activities (3) Standards Application Strength Concern ITU-T G.hn/G.9960 Home Networking - In-home networking over power lines, - Designed especially for HAN - Does not address PLC access application

Standard phone lines, and coaxial - Use single fast Fourier - Does not support HomePlug standard

cables transform (FFT) OFDM modulation and low-

density parity-check code (LDPC) forward

error correction (FEC) code

ISO/IEC 15045, A residential gateway - Defines specification for residential - Defines functional requirement & - Still under consideration by independent

model for Home electronic gateway (RG) that connects HAN to network architecture for RG organization

system domain outside the home basically last mile - Defines security requirements for connecting

connection to WANs

ISO/IEC 15067-3, Model for an energy - Defines a model for energy management - Specifies methods for demand response that

management system for Home system that accommodates a range of load may be implemented by an electric utility or

electronic system control strategies by a third-party supplier of energy

management services

- Supports various smart appliances

IEEE GLOBECOM'11 60 Communication Architectures Cognitive Radio [Yu_2011] (1) • Cognitive radio based communications architecture is presented for the smart grid • Cognitive radio allows unlicensed (secondary) user to access spectrum licensed to licensed (primary) user – Improve spectrum utilization – Improve spectrum efficiency • The proposed architecture is motivated by – Explosive data volume – Diverse data traffic – Need for QoS support

IEEE GLOBECOM'11 61 Communication Architectures Cognitive Radio [Yu_2011] (2) • Proposed Network Architecture

IEEE GLOBECOM'11 62 Communication Architectures Cognitive Radio [Yu_2011] (3)

Cognitive area Home area network (HAN) Neighborhood area Wide area network network network (NAN) (WAN) Spectrum band Unlicensed band Licensed band Licensed band

Network topology Centralized/decentralized Centralized Centralized

Network users Smart HGWs, NGWs spectrum broker meters/sensors/acuators HGW Featured strategy Cross-layer spectrum sharing Hybrid dynamic spectrum Optimal spectrum access leasing Key techniques Access control, power Guard channel, spectrum Join spectrum coordination handoff management

IEEE GLOBECOM'11 63 Communication Architectures Cognitive Radio [Yu_2011] (4) • Dynamic Spectrum Sharing in Cognitive HAN – HGW will connect to the HAN, which in turn will connect to external networks (e.g., Internet and NAN) – Within a HAN, the HAN cognitive gateway (HGW) manages the license-free spectrum bands to provide optimal data rate with low interference – HGW enables other devices and sensors to join the network, assigns channel and network addresses to each device, and coordinates the communications between the devices within the HAN

IEEE GLOBECOM'11 64 Communication Architectures Cognitive Radio [Yu_2011] (5) • Cognitive Communications in Neighborhood Area Network (NAN) – NAN Cognitive gateway (NGW) connects several HGWs from multiple HANs together – Hybrid dynamic spectrum access (H-DSA) is proposed – Some licensed spectrum bands are leased/bought from a telecommunication operator, and these bands are used as licensed access for the HGWs to ensure the QoS of data communications – The NGW distributes these licensed bands to the HGWs according to the transmission demand – However, if licensed spectrum bands are not enough to meet the demand, unlicensed access is also needed for the HGWs to improve the capacity and throughput of the NAN – In unlicensed access, the HGWs and NGW could be considered secondary users IEEE GLOBECOM'11 65 Communication Architectures Cognitive Radio [Yu_2011] (6) • Cognitive Communications in Wide Area Network (WAN) – In WAN, each NGW is a cognitive node with the capability to communicate with the control center through frequency space unused by a licensed primary user – Control center is connected with cognitive radio base stations – Spectrum broker controls sharing the spectrum resources among different NANs to enable coexistence of multiple NANs – Joint WAN/NAN spectrum management is proposed by minimizing the maximum dropping probability of data connection in NAN

IEEE GLOBECOM'11 66 Sensor and Actuator Networks (SANETs) Applications of Data Sensing in Smart Grid • Power Generation – WSN called WiMMS unit is deployed in the wind turbine structure [Wang_2007] to provide information about dynamic behavior of wind turbine and response to loading – For energy storage, lead-acid batteries will be used, and sensor network can be used to monitor temperature, voltage, and current • Power Transmission and Distribution – Data sensing can be used to monitor substations, transformers, underground lines, and overhead lines • Power Consumption – Smart meter acts as a sensor node and records the electricity consumption (kilo watt hour [kWh]) and time of use (TOU)

IEEE GLOBECOM'11 67 Sensor and Actuator Networks (SANETs) Requirements for Data Sensing and Communication • Sensor and Actuator Requirements – Longer life span – Reliability and energy-efficiency – Cost-effectiveness and secured operation • Data Collection Requirements – Machine readable format – Contain the temporal information including the time-stamp – Identification of location • Requirements for Communication Networks – Distributed operation – Interoperability – Scalability – Security

IEEE GLOBECOM'11 68 Sensor and Actuator Networks (SANETs) SANET in Transmission Line Monitoring [Hung_2010] • The linear sensor network for transmission line is analyzed • Accelerometer (inclination and cable position and tilt), magnetic field sensor (current and power quality), strain sensor, and temperature sensor are considered

IEEE GLOBECOM'11 69 Sensor and Actuator Networks (SANETs) Approaches for Data Sensing • Phasor Measurement Units – Phasor measurement units (PMUs) (also referred as synchrophasors) measure the electrical waves, using a common time source for synchronization – IEEE Standard C37.118-2005 deals with issues concerning the use of PMUs in electric power systems • Compressive Sensing – Compressive sensing (CS) is proposed which links data acquisition, compression, dimensionality reduction, and optimization together – CS senses less and computes more to obtain the useful data • Decentralized and Cooperative Sensing – Distributed information processing and control are needed in power system operations – For example, distributed state estimation methods have been considered for decades with the goal of reducing the computational burden at the central control by distributing the tasks across the system.

IEEE GLOBECOM'11 70 Sensor and Actuator Networks (SANETs) Approaches for Data Communication • Cooperative Communications – Cooperative communications refer to the techniques in which multiple nodes help each other (e.g., in wireless mesh, ad hoc, and sensor networks) to relay or forward data packets to their destinations – Cooperative wireless sensor network (IEEE 802.15.4 ZigBee) is used to provide data transmission in urban-scale smart grid environment [Ullo_2010] – Secure and reliable collaborative communication scheme for advanced metering infrastructure (AMI) is introduced [Yan_2011] – Multihop wireless network is used to connect smart meters with AMI to transfer meter data to a local collector • Cognitive Radio – CR-based wireless sensor network using the 802.15.4 ZigBee standard is proposed in [Sreesha_2011] – In the design, a coordinator is used to provide the synchronization and control of data transmission, while a spectrum sensor is used to support frequency agility so that the transmission can be adapted based on the wireless channel condition

IEEE GLOBECOM'11 71 Data Communications and Networking in Smart Grid • Demand Response Management (DRM) • Home Energy Management System (HEMS) • Advanced Metering Infrastructure (AMI) • Wide-Area Measurement Systems (WAMSs)

IEEE GLOBECOM'11 72 Demand Response Management (DRM)

• DRM is the programs implemented by utility companies to control the energy consumption at customer side

Optimality

Temporary Spinning adjustment reserve Physical DR Market Optimized DR schedule TOU

Optimized Energy infrastructure efficiency Time Permanent Days Seconds

IEEE GLOBECOM'11 73 Demand Response Management (DRM) • Energy efficiency focuses on users and behavioral changes to achieve more efficient energy usage – Users buy appliance with energy reduction feature

IEEE GLOBECOM'11 74 Demand Response Management (DRM) • Smart pricing or time of use (TOU) – Customers (re)arrange their energy consumption to minimize costs • Market demand response – Direct load control (DLC): utility or grid operator control energy consumption of consumers – Interruptible/curtailable rates: customers has a contract with limited sheds feature from utility – Emergency demand response programs: customers voluntarily adjust energy consumption based on emergency signals (e.g., blackout) – Demand bidding programs: customers can bid for curtailing at attractive price

IEEE GLOBECOM'11 75 Demand Response Management (DRM) • Physical demand response – Grid management and emergency signals (on the utility side) – Signal if the grid (power lines, transformers, and substations) are in a reduced performance due to maintenance or failure • Spinning Reserves (SR) – Generators are online, synchronized to the grid, that can increase output immediately in response to a major outage and can reach full capacity [Hirst_1998]

IEEE GLOBECOM'11 76 Demand Response Management (DRM) • Energy efficiency vs. demand response

Energy consumption Original load

Demand response without rebound Demand response with rebound

Energy efficiency

Time

IEEE GLOBECOM'11 77 Demand Response Management (DRM)

Residential load management [Mohsenian-Rad_2010] • Residential load management programs usually are to reducing consumption and shifting consumption • In direct load control (DLC), utility company sets up an agreement with its customers • Utility company can manage and control remotely the operations and energy consumption of certain household appliances – Lighting – Thermal and cooling system – Refrigerators – Pumps

IEEE GLOBECOM'11 78 Demand Response Management (DRM)

Smart Pricing [Mohsenian-Rad_2010] • With smart pricing, energy consumers are encouraged to individually and voluntarily manage their loads – Reducing their consumption at peak hours • Critical-peak pricing (CPP), time-of-use pricing (ToUP), and real-time pricing (RTP) can be used • For example, in RTP, the price of electricity varies at different hours of the day – Prices are usually higher during the afternoon, on hot days in the summer, and on cold days in the winter

IEEE GLOBECOM'11 79 Home Energy Management System (HEMS) • HEMS acts as the subset of energy management system (EMS) and together with smart meter provides a necessary interface to the HAN for better energy management

Other 8% Electronics 7% Heating and cooling 49% Lighting 10% Clothes washer & Dryer 6% Dishwasher 2% Refrigerator 5%

Water heater 13%

http://www.energystar.gov/ IEEE GLOBECOM'11 80 Home Energy Management System (HEMS) • HEMS (or EMS) sets a certain user limit threshold based on the information about real-time price-responsive load management and consumption history (i.e., collected from smart meter) to control the energy usage of appliances • HEMS is generally integrated into HAN to offer a channel for the consumers to interact with the electrical power grid • HEMS may reside in the smart meter or in an independent gateway such as residential gateway and network adapters • HAN contains many electrical appliances (e.g., routers, TV, AC, computers, etc) which provide different services, e.g., wireless access, VoIP calls, ambient temperature control • These services can be controlled by using different power control elements (PCEs) such as Ethernet switch, PSTN, and DSL modem IEEE GLOBECOM'11 81 Home Energy Management System (HEMS) • Example: GE Demand Reduction Approach

Price Event Signal to Electric supply from Transmission Smart Appliance Data communications Smart Smart Devices Devices

Actuators Actuators

In-Home Wired/Wireless Display HAN Connection (e.g., Zigbee, Smart Appliance will indicate DAU/NAN Gateway BACnet (Smart Meter or Light Actuators to consumer Dedicated in-home Sensors Smart Gateway) Devices Price Event has occurred Voltage Temperature Smart Appliance will recommend to delay start

Over Initiate delayed start function Ride? Consumer Choice

Over Initiate peak reduction mode Ride?

Run Normal operating mode

IEEE GLOBECOM'11 82 Home Energy Management System (HEMS) Machine-to-Machine Communications [Niyato2011] • Network design issue of M2M communications for a home energy management system (HEMS) is considered • The network architecture for HEMS to collect status and power consumption demand from home appliances is introduced • Optimal HEMS traffic concentration is presented and formulated as the optimal cluster formation

IEEE GLOBECOM'11 83 Home Energy Management System (HEMS) Machine-to-Machine Communications • Network model

Service area with wide area network (WAN)

Neighborhood area network (NAN)

Concentrator

Control center Internet Base station backhaul

Smart meter

Home area network (HAN)

IEEE GLOBECOM'11 84 Home Energy Management System (HEMS) Machine-to-Machine Communications • Optimal cluster is determined • The average cost per node under different packet generation rates is shown

20 Cluster Cluster 18 size = 10 size = 5 16

14 Cluster size = 1

12 Cluster Cluster 10 size = 4 size = 3 Cluster size = 2 8

Average costAverage node per 6

4

2 Optimal formation Fixed formation 0 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 Packet generation rate (packets/minute) IEEE GLOBECOM'11 85 Advanced Metering Infrastructure • AMI acts as the gateway for access enabling the bidirectional flow of information and power in support of distributed energy resource (DER) management or distributed generation (DG) and consumer participation • AMI will provide near real-time consumption data including fault and outage to the utility control center • AMI supports time-based and dynamic tariffs such as Time of Use (TOU), Real-Time Pricing (RTP), and Critical Peak Pricing (CPP) • AMI consists of several different components – Smart meters and data aggregator units (DAUs)) – Hierarchical area networks (e.g., home-area networks (HANs) and neighborhood-area networks (NANs), and wide-area networks (WANs))

IEEE GLOBECOM'11 86 Advanced Metering Infrastructure • Comparison

Manual/Automatic Meter AMI Reading (AMR) Pricing Fixed price and measure total Total consumption consumption only Time-of-use Critical peak pricing Real-time pricing Other demand response None Load control Demand bidding Demand reserves Critical peak rebates Customer feedback Monthly bill Monthly bill Monthly detailed report Web display In-home display Customer bill savings Turn off appliances manually Turn off appliances Shift appliances off peak Manual or automatic control Outages Customer phone calls Automatic detection Verification of restoration at individual home level Distribution operations Use engineering models Dynamic, real-time operations

IEEE GLOBECOM'11 87 Advanced Metering Infrastructure Benefit of AMI [Liu_2010] • Fault Location, Isolation and Service Restoration (FLISR) – AMI will be able to automatically report loss of power, and the information can be used to assist locating the fault location • Emergency Load Shedding – AMI helps to shed large amounts of load very quickly (within seconds) to avoid power system instability and loss of system integrity (e.g., during bulk power grid emergencies) • Distribution System Planning and Analysis – AMI provides accurately metered data for all customers on the feeder from billing records, and this information will enable the system to prepare much more accurate short term load forecast • Continuous Condition Monitoring • Equipment and System Performance Forecasting • Automated “Triggering” for Maintenance and Work Assignments • Substation and Line Monitoring

IEEE GLOBECOM'11 88 Advanced Metering Infrastructure Wireless Broadband Architecture [Mao_011] and Key Design Issues • Address Depletion – For AMI, a very large number of new subscriber devices, i.e. smart meters, will need address for communications • Traffic Scheduling – Critical alarm indication data should be reported immediately and not be queued until the next scheduled connected period • Congestion Control – A very large numbers of SM give rise to potential “traffic burst” scenarios which arise when large numbers of devices are simultaneously (or near simultaneous) reporting or reacting to a common event

IEEE GLOBECOM'11 89 Advanced Metering Infrastructure Service-Oriented AMI [Chen_2010] • Service-oriented approach to AMI aiming at solving the intercommunication problem and meanwhile providing a trust and secure environment for smart grids – System integration and cooperation are done through service composition. – Generic service interfacing method is designed to develop standardized – services for heterogeneous power systems – Role-based access control mechanism is used to guarantee secure access

IEEE GLOBECOM'11 90 Advanced Metering Infrastructure Reliability Analysis • Reliability analysis of the wireless communications system in the smart grid can be performed • Availability performance can be obtained given the random failure of the system devices • Availability measure can be used to calculate the cost of power-demand estimation error and damage of power distribution equipment if its failure cannot be reported • Redundancy design approaches can be developed to minimize the cost of failure as well as the cost of deployment of the wireless communications system in the smart grid

IEEE GLOBECOM'11 91 Advanced Metering Infrastructure Reliability Analysis

Meter data-management system (MDMS)

Home area network (HAN) NAN with gateway Neighborhood area redundancy network (NAN)

HAN gateway and NAN gateway Data aggregator Power distribution smart meter unit (DAU) equipment

IEEE GLOBECOM'11 92 Advanced Metering Infrastructure Reliability Analysis: Operation of a power system

Smart meter estimates power demand in the next period (e.g., using power scheduling)

HAN gateway sends power demand collected from smart meter to the corresponding NAN gateway to forward to DAU and subsequently MDMS MDMS checks if power Yes supply is enough or not?

Power demand of each house No is received by MDMS? No MDMS buys additional Yes MDMS uses mean power power supply in consumption of that house to economic dispatch stage Power demand is added into compute amount of amount of power to be supplied power to be supplied

MDMS buys power supply in unit commitment stage

IEEE GLOBECOM'11 93 Advanced Metering Infrastructure Reliability Analysis: Operation of a power system • If the power demand of any house is not received by the MDMS (e.g., due to failure of the HAN gateway, the NAN gateway, or the DAU), the MDMS uses historical data to compute the aggregated power demand • x% of mean power-consumption1 of those houses is used as the estimated demand

Estimated power demand (i.e., reserved power from unit commitment stage) for

x=100% of mean Probability distributionProbability 0 Power consumption (kWh) Cost of under-reservation Cost of over-reservation

IEEE GLOBECOM'11 94 Advanced Metering Infrastructure Reliability Analysis: Availability • Availability of a component/device/system is the probability that the component/device/system has not failed or repaired and it can operate normally

• Uptime is also known as the mean time between failure (MTBF) • Downtime is known as the mean time between repair (MTBR) • Failure rate can be obtained a 1-Availability

IEEE GLOBECOM'11 95 Advanced Metering Infrastructure Reliability Analysis: Availability • Dependence diagram (DD) determines the contribution of each component to the availability of the system • The components can be connected in parallel and/or series

Metering engine Control unit Power Radio interface

Dependence diagram of smart meter and home area network gateway

Power Radio interface Single board computer Adaptor Software Power Dependence diagram of neighborhood area network gateway

Radio network Service gateway GPRS gateway support Node B controller (RNC) support node (SGSN) node (GGSN)

Dependence diagram of UMTS network

IEEE GLOBECOM'11 96 Advanced Metering Infrastructure Reliability Analysis: Availability • HAN gateway and a smart meter can be integrated into a single device.

The availability of a HAN gateway is computed from AHAN = availability of metering engine × availability of control unit × availability of power module × availability of radio interface

• Availability of a NAN gateway is computed from ANAN = availability of radio interface × availability of single board computer × availability of adaptor × availability of software × (1 − (1−availability of power module)2) • 3G cellular base station is assumed to have the DAU functionality

whose availability is computed from: ADAU = availability of node B × availability of radio network controller (RNC) × availability of service gateway support node (SGSN) × availability of GPRS gateway support node (GGSN)

IEEE GLOBECOM'11 97 Advanced Metering Infrastructure Reliability Analysis: Cost of Network Unavailability • Cost of demand-estimation error of individual house i whose connection to the MDMS is unavailable can be obtained from

• Ei = x/100 × Meani is the power supply reserved in the unit commitment stage

• Meani is the mean power-consumption of house I

• Maxi is the maximum power-consumption (i) • fA (a) is the PDF of actual power demand a

• puc and ped denote the power prices in the unit commitment and in the economic dispatch stages, respectively

IEEE GLOBECOM'11 98 Advanced Metering Infrastructure Reliability Analysis: Cost of Network Unavailability • Number of houses • Number of redundant NAN gateways

70 300 Failure rate of HAN gateway = 2 days in 1 years Failure rate of NAN gateway = 2 day in 2 years Failure rate of HAN gateway = 2 days in 2 years Failure rate of NAN gateway = 2 day in 3 years 60 280 Failure rate of HAN gateway = 2 days in 3 years Failure rate of NAN gateway = 2 day in 4 years Failure rate of HAN gateway = 2 days in 4 years 50 260

40 240

30 220

20 200 Average totalAverage cost month ($) per

10 180 Cost of demand estimation error per month ($) Costper estimationof demand error

0 160 20 40 60 80 100 120 140 160 180 200 0 1 2 3 4 5 6 7 8 9 10 Number of houses in NAN Number of redundant NAN gateways

IEEE GLOBECOM'11 99 Wide-Area Measurement Systems (WAMSs) • WAMS is used to conduct real time monitoring and control in dynamic power system states • WAMS uses a synchronized phasor measurement unit (PMU) to guarantee for security and stability of power systems • WAMS is typically composed of PMUs, phasor data concentrator (PDC), control center (CC), as well as the high-speed data communication networks

IEEE GLOBECOM'11 100 Wide-Area Measurement Systems (WAMSs) Applications [Naduvathuparambil_2002] • State estimation: PMUs can measure and relay information on a continuous basis to the control centers, and control center will generate a state vector of system dynamics • Instability prediction: Synchronized phasor measurements can enable real-time stability analysis and instability prediction • Improved control of power systems: Controllers (e.g., variable series capacitors [VSC], universal power flow controllers [UPFCs] and power system stabilizers) can receive feedback from control center to regulate the grid

IEEE GLOBECOM'11 101 Wide-Area Measurement Systems (WAMSs)

Data Communication • Telephone lines – Easy to set up and economical to use, but low speed • Fiber-optic cables – Immunity to RF & atmospheric interference – Large bandwidth • Satellites: low-earth orbiting (LEO) – Large coverage area, – High cost, narrow bandwidth, and large delays • Power lines – Uses the medium and low voltage electric supply grid for transmission of data and voice • Microwave links – Easy to set up and are highly reliable – Signal fading and multipath propagation

IEEE GLOBECOM'11 102 Wide-Area Measurement Systems (WAMSs) Centralized WAMS [Shahraeini_2011] • All data resources send data to control center (CC) • After processing the received data, appropriate decisions are made and related commands are sent back to controllable devices

IEEE GLOBECOM'11 103 Wide-Area Measurement Systems (WAMSs) Decentralized WAMS [Shahraeini_2011] • System is divided into multiple areas • Each area has its own are control center area (ACC) • In each area, ACC processes the acquired data and perform control • For the control of a system, ACCs share information among each other through communication systems

IEEE GLOBECOM'11 104 Wide-Area Measurement Systems (WAMSs)

Reliability Analysis • [Bruce_1998], [Xie_2002], [Wang_2010] – Synchronized phasor measurement unit (PMU) – Phasor data concentrator (PDC) – Ring interface unit (RIU) – Control center (CC)

Fault tree analysis of WAMS

IEEE GLOBECOM'11 105 Wide-Area Measurement Systems (WAMSs)

Reliability Analysis • Availability is calculated from

• Ai is the availability of the th PMUs-PDC working group PMU • Aij is the availability of PMU j in PMUs-PDC working group i

• Mi is the number of PMUs in group I RN • Ai is availability of regional communication network PDC • Ai is availability of PDC device

IEEE GLOBECOM'11 106 Cyber Security for Smart Grid

• Introduction • Why do we need cyber security • Adversaries • Threats • Impacts • How to achieve cyber security • Survey some solutions

IEEE GLOBECOM'11 107 Current Electric Grid – Islands of Technology

Generation Transmission Distribution Customers

GEN1 - Operational Information TOP1 – Operational Information DIST1 - Operational Information

GENx - Operational Information TOPx – Operational Information DISTx – Operational Information

IEEE GLOBECOM'11 108 Convergence of Enterprise & Operations IT Convergence of Enterprise & Operations IT

Information Technology Operations Technology Smart Grid Technology

Enterprise Systems Control Systems Web Applications Protection Systems AMI DSM OMS GIS

Cyber Secure

Integration counters key security principals of isolation and segregation

IEEE GLOBECOM'11 109 Smart Grid – Connectivity with Security

End-to-End Communications, Intelligence, and Defense-in-Depth Security

Generation Transmission Distribution Customers

AMI DSM

System Conservation Operators Authorities

IEEE GLOBECOM'11 110 Why do we need cyber security ?

• Network security is a priority and not a add on for smart grids • Protecting control center alone - not enough • Remote access to devices • QoS requirement from security system • Safety (line worker public and equipment) • Reliability and availability

111 Drivers

Increasing New 2-Way Interconnection Systems and Integration (e.g. AMI, DSM)

Increasing Use of New Customer COTS Hardware Touch Points into and Software Utilities

Control Systems Increasing Number Not Designed with Of Systems and Security in Mind Size of Code Base

Increased Attack Surface Increased Risk to Operations

112 Threats-I

PerformSQL Admin Operator Admin ARPEXEC Scan Opens Send e-mail Email with with malware Malware

Internet Acct Operator

1. Hacker sends an e-mail with malware 2. E-mail recipient opens the e-mail and the malware gets installed quietly Master DB 3. Using the information that malware gets, hacker is able to take control of the e-mail Slave recipient’s PC! Database RTU 4. Hacker performs an ARP (Address Resolution Protocol) Scan 5. Once the Slave Database is found, hacker sends an SQL EXEC command 6. Performs another ARP Scan 7. Takes control of RTU Example from 2006 SANS SCADA Security Summit, INL 113 Threats-II Attacker Controls the Cyber Head End Penetration Attacker Performs Communications Remote Network (WAN) Disconnect Attacker AMCC (Advanced Metering Control Computer) Communications Network (WAN) Retailers 3rd Parties

AMI WAN AMI WAN AMI WAN

Data Management Systems (MDM/R)

U N I V E R S I T Y Example from AMRA Webinar, Nov ’06 “The Active Attacker”

114 Impacts-I

Utility Energy Back Office Service Provider

The Impact of a Security Breach* AMI Wide Effect, High Impact on the Grid, Attacker may be Network Remote EMS HAN Local Effect, Narrow Impact,

Attacker Needs to be Local

* Does not represent the difficulty or ease of executing the breach. Meter Energy Consumption Data Demand Response Trigger Direct Energy Information Access from Meter and Local Control in Customer Premises has Lowest Risk 115 Impact-II

Attacker Impact Impact Threat Location Spread Effect

AMI Network Network Remote Wide Compromised Stability

DR Manipulated in Network ―Cloud‖* Remote Wide Stability

Customer Privacy Breached in ―Cloud‖* Remote Wide Loss of Privacy

HAN Compromised Local Local Narrow Nuisance

* “Cloud” refers to both a Utility Back Office and Energy Service Provider IEEE GLOBECOM'11 116 Northeast Blackout – August 14, 2003 • Affected 55 million people • $6 billion lost • Per year $135 billions lost for power interruption

~$6 billion lost due to 8/14/03 blackout Cost of Power Disturbances: $25 - $188 billion per year http://en.wikipedia.org/wiki/Northeast_Blackout_of_2003

IEEE GLOBECOM'11 117 Adversaries

• Hostile States • Hackers • Terrorist /Cyber terrorists • Organized crime • Other criminal elements • Industrial competitors • Disgruntled employees • Careless and poorly trained employees

IEEE GLOBECOM'11 118 Classification of attacks

• Component based attacks

• Protocol based attack

IEEE GLOBECOM'11 119 COMPONENT BASED ATTACK -STUXNET

• Specifically programmed to attack SCADA and could reprogram PLC‟s • Zero day attack • Highly complex • 0.5 Mb file transferred able to multiply • Targets- Iran nuclear plants ,Process plants in Germany and ISRO India Source: wikipedia

IEEE GLOBECOM'11 120 COMPONENT BASED ATTACK - SCADA attacks • Internal attacks  Employee  Contractor • External attacks  Non specific- malware , hackers  Targeted  Special knowledge – former insider  No special knowledge –hacker terrorist  Natural disaster  Manmade disasters

IEEE GLOBECOM'11 121 SCADA – vulnerability points

• Unused telephone line – war dialing

• Use of removable media – stuxnet

• Infected Bluetooth enabled devices

• Wi-Fi enabled computer that has Ethernet connection to scada system

• Insufficiently secure Wi-Fi

• Corporate LAN /WAN

• Corporate web server email servers internet gateways

IEEE GLOBECOM'11 122 SCADA-CYBER ATTACKS

• Web servers or SQL attacks

• Email attacks

• Zombie recruitment

• DDOS attacks

IEEE GLOBECOM'11 123 Protocol based attacks

• All protocols runs on top of IP protocol and IP protocol has its own set of weakness

• DNP3 implements TLS and SSL encryption which is weak

• The protocol is vulnerable to out-of-order, unexpected or incorrectly formatted packets

• A significant weakness for IEC 61850 is that it maps to MMS (Manufacturing message specification)as the communications platform, which itself has a wide range of potential vulnerabilities

IEEE GLOBECOM'11 124 Challenges

• The challenge is complex and continuously changing • Legacy systems need to be protected • Number and geographic location of end points • Relationship to physical security • Systems are 7x24 and critical • The human element / social engineering

IEEE GLOBECOM'11 125 Challenges („cont.)

• Scale • Legacy devices • Field location • Culture of security through obscurity • Evolving standards and regulations

IEEE GLOBECOM'11 126 How to achieve cyber security? • Security by obscurity

• Trust no one

• Layered security framework

• Efficient firewall

• Intrusion detection

• Self healing security system

IEEE GLOBECOM'11 127 Types of Cyber Security Solutions

• Reactive vs. Proactive – Reactive o Incident response plan o Applied for general purpose computers more – Proactive Security for embedded computers • High assurance boot • Secure software validation • Secure association termination if found infected • Device assentation

IEEE GLOBECOM'11 128 Solution - Incidence response plan

Attack

Prevention Services

Containment Services

Detection & Notification Services

Recovery & Restoration Services

IEEE GLOBECOM'11 129 Solution - Defense in Depth • Perimeter Protection – Firewall, IPS, VPN, AV – Host IDS, Host AV – DMZ – Physical Security • Interior Security – Firewall, IDS, VPN, AV – Host IDS, Host AV – IEEE P1711 (Serial Connections) – NAC – Scanning IDS Intrusion Detection System • Monitoring IPS Intrusion Prevention System DMZ DeMilitarized Zone • Management VPN Virtual Private Network (encrypted) • Processes AV Anti-Virus (anti-malware) NAC Network Admission Control

IEEE GLOBECOM'11 130 Solution –Control Network

Key Points: Internet • Defense in Depth • Access Control • Secure connections • Link to Physical • Security Management Enterprise Network • Apply same approach to other Smart Grid elements VPN FW Proxy AV IPS IPS Host IPS Host AV FW IDS Control Network Partner NAC Scan Site FW Host IDS Host AV

VPN P1711 FW IDS AV Field Site Field Site Field Site Scan NAC

131 Solution – Key management

• Issue of key management – Scale

• PKI with trusted computing elements- considerable amount of security

• Embedded vs. general-purpose computing

IEEE GLOBECOM'11 132 PKI Infrastructure

133 Issues with PKI

• Updating the keys

• Parameter generation

• Key distribution

• Staffing for key management

IEEE GLOBECOM'11 134 Solution – Attack trees

IEEE GLOBECOM'11 135 Calculation of cyber security conditions (omega)

Rules for Conditions 1, 2, and 3 Conditions Rules Condition 1 The system is free of intrusion attempt that is concluded from the electronic evidences in the system Condition 2 At least one or more countermeasures are implemented to protect an attack leaf. Condition 3 At least one or more password policies are enforced corresponding to each attack leaf.

IEEE GLOBECOM'11 136 Weighing factor for password policy

IEEE GLOBECOM'11 137 Calculations of vulnerability index

• Leaf VI : max( total countermeasures implemented /total countermeasures available x ω , ω x weighing factor of password policy) • Scenario vulnerability index : Product of its leaf vulnerability indices • System vulnerability index is the max of all scenario vulnerabilities indices

IEEE GLOBECOM'11 138 State estimation attack - introduction • State estimation is to determine the optimal estimate for the complex voltages at each bus based on real-time analog measurements. – The state typically refers to bus voltage magnitudes and phase angles • Bad data processing is to detect measurement errors, and identify and eliminate them if possible. – It is effective against random noises, but – It lacks the ability to detect intentionally coordinated bad data • That conforms to the network topology and physical laws

IEEE GLOBECOM'11 139 State estimation attack - 1 • Attack on state estimation [Giani_2011] – By compromising some line meters, sending wrong information about voltage / current status • Force the energy management system to make wrong balancing operations that causes outage – Main characters of the attack • Sparse attacks are common (unobservable attacks) – Large number of coordinated attacks can be detected by a bad data detection algorithm

• [Giani_2011] A. Giani, E. Bitary, M. Garciay, M. McQueenz, P. Khargonekarx, and K. Poolla, “Smart Grid Data Integrity Attacks: Characterizations and Countermeasures”, Proceedings of IEEE SmartGridComm 2011.

IEEE GLOBECOM'11 140 Main contributions • An efficient detection algorithm for – Case I : the attackers compromise • Two power injection meters coordinately • Arbitrary number of line meters • The algorithm require O(n2×m) flops – n is the number of buses , m is the number of line meters – Case II • Limited number of coordinated meters for attack (i.e., 3, 4, or 5) • All lines are metered • The algorithm requires O(n2) flops • Countermeasures – Using known-secure PMUs for counteracting the attacks – Demonstrate that p+1 PMUs are enough to neutralize a collection of p cyberattacks • The positions of PMUs need to be carefully chosen

IEEE GLOBECOM'11 141 State estimation attack - 2 • Study the vulnerability of the state estimator to attacks performed against the communication infrastructure [Vukovic_2011] • Use the security metrics defined by them to show – how various network and application layer mitigation strategies can be used • to decrease the vulnerability of the state estimator • Background – An attacker that wants to change the measurement on one substation might have to change several other measurements • To avoid a bad data detection (BDD) alarm

• [Vukovic_2011] O. Vukovic, K-C Sou, G. Dan, and H. Sandberg, “Network-layer Protection Schemes against Stealth Attacks on State Estimators in Power Systems”, Proceedings of IEEE SmartGridComm 2011.

IEEE GLOBECOM'11 142 Main ideas • Substation is the weak point – Measurement data are usually collected through substations – An attacker can access and modify all data that traverses a substation – The authors proposed to assess the importance of each substation with respect to state estimation • Security metrics – Substation attack impact • The number of measurements on which an attack can perform a stealth attack – Measurement of attack cost • Minimum number of substations that have to be attacked in order to perform attack against the measurement

IEEE GLOBECOM'11 143 Main contributions • Protective methods – Network layer solutions • Single-route routing vs. Multi-path routing • Modify single-route path to decrease the vulnerability of the system • Multi-path routing could reduce the maximum attack impact by 50% – Application layer solutions • Data authentication increases the attack cost – The solutions are very realistic

IEEE GLOBECOM'11 144 State estimation attack - 3 • This paper introduced a procedure that aims to achieve network-wide optimal attack detection and state estimation [Tajer_2011] • The procedure is distributed – Different controlling agents distributed across the network carry out the attack detection and system recovery tasks through • local processing and message passing, and • An iterative process – Distributed state estimation method can reduce the computational burden on the centralized control system • Using a decompose-merge approach

• [Tajer_2011] A. Tajer, S. Kar, V. Poor, and S. Cui, “Distributed Joint Cyber Attack Detection and State Recovery in Smart Grids”, Proceedings of IEEE Globecom 2011.

IEEE GLOBECOM'11 145 Main contributions • Reliable detection + reliable estimate of the false injected data – Means that the system can still obtain relatively accurate estimation of the data in spite of attacks – Different from works that avoid data to be compromised – Used an information theoretic method

IEEE GLOBECOM'11 146 State estimation attack - 4 • This paper [Esmalifalak_2011] demonstrate an attack method that – Inject false data with low detectability – Without knowledge of the network topology – Makes the inference from the correlations of line measurements • But assume that the attackers can break into the SCADA system • Main contributions – Demonstrate that an attacker can estimate both the system topology and power states just by observing the power flow measurements – Independent component analysis (ICA) is used • to infer the linear structure of the power flow measurements

• [Esmalifalak_2011] M. Esmalifalak, H. Nguyen, R. Zheng, and Z. Han, “Stealth False Data Injection using Independent Component Analysis in Smart Grid”, Proceedings of IEEE SmartGridComm 2011.

IEEE GLOBECOM'11 147 Performance evaluation

• The authors demonstrated that – The ICA based attack is almost unobserserable – The random attack is easy to be detected • Real – no attack • Estimated – ICA based attack

IEEE GLOBECOM'11 148 General security technique used in smart grid

IEEE GLOBECOM'11 149 Message authentication code aggregation • Message Authentication Code (MAC) is used to authenticate each message [Kolesnikov_2011] – To prevent en route accidental and malicious data corruption – Aggregate MAC is often used • Since the communication channel capacity is often small, and • The data size is short compared to the MAC code – The aggregate MAC is not resilient to denial-of- service (DOS) attacks

• [Kolesnikov_2011] V. Kolesnikov, W. Lee, and J. Hong, “MAC Aggregation Resilient to DoS Attacks”, Proceedings of IEEE SmartGridComm 2011.

IEEE GLOBECOM'11 150 Main contributions • The authors proposed a new authentication mechanism for the wireless sensor data – Securely combine authentication tags computed by sensors • So that the aggregate tag is much shorter than the concatenation of the constituent tags, but • Provides same strong security guarantees – Resilient to denial-of-service (DOS) attacks • A DoS attacker will only be able to disrupt a portion of the data – Only the data he relays • His point of insertion can be estimated based on which part of aggregate MAC is corrupted.

IEEE GLOBECOM'11 151 Secure energy routing • The authors of [Zhu_2011] developed a novel secure energy routing mechanism – for securely and optimally sharing renewable energy in smart microgrids – It can detects most internal attacks by using message redundancy • Spoofed route signaling • Fabricated routing messages

• [Zhu_2011] T. Zhu, S. Xiao, Y. Ping, D. Towsley, and W. Gong, “A Secure Energy Routing Mechanism for Sharing Renewable Energy in Smart Microgrid”, Proceedings of IEEE SmartGridComm 2011.

IEEE GLOBECOM'11 152 Intrusion detection systems for home area networks • This paper [Jokar _2011] presents a layered specification-based intrusion detection system (IDS) – Designed to target ZigBee technology – Addressed the physical and MAC layer • Normal behavior of the network is defined through selected specifications extracted from the IEEE 802.15.4 standard • Deviations from the defined normal behavior is viewed as a sign of malicious activities • The performance analysis demonstrated that the designed IDS provides a good detection capability against known attacks – The same is expected for unknown attacks • Since the design of the IDS is based on anomalous event detection

• [Jokar _2011] P. Jokar, H. Nicanfar, V. Leung, “Specification-based Intrusion Detection for Home Area Networks in Smart Grids”, Proceedings of IEEE SmartGridComm 2011.

IEEE GLOBECOM'11 153 Privacy of electricity usage information

IEEE GLOBECOM'11 154 Privacy-preserving authentication

• Privacy requirement: to preserve the privacy of the consumers, the electric usage information is hidden from the substations [Chim_2011] – But it should be known by the control center • Pseudo identity is used • Authentication requirement on each smart meter – To ensure requests are sent from valid users • The authentication process is made very efficient by means of Hash-based Message Authentication Code (HMAC) – The overhead is only 20 bytes per request message • Under attack, the substation allows 6 times more valid messages to reach the control center – when compared to the case without any verification

• [Chim_2011] T. Chim, S. Yiu, L. Hui, and V. Li, “PASS: Privacy-preserving Authentication Scheme for Smart Grid Network”, Proceedings of IEEE SmartGridComm 2011.

IEEE GLOBECOM'11 155 Privacy-utility tradeoff

• Existing privacy preservation solutions for user‟s electricity usage data have also not quantified the loss of benefit (utility) of data dissemination [Rajagopalan_2011] • Using tools from information theory, a new framework is presented that abstracts both the privacy and the utility requirements of smart meter data. • For a stationary Gaussian Markov model of the electricity load, it is shown that the optimal utility-and-privacy preserving solution requires filtering out frequency components that are low in power – this approach encompass most of the proposed privacy approaches

• [Rajagopalan_2011] S. Rajagopalan, L. Sankar, S. Mohajer, and V. Poor, “Smart Meter Privacy: A Utility-Privacy Framework”, Proceedings of IEEE SmartGridComm 2011.

IEEE GLOBECOM'11 156 Cooperative state estimation for preserving privacy • This paper [Kim_2011] presents a cooperative state estimation technique that protects the privacy of users‟ daily activities. – By exploiting the kernel of an electric grid configuration matrix – Obfuscate the privacy-prone data without compromising the performance of state estimation • The power consumption measurement is well obfuscated such that the consumers do not fully disclose their private behavioral information in the first place, and • the obfuscated data retain the necessary information such that the state vector can be accurately estimated from the perturbed measurement • [Kim_2011] Y. Kim, E. Ngai, and M. Srivastava, “Cooperative State Estimation for Preserving Privacy of User Behaviors in Smart Grid”, Proceedings of IEEE SmartGridComm 2011.

IEEE GLOBECOM'11 157 Summary on Cyber Security for Smart Grid

• Different security constraints that makes securing smart grids a difficult problem • Several highly efficient adversaries • Use existing protocols like IP with known vulnerabilities and work around to using new protocols with unknown vulnerabilities • Use of layered security architecture and attack tree‟s for efficient security and risk assessment

IEEE GLOBECOM'11 158 Field Trials and Case Studies for Smart Grid Communication Infrastructures . Smart Power Grid • SDC . Smart Renewable • W2B . Smart Electricity Service • S&C‟s CES . Smart Transportation • PHEV/EV . Smart Consumer • MDM • MYPOWER

IEEE GLOBECOM'11 159 SmartGridCityTM – Boulder, Colorado Collaborating to Build the Next Generation Utility “The fundamental component for making the smart grid work will be a robust and dynamic communications network; providing the utility the ability for real-time, two-way communications throughout the grid and enabling interaction with each component from fuel source to end use” (Xcel Smart Grid White Paper)

160 Status of SGC City - City of Boulder - 100,000 people, 50,000 homes Smart Meters - 14,398 as of 1/28/09 Premises - 16,616 BPL enabled homes as of 1/28/09 Telecom Fiber - 120 miles planned by June 2009 Delivery Dates - build out complete by 6/30/2009 Systems - plug and play demand and generation response (in process)

161 SMARTGRIDCITY – Key Values

Demand Management

• Reduce spinning Renewablesreserves Management • Generation following (not demand response) • Availability-based• pricing Align demand to availability Asset Management • Automated generation• Manage dispatch intermittency • Opt for type of energy use • Supply-based• Improve pricing field efficiency • Real-time assetPremise status Management & control • Expanded reliability • Extended asset• Automated life device response control • Real-time pricing (device-level) • New services and products • Enable customer choice

162 SmartGridCity-Objectives

Xcel Objective Measurement CURRENT Smart Grid Impact Improving Customer Satisfaction by Reduce SAIDI by 10% Distribution Automation reducing customer minutes out of service Analysis & Reporting of: . Incipient transformer failure . Secondary neutral failure . Voltage exceptions . Transformer Overload . Underground remote fault detection . Outage notification & restoration Empowering Customers to Reduce Decrease usage by 2.5% . 2-way thermostat control Electricity Usage . Demand response portals . Meter consumption reporting

Reduce Service and Billing Expense, Up to 50% annually . Call center meter pings Increase Revenue Assurance . Automated meter reading . Proactive maintenance (reduced O&M)

Decrease System Losses Reduce CO2 emissions up to 500,000 tons System Optimization annually . Conservation voltage reduction . Volt/Var Control . Phase Load Balancing Asset Optimization Reduce capital investment and . Substation monitoring distribution/substation maintenance up to $32 . Targeted asset replacement (system mil annually reports)

Develop a Smart Grid City Consortium Seamless integration of applications and Open GridTM Platform Framework business process

Develop a Regulatory Framework to TBD Smart Grid Value Model Recover Smart Grid Investment

163 Smart Grid Operational Impact Examples of items detected by a Smart Grid: Secondary Neutral Smart Grid Solutions: Connection ● 24x7 real-time distribution Sub LV Bus 6% network monitoring in use Xformer Tap Voltage Trees ● Dispatching work crews to repair 6% 1% problems detected by CURRENT 8% Secondary Brkr Smart GridTM Capacitor Tripped ● Underground fault detection 2% 30% installed ● Successful distribution automation switching trial Xformer Arrestor 25% 1%

Secondary Xformer 2% Lead/Connection Secondary/Xformer 10% 9%

 94% of the incidents detected avoided customer complaints  54% of the incidents detected avoided outages

SmartGridCityTM Consortium

164 Smart Renewable Grid Balancing Renewable Integration Outage Support Capital Cost Avoidance Emissions Savings Transmission Support Firm Renewable Power Pricing

Graph from John P. Benner, Manager, PV Industry Partnerships, National Renewable Energy Laboratory, 303-384-6496 165 Wind 2 Battery (W2B) Project Description

• 1 MW NaS Battery System • Can deliver 1 MW for 7 hrs • Power Conditioning Equipment • Wind farm/grid interconnection • Local and remote data and communication equipment

• Two Phases of Study • Understand how system could optimize wind farm economies • Understand how system could optimize utility integration of wind resources

166 Smart Electricity Service

167 COMMUNITY ENERGY STORAGE (CES)

Growth of Customer-Owned DG (solar) • Availability? • Reliability? • Safety? • Dispatch?

“Net Zero” or “Near Zero” Customers and Areas • Own their generation (solar or wind) • Grid-Independent (with storage) • Third-party storage service could take them off the utility grid

168 S&C‟S CES PROJECT-HARDWARE OVERVIEW CES is a small distributed energy storage unit connected to the secondary of transformers serving a few houses or small commercial loads

Key Parameters Value Power (active and reactive) 25 kVA Energy 25-75 kWh Voltage - Secondary 240 / 120V Battery - PHEV Li-Ion Round Trip AC Energy > 85% Efficiency

25 KVA

169 S&C‟s CES Project-A “Virtual” Substation Battery

CES is Operated as a Fleet providing Multi-MW, Multi- hour Storage

Local Benefits: Grid Benefits: • Backup power • Load Leveling at substation • Voltage correction • Power Factor Correction • Renewable Integration • Ancillary services

Integration CES Control Hub Utility Dispatch Communication Platform Center/ SCADA and Control Layout for CES

Substation

CES CES CES CES

Power Lines Communication and Control Links 170 SpeedNET Network Management

171 SpeedNet™ Radios: A Leading Solution for Self-Healing Applications Features Benefits Self healing—peer-to-peer Reliable performance even if a mesh network communication point is lost Improved performance, less susceptible Multi-level security to interference High speed communications—shorter Low latency restoration times

Assignable messaging Effectively serves both AMI backhaul priority and DA applications

172 Smart Transportation

Electric Drive & Energy / Power Storage Electronic Components Systems Integration of the FEV in cooperative transport Vehicle 2 Grid Infrastructure Interface

Communication Functional Safety Architecture for Energy, Vehicle Stability & durability of the FEV Communication & Control thermal management,

173 Electric Drive Vehicles • Until now, base growth of 1% per year for USA system – At 25% of US vehicle fleet is “only” 2% of total MW*hr (but billions of $ in generation and distribution costs) – On distribution a car‟s 6 KW connection for an average home‟s peak usage of 3 KW is +200% & is very significant

http://www.ornl.gov/info/ornlreview/v41_1_08/regional_phev_analysis.pdf

174 2007 Xcel Energy / NREL PHEV Study

* Could be mitigated with control technology / incentives

Scenarios Production Cost Capacity Cost Avoided Gasoline Emissions Distribution Impacts

Do Nothing Good Worse* Good Better Worse*

Delay to 10pm Better Best Good Good Best

Optimized to Off-peak Best Best Good Worse Best

Opportunity Charging Worse Worse* Best Best Worse*

• For any utility: For Xcel Energy with night time coal base  Time of charging matters… load:  Coincident peak loading matters…  Tailpipe versus upstream Smart Charge after 10 PM avoids Capital emissions matter… Costs and Green House Gasses 175 2008 Xcel Energy / NREL PHEV Study

• 6 Converted Ford Escapes (3 fleet, 3 personal use) and driven 40 miles per day (as do 85% of US commuters) at $7500 / car

• Results (yet not statistically significant) Used only top 1/3 of 25 mile battery pack (parallel hybrid) Averaged over 6 months, 56.84 MPG in a SUV at $0.03 vs $0.11* Extremely consistent availability (except Sunday post 5:00 PM) Plugged In MORE often over time (from 50% to 80% over 6 months) Availability to utility at 60% - 85% with all factors considered Infrastructure is EVERYWHERE - “power to the curb” is there but what is the “tipping point”?

* at $2.00 / gal gas for 18 MPG for 12,000 per year at with $0.08 / kW*hr * payoff at $7,500 cost to implement is 93,750 miles or 7.8 years while GM’s Volt is expected to have 140 MPG or 3.2 year payoff

Photo by ASC Designs 303-522-0066

176 Impacts from PHEVs & EVs

 Without SmartCharging: 130 new power plants needed with 25% PHEV/EV penetration (source: ORNL), but still 40% less emissions when “filled” with coal based generation

 With SmartCharging: Theoretically ZERO new power plants needed (source: ORNL) until 73% of total fleet with generation “valley fill”

 With SmartCharging: Reduce to 85% fewer car emissions by reducing total number of power plants (source: NREL, and being studied by Xcel Energy)

177 Smart Consumer

178 Meter Data

179 Meter Data Management (MDM)

Multiple Accurate Secure Create and data and timely data disseminate sources data storage information

• Securely manages • Validating, • Interface to billing 1,000 times more Editing and systems data/meter than CIS • AMI Estimating (for • Interface for Customer or AMI systems can. • Manual hourly data) Service Reps • Tags for weather, Readings • Standards and • Create TOU billing demographic and • SCADA rules for service summaries other operational • OMS order creation • Provide summary data characteristics • MWF • Proactive • Support operation & • Manage and access • Other assurance of planning needs non-traditional meter data availability • Platform for customer data, e.g., PQ, volts, • Audit trail web presentment etc.

180 Combined data flow

G&T: • Data for M&V of load control • Class level data from each EMC • Demographic data for planning • Other AMI/AMR CIS Systems: Systems: • NISC • RF • SEDC • PLC Data input, • Daffron • Drive-by • Others Other inputs: validation and • Handhelds warehouse • SCADA • Manual data Analytics: Operational Web • Weather data • Revenue Support for AMI: Presentment: • ??? Protection • Business rules • Meter data • System loss • Service order • Customer and analysis interpretation billing data • Planning • Demographic or • Cost of Service MDMS other data • Others

181 MDM Vendors

SIEMENS

182 myPower Pricing Pilot Overview

Control Group myPower Sense myPower Connection

Customers 450 Residential 379 Residential 319 Residential

Rate* RS TOU-CPP (RSP) TOU-CPP (RSP)

Electric interval meter Electric interval meter Electric interval meter Programmable thermostat Equipment Two-way communications infrastructure - PLC, RF, Hybrid

N/A Mail Mail E-mail E-mail Customer Education Telephone Telephone and Communication Signal to thermostat

Usage and Billing N/A Internet Internet Information

* RS = Residential Service, TOU-CPP = Time-of-Use, Critical Peak Pricing

183 myPower Time-of-Use – Critical Peak Pricing (TOU-CPP)Summer 2007 Pricing Plan

Weekdays Weekends June - September June - September $1.46 28 Critical 28 Price

24 24

20 23.7 ¢ 20 High Price 16 (On- Peak) 16

Standard Residential Rate Standard Residential Rate

Price in cents Priceperin kWh 12 12 Price in cents per kWh per Price cents in

8 8 8.7¢ 8.7¢ 8.7¢ Medium Medium Medium Price Price Price 4 (Base (Base 4 (Base Price) Price) Price) 3.7¢ 3.7¢ Low Price Low Price (Night Discount) (Night Discount) 0 0 6P M 10 PM 9 AM 9 AM 1 PM 9 AM 10 PM 9 AM Time of Day Time of Day 184 myPower Sense Customers Time-of-Use and Critical Peak Impacts

With Central AC on Summer Peak Days Without Central AC on Summer Peak Days

4.0 4.0 Average kW Average kW per Customer per Customer 3.5 3.5 CPP TOU 3.0 3.0 Baseline

2.5 2.5

2.0 2.0

1.5 1.5 CPP TOU 1.0 1.0 Baseline

0.5 0.5

Night Base On-Peak Base Night Base On-Peak Base 0.0 0.0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Hour Ending Hour Ending

Source: myPower Pricing Pilot results based on 2006 and 2007 data through September 30, 2007 Customers who received no in-home technology were able to reduce On-Peak period demand on critical peak days by up to 20%, even if they do not have Central AC. 185 myPower Connection and myPower Sense Customers Summer Period Energy Savings Estimates

Control Participant Summer Total Summer Group Group Energy Savings Energy Savings Variable Change in Change in from TOU from TOU Use Use (Percent) (kWh per Cust) myPower Connection 5.2% - 1.9% = 3.3% 139 myPower Sense with 5.2% - 1.5% = 3.7% 144 Central AC myPower Sense without 6.4% - 2.1% = 4.3% 127 Central AC

Source: myPower Pricing Pilot results based on 2006 and 2007 data through September 30, 2007

• Both the myPower participant and the Control Group customers showed increases in summer usage compared to prior years • The increase in usage in the myPower participants‟ segments was significantly smaller than the Control Group. • An overall energy savings estimate is developed by examining the difference between the Control Group‟s and participant groups‟ increase in energy use. Customers who participated in myPower achieved summer period energy savings in the range of 3-4%. 186 Prototype on WSN for line monitoring • Use a hierarchical communication topology [Casey_2011] – Avoid single point failure of sensors that is common in a multi- hop sensor network • Main characters of the implemented system – Sensors • Self –Configurable • Remote-controllable • Able to adjust the data sampling frequency automatically – E.g. Increase the sampling frequency from 10 minutes to 5 seconds when a fault is detected – Gateway • Does not forward sensor packets until – A full WLAN packet (about 18 sensor packets) has been accumulated, or – Timeout happens

• [Casey_2011] P. Casey, N. Jaber, and K. Tepe, “Design and Implementation of a Cross-Platform Sensor Network for Smart Grid Transmission Line Monitoring”, Proceedings IEEE SmartGridComm 2011.

187 Hardware and software implementation • Hardware – Gateway • Encompasses a ZigBee mote on Crossbow MIB510 programming board connected to a laptop • ZigBee mote: Crossbow Micaz mote that utilizes the Chipcon CC2420 radio • Linksys WUSB54GC as the WLAN interface – Sensor node • A standalone ZigBee mote with a sensor board (Crossbow MTS300CA) • Software – TinyOS-2.x for the sensor – Ubuntu 8.10 for the laptop (gateway)

188 Hierarchical communication topology

• Using ZigBee for communications between sensors and gateways

• Using 802.11 to build a mesh network among gateways • Control center is the sink node • The communication system for line monitoring is reliable since – Both ZigBee and WLAN are reliable for this smart grid application

189 Field trial on PLC for smart meter applications

• PRIME (PoweRline Intelligent Metering Evolution) – A narrowband power line communications (PLC) technology targeted for use in smart metering applications – Use OFDM techniques and well-known forward error correction mechanisms, novel discovery and network-building MAC procedures – Allow for cost-effective, seamless integration with recognized standard metering protocols such as DLMS/COSEM – could become a globally recognized industry standard

• This paper [Berganza_2011] presents results obtained from real- field multi-vendor deployments with PRIME-compliant interoperable implementations at Iberdrola network in Spain.

• [Berganza_2011] I. Berganza, A. Sendin, A. Arzuaga, M. Sharma, and Badri Varadarajan, “PRIME on-field deployment - First summary of results and discussion”, Proceedings IEEE SmartGridComm 2011.

190 Main lessons learned from the field trials

• Signal interference due to misconfigurations – Two concentrators were deployed on the two transformers in a same substation • Beacons collide in the time domain – Some service nodes are jumping between the two subnetworks • Should only set one concentrator, and set others as switches • Unreliable communications when not all meters governed by a substation are PRIME meters – The signal-to-noise ratio might not be high enough

191 PLC communication for remote areas • This paper [Kikkert_2011]describes an accurate SWER line model – Single Wire Earth Return (SWER) lines are used in Australia, USA, South Africa and many other countries to provide power to remote communities – The model demonstrate the severe signal channel degradation that can occur due to line branches and coupling networks – The model is verified with measurements from two sites in Australia. – Data rates are at 22.8 kbps on a 14 km SWER line • when the attenuator is set to less than or equal to 15 dB attenuation – Predict that PLC communication systems using G3-PLC modems on SWER lines in excess of 2000 km are feasible

• [Kikkert_2011] C. Kikkert, “Effect of Couplers and Line Branches on PLC Communication Channel Response”, Proceedings IEEE SmartGridComm 2011.

192 Device communications using SCADA systems • Communications in traditional power grid are mainly enabled by a centralized supervisory control and data acquisition (SCADA) system • In [Lu_2011], they establish a monitoring system for a Solid State Transformer (SST) in a micro smart grid - Green Hub – To verify that SCADA system can be used to support such an application – The one megawatt Green Hub system is a power electronics based power system in the FREEDM systems center at the North Carolina State University. • It is established to demonstrate salient features and capabilities of the FREEDM system on renewable energy generation, distribution, storage and management

• [Lu_2011] X. Lu, W. Wang, A. Juneja, and A. Dean , “Talk to Transformers: An Empirical Study of Device Communications for the FREEDM System”, Proceedings IEEE SmartGridComm 2011.

193 Implementation of SST monitoring system

• In the network domain, a control center is connected to the SST controller – via a Local Area Network (LAN) • DNP3 is overlayed over TCP/IP in he implementation – DNP3 (distributed network protocol 3.0) is a widely-adopted SCADA protocol

194 Conclusions and lessons • Conclusion – The DNP3 based SCADA system can be used in the smart grid • for the device monitoring and control • Lessons – A careful optimization is crucial to reduce the total delay • By optimizing every time-consuming part of every system component • Delay is the primary concern for most smart grid applications – The DNP3-based monitoring system is not suitable for more time stringent applications like relay protection • The architecture is too complex and induce extra delay

195 Open Research Issues • Cost-Aware Data Communication and Networking Infrastructure • Quality-of-Service (QoS) Framework • Optimal Network Design • Need of Secured Communication Network Infrastructure • Plug-in Hybrid Electric Vehicle (PHEV)

IEEE GLOBECOM'11 196 Open Research Issues Cost-Aware Data Communication and Networking Infrastructure • There is a cost in retrieving the real-time information (e.g., power pricing, metering data, and surveillance data), which increases with the increase in frequency of inquiry • However, the performances such as latency, bandwidth, reliability must be met • The cost optimization for data monitoring and transferred must be performed

IEEE GLOBECOM'11 197 Open Research Issues Quality-of-Service (QoS) Framework • The QoS in smart grid can be defined by accuracy and effectiveness with which different information such as equipment‟s state, load information, and power pricing are delivered timely to the respective parties • QoS framework can be developed by identifying the specific QoS requirements and priorities for specific communication network in smart grid

Maximum Latency Communication Type ≤ 4 ms Protective relaying Sub-seconds Wide area situational awareness monitoring Seconds Substation and feeder supervisory control and data acquisition (SCADA) Minutes Monitoring noncritical equipment and marketing pricing information Hours Meter reading and longer-term pricing information Days/Weeks/Months Collecting long-term usage data

IEEE GLOBECOM'11 198 Open Research Issues Optimal Network Design • Dedicated network can be built to support the QoS- and security-sensitive smart grid applications (e.g., status monitoring and time-of-use report) • Optimal network devices, their connections, and protocols have to be chosen to avoid congestion and failure • Shared network (e.g., cellular service) can be used to support noncritical smart grid applications (e.g., billing) • Integration of dedicated and shared network can be explored

IEEE GLOBECOM'11 199 Open Research Issues Need of Secured Communication Network Infrastructure • If smart grid is attacked, the hackers can penetrate the network and alter critical system parameters which could destabilize the grid in an unpredictable way causing nationwide crisis • Intrusion detection and prevention for smart grid (e.g., AMI and WAMS) • Public key infrastructure (PKI) for smart grid

IEEE GLOBECOM'11 200 Open Research Issues Plug-in Hybrid Electric Vehicle (PHEV) • With the use of electric power, PHEV has lower operational

cost and smaller emission of CO2 • PHEV requires electric charging from charging station • To ensure stabilized load, electric power has to be supplied according to the demand from PHEV • Communications intrastructure for PHEV charging can be proposed (e.g., [Erol-Kantarci 2011]) – Utility company communicates with substation control center (SCC) using WiMAX and charging station using wireless mesh network – SCC decides to accept or refuse the charging request from PHEV

IEEE GLOBECOM'11 201 Conclusion

• Smart grid will be a crucial technology to improve the efficiency of the power grid • There are many issues related to data communications and networking

IEEE GLOBECOM'11 202 References (1)

[Niyato_2011] D. Niyato, L. Xiao, and P. Wang, "Machine-to-machine communications for home energy management system in smart grid," IEEE Communications Magazine, vol. 49, no. 4, pp. 53-59, April 2011. [Bruce_1998] A. G. Bruce, “Reliability analysis of electric utility SCADA systems,” IEEE Transactions on Power Systems, vol. 13, no. 3, pp. 844-849, August 1998. [Xie_2002] Z. Xie, G. Manimaran, V. Vittal, A. G. Phadke, and V. Centeno, “An information architecture for future power systems and its reliability analysis,” IEEE Transactions on Power Systems, vol. 17, no. 3, pp. 857- 863, August 2002. [Wang_2010] Y. Wang, W. Li, and J. Lu, “Reliability analysis of wide-area measurement system,” IEEE Transactions on Power Delivery, vol. 25, no. 3, pp. 1483-1491, July 2010. [Erol-Kantarci 2011] M. Erol-Kantarci, J. H. Sarker, and H. T. Mouftah, "Communication-based Plug-In Hybrid Electrical Vehicle load management in the smart grid," IEEE Symposium on Computers and Communications (ISCC), pp. 404-409, June 2011-July 2011. [Liu_2010] E. Liu, M. L. Chan, C. W. Huang, N. C. Wang, and C. N. Lu, "Electricity grid operation and planning related benefits of advanced metering infrastructure," International Conference on Critical Infrastructure (CRIS), pp.1-5, September 2010. [Mao_011] R. Mao and V. Julka, "Wireless Broadband Architecture Supporting Advanced Metering Infrastructure," IEEE Vehicular Technology Conference (VTC Spring), pp.1-13, May 2011. [Chen_2010] S. Chen, J. Lukkien, and L. Zhang, "Service-oriented Advanced Metering Infrastructure for Smart Grids," Asia-Pacific Power and Energy Engineering Conference (APPEEC), pp.1-4, March 2010. [Yu_2011] R. Yu, Y. Zhang, S. Gjessing, C. Yuen, S. Xie, and M. Guizani, "Cognitive radio based hierarchical communications infrastructure for smart grid," IEEE Network, vol. 25, no. 5, pp. 6-14, September-October 2011. [Wang_2007] Y. Wang, J. P. Lynch, and K. H. Law, “A wireless structural health monitoring system with multithreaded sensing devices: Design and validation,” in Structure and Infrastructure Engineering, vol. 3(2), pp. 103-120, 2007. [Ullo_2010] S. Ullo, A. Vaccaro, and G. Velotto, “The role of pervasive and cooperative sensor networks in smart grids communication,” in Proceedings of IEEE Mediterranean Electrotechnical Conference (MELECON), pp. 443-447, April 2010. [Yan_2011] Y. Yan, Y. Qian, and H. Sharif, “A secure and reliable in-network collaborative communication scheme for advanced metering infrastructure in smart grid,” in Proceedings of IEEE Wireless Communications and Networking Conference (WCNC), pp. 909-914, March 2011. [Sreesha_2011] A. A. Sreesha, S. Somal, and I-Tai Lu, “Cognitive radio based wireless sensor network architecture for smart grid utility,” in Proceedings of IEEE Long Island Systems, Applications and Technology Conference (LISAT), May 2011.

IEEE GLOBECOM'11 203 References (2)

[Hung_2010] K. S. Hung, W. K. Lee, V. O. K. Li, K. S. Lui, P. W. T. Pong, K. K. Y. Wong, G. H. Yang, and J. Zhong, "On Wireless Sensors Communication for Overhead Transmission Line Monitoring in Power Delivery Systems," IEEE International Conference on Smart Grid Communications (SmartGridComm), pp. 309-314, October 2010. [Naduvathuparambil_2002] B. Naduvathuparambil, M. C. Valenti, and A. Feliachi, "Communication delays in wide area measurement systems," Proceedings of the Thirty-Fourth Southeastern Symposium on System Theory, pp. 118- 122, 2002. [Shahraeini_2011] M. Shahraeini, M. H. Javidi, and M. S. Ghazizadeh, "Comparison Between Communication Infrastructures of Centralized and Decentralized Wide Area Measurement Systems," IEEE Transactions on Smart Grid, vol. 2, no. 1, pp. 206-211, March 2011. [Mohsenian-Rad_2010] A. Mohsenian-Rad, V. W. S. Wong, J. Jatskevich, R. Schober, and A. Leon-Garcia, "Autonomous Demand-Side Management Based on Game-Theoretic Energy Consumption Scheduling for the Future Smart Grid,” IEEE Transactions on Smart Grid, vol. 1, no. 3, pp. 320-331, December 2010. [Hirst_1998] E. Hirst and B. Kirby, “Unbundling Generation and Transmission Services for Competitive Electricity Markets: Ancillary Services”, NRRI-98-05, National Regulatory Research Institute, Columbus, OH, Jan. 1998 [Giani_2011] A. Giani, E. Bitary, M. Garciay, M. McQueenz, P. Khargonekarx, and K. Poolla, “Smart Grid Data Integrity Attacks: Characterizations and Countermeasures”, Proceedings of IEEE SmartGridComm 2011. [Vukovic_2011] O. Vukovic, K-C Sou, G. Dan, and H. Sandberg, “Network-layer Protection Schemes against Stealth Attacks on State Estimators in Power Systems”, Proceedings of IEEE SmartGridComm 2011. [Tajer_2011] A. Tajer, S. Kar, V. Poor, and S. Cui, “Distributed Joint Cyber Attack Detection and State Recovery in Smart Grids”, Proceedings of IEEE Globecom 2011. [Esmalifalak_2011] M. Esmalifalak, H. Nguyen, R. Zheng, and Z. Han, “Stealth False Data Injection using Independent Component Analysis in Smart Grid”, Proceedings of IEEE SmartGridComm 2011. [Kolesnikov_2011] V. Kolesnikov, W. Lee, and J. Hong, “MAC Aggregation Resilient to DoS Attacks”, Proceedings of IEEE SmartGridComm 2011. [Zhu_2011] T. Zhu, S. Xiao, Y. Ping, D. Towsley, and W. Gong, “A Secure Energy Routing Mechanism for Sharing Renewable Energy in Smart Microgrid”, Proceedings of IEEE SmartGridComm 2011. [Jokar _2011] P. Jokar, H. Nicanfar, V. Leung, “Specification-based Intrusion Detection for Home Area Networks in Smart Grids”, Proceedings of IEEE SmartGridComm 2011.

IEEE GLOBECOM'11 204 References (3)

[Chim_2011] T. Chim, S. Yiu, L. Hui, and V. Li, “PASS: Privacy-preserving Authentication Scheme for Smart Grid Network”, Proceedings of IEEE SmartGridComm 2011. [Rajagopalan_2011] S. Rajagopalan, L. Sankar, S. Mohajer, and V. Poor, “Smart Meter Privacy: A Utility-Privacy Framework”, Proceedings of IEEE SmartGridComm 2011. [Kim_2011] Y. Kim, E. Ngai, and M. Srivastava, “Cooperative State Estimation for Preserving Privacy of User Behaviors in Smart Grid”, Proceedings of IEEE SmartGridComm 2011. [Casey_2011] P. Casey, N. Jaber, and K. Tepe, “Design and Implementation of a Cross-Platform Sensor Network for Smart Grid Transmission Line Monitoring”, Proceedings IEEE SmartGridComm 2011. [Berganza_2011] I. Berganza, A. Sendin, A. Arzuaga, M. Sharma, and Badri Varadarajan, “PRIME on-field deployment - First summary of results and discussion”, Proceedings IEEE SmartGridComm 2011. [Kikkert_2011] C. Kikkert, “Effect of Couplers and Line Branches on PLC Communication Channel Response”, Proceedings IEEE SmartGridComm 2011. [Lu_2011] X. Lu, W. Wang, A. Juneja, and A. Dean , “Talk to Transformers: An Empirical Study of Device Communications for the FREEDM System”, Proceedings IEEE SmartGridComm 2011.

IEEE GLOBECOM'11 205