Development of a Control Platform for a Building-Scale Hybrid Solar PV-Natural Gas Microgrid

energies

Article Development of a Control Platform for a Building-Scale Hybrid Solar PV-Natural Gas Microgrid

Parhum Delgoshaei 1,* and James D. Freihaut 2

1 Engineering Division, Penn State Great Valley, Malvern, PA 19355, USA 2 Department of Architectural Engineering, Penn State, University Park, Philadelphia, PA 16802, USA; [email protected] * Correspondence: [email protected]

Received: 8 September 2019; Accepted: 16 October 2019; Published: 4 November 2019

Abstract: Building-scale microgrids are a type of behind-the-meter microgrids where the building operator has control of the distributed energy resources, including, in this case, a natural gas-fired microturbine in addition to solar PV and battery energy storage systems. There is a growing trend in deploying behind-the-meter microgrids due to their benefits including the resiliency of serving critical loads, especially in regions with abundant natural gas. In order to ensure distributed energy resources are dispatched optimally for the desired mode of operation, a hierarchical control platform including a centralized controller was developed and installed. The platform includes communication and control infrastructure that interface with controllers for distributed energy resources and the building automation system of a recently built energy efficient commercial building. Based on desirable outcomes under different grid and building conditions, operational modes were defined for the microgrid controller. The controller is programmed to map each mode to respective operational modes for distributed energy resources controllers. Experimental data for test runs corresponding to two operational modes confirm the communication and control infrastructure can execute hierarchical control commands. Finally, dispatch optimization for a year-long simulation of system operation is presented and the benefits of the hybrid solar PV-natural gas setup are evaluated.

Keywords: microgrids; hierarchical control; microturbines; hybrid solar PV-gas systems; distributed energy resource dispatch optimization; battery management systems; demand response

1. Introduction Microgrids are electricity distribution systems containing loads and distributed energy resources (such as distributed generators, storage devices, or controllable loads) that can be operated in a controlled, coordinated way either while connected to the main power network or while islanded [1]. Integrating distributed energy resources (DERs) including renewable sources such as solar PV, nonrenewable sources such as natural gas-powered microturbines, and energy storage in the boundaries of a microgrid (behind-the-meter) in the low voltage (LV) and (MV) scale can potentially provide a large number of beneﬁts to the main power utility by increasing its eﬃciency of operation and to the customer by improving reliability and quality of service [2]. Reductions in solar photovoltaic (PV) installation costs [3,4] have made it a viable, albeit intermittent DER when favorable interconnection conditions exist and space is available. Factoring in short-term stochastic variability of the solar resource (such as passing clouds) leads to a 6% to 15% decrease in capacity values for solar PV as a DER [5]. Forecasting approaches have been proposed [6–8], and [9] to predict the variability of the solar resource and assist in more accurate sizing of system

Energies 2019, 12, 4202; doi:10.3390/en12214202 www.mdpi.com/journal/energies Energies 2019, 12, 4202 2 of 30 components including battery energy storage systems that are integrated as a DER to compensate for the intermittency of renewables among other reliability and resiliency benefits that they provide [10–12]. Although non-renewable DERs such as natural gas lack the carbon reduction benefits, in situations that continuity of serving the load can be economically justified, they offer added reliability and resiliency and added energy efficiency benefits when they are utilized in combined heat and power (CHP) applications as they don’t suffer from the intermittency of renewables [13,14]. The benefits of combining availability of renewable resources, battery energy storage systems to mitigate their intermittency, and natural gas microturbines with reliable dispatchability as DER resources of a microgrid can be best realized when the deployment and dispatch command for each DER is controlled behind the meter in coordination with managing the load of the microgrid, however, come at cost of the challenges associated with local control and dispatch of these DERs. With reductions in the cost of DER installations and the increased value placed on the benefits of integrating them in industrial, military, academic, and commercial microgrids (“behind-the-meter” campus/community-scale microgrids), increasingly these facilities have operated their DERs using their microgrid control systems [15–20]. On a smaller scale, the same concept has been implemented on a low-voltage (LV) scale in the confines of a single building. For example, in [21], monitoring and control of solar PV, a microturbine, and battery energy storage as DERs of a low voltage microgrid in a university laboratory setup are documented. Authors in [22] demonstrate controlling the operation of an LV microgrid consisted of a generator and battery energy storage, and in [23], the dynamics of operating a microturbine in an LV network are simulated. In [24], the dynamics of operating a microturbine and a battery energy storage system are studied; this does not cover how those DERs interact with a building as their load. While these microgrids have many similarities to their larger community-scale counterparts, a common observation of these systems as a representative of LV microgrids is that they are experimental setups and do not serve main building loads. An exception is [25], where the operation of both renewable and non-renewable DERs in an LV microgrid is covered in both islanded and grid-connected setups although the analysis is limited to electrical variables (voltage and frequency) and not how the control system manages DER power generation and load power consumption. There is a need to study how the control platform in building-scale microgrids can perform energy management for renewable and non-renewable energy resources, battery energy storage, building loads, and electric utility at the same time. The building-scale microgrid associated with the control platform in this paper fills this knowledge gap in the context of demonstrating how all DERs can be controlled for a common application (i.e., demand curtailment). It should be noted that in the smaller building-scale microgrids, the communication and control infrastructure is dependent upon interfacing building-level controls with controls utilized in distributed generation such as inverters and microturbines whereas the larger “feeder-level” microgrids rely on the mature communication and control infrastructure utilized in electrical distribution networks for years. Moreover, perhaps not surprising, economies of scale, including the collective efficiency and resiliency benefits of the DERs, work more in favor of community-scale microgrids. The main aim of this paper is to provide the details of developing a hierarchical control platform for a low-voltage, behind-the-meter building-scale microgrid in an energy-efficient commercial building from the central microgrid controller to renewable and non-renewable DERs and the battery energy storage system. In addition, test and simulation of the system in action is covered. In the following sections, the components, communication infrastructure, and control(simulated) resulting in the optimized dispatch of the DERs of this building-scale microgrid will be contrasted against the similar areas of their larger scale counterparts (feeder-level microgrids) to achieve an economic goal (demand peak reduction) and resiliency. This paper is organized as follows: Section2 covers microgrid communication and control. In Section3, the building-scale microgrid implementation is presented. Section4 covers the description of the components of the installed control platform for the microgrid, their function, and different system Energies 2019, 12, 4202 3 of 30 and component modes operation. Section5 covers two tests of the control system and simulation of one of its modes of operation. Principal conclusions are drawn in Section6. Future directions are finally discussed in Section7.

2. Hierarchical Control in Microgrids In this section, we will cover why there are multiple conceptions of microgrid control in literature and how control is accomplished in diﬀerent layers from the utility side all the way to the control of distributed energy resources. Using the terminology covered in this section, we will specify the control layers that the developed platform in this paper can execute in its current form and its extended form as additional funding is secured for the project (covered in the future directions section).

2.1. Functions of a Microgrid Controller We should first clarify the functions that a controller can coordinate to be fulfilled [26], including the following five main functions:

1. Uninterruptible supply of sensitive loads 2. Seamless disconnection (island mode) and reconnection: 3. Delivery of active power and reactive power per the needs of the microgrid and/or the distribution system 4. DER operation per assigned set point within their operational limits 5. DER generation optimization while factoring in exchanges with the utility and market participation

These functions can be grouped into broad categories: those that are associated with connecting or disconnecting and stable operation (islanding) within a predeﬁned time per standards (1 and 2) and those that are associated with assigning set points for DERs and loads (3, 4, and 5). When referencing microgrid control, a product [27,28], for example, or a scholarly work [29], as a sample, may refer to the ﬁrst category (controlling connection/disconnection) as the function of a microgrid controller. Alternatively, scholarly works such as [30] (utilizing a dedicated controller in this category from the same vendor in [28]) subscribe to the second conception (controlling set points and optimizing DER operation) for a microgrid controller. The controller for this building-scale microgrid discussed, in Section4, also falls in this category. Highlighting this distinction is important as each conception has its own design and implementation challenges which dictates the focus on the topics covered in the rest of this section.

2.2. Hierarchical Control Layers in Microgrids A hierarchical control strategy is a common approach for controlling microgrids [26,31]. The first level in the hierarchy, primary control, is implemented in local controllers (also called primary controllers, Figure1) of distributed energy resources (DERs) for controlling either voltage and frequency (in islanded mode)—characterized as “Primary Control-Power Sharing Control” in [31] or controlling active and reactive power (in grid-connected mode), characterized as “Primary Control-Inverter Output Control” in [31]. The secondary level, when implemented by a central controller as is the case in this work, is tasked with making decisions of the dispatch of DERs and serving loads, dictating their power set points, and economic optimization of the dispatch. It should be noted that the mapping of functions to hierarchy levels differs between authors. For example, [32] affixes the functions listed earlier as secondary control (also classified as such in [2,31]) to the tertiary level of the hierarchy. Energies 2019, 12, x FOR PEER REVIEW 4 of 30

Energies 2019, 12, x FOR PEER REVIEW 4 of 30 Energies 2019, 12, 4202 4 of 30

Figure 1. Multilevel control of microgrids. FigureFigure 1.1. MultilevelMultilevel controlcontrol ofof microgrids.microgrids. Closely related to the primary-secondary-tertiary classification of the hierarchical layers of controlCloselyClosely [31], is related related the three to to- thelevel the primary-secondary-tertiary primarydelineation-secondary in [2] (Figure-tertiary 2). classification classification The Microgrid of of Optimizer the the hierarchical hierarchical Layer in layers layers Figure of of 2controlcontrol corresp [[31]31onds],,is is theto the functions three-level three-level in delineation thedelineation secondary in in [ 2and[2]] (Figure (Figure tertiary2). 2). Thelevel The Microgrids .Microgrid The Dispatch Optimizer Optimizer Layer Layer correspondsLayer in Figurein Figure 2to thecorresponds2 corresp secondaryonds to andtofunctions functions primary in thein level the secondary ssecondary (inverter and and output tertiary tertiary control). levels. level The s Finally,. The Dispatch Dispatch the LayerConsortium Layer corresponds corresponds for Electric to the to Reliabilitysecondarythe secondary and Technology primary and primary levels Solutions (inverterlevel s ( CERTS(inverter output) Autonomous control). output control). Finally, Layer the Finally, Consortium(discusse the dConsortium forlater Electric in this for Reliability Electricpaper) correspondsTechnologyReliability SolutionsTechnologyto the primary (CERTS) Solutions level Autonomous (power (CERTS-sharing) LayerAutonomous control) (discussed and Layer functions later in(discusse this related paper)d to later transition corresponds in this between paper to the) gridprimarycorresponds-connected level to (power-sharing the and primary islanded level control) modes (power and of- sharing functions operation control) related including and to transitionfunctions switches, betweenrelated disconnect to grid-connected transition devices, between and and protectionislandedgrid-connected modes. The of and three operation islanded-layer including approach modes switches, of of operation [2] disconnectwill including be adopted devices, switches, in and this protection. disconnect paper to The describe devices, three-layer andthe implementationapproachprotection of. [ 2 The] willof the three be control adopted-layer platform approach in this paper in ofSection to[2] describe will 5 as bethis the adopted approach implementation in of this grouping paper of the the control to layers describe platform of the the hierarchyinimplementation Section 5closely as this ofaligns approach the controlwith ofthe groupingplatform current (Dispatchin the Section layers Layer)5 of as the this and hierarchy approach future closely (Microgrid of grouping aligns Optimizer, with the layers the currentSection of the 7)(Dispatchhierarchy state of the Layer)closely project. and aligns The future with challenges (Microgrid the current and Optimizer, benefits(Dispatch of Section Layer)a potential7 and) state future of the implementation(Microgrid project. The Optimizer, challenges of the CERTSSection and Autonomousbenefits7) state of the a potential Layer project. willfuture The be challenges discussed implementation andin the benefits ofcontext the CERTSof ofa potential the Autonomous Microgrid future Control implementation Layer will States be discussedin ofSection the CERTS in4. theIn thecontextAutonomous following, of theMicrogrid thisLayer layer will Controlwill be discussedbe briefly States indiscussed.in Section the context4. In theof the following, Microgrid this layerControl will States be briefly in Section discussed. 4. In the following, this layer will be briefly discussed.

Figure 2. Architecture of the control layers for the resilient microgrid controller reprinted by Figure 2. Architecture of the control layers for the resilient microgrid controller reprinted by permission permissionofFigure the publisher 2. Architectureof the (Taylor publisher & of Francis (Taylor the control Ltd, & Francis http: layers//www.tandfonline.com Ltd, for http://www.tandfonline.com) the resilient microgrid)[2]. controller [2]. reprinted by permission of the publisher (Taylor & Francis Ltd, http://www.tandfonline.com) [2].

Energies 2019, 12, 4202 5 of 30

During the stand-alone (islanded) mode of operation, voltage and frequency of the microgrid are no longer determined by the host grid and thus should be controlled by primary controllers (power-sharing control, [31]) of DERs—a control function complimentary to DERs primary controller (inverter output control, [31]) used to control real and reactive power levels in the grid-connected mode of operation. In the islanded mode, the central microgrid controller oversees load sharing mechanisms and communicates appropriate set points to diﬀerent local low-level DER controllers and controllable loads. A small mismatch of the amplitude, phase angle or frequency of the output voltage of any DER unit can lead to a high circulating current. The implementation of Consortium for Electric Reliability Technology Solutions (CERTS) [18,19] microgrid concept has demonstrated how the reliability of a microgrid can be increased by enabling plug-and-play DER addition without extra communications. Each DER in a CERTS paradigm regulates voltage and frequency and balances real power through a power versus frequency droop controller (implemented in the primary controller of all DERs). To control the reactive power, a voltage versus reactive power droop controller is implemented in the primary controller of all DERs. In order for an inverter to operate in the CERTS paradigm, it would need to be able to operate as a voltage-source inverter and have a stable dc bus voltage during load transients [33]. In this project, this is ensured for the built-in microturbine inverter. While most grid-connected PV inverters are only capable of functioning as current-source inverters, the selected inverter in this project can cover this requirement and also operate as a voltage-source inverter (details of both to follow in Section4).

2.3. Communications for Hierarchical Control Since different microgrid control functions are mapped to the different levels of the hierarchy, which are implemented in different devices, including the microgrid controller, there should exist the means to carry control signals between them, especially between the microgrid controller and DERs. Microgrids can be implemented in different scales based on their point of interconnection: full substation microgrid, full feeder microgrid, partial feeder microgrid, and single customer microgrid. Accordingly, they have different hardware integration and communication and control requirements. Although the foundation of a microgrid is based on lateral integration of its fundamental assets, intelligence in a smart microgrid is based on vertical integration of upper-layer applications with lower-layer components. For example, an application such as demand response may not be feasible without tight integration between assets functioning and the different layers of the hierarchy, which provide inputs and are influenced by the real-time status of the system. This tight integration is possible by a microgrid communication system utilizing different protocols at different layers. At the highest level, a microgrid supervisory control and data acquisition (SCADA) [2] collects the real-time status and through interaction with the system operator, and the microgrid management software enables the microgrid to fulfill different applications such as load shedding, demand response, and islanding/reconnection. Different protocols enable communication between different microgrid layers as shown in Figure3[2]. As seen in Figure1, in the lowest level (interaction with loads), the Open ADR (Open Automated Demand Response) is used for communication for the microgrid system in [2]. In the next level, Modbus is used for inverter and generator control systems and human machine interface (HMI). It can be transmitted over different communication links such as RS-232/RS-485 and ethernet. Moving up in the hierarchy, IEC 61850 is a substation automation standard. This relatively new standard has been touted as one of the key solutions to microgrid control and communication. As seen in Figure3, the central gateway that implements microgrid control at the highest level (SCADA) communicates with the microgrid controller and individual DER controllers using the IEC 61850 protocol. This is the same approach followed by Eaton in implementing their microgrid controller, including the system installed at Fort Sill [2]. The next higher-level protocol is DNP3. The Distributed Network Protocol (DNP3) is mainly used in utilities such as electric and water companies. As seen in Figure3, this protocol is key in enabling the electric utility to communicate with the microgrid controller. It is also the protocol used Energies 2019, 12, 4202 6 of 30 by PJM (PJM Interconnection, Valley Forge, PA, USA), a regional transmission organization overseeing electricity distribution in the USA, to share the frequency regulation signal with its customers. IEC 60870-5 (primarily used in Europe and the Middle East) is another SCADA protocol closely related to DNP3 (widely used in the U.S.) [34]. Compared to DNP3, the application layer in IEC 60870-5-104 is integrated with Transmission Control Protocol/Internet Protocol (TCP/IP) stack, allowing convenient solutions for interfacing server devices using this protocol with local Ethernet/IP networks [35]. Energies 2019, 12, x FOR PEER REVIEW 6 of 30

Figure 3. Configuration of an example microgrid communication reprinted by permission of the Figure 3. Configuration of an example microgrid communication reprinted by permission of the publisher (Taylor & Francis Ltd, http://www.tandfonline.com) [2]. publisher (Taylor & Francis Ltd, http://www.tandfonline.com)[2]. As seen in Figure 1, in the lowest level (interaction with loads), the Open ADR (Open Automated CommunicationDemand Response) protocols is used for utilized communication in microgrids for the microgrid including system those in listed [2]. In earlier, the next rely level, on the underlyingModbus physicalis used for communication inverter and generat links,or control which systems are key and to human effective machine monitoring interface and (HMI). control It of DERscan [2, 36be ].transmitted In addition over to different wired communication communication links such that as commonly RS-232/RS enable-485 and protocols ethernet. listedMoving earlier in thisup section,in the hierarchy, wireless IEC communication 61850 is a substation links automation have been standard. introduced This relatively to smart new grid standard and microgrid has applicationsbeen touted in recent as one years.of the key In addition solutions to to wirelessmicrogrid cellular control technologiesand communication. [2,37], thereAs seen are in two Figure notable recent3, the advances central gateway in this area.that implements The first microgrid is combining control wireless at the highest and level power (SCADA) line carrier communicates (PLC). This technologywith the lends microgrid itself controller well to smart and individual grid/microgrid DER controllers data acquisition using the applications IEC 61850 protocol. in rural This or remoteis areas.the While samePLC approach works followed best for by smaller Eaton distances, in implementing wireless their can micro covergrid long controller distances, including with repeaters; the system installed at Fort Sill [2]. The next higher-level protocol is DNP3. The Distributed Network however, unlike wireless, PLC is not impacted by weather or natural obstacles. As a result, the Protocol (DNP3) is mainly used in utilities such as electric and water companies. As seen in Figure 3, heterogeneousthis protocol radio-PLC is key in enabling network the was elec demonstratedtric utility to communicate to support stablewith the communication microgrid controller. overdistances It is of kilometersalso the protocol without used the by necessity PJM (PJM for Interconnect repeatersion, [38]. Valley Another Forge, recent PA, USA), advance a regional in wireless transmission technology that hasorganization found its overseeing way in smart electricity grid and distribution microgrid in applicationsthe USA, to share is the the deployment frequency regulat of Lowion Power signal Wide Areawith Networks its customers. (LPWAN) IEC [6087039] which-5 (primarily in addition used in to Europe being lowand power,the Middle offer East) long is range,another high SCADA capacity, and highprotocol quality closely of servicerelated to connections DNP3 (widely between used in utilities the U.S.) and [34] microgrid. Compared devices to DNP including3, the application DERs based on existinglayer in cellular IEC 60870 long-term-5-104 is integrated evolution with (LTE) Transmission functionalities. Control Protocol/Internet Protocol (TCP/IP) Historically,stack, allowing security convenient concerns solutions have forbeen interfacing an impediment server devicesto utilizing using wireless this protocol communication with local links Ethernet/IP networks [35]. in microgrid applications [2]. It is due to the importance of cybersecurity considerations that standards Communication protocols utilized in microgrids including those listed earlier, rely on the and recommendations such as IEC 62351, IEC 62443, and NIST 800-57 have been developed [40]. Their underlying physical communication links, which are key to effective monitoring and control of DERs implementation,[2,36]. In addition however, to wired comes communication with a performance links that co penalty,mmonly including enable protocols bandwidth listed earlier and delay in this issues that shouldsection, bewireless taken communication into account [ links41]. have been introduced to smart grid and microgrid applications Itin should recent years. be noted In addition that the three to wireless protocols cellular listed technologies earlier (DNP, [2,37] IEC, there 61850 are and two 60870) notable are recent deployed primarilyadvances in the in this electrical area. The distribution first is combining system wireless and are and not power suitable line for carrier behind-the-meter (PLC). This technology adoption to carrylends control itself signals well to issued smart bygrid/microgrid the microgrid data controller acquisition in applications the building. in rural They or were remote reviewed, areas. While however, as theyPLC should works bebest relied for smaller upon indistance the contexts, wireless of any can future cover interactionslong distances between with repeaters a curtailment; however, service unlike wireless, PLC is not impacted by weather or natural obstacles. As a result, the heterogeneous radio-PLC network was demonstrated to support stable communication over distances of kilometers without the necessity for repeaters [38]. Another recent advance in wireless technology that has found its way in smart grid and microgrid applications is the deployment of Low Power Wide Area Networks (LPWAN) [39] which in addition to being low power, offer long range, high capacity, and

Energies 2019, 12, x FOR PEER REVIEW 7 of 30

high quality of service connections between utilities and microgrid devices including DERs based on existing cellular long-term evolution (LTE) functionalities. Historically, security concerns have been an impediment to utilizing wireless communication links in microgrid applications [2]. It is due to the importance of cybersecurity considerations that standards and recommendations such as IEC 62351, IEC 62443, and NIST 800-57 have been developed [40]. Their implementation, however, comes with a performance penalty, including bandwidth and Energiesdelay2019 ,issues12, 4202 that should be taken into account [41]. 7 of 30 It should be noted that the three protocols listed earlier (DNP, IEC 61850 and 60870) are deployed primarily in the electrical distribution system and are not suitable for behind-the-meter provider (CSP) or the utility with the microgrid controller in the context of tertiary control. The adoption to carry control signals issued by the microgrid controller in the building. They were authorsreviewed, in [37 ,however42] share, as the they architectures, should be relied information upon in the exchange context of details,any future technical interactions communication between details,a curtailment and organizational service provider guidelines (CSP) to or establish the utility such with interactions. the microgrid In controller Section4, in more the detailscontext onof the communicationtertiary control. network The authors that enablesin [37,42]the share microgrid the architectures, controller information to communicate exchange details, with DER technical primary controllerscommunication behind the details meter, and are organizational discussed. guidelines to establish such interactions. In Section 4, Beforemore details contextualizing on the communication this review ofnetwork microgrid that hierarchicalenables the micro controlgrid and controller associated to communic communicationsate in thewith developed DER primary control controllers platform behind for the the building-scale meter are discussed. microgrid (Section5), we will ﬁrst discuss in the next section,Before thecontextualizing sources and this sinks review of energy of microgridin the microgrid: hierarchical the building, control and distributed associated energy communications in the developed control platform for the building-scale microgrid (Section 5), we resources, and battery energy storage systems. will first discuss in the next section, the sources and sinks of energy in the microgrid: the building, 3. Microgriddistributed Components energy resources, and Layoutand battery energy storage systems. The3. Microgrid Penn State Components Building a 7Rnd microgridLayout system consists of the following components: a natural gas-fueledThe Capstone Penn State 65 Building kW microturbine 7R microgrid which system is consists capable of ofthe generating following components: both electricity a natural and hot water,gas 10-fueled kW solar Capstone photovoltaic 65 kW microturbine array, lead-acid which and is capable lithium-ion of generating battery banks, both electricity a ‘smart’ and inverter, hot a microcontroller,water, 10 kW and solar a weatherphotovoltaic station. array, It lead is designed-acid and tolithium operate-ionboth battery in tandembanks, a ‘smart’ with the inverter, electric a grid, as wellmicrocontroller, as independently and a inweather ‘islanded’ station. mode It is (simulatingdesigned to operate a grid outage).both in tandem Figure with4 shows the electric the interaction grid, of diﬀaserent well componentsas independently in the in ‘islanded’ 7R microgrid. mode The(simulating description a grid of outage). the building Figure and4 shows these the components interaction are coveredof different in the following components subsections. in the 7R microgrid. The description of the building and these components are covered in the following subsections.

Figure 4. 7R Microgrid System Schematic.

3.1. Microgrid Host Site-Building 7R Building 7R, a part of Penn State at the Navy Yard Campus, is located in the Philadelphia Navy Yard and was constructed as an energy-efficient building [43,44].The location is significant as The Navy Yard has nearly complete autonomy in managing electricity distribution within its boundaries including allowing and enabling third party management of reverse power flow from buildings including 7R back to the grid. In addition to serving its academic purpose, the building is designed to function as a living laboratory for advanced building energy efficiency and sustainability measures (Figure5). The building was built in 2015 with a gross floor area of 2341 m 2 (25,200 ft2). Key energy efficiency and sustainability measures implemented include geothermal heat pump system, green roof, Energies 2019, 12, x FOR PEER REVIEW 8 of 30

Figure 4. 7R Microgrid System Schematic.

3.1. Microgrid Host Site-Building 7R Energies 2019, 12, 4202 8 of 30 Building 7R, a part of Penn State at the Navy Yard Campus, is located in the Philadelphia Navy Yard and was constructed as an energy-efficient building [43,44].The location is significant as The demand control ventilation, and energy recovery ventilation, and advanced integrated daylighting Navy Yard has nearly complete autonomy in managing electricity distribution within its boundaries strategies. The building-to-grid integration capabilities including the inverter and solar PV, were added including allowing and enabling third party management of reverse power flow from buildings during and after the building construction in relation to another project from the U.S. Department of including 7R back to the grid. In addition to serving its academic purpose, the building is designed Energy [45]. Subsequently, microgrid components were installed as part of a project funded by the to function as a living laboratory for advanced building energy efficiency and sustainability measures Pennsylvania State Department of Environmental Protection (DEP) [46]. The focus of this paper is the (Figure 5). The building was built in 2015 with a gross floor area of 2341 m2 (25,200 ft2). Key energy development of the microgrid control platform for building 7R which is tasked with dictating control efficiency and sustainability measures implemented include geothermal heat pump system, green decisions to installed microgrid components, including a 65 kW natural gas-fired microturbine, a 10 kW roof, demand control ventilation, and energy recovery ventilation, and advanced integrated Solar Photovoltaic (PV) arrays, lithium-ion and lead-acid energy storage systems, a microcontroller, daylighting strategies. The building-to-grid integration capabilities including the inverter and solar and the building automation system. The reminder of this section is devoted to briefly describing the PV, were added during and after the building construction in relation to another project from the building followed by the components of its distributed energy resources listed earlier. U.S. Department of Energy [45]. Subsequently, microgrid components were installed as part of a The building is composed of offices, classrooms, and other educational spaces and an auditorium project funded by the Pennsylvania State Department of Environmental Protection (DEP) [46]. The with a diversity of heating, cooling, and electric loads. While the office spaces have consistent baseline focus of this paper is the development of the microgrid control platform for building 7R which is loads, the educational spaces including, lobby (maximum occupancy 50 people), classrooms (maximum tasked with dictating control decisions to installed microgrid components, including a 65 kW natural occupancy 100 people), and an auditorium (maximum occupancy 165 people), have transient loads. To gas-fired microturbine, a 10 kW Solar Photovoltaic (PV) arrays, lithium-ion and lead-acid energy meet the heating and cooling loads in this two-story building, the building has a water to water source storage systems, a microcontroller, and the building automation system. The reminder of this section heat pump (WSHP) entailing 32 wells with 350 feet deep (Figure5b). The building utilizes a number of is devoted to briefly describing the building followed by the components of its distributed energy daylighting strategies and is powered by LED lights. resources listed earlier.

(a)

Figure 5. Cont.

Energies 2019,, 12,, 4202x FOR PEER REVIEW 9 of 30

(b)

Figure 5 5.. ((a)).. The The case case study study building building energy energy efficiency eﬃciency and and sustainability sustainability features features Reproduced with permission from from [47 [47]]. Kieran. Kieran Timberlake, Timberlake, 2019; 2019 (b) The; (b case) The study case building: study building brick screen: brick and screen translucent and translucentglazing panels. glazing panels.

3.2. Distributed Energy Resources (DERs) The building is composed of offices, classrooms, and other educational spaces and an auditoriumThe distributed with a diversity energy resources of heating, (DERs) cooling, consisted and of electric physical loads. assets While (Solar thePV, energy office spaces storage, have and consistentmicroturbine) baseline in conjunction loads, the witheducational virtual assetsspaces(building including, demand lobby (maximum management) occupancy assist the 50 electricalpeople), classroomsgrid in serving (maximum building occupancy loads when 100 the people), building and isan connected auditorium to (maximum the grid and occupancy are the sole 165 provider people), haveto critical transient loads loads. when To it ismeet not. the As heating we gradually and cooling reduce loads our reliancein this two on- centralizedstory building, bulk the generation, building hasDERs a water are changing to water thesource way heat power pump is generated (WSHP) entailing and distributed 32 wells inwith the 350 electric feet deep grid. (Figure Moreover, 5b). theyThe providebuilding resiliencyutilizes a number and/or relyof daylighting on more sustainable strategies and energy is powered sources by and LED defer lights. capital investment in demand-constrained areas. In the following, DERs and associated technologies utilized in building 7R 3.2.are discussed.Distributed Energy Resources (DERs) The distributed energy resources (DERs) consisted of physical assets (Solar PV, energy storage, 3.2.1. Solar PV and microturbine) in conjunction with virtual assets (building demand management) assist the electricalThe buildinggrid in serving green roofbuilding hosts loads a 10 kWwhen solar the photovoltaic building is connected (PV) array, to as the shown grid and in Figure are the6. Thesole providerarray will to produce critical 10.1loads kW when of powerit is not. at itsAs peak.we graduall The averagey reduce electricity our reliance output on iscentralized 13,000 kWh bulk/yr. generation,During the winter DERs months, are changing there is the an 8–13% way power loss in is generation generated due and to shading distributed and inlack the of sunlight.electric grid. The Moreover,shading losses they during provide spring resiliency and summer and/or rely are negligibleon more sustainable because there energy is more sources than and enough defer sunlight capital investmentto generate in electricity. demand- Theconstrained solar array areas. can In operate the following in the, temperature DERs and associated range of technologies15 C to 40 utilizedC with − ◦ ◦ ineﬃ buildingciency around 7R are 15.7%discussed. [48].

3.2.1. Solar PV The building green roof hosts a 10 kW solar photovoltaic (PV) array, as shown in Figure 6. The array will produce 10.1 kW of power at its peak. The average electricity output is 13,000 kWh/yr. During the winter months, there is an 8–13% loss in generation due to shading and lack of sunlight. The shading losses during spring and summer are negligible because there is more than enough sunlight to generate electricity. The solar array can operate in the temperature range of −15 °C to 40 °C with efficiency around 15.7% [48].

Energies 2019, 12, 4202 10 of 30 Energies 2019, 12, x FOR PEER REVIEW 10 of 30 Energies 2019, 12, x FOR PEER REVIEW 10 of 30

FigureFigure 6. 6. TheThe 10 10 kW kW installed installed PV PV panels panels.panels..

TheThe array array was was installed installed at at a a 1515◦°° tilttilt angleangle isis andanandd 174.3174.3◦°° solarsolar azimuth.azimuth. It It is is structured structured into into 40 40 mountedmounted modules modules with with 60 60 monocrystalline monocrystalline silicon silicon cells cells per per panel. panel. There There are are 4 4 rows rows of of 10 10 panels panels each each onon toptop ofof of 7R.7R. 7R. TheThe The wholewhole whole systemsystem system isis wired iswired wired asas 3 as3 stringsstrings 3 strings ofof 1313 of modules,modules, 13 modules, withwith with oneone unusedunused one unused modulemodule module toto buildbuild to upbuildup thethe up properproper the proper voltage voltage voltage to to match match to match the the the inverter inverter inverter operating operating operating range. range. range. In In In order order order to to to combine combine combine the the the output output of of multiplemultiple strings strings strings of of of PV PV PV into into into one one one main main main feed feedthat feed is that that fed tois is the fed fed inverter, to to the the inverter, ainverter, combiner a a boxcombiner combiner [49] by box box SOLARBOS [49][49] byby SOLARBOS(SolarBOS,SOLARBOS Grand ((SolarBOSSolarBOS Rapids,,, GrandGrand MI, Rapids, USA)Rapids, was MI,MI, used, USA)USA) as waswas shown usedused in,, asas Figure shownshown7. Theinin FigureFigure combiner 7.7. TheThe box combinercombiner is installed boxbox isbetweenis installedinstalled the betweenbetween solar array thethe andsolarsolar the arrayarray inverter, andand thethe and inverterinverter in addition,, andand to inin combining additionaddition toto the combiningcombining output of thethe the outputoutput three strings, ofof thethe threeprovidesthree stringsstrings overcurrent,, provideprovide andss overcurrentovercurrent overvoltage andand protection overvoltageovervoltage to protect protectionprotection the inverter toto protectprotect as well thethe asinverterinverter the means asas wellwell to quickly asas thethe meansshutmeans oﬀ totothe quicklyquickly power shutshut on one offoff thethe section powerpower of theonon one PVone circuitsectionsection (DC ofof thethe disconnect). PVPV circuitcircuit (DC(DC disconnect).disconnect).

FigureFigure 7. 7. CombinerCombiner box box.box..

3.2.2.3.2.2. EE Energynergynergy Storage Storage SinceSince the the microgrid microgrid utilizes utilizes intermittent intermittent renewable renewable sources sources of of generation generation or or other other distributed distributed generationgeneration sources sources that that can can be be operated operated for for a a limited limited number number of of hours hours annually, annually, an an energy energy storage storage systemsystem (ESS) (ESS) should should be be deployed deployed to to enable enable the the mic microgridmicrogridrogrid to to function function while while in in different didifferentﬀerent modes modes of of operationoperation when when it it is is in in connected connected to to the the grid grid and and when when it it isis islanded.islanded. ESS ESS enables enables matching matching the the load load withwith generation generation and and is is a a requirement requirement where where intermittent intermittent resources resources are are utilized utilized,utilized,, especially especially since since peak peak generagenerationtion andand peakpeak loadload maymay occuroccur atat differentdifferent times.times. Additionally,Additionally, anan ESSESS cancan actact asas aa bufferbuffer toto absorbabsorb power power from from renewable renewable sources sources while while the the microturbine microturbine operates operates at at its its optimum optimum operating operating

Energies 2019, 12, x FOR PEER REVIEW 11 of 30 efficiency. Finally, ESS is essential for a smooth transition from/to grid-connected to/from islanded operation. A battery energy storage system (BESS) utilizes different electrochemical batteries arranged in series and parallel to generate desired voltage and current respectively and a dedicated control systemEnergies for2019 each, 12, 4202chemistry to manage its state of charge (SOC) and physical and operational variables11 of 30 including temperature, rate of charge, and rates of discharge. In addition, the BESS should be able to communicate with a charge controller (normally a part of the inverter) to enforce the proper values generation and peak load may occur at different times. Additionally, an ESS can act as a buffer to absorb for those variables (especially rate of charge and rates of discharge and changing them when it power from renewable sources while the microturbine operates at its optimum operating efficiency. approaches high or low SOC). Different applications (i.e., power vs. energy) require different Finally, ESS is essential for a smooth transition from/to grid-connected to/from islanded operation. chemistries based on their operational parameters. A battery energy storage system (BESS) utilizes different electrochemical batteries arranged In the building hybrid microgrid implementation, two different battery chemistries (lead-acid in series and parallel to generate desired voltage and current respectively and a dedicated control and lithium-ion for energy and power applications respectively) are deployed. The lead-acid BESS system for each chemistry to manage its state of charge (SOC) and physical and operational variables utilizes Enersys Nexsys 12NXS186 (Enersys, Reading, PA, USA) batteries [50] shown in Figure 8a and including temperature, rate of charge, and rates of discharge. In addition, the BESS should be able to is a valve-regulated lead-acid battery (VRLA). It is of the absorbed glass mat (AGM) type and as a communicate with a charge controller (normally a part of the inverter) to enforce the proper values for result, has enhanced cyclic capability (up to 1200 cycles at 60% Depth of Discharge) compared to those variables (especially rate of charge and rates of discharge and changing them when it approaches conventional (gel or flooded) lead-acid batteries. The bank has 45 batteries in series and has the high or low SOC). Different applications (i.e., power vs. energy) require different chemistries based on capacity to supply up to 17 kW of power and can store 60 kWh of energy and 186 Ah of electric their operational parameters. charge. The function of this unit is to safely store excess energy generated by the solar array or In the building hybrid microgrid implementation, two different battery chemistries (lead-acid and microturbine, providing power when the system calls for it. However, this type of battery chemistry lithium-ion for energy and power applications respectively) are deployed. The lead-acid BESS utilizes cannot respond rapidly enough to changes in solar PV output to handle some common conditions, Enersys Nexsys 12NXS186 (Enersys, Reading, PA, USA) batteries [50] shown in Figure8a and is a such as when clouds abruptly shade the PV array. valve-regulated lead-acid battery (VRLA). It is of the absorbed glass mat (AGM) type and as a result, The lithium-ion BESS, manufactured by ALLCell, shown in Figure 8b, utilizes LG 18650 MH1 has enhanced cyclic capability (up to 1200 cycles at 60% Depth of Discharge) compared to conventional (LG, Seoul, South Korea) cells based on a Lithium Metal Oxide proprietary chemistry [51]. The (gel or flooded) lead-acid batteries. The bank has 45 batteries in series and has the capacity to supply lithium-ion BESS is nominally discharged three times as fast compared to the lead-acid BESS. A up to 17 kW of power and can store 60 kWh of energy and 186 Ah of electric charge. The function of summary of cell and pack specifications is provided in [52]. The BESS can respond very rapidly when this unit is to safely store excess energy generated by the solar array or microturbine, providing power dispatched by the inverter with up to 50 kW of power with its 39 kWh energy capacity. The BESS can when the system calls for it. However, this type of battery chemistry cannot respond rapidly enough to also perform frequency regulation when called upon by a curtailment service provider by injecting changes in solar PV output to handle some common conditions, such as when clouds abruptly shade power to the grid when frequency drops and absorbing it when it rises, a service with monetary the PV array. compensation by the regional transmission organization (RTO).

(a) (b)

FigureFigure 8.8. (a()a The) The lead lead-acid-acid BESS BESS utilizing utilizing Enersys Enersys Nexsys Nexsys 12NXS186 12NXS186 batteries batteries and and (b ()b )Lithium Lithium-ion-ion BESS,BESS, manufactured manufactured by by ALLCell ALLCell..

3.2.3. MultiportThe lithium-ion Inverter BESS, manufactured by ALLCell, shown in Figure8b, utilizes LG 18650 MH1 (LG, Seoul, South Korea) cells based on a Lithium Metal Oxide proprietary chemistry [51]. The lithium-ion BESS is nominally discharged three times as fast compared to the lead-acid BESS. A summary of cell and pack speciﬁcations is provided in [52]. The BESS can respond very rapidly when dispatched by the inverter with up to 50 kW of power with its 39 kWh energy capacity. The BESS can also perform frequency regulation when called upon by a curtailment service provider by injecting power to the Energies 2019, 12, 4202 12 of 30 grid when frequency drops and absorbing it when it rises, a service with monetary compensation by the regional transmission organization (RTO).

3.2.3. Multiport Inverter The 50 kW inverter [53] enables DC-coupling of solar and battery energy storage as opposed to AC-coupling the DERs through separate inverters. The multiport inverter supports three DC ports and two AC ports. In this application, the first DC port is connected to the solar PV array, the second DC port is connected to the lithium-ion battery system (main battery), and the third DC port is connected to the lead-acid battery system. The first AC port is the grid connection, and the second AC port is the load connection. Energy can flow from DC to DC and AC side, from DC to DC side only (PV charging either battery) or from AC side (grid) to AC load and DC side (batteries). Finally, the AC grid and any combination of DC ports can supply power to the load. The inverter operation will be covered in more detail when the microgrid modes of operation are discussed in Section4.

3.2.4. Microturbine Combined Heat and Power (CHP)—simultaneous production of electricity and heat from a single fuel source—as a distributed energy resource can enhance reliability and resiliency in buildings since it has proved its value as an alternative source of power and thermal energy (heating and cooling) during extreme weather emergencies [13]. CHP has many benefits, including a substantial increase in energy efficiency compared to separate power and heat generation. To help realize these benefits and advance its adoption, in the U.S., regional CHP Technical Assistance Partnerships (CHP-TAP) have been funded to provide expert advice and to determine the suitability and economic impact of CHP based on site conditions for different applications including building-scale microgrids [13,14]. Microturbines are relatively small combustion turbines that can use gaseous or liquid fuels and in smaller sizes (30 to 330 kilowatts (kW)) have emerged as a CHP option (with lower emissions compared to other technologies) since the 1990s [54]. This size range has made them a feasible option as a distributed energy resource in the building scale. A Capstone 65 kW combined heat and power (CHP) microturbine (c65) (Capstone Turbine, Los Angeles, CA, USA) [55,56] was selected as the electric generator deployed outside building 7R, as shown in Figure9a. The microturbine is fueled by natural gas and is limited to 2000 hours per year of operation per its permit. The 7R microturbine is a dual-mode DER, it can run by itself (stand-alone) or can be connected to the grid. In order to enable the microturbine to transition automatically from grid-connect to stand alone mode of operation during an unplanned outage, a dual-mode integrator (DMI) (E-Finity Distributed Generation, Wayne, PA, USA) shown in Figure9b has been utilized [ 57]. When the DMI senses power outage, it disconnects the microturbine and its critical loads from the electric grid. After a delay, the microturbine is powered up and supplies stand-alone power. Finally, when the power comes back on, the DMI operates the microturbine in grid-connect mode after a delay. The default mode of operation for the DMI is operating the microturbine parallel to the grid. The details of mode transitions when the solar PV inverter is also involved and controlled under the microgrid controller is covered in Section4. EnergiesEnergies 20192019, 12, ,12 x, FOR 4202 PEER REVIEW 13 13of of30 30

(a) (b)

FigureFigure 9. ( 9.a)( Capstonea) Capstone C65 C65 Microturbine Microturbine and and (b) ( bE)-finity E-ﬁnity Dual Dual Mode Mode Integrator Integrator..

InIn a potent a potentialial future future system system extension extension (contingent (contingent upon upon availability availability of offunding), funding), the the waste waste heat heat ofof the the microturbine microturbine can can be be utilized utilized in in a acombined combined heat heat and and power power (CHP) (CHP) application application which which would would significantlysignificantly increase increase the the efficiency efficiency of of the the microturbine microturbine from from the the current current 30% 30% to total system eefffificiencyciency of ofbetween between 60% 60% and and 80% 80% [56 [56]]. This. Thi cans can be be accomplished accomplished by installingby installing an integrated an integrated heat recoveryheat recovery system systemthat can that provide can provide considerable considerable thermal energythermal by energy capturing by capturing waste heat waste generated heat by generated the combustion by the of combustionnatural gas. of Sincenatural this gas. is an Since all-electric this is an building, all-electric one possiblebuilding, use one of possible the waste use heat of isthe for waste space heat heating is forof space neighboring heating buildingsof neighboring in the buildings Navy Yard in campusthe Navy through Yard campus a district through hot water a district system. hot Anotherwater system.possible Another use of possible the waste use heat of couldthe waste be in heat supplying could be the in heatsupplying to air-cooled the heat absorption to air-cooled chillers absorption for space chillerscooling. for Due space to the cooling. proximity Due of to this the building proximity to other of this buildings building with to di otherfferent buildings heating andwith cooling different load heatingrequirements, and cooling all of load these requirements, solutions are all feasible, of these and solutions future workare feasible can assess, and difuturefferent work design can scenarios assess differentto utilize design the microturbine scenarios to wasteutilize heat.the microturbine waste heat.

4. 4.Implementing Implementing Control Control for for a Building a Building-Scale-Scale M Microgridicrogrid ThereThere are are several several examples examples of ofmicrogrid microgrid control control im implementationsplementations in inthe the literature literature [16,17,58 [16,17,58–62]–62, ], toto name name a few a few.. All Allof these of these implementations implementations rely relyon communication on communication networks networks and protocols and protocols that are that wellare-estab well-establishedlished in electrical in electrical distribution distribution systems, systems, covered covered in S inection Section 4. In4. Incontrast, contrast, in in order order to to implementimplement behind behind-the-meter-the-meter control control in ina building, a building, a dedicated a dedicated communication communication network network should should be be designeddesigned and and implemented implemented.. In Inthe the following following,, the the communication communication infrastr infrastructure,ucture, the the components components and and theirtheir function, function, information information collected collected and and sent sent to tothe the controller controller (inputs (inputs),), and and controller controller commands commands (outputs)(outputs) using using this this communication infrastructure infrastructure and and their their functional functional signiﬁcance significance in microgrid in microgrid control controlare discussed. are discussed. 4.1. Microgrid Control Components, Their Function, and Interconnections 4.1. Microgrid Control Components, Their Function, and Interconnections The communication infrastructure for the microgrid control platform is depicted in Figure 10. The communication infrastructure for the microgrid control platform is depicted in Figure 10. This platform was installed by the project contractor [63] by running category 6 Ethernet cable enclosed This platform was installed by the project contractor [63] by running category 6 Ethernet cable in metal conduits connecting all indoor and outdoor components which lie on diﬀerent networks enclosed in metal conduits connecting all indoor and outdoor components which lie on different separated from each other physically or virtually, including a virtual private network for microturbine networks separated from each other physically or virtually, including a virtual private network for remote monitoring and maintenance. Some components are powered through a Power Over Network microturbine remote monitoring and maintenance. Some components are powered through a Power (POE) switch, as noted. The protocols used to connect the component and the microgrid controller Over Network (POE) switch, as noted. The protocols used to connect the component and the microgrid controller are listed below, next to each component. Since this project involves controlling

Energies 2019, 12, 4202 14 of 30 are listed below, next to each component. Since this project involves controlling building loads through collecting information from the building automation system (BAS), a method had to be devised to enable the BAS to communicate with DER controllers and the microgrid controller. Since BACnet is a communication protocol developed by the American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) for the BAS to interact with building systems, a BACnet to Modbus (Modicon, North Andover, MA, USA) gateway is utilized, so BACnet [64] and Modbus signals can be translated into each other. Energies 2019, 12, x FOR PEER REVIEW 15 of 30

FigureFigure 10. Microgrid10. Microgrid Control Control Component Component Interconnection.Interconnection. Reproduced Reproduced with with permission permission from [63] from. [63]. ProtoGenProtoGen (project (project contractor), contractor) 2018., 2018.

4.2It can. Data be and observed Command that Flow compared between toController the larger and scaleComponents microgrids implemented in electric distribution networkThe with data a dedicated and command communication flow of a typical infrastructure microgrid is anddepicted established in Figure protocols 11. It can be implemented observed in industrial-gradethat data is collected components from, and [16 control,58,60] signal to names are aissued few, to a building-scaleall three layers of microgrid the control hashierarchy to rely on oﬀ-the-shelfcovered in commercial Section 2 (Figure components 2): microgrid and standards optimizer, fromdispatch, diﬀ erentand CERTS domains autonomous which necessitates layer. These more complexdata maintenance. and commands The are control encoded components into communication for this building-scale signals in microgrid different are protocols listed below: and communicated between the controller and different components in Figure 10. EnergyIn other IQ Controller words, the (Energycommand IQ, the Boca data Raton,and flows FL, in USA; Figure The 11 can Microgrid be mapped Controller): to compone Centralizednts in • FigureSCADA 10, siteas shown controller. with the A following programmable examples. Industrial Each mapping PC [65 corresponds] running anto a open programming driver-based scenarioand JAVA-based for the controller. supervisory (More platformdetails on usedthat will in buildingbe provided automation later in this systems paper.) The (Niagara one to N4)the [66] last horizontal arrow in the lower-left section of Figure 11 corresponds to the meteorological station providing integration to various Modbus-based DER components, listed below. that includes a pyranometer that measures solar irradiation and sky camera (a fisheye camera with Dynapower Inverter Power Control System (PCS;South Burlington, VT, USA) [53]: The • 180-degree view) that reports on cloud covering sending data to the controller that it can compile into informationModbus-based on the primary availability controller of the solar for the resource. Dynapower On the inverter. other hand, The the PCS bottom is in-left charge data offlow switching in Figurebetween 11 corresponds its grid-connected to the Building and islanded Automation modes System of operation. sending information on building loads to theMicroturbine controller. Interface(The controller Module utilizes (mTIM; the E-Finity,Wayne, discovery function PA,USA) of BACnet [67]: to The configure Modbus-based data points primary • availablecontroller in for the the Building Capstone Automation microturbine. System.) This componentAll modules coordinates including the with “Unit the microturbine Commitment dual Dispatchmode integrator and Load (DMI)Shed” m inodule charge in Figure of switching 11, are implemented between its in grid-connected a Java programming and islandedblock based modes onof operation. the Niagara platform installed on the controller. Based on these on other data flows, the controller’s dispatch module issues commands containing set points for real and reactive power for elithion AllCell Battery Management System (BMS; elition, Boulder, CO, USA) [51]: The CAN • the distributed energy resources (“P & Q Dispatch” Arrow in Figure 11). bus-basedAs other primary example controllers of the data for flow the reported AllCell lithium-ionto the microgrid battery controller, energy the storage other blocks system. in the left portion of Figure 11 report the voltage (V), current (I), both AC and DC (in the case of the inverters), real (P) and reactive (Q) power of the DERs and in case of battery energy storage, the state of charge (SoC) as well, to the microgrid controller. In the current state of the project, all these pieces of information are obtained by the controller addressing the Modbus registers of the DERs. In the future, additional metering will be installed. Another important example is the “Historical Database”

Energies 2019, 12, 4202 15 of 30

Cellwatch PC Battery Management System (BMS; Cellwatch, Raleigh, NC, USA) [52]: The • Modbus-based primary controller for the lead-acid battery energy storage system. JACE controller (Tridium, Richmond, VA, USA) [66]: The BACnet-based controller/server for the • Building Automation System (BAS).

In addition to the above main components in the control hierarchy (microgrid controller—secondary and tertiary level) and DER (primary controllers) that interact with the building automation system, there are a number of other network components as depicted in Figure 10 that collect information useful for microgrid control which will be addressed in the next subsection. These components are scheduled to be configured in the next phase of the project, including the General Electric (GE) Microgrid Controller—an islanding controller—integrated into this platform by GE, a project partner.

4.2.Energies Data 2019 and, 12 Command, x FOR PEER Flow REVIEW between Controller and Components 16 of 30 The data and command flow of a typical microgrid is depicted in Figure 11. It can be observed module, implemented in the platform by utilizing a PI server (OSIsoft, San Leandro, CA, that data is collected from, and control signals are issued to all three layers of the control hierarchy USA;https://www.osisoft.com). covered in Section2 (Figure2): microgrid optimizer, dispatch, and CERTS autonomous layer. These Now that the layout, components, and data and command flow of the control platform are data and commands are encoded into communication signals in different protocols and communicated explained, we will discuss how the platform is utilized to implement the three layers of the control between the controller and different components in Figure 10. hierarchy (Figure 2.), from the bottom to the top.

FigureFigure 11.11. BlockBlock diagram diagram of of data data and and command command flow flow in ain microgrid a microgrid controller controller (RT stands (RT stands for Real-Time). for Real- ReprintedTime). Reprinted by permission by permission of the publisher of the publisher (Taylor (Taylor & Francis & Francis Ltd, http: Ltd,//www.tandfonline.com http://www.tandfonline.com))[2]. [2]. In other words, the command the data and flows in Figure 11 can be mapped to components in4.3. Figure Implementation 10, as shown of Lowest with theLayer: following CERTS Autonomous examples. Each Layer mapping corresponds to a programming scenario for the controller. (More details on that will be provided later in this paper.) The one to the We recall from Section 2 that this layer is tasked with power-sharing control of multiple DERs last horizontal arrow in the lower-left section of Figure 11 corresponds to the meteorological station and functions related to the transition between grid-connected and islanded modes of operation, that includes a pyranometer that measures solar irradiation and sky camera (a fisheye camera with including switches, disconnect devices, and protection and have cycle-scale timing (10–100 180-degree view) that reports on cloud covering sending data to the controller that it can compile milliseconds). Although these functions take place autonomously during unplanned islanding and into information on the availability of the solar resource. On the other hand, the bottom-left data without controller intervention, the controller collects status information from transfer switches and flow in Figure 11 corresponds to the Building Automation System sending information on building breakers operating at this level and has the ability to command them during a planned islanding event. There are three devices in this layer, as listed below. The first device in this layer is a triple-pole motor-operated breaker (3-phase, 125A; Eaton, Beachwood, OH, USA) [68], acting as the islanding contactor for the building-scale microgrid. In the event of grid power loss, the breaker opens, commands the microturbine to power off (if running through) the DMI and mTIM modules discussed earlier and disconnects the microturbine from the grid. It will serve the critical building loads through the ATS, below. The second device in this layer is an automatic transfer switch (ATS; Eaton, Beachwood, OH, USA) [69] that, in the event of an outage, switches from the grid (primary source) to the microturbine (backup source). The load side of the ATS is wired to the first AC port of the inverter. The second AC port is connected to the critical load panel. The third device in this layer is the SEL-547 directional power relay (Schweitzer Engineering Laboratories, Pullman, WA, USA) to avoid reverse power [70]. The reverse power protection is used to protect the microturbine, since if a generator takes in power, it will act as a synchronous motor which could lead to turbine blade failure.

Energies 2019, 12, 4202 16 of 30 loads to the controller. (The controller utilizes the discovery function of BACnet to configure data points available in the Building Automation System.) All modules including the “Unit Commitment Dispatch and Load Shed” module in Figure 11, are implemented in a Java programming block based on the Niagara platform installed on the controller. Based on these on other data flows, the controller’s dispatch module issues commands containing set points for real and reactive power for the distributed energy resources (“P & Q Dispatch” Arrow in Figure 11). As other examples of the data flow reported to the microgrid controller, the other blocks in the left portion of Figure 11 report the voltage (V), current (I), both AC and DC (in the case of the inverters), real (P) and reactive (Q) power of the DERs and in case of battery energy storage, the state of charge (SoC) as well, to the microgrid controller. In the current state of the project, all these pieces of information are obtained by the controller addressing the Modbus registers of the DERs. In the future, additional metering will be installed. Another important example is the “Historical Database” module, implemented in the platform by utilizing a PI server (OSIsoft, San Leandro, CA, USA; https://www.osisoft.com). Now that the layout, components, and data and command flow of the control platform are explained, we will discuss how the platform is utilized to implement the three layers of the control hierarchy (Figure2), from the bottom to the top.

4.3. Implementation of Lowest Layer: CERTS Autonomous Layer We recall from Section2 that this layer is tasked with power-sharing control of multiple DERs and functions related to the transition between grid-connected and islanded modes of operation, including switches, disconnect devices, and protection and have cycle-scale timing (10–100 milliseconds). Although these functions take place autonomously during unplanned islanding and without controller intervention, the controller collects status information from transfer switches and breakers operating at this level and has the ability to command them during a planned islanding event. There are three devices in this layer, as listed below. The first device in this layer is a triple-pole motor-operated breaker (3-phase, 125A; Eaton, Beachwood, OH, USA) [68], acting as the islanding contactor for the building-scale microgrid. In the event of grid power loss, the breaker opens, commands the microturbine to power off (if running through) the DMI and mTIM modules discussed earlier and disconnects the microturbine from the grid. It will serve the critical building loads through the ATS, below. The second device in this layer is an automatic transfer switch (ATS; Eaton, Beachwood, OH, USA) [69] that, in the event of an outage, switches from the grid (primary source) to the microturbine (backup source). The load side of the ATS is wired to the first AC port of the inverter. The second AC port is connected to the critical load panel. The third device in this layer is the SEL-547 directional power relay (Schweitzer Engineering Laboratories, Pullman, WA, USA) to avoid reverse power [70]. The reverse power protection is used to protect the microturbine, since if a generator takes in power, it will act as a synchronous motor which could lead to turbine blade failure. Testing the islanded modes and transition to/from them is pending the building owner’s (Penn State) Office of Physical Plant approval, and as the result, at the time of this writing, the two tests conducted on the system in its current state were in the grid-connected modes of operation. It should be noted that the tests conducted in Section5 are in grid-connected modes and do not involve operating the two DERs in a CERTS configuration (Section 2.2, not to be confused with the name of the lowest layer in the control hierarchy) where they power-share serving the loads through droop control, as discussed earlier. Either DER (microturbine or Solar PV inverter) can serve all the critical loads in building 7R, and in this case, there is no need to program the DERs to operate in the CERTS configuration. Energies 2019, 12, 4202 17 of 30

4.4. Implementation of the Dispatch and Optimizer Layers Currently, only the dispatch layer is programmed in the controller. In this mode, power dispatch commands can be issued to the DERs so they can be operated per manufacturer’s speciﬁcations. However, the decision to select a controller [65] that utilizes an open driver-based system [66] where it can readily communicate with the building automation system (BAS) compared to a proprietary system, such as [62], enables ongoing development in the context of university research as additional funding is secured. Additionally, programming for demand curtailment scenarios can also involve changing building set points at peak demand times and reusing programming blocks between the microgrid controller and the BAS.

4.4.1. Control of Microgrid Operational Modes In the following, grid-connected and islanded modes of operation will be discussed along with the unique resiliency or economic goal the microgrid should fulfill in each scenario based on the grid, DER, and load situation. It should be noted that scenarios implemented in the optimizer layer have not been tested (please see the Future Directions Section), and only those implemented at the dispatch layer have been tested, as documented later in this paper. For the grid-connected modes, the utility or curtailment service provider (Tangent, Figure 10) has the ability to coordinate the execution of the mode with the controller. Grid-connected Modes • In all grid-connected modes, the dispatched DERs will operate in grid-following mode (PQ) since the grid ensures stable voltage and frequency values. Accordingly, the controller assigns real and reactive power values to each dispatched DER. The internal controller of each DER ensures these power values are maintained. In each of the following modes, the decision to dispatch the microturbine, solar PV inverter, or both and the assigned power values to each is dependent upon the mode, solar radiation, battery state of charge, grid conditions (i.e., peak demand times), and building demand during each calculation horizon. In the next phase of implementation, the microgrid controller will be programmed to solve and accordingly assign power set points for each calculation horizon. The modes followed by a brief description of each are listed below: Base loading #For the duration of this mode, the controller assigns fixed power setpoints for the microturbine and the inverter. The remaining power for the building is provided by the electric utility. Load following #In this mode, the controller in successive time intervals alters set points assigned to the microturbine and inverter up to an assigned maximum value to meet varying building demand. As the DERs are ramped up, the electric utility supplies the difference between demand and total generation. Peak shaving and electric sales # When demand is above a certain value for certain days, the controller reduces the building demand by a combination of the following: dispatching the DERs, commanding the building automation to apply its pre-programmed demand reduction strategy and to shed loads in the following order: lighting reduction [71], unloading of pumps and fans, and changes in HVAC set points. It should be noted that if due to high demand, grid prices are higher than the microgrid’s cost to produce power, the controller can increase the set points for the microturbine and inverter, so excess electricity is generated and sold back to the grid. Frequency regulation #In this mode, by dynamically changing the set point for the inverter such that power from the batteries is injected into or absorbed from the grid, the controller can increase/reduce the frequency and effectively act as fast-acting frequency regulator and generate revenue for the system. Energies 2019, 12, 4202 18 of 30

Islanded Modes • When grid-wide power failures occur, they are sensed by both the Panel TP and ATS, and as a result, the Panel TP islands oﬀ the grid, and the ATS switches to the critical load panel. Inverter as grid-forming source, setting U (output voltage) and F (frequency): UF Mode, microturbine# oﬀ. In this mode, the primary controller of the inverter controls voltage and frequency. Microturbine as grid-forming source (UF Mode), inverter as grid-following source (PQ Mode), setting# P (real power) and Q (reactive power). In this mode, the microturbine primary controller controls voltage and frequency, and the inverter executes a dispatch command from the microgrid controller, and its primary controller ensures real and reactive power levels are maintained.

4.4.2. Inverter Operating Modes The microgrid controller commands bidirectional current flow and consequently power on each DC port in order to maintain an optimum state of charge and can command the inverter to transition between grid-connected (PQ) mode where real and reactive power setpoints are assigned to the inverter and standalone (UF) mode road inverter maintains output voltage (480 V 3 Phase) and frequency (60 Hz). The DC-coupled setup avoids unnecessary conversion losses associated with AC-coupled systems. In addition, it enables fast output response on load step-up/down in both grid-connected and standalone modes. In addition to serving as the primary controller for all distributed energy resources connected to its three ports, the inverter’s domain controller handles all the communication protocols and directs commands to the appropriate embedded systems as requested by the end-user. The main operational limitation of the inverter that the microgrid controller should be closely monitoring the DC bus voltage. The voltage should not drop below 450 V, which would cause the inverter to trip under battery under-voltage. On the other hand, if a battery reaches its high voltage limit (697 V), the controller should stop the PV DC/DC converter for a while (until the main battery is discharged halfway) and power to the grid/load is supplied by the main battery only. In grid-connected (PQ) mode, the power from/to the battery is defined by subtracting the power delivered by solar PV from the absolute value of the +/ kW set point signal issued by the microgrid − controller. During standalone (PV) operation of the inverter, if the main battery is the only source and the other two DC ports are disconnected, the output power is determined by the load demand (subject to a limit set by microgrid controller) while voltage and frequency are fixed. The inverter can support this mode if the battery voltage does not drop below 450 V. If the main battery and solar PV are present in standalone (PV) operation of the inverter, the power in excess of the load demand charges the battery. Once the batteries charge up to limit, the microgrid controller disables the solar PV DC/DC converter until the main battery is charged halfway.

4.4.3. Microturbine Operating Modes The Capstone C65 microturbine operates in one of the following two operational states [57]: Stand Alone (SA) and Grid Connect (GC). In normal GC operation, the microturbine contactor is closed, and the microturbine is capable of exporting power in excess of building loads to the grid. When a disturbance is detected on the utility grid, voltage and current sensing in the microturbine will cause the internal contactor to open, and the microturbine will go through a shutdown procedure. When the conditions have been met to allow reconnection to the grid, the output contactor will automatically close, and the microturbine will be restarted. Energies 2019Transition, 12, 4202 to the SA operational mode takes place in less than 7 seconds after the SA enable19 of 30 signal is issued by the microturbine control system. External control actions, including those initiated by the microgrid control system, should proceed with this action. The transition back to the GC mode, Afterward,on the the other microturbine hand, involves enters a 0– the30 Hot minutes Standby adjustable operational wait timer mode during and remains which load inthis voltage mode is until it is ensuredmaintained. the utilityAfterward, voltage the remainsmicroturbine within enters the the protective Hot Standby relay operational setting limits mode for and the remains reconnect in time delay,this adjustable mode until between it is ensured 5 (per the IEEE utility 1547.1 voltage and remains UL 1741 within standards) the protective and 30relay minutes—afterward setting limits for it switchesthe reconnect to the GC time mode. delay, adjustable between 5 (per IEEE 1547.1 and UL 1741 standards) and 30 minutes—afterward it switches to the GC mode. 4.5. Testing Hierarchical Control 4.5. Testing Hierarchical Control The platform was used to test hierarchical control at the dispatch layer in the grid-connected The platform was used to test hierarchical control at the dispatch layer in the grid-connected mode of the controller, by assigning power set points to the DERs and observing the DERs operation. mode of the controller, by assigning power set points to the DERs and observing the DERs operation. The ﬁrstThe testfirst wastest was conducted conducted on on the the inverter, inverter, and and thethe second test test was was conducted conducted on onthe themicroturbine. microturbine.

4.5.1.4.5.1. Experiment Experiment 1: Dispatch1: Dispatch Layer Layer (Solar (Solar PVPV + StorageStorage Inverter) Inverter) The controllerThe controller operates operates the the inverter inverter in in PQ PQ mode mode (grid-connected). (grid-connected). The set set point point for for real real power power (P) is changed,(P) is changed and the, and inverter the inverter reacts reacts based based on new on newP values. P values. The The graphical graphical user user interfaceinterface ( (GUI)GUI) from from the microgridthe microgrid controller, controller, Energy IQ,Energy conﬁrms IQ, confirmsthe dispatch the layerdispatch was layer successful. was successful Figure 12. Figure demonstrates 12 the grid-connecteddemonstrates the base-loading grid-connected mode base where-loading the mode controller where commands the controller the invertercommands with the the inverter positive P valuewith (inverter the positive exporting P value power (inverter to the exporting grid) as power a set to point. the grid) As canas a beset observedpoint. As can in Figurebe observed 12, both in the Figure 12, both the discharging battery as solar output contribute to providing the power set point (P discharging battery as solar output contribute to providing the power set point (P value). value).

Figure 12. Positive set point assigned by the controller (baseload mode).

Figure 13 demonstrates the grid-connected base loading mode where the controller commands the inverter with the negative P value (inverter importing power to the grid) as a set point. As can be observed in Figure 13, both the grid and solar contribute to charging the battery. Energies 2019, 12, x FOR PEER REVIEW 20 of 30

Figure 12. Positive set point assigned by the controller (baseload mode). Energies 2019, 12, x FOR PEER REVIEW 20 of 30

Figure 13 demonstratesFigure 12. P ositivethe grid set- connectedpoint assigned base by loadingthe controller mode (baseload where mode the controller). commands the inverter with the negative P value (inverter importing power to the grid) as a set point. As can be observedFigure in Figure 13 demonstrates 13, both the the grid grid and-connected solar contribute base loading to charging mode where the battery.the controller commands Energiesthe inverter2019, 12, with 4202 the negative P value (inverter importing power to the grid) as a set point. As can20 be of 30 observed in Figure 13, both the grid and solar contribute to charging the battery.

Figure 13. Negative set point assigned by the controller (baseload mode). Figure 13. Negative set point assigned by the controller (baseload mode). Figure 13. Negative set point assigned by the controller (baseload mode). 4.5.2. Experiment 2: Dispatch Layer (Microturbine) 4.5.2.4.5.2. Experiment Experiment 2: 2: Dispatch Dispatch LayerLayer (Microturbine)(Microturbine) In Figure 14, interval data from the main building meter shows the microturbine was active In Figure 14, interval data from the main building meter shows the microturbine was active betweenIn Figure 10:35 14AM, interval and 11:30 data AM from during the the main test building day, where meter it was shows operated the microturbine with the P wasvalue active of 20 between 10:35 AM and 11:30 AM during the test day, where it was operated with the P value of 20 betweenkW. As the 10:35 result AM of and its 11:30 operation, AM during the average the test power day, where imported it was from operated grid dropped with the Pfromvalue 32.7 of 20kW kW. to kW. As the result of its operation, the average power imported from grid dropped from 32.7 kW to As12.8 the kW result and ofat its 10:55 operation, AM, when the average it was ramped power imported up to 65 fromKW gridfor a dropped short period from of 32.7 time, kW this to 12.8 new kW P 12.8 kW and at 10:55 AM, when it was ramped up to 65 KW for a short period of time, this new P andvalue at resulted 10:55 AM, in whenan average it was (calculated ramped up over to 65 5 KW-min for in atervals) short period of 6 kW of net time, power this new exportedP value to resultedthe grid value resulted in an average (calculated over 5-min intervals) of 6 kW net power exported to the grid in an average (calculated over 5-min intervals) of 6 kW net power exported to the grid at 10:55 AM. at at10:55 10:55 AM AM. .

4040 3232 2424 1616 8 8 Avgerage kW Avgerage Avgerage kW Avgerage 0 0

Building Load Turbine Generation Sent to PIDC Grid Building Load Turbine Generation Sent to PIDC Grid

FigureFigure 14.14. BuildingBuilding load and and microturbine microturbine power power delivery delivery.. Figure 14. Building load and microturbine power delivery.

5. Simulation of Optimized Dispatch for Demand Charge Reduction Before discussing the HOMER Grid simulation, the power limits the software observes for charging and discharging the battery and is applied to the simulation is explained. Energies 2019, 12, x FOR PEER REVIEW 21 of 30

5. Simulation of Optimized Dispatch for Demand Charge Reduction

EnergiesBefore2019, 12 discussing, 4202 the HOMER Grid simulation, the power limits the software observes21 of for 30 charging and discharging the battery and is applied to the simulation is explained.

5.1. Maximum Maximum Battery Discharge and Charge Power The maximum amount of power that the storage bank can discharge (charge) over a speciﬁcspecific length of time can be calculated based on the Kinetic Storage model [[72]72] illustratedillustrated inin FigureFigure 1515..

Figure 15.15. Kinetic storage model model..

Based on the model, the following equations can bebe derived: dQ Q Q 1 = −() + (ℎ − ℎ) = −() + ( 2 −1 ) (1) = P(t) + k(h2 h1) = P(t) + k 1 − (1) dt − − − 1 c − c − dQ2 Q2 Q == −k(h(ℎ −h ℎ)=) = −k ( −1 ) (2) dt − 2 − 1 − 1 1 c−− c − Q1 isis the the available energy [kWh][kWh] inin thethe Storage Storage Component Component at at the the beginning beginning of of the the time time step, step,Q 2 is isthe the total total amount amount of energyof energy [kWh] [kWh] in the in Storage the Storage Component Component at the at beginning the beginning of the timeof the step time (bound step 1 ℎ ℎ (boundenergy), energy),c is the storage is the capacitystorage capacity ratio [unitless], ratio [unitless],k is the storage k is the rate storage constant rate constant [h− ], h1 is[ the], height is the of heightavailable of available charge well, charge and well,h2 is theand height ℎ is the of boundheight chargeof bound well. charge The well initial. The conditions initial conditions are: are:

(0) = & (0) = (1 − ) (3) Q (0) = cQmax & Q (0) = (1 c)Qmax (3) 1 2 − where is the total capacity [kWh] of the storage bank. Solving the above differential equations whereyields theQmax followingis the total equation capacity for [kWh] maximum of the discharge storage bank. power: Solving the above differential equations yields the following equation for maximum discharge power:∆ ∆ − + + (1 − ) ,, = ∆ ∆ (4) 1 − + k(∆t ∆ − 1+ k∆)t kcQmax + kQ e + Qkc 1 e − 1 − − − Pbatt, dmax,kbm = (4) Through a similar approach, we arrive 1at thee k ∆followingt + c(k∆t equation1 + e k∆ fot)r maximum charge power: − − − − ∆ + (1 − ∆) Through a similar approach, we arrive= at the following equation for maximum charge power:(5) ,, 1 − ∆ + (∆ − 1 + ∆) k∆t k∆t where ∆ is the length of the time step [h]. kQ1e− + Qkc 1 e− P = − (5) batt,cmax, kbm k∆t k∆t 1 e− + c(k∆t 1 + e− ) 5.2. HOMER Grid Simulation − − where ∆t is the length of the time step [h]. The main focus in this HOMER Grid simulation [73–75] is optimizing distributed energy resource5.2. HOMER dispatch Grid Simulation with the main objective of demand cost reduction. The primary advantage of Homer Grid is its integration with the Genability tariff database [76] to efficiently model the economic The main focus in this HOMER Grid simulation [73–75] is optimizing distributed energy resource dispatch with the main objective of demand cost reduction. The primary advantage of Homer Grid Energies 2019, 12, 4202 22 of 30 is its integration with the Genability tariff database [76] to efficiently model the economic effect of variousEnergies control 2019, 12, x strategies FOR PEER REVIEW for distributed energy resources. HOMER Grid aims to limit the monthly22 of 30 demand charges and therefore reduces the total operational cost. effectHOMER of various simulates control optimal strategies control for distributed strategies withenergy the resources. goal of the HOMER most economic Grid aims DER to limit (i.e., the solar, batteries,monthly and demand microturbine charges CHP)and therefore dispatch reduce to avoids the significant total operational demand cost. charges. The CHP can operate only ifHOMER it is loaded simulates to at leastoptimal 30% control of its strategies nominal with capacity—a the goal of constraint the most economic taken into DER account (i.e., solar, by the batteries, and microturbine CHP) dispatch to avoid significant demand charges. The CHP can operate software. The natural gas price is $0.4384/m3. The initial cost, replacement cost, and operation and only if it is loaded to at least 30% of its nominal capacity—a constraint taken into account by the maintenance (O&M) cost of this CHP unit is $75,000, $75,000, and $1.95/hour, respectively. The model software. The natural gas price is $0.4384/m3. The initial cost, replacement cost, and operation and setup allows simulating dispatch with and without the CHP generator. The convertor model has an maintenance (O&M) cost of this CHP unit is $75,000, $75,000, and $1.95/hour, respectively. The model efficiency of 94% for both inverter and rectifier, while the relative capacity of the rectifier is 100%. The setup allows simulating dispatch with and without the CHP generator. The convertor model has an capital cost and replacement cost of the inverter are both $50,000, and this study assumes a $0/year efficiency of 94% for both inverter and rectifier, while the relative capacity of the rectifier is 100%. O&MThe cost. capital For cost the and storage replacement configuration, cost of the the minimuminverter are and both the $50,000, initial state and of this charge study are assumes 10.2% anda 100%,$0/year respectively. O&M cost. The For capital the storage and replacement configuration, costs the are minimum $30,000, and the O&Minitial coststate isof $0. charge Forthe are PV panels,10.2% this and simulation 100%, respectivel assumesy. The a derating capital and factor replacement of 80% with costs initial are $30,000, and replacement and the O&M cost cost of $20,000is $0. andFor O&M the PV cost panel of $390s, this per simulation year. assumes a derating factor of 80% with initial and replacement cost of The$20,000 utility and rates O&M are cost (i) of the $390 energy per year. efficiency charge of 0.00223 $/kWh net purchase, (ii) the variable distributionThe utility service rates charge are of (i) 0.0006 the energy $/kWh e netfficiency meter, charge and (iii) of the 0.00223 flatgeneration $/kWh net charge purchase, of 0.12 (ii) $ the/kWh netvariable purchase. distribution The variable service distribution charge of 0.0006 service $/kWh charge net is meter, 7.93 $/ kW.and (iii) the flat generation charge of 0.12 $/kWh net purchase. The variable distribution service charge is 7.93 $/kW. Simulation Results Simulation Results Summer Peak Demand Reduction •  Summer Peak Demand Reduction In Figure 16, three consecutive summer days where the grid demand limit of 40 KW was exceeded is depicted.In Figure As it 16, can three be seen, consecutive the software summer dispatches days where all three the DER’s grid demand in the first limit and of second 40 KW day was and onlyexceeded the battery is depicted. and solar As PVit can in thebe seen, third the day software which hasdispatches a noticeably all three lower DER peak’s in comparedthe first and to second the other twoday days, and yetonly still the above battery the and grid sol demandar PV in limit.the third In allday three which days, has demanda noticeably charges lower are peak avoided compared as total to the other two days, yet still above the grid demand limit. In all three days, demand charges are grid purchases demonstrate in the figure. avoided as total grid purchases demonstrate in the figure.

Figure 16. Simulation results for three consecutive summer days. Figure 16. Simulation results for three consecutive summer days.  Summer peak demand reduction, a closer look Summer peak demand reduction, a closer look • As it can be observed in Figure 17, on 6 August 2016, DER's are dispatched optimally by the As it can be observed in Figure 17, on 6 August 2016, DER’s are dispatched optimally by the software avoid a peak demand charge during the period that the building load remains above the software avoid a peak demand charge during the period that the building load remains above the grid demand limit starting a little after 7:00 all the way until 20:00. Electrical demand is significantly gridhigher demand at 8:00 limit in starting this summer a little day after compared 7:00 all the to way 7:00 untiland 20:00. as a result, Electrical the batterydemand is is dispatched signiﬁcantly highermaximally at 8:00 from in this a starting summer state day of compared charge of to 100% 7:00 while and as the a result,solar ramps the battery up (it isslowly dispatched ramps maximallyup until fromnoon a startingas normally state expected) of charge and of as 100% seen while in the the figure, solar the ramps state of up charge (it slowly is reduced ramps for up the until next noon two as hours until the minimum allowed of 12.5%. Although the demand at 10:00 is slightly lower compared to 9:00, the microturbine is fired up and operational at 10:00. Not surprisingly, its operation coincides

Energies 2019, 12, 4202 23 of 30 normally expected) and as seen in the figure, the state of charge is reduced for the next two hours until the minimum allowed of 12.5%. Although the demand at 10:00 is slightly lower compared to 9:00, theEnergies microturbine 2019, 12, x FOR is fired PEER upREVIEW and operational at 10:00. Not surprisingly, its operation coincides23 of with30 charging the battery up to 100% state of charge. Afterward, the battery discharges, first slowly until 15:00with and charging quickly the afterwards battery up as to the 100% solar state PV generationof charge. Afterward decreases, inthe the battery later afternoondischarges, hours. first slowly Starting until 15:00 and quickly afterwards as the solar PV generation decreases in the later afternoon hours. at 20:00,Energies as 2019 the, building12, x FOR PEER exits REVIEW the grid demand limit interval, the battery is charged with grid23 of power30 Starting at 20:00, as the building exits the grid demand limit interval, the battery is charged with grid until fully charged at 22:00. powerwith until charging fully the charged battery at up 22:00 to 100%. state of charge. Afterward, the battery discharges, first slowly until 15:00 and quickly afterwards as the solar PV generation decreases in the later afternoon hours. Starting at 20:00, as the building exits the grid demand limit interval, the battery is charged with grid power until fully charged at 22:00.

Figure 17. Simulation results for a summer day. Figure 17. Simulation results for a summer day.

 Winter Peak Demand ReductionFigure 17. Simulation results for a summer day. Winter Peak Demand Reduction • In Figure 18, three consecutive winter days where the grid demand limit of 40 KW was exceeded  Winter Peak Demand Reduction is depicted.In Figure As 18 ,it three can be consecutive seen, the software winter daysdispatches where all the three grid DER's demand in the limit second of 40 and KW third was day exceeded and is depicted.only theIn battery Figure As it can 18,and three be solar seen, consecutive PV the in software the winter first dispatchesday days which where allhas the three grida noticeably demand DER’s in limitlower the of second peak40 KW compared and was thirdexceeded to day the and onlyother theis depicted. two battery days, As and yet it solarcan still be above PV seen, in the the softwaregrid ﬁrst demand day dispatches which limit. has allIn a threeall noticeably three DER's days in lower ,the demand second peak chargescompared and third are day to avoided theand other twoas days,totalonly gridthe yet battery stillpurchases above and thedemonstratesolar grid PV demandin th ine firstthe limit. figureday Inwhich. all three has a days,noticeably demand lower charges peak compared are avoided to the as total other two days, yet still above the grid demand limit. In all three days, demand charges are avoided grid purchases demonstrate in the ﬁgure. as total grid purchases demonstrate in the figure.

Figure 18. Simulation results for three consecutive winter days. Figure 18. Simulation results for three consecutive winter days. Figure 18. Simulation results for three consecutive winter days.  Winter peak demand reduction, a closer look  Winter peak demand reduction, a closer look WinterAs observed peak demand in Figure reduction, 19, on 6a December closer look 2016, DER's are dispatched optimally by the software • to avoidAs a observedpeak demand in Figure charge 19, on during 6 December the period 2016, DER'sthat the are building dispatched load optimally remains by above the software the grid demandto avoid limit a peak starting demand at 7:00 charge and lastingduring theuntil period a little that after the 16:00 building. Electrical load remains demand above is significantly the grid demand limit starting at 7:00 and lasting until a little after 16:00. Electrical demand is significantly higher at 8:00 as the building is occupied in this winter day compared to 7:00 and as the result, the higher at 8:00 as the building is occupied in this winter day compared to 7:00 and as the result, the battery is dispatched maximally from a starting state of charge of 100% while the solar slowly ramps battery is dispatched maximally from a starting state of charge of 100% while the solar slowly ramps

Energies 2019, 12, 4202 24 of 30

As observed in Figure 19, on 6 December 2016, DER’s are dispatched optimally by the software to avoid a peak demand charge during the period that the building load remains above the grid demand limit starting at 7:00 and lasting until a little after 16:00. Electrical demand is significantly higher at 8:00 as the building is occupied in this winter day compared to 7:00 and as the result, the battery is dispatchedEnergies 2019 maximally, 12, x FOR PEER from REVIEW a starting state of charge of 100% while the solar slowly ramps24 upof 30 (in this particular day, peak PV generation is achieved at 10:00) and as seen state of charge is reduced significantlyup (in this atparticular 8:00 due day, to the peak battery PV generation discharge. is At achieved this time, at 10 even:00) thoughand as seen the building state of charge demand is is slightlyreduced lower significantly than the previousat 8:00 due time to the interval, battery the discharge. microturbine At this is operationaltime, even though at almost the fullbuilding power. Thedem nextand time is slightly interval lower (10:00), than we the can previous see that time the batteryinterval, is the being microturbine discharged is againoperational and continues at almost to dofull sountil power. noon The when next time it starts interval to charge (10:00 up), we until can 14:00 see that asthe the solarbattery PV is is being steadily discharged available again (although and continues to do so until noon when it starts to charge up until 14:00 as the solar PV is steadily available noticeably less than that summer peak). It can be seen that solar drops off after 14:00, and as the (although noticeably less than that summer peak). It can be seen that solar drops off after 14:00, and result, the battery is dispatched again afterward as its state of charge drops until 16:00, after which it is as the result, the battery is dispatched again afterward as its state of charge drops until 16:00, after charged again as the building exits the grid demand limit interval a little after 16:00. By 17:00, the which it is charged again as the building exits the grid demand limit interval a little after 16:00. By battery is fully charged again. 17:00, the battery is fully charged again.

FigureFigure 19. 19.Simulation Simulation results for a a winter winter day. day.

6. Conclusions6. Conclusions TheThe literature literature on on microgrid microgrid control control waswas reviewedreviewed to highlight an an existing existing gap gap in in design design and and implementationimplementation of of microgrids microgrids that that are are small small enoughenough inin size that that can can be be installed installed behind behind the the meter meter of of commercialcommercial buildings buildings which which utilize utilize both both renewablerenewable and nonrenewable distributed distributed energy energy resources resources highlightinghighlighting the the resiliency resiliency benefits benefits of of the the latter,latter, especiallyespecially in locations locations where where natural natural gas gas is isabundant. abundant. TheThe distributed distributed energy energy resources resources consisted consisted ofof solarsolar PV, naturalnatural gas gas-fired-fired microturbine, microturbine, lithium lithium-ion-ion energy storage, and associated communication and control components of this recently installed energy storage, and associated communication and control components of this recently installed microgrid and their individual functions were explained. For the development of the control microgrid and their individual functions were explained. For the development of the control platform, platform, the existing, commonly used, hierarchical control architecture of the larger scale feeder the existing, commonly used, hierarchical control architecture of the larger scale feeder level microgrids level microgrids was reviewed and adapted to the low-voltage setup of a recently constructed energy was reviewed and adapted to the low-voltage setup of a recently constructed energy efficient commercial efficient commercial building. The control layout demonstrates a chain of command from the central building.microgrid The controller control layout to individual demonstrates DER controllers a chain of commandand bidirectional from the communication central microgrid between controller the to individualcentral controller DER controllers and the and building bidirectional automation communication system, each between with itsthe own central industry controller-standard and the buildingcommunication automation protocol. system, each with its own industry-standard communication protocol. TheThe control control platform platform enables enables the DERs the DERs to be to timely be timely responsive responsive to the microgrid to the microgrid modes of modes operation of andoperation grid and and load grid conditions. and load conditions. The implementation The implementation of this platform of this includedplatform included mapping mapping the microgrid the modesmicrogrid of operation modes toof the operation microturbine to the andmicroturbine inverter (managing and inverter solar (managing PV and batteries). solar PV and The batteries). functioning of bothThe functioning the microturbine of both the and microturbine the inverter and was the tested inverter and was documented tested and documented with collected with data collected for the grid-connecteddata for the mode.grid-connected Finally, using mode. the Finally, recently using released theHOMER recently Grid released software, HOMER dispatch Grid optimization software, fordispatch a year-long optimization simulation for of a theyear system-long simulation operation of was the presented.system operation was presented. The simulation of the grid-connected mode of operation highlighted the benefits of the hybrid solar PV, battery energy storage, and natural gas setup in avoiding utility peak demand charges. Specifically, it was demonstrated how optimal dispatch could dictate efficient coordination between the three DER's to avoid rates associated with exceeding the utility's demand limit. In the context of real building data, it was demonstrated how the microturbine can mitigate the intermittency of solar

Energies 2019, 12, 4202 25 of 30

The simulation of the grid-connected mode of operation highlighted the benefits of the hybrid solar PV, battery energy storage, and natural gas setup in avoiding utility peak demand charges. Specifically,Energies 2019, it12 was, x FOR demonstrated PEER REVIEW how optimal dispatch could dictate efficient coordination between25 of 30 the three DER’s to avoid rates associated with exceeding the utility’s demand limit. In the context ofPV real when building the battery data, itis wasno longer demonstrated able to sustain how the this microturbine function and can how mitigate the combination the intermittency of the three of solarenables PV whenthe microturbine the battery isto no operate longer at able peak to sustainefficiency this for function a fewer and number how theof hours combination as opposed of the to threemore enables frequent the less microturbine efficient operation to operate if atit peakwere e ffitheciency sole DER for a fewerof the numbermicrogrid. of hoursInvestigating as opposed how tothese more findings frequent would less e ffiapplycient to operation a larger scale, if it were a topic the for sole future DER ofresearch, the microgrid. is increasingly Investigating applicable how to thesegrid findingsmodernization would applyefforts toas a both larger natural scale, agas topic-powered for future distributed research, generation is increasingly and battery applicable energy to gridstorage modernization are projected eff ortsto grow as both, and natural their interplay gas-powered is of distributedinterest to grid generation operators, and especially battery energy during storagepeak demand are projected periods. to grow, and their interplay is of interest to grid operators, especially during peak demandThe periods. next step in the implementation of the microgrid is completing developing control modules for The other next modes step in of the operation implementation and programming of the microgrid existing is completing and new developing control modules control to modules enforce foroptimized other modes dispatch of operation under anddifferent programming modes of existing operation and as new detailed control in modules the future to enforcedirections optimized section, dispatchbelow. under different modes of operation as detailed in the future directions section, below.

7.7. Future Future Directions Directions TheThe next next step step based based on on the the simulation simulation results results is is utilizing utilizing the the HOMER HOMER controller controller API API (application (application programmingprogramming interface; interface; Figure Figure 20) to 20) develop, to develop, test, and test, validate and optimizer-layer validate optimizer algorithms,-layer algor includingithms, thoseincluding based those on the based scenarios on the scenarios discussed discussed below. HOMERbelow. HOMER utilizes utilizes a proprietary a proprietary “derivative-free” “derivative- optimizationfree” optimization algorithm, algorithm and, as and the as result, the result is computationally, is computationally advantageous advantageous compared compared to to other other alternativesalternatives in in this this area. area.

Figure 20. HOMER Controller API. Reproduced with permission from [74]. HOMER Energy, 2019. Figure 20. HOMER Controller API. Reproduced with permission from [74]. HOMER Energy, 2019.

BuildingBuilding automation automation systems systems (BAS) (BAS) can can be be fine-tuned fine-tuned to to optimize optimize their their participation participation in in energy energy andand ancillary ancillary services services such such as as frequency frequency regulation regulation and and demand demand response. response. For For example, example, one one way way to to accomplishaccomplish this this is tois programto program and and operate operate the buildingthe building under under a special a special demand demand curtailment curtailment mode withmode newwith temperature new temperature set points, set mappedpoints, mapped to lower to demand lower levelsdemand [77 ]levels at the [77] expense at the of expense minimal of occupant minimal discomfortoccupant todiscomfort reduce demand to reduce map. demand The microgrid map. The controller microgrid can controller be programmed can be to programmed communicate to lowercommunicate demand levels lower to demand the building levels automation to the building system where automation the latter system can accordingly where the map latter to can a newaccordingly set point. map In otherto a new words, set point the combination. In other words, of the the microgrid combination controller of the microgrid and BAS cancontroller dispatch and buildingBAS can loads dispatch in tandem building with loads distributed in tandem energy with distributed resources (asenergy detailed resources earlier (as in detailed the simulation earlier in section).the simulation An integrative section). (BAS An + integrativeDER dispatch) (BAS approach + DER dispatch)to improving approach network to security improving (addressed network brieflysecurity in Section(addressed2) should briefly be in implemented Section 2) should as well. be implemented as well. InIn the the optimizer optimizer layer, layer, a a schedule schedule (lookahead) (lookahead) optimization optimization can can be be programmed programmed in in the the microgrid microgrid controllercontroller based based on on forecasting forecasting building building load load and and meteorological meteorological conditions. conditions. In In [ 78[78]], the, the authors authors proposepropose a method, a method, implemented implemented through through simulation, simulation, for coordinating for coordinating controls controls between between the two the layers two forlayers both for grid-connected both grid-connected and stand-alone and stand modes-alone by modes maximizing by maximizing revenue according revenue according to the DER to bids the andDER marketbids and price market in grid-connected price in grid- modeconnected and bymode maximizing and by maximizing satisfaction satisfaction rate of the load rate withof the minimum load with operationminimum cost operation in the stand-alonecost in the stand mode.-alone A similarmode. A double-layer similar double setup-layer is proposedsetup is proposed in [79] in in the [79] contextin the of context an experimental of an experimental setup with setup solar PV with and solar batteries PV and as DERs. batteries Utilizing as DERs. the implemented Utilizing the controlimplemented platform control in this platform work to in execute this work a similar to execute control a similar scheme control adds thescheme value adds of also the includingvalue of also a naturalincluding gas-powered a natural microturbine.gas-powered microturbine. The approach presentedThe approach in [80 presented] where two in [80] different where horizons two different were horizons were considered for optimization is applicable to the next phase of this work as prediction based on the long-term horizon can be applied to the battery state of charge and a fuzzy-based approach can be used in the short-term horizon. Also, electric vehicle charging facilities, planned for installation when additional funding is secured, can be utilized as a DER and be dispatched in the optimizer layer to participate in grid ancillary services, as an advanced version of a setup tested in a nearby facility [81].

Energies 2019, 12, 4202 26 of 30 considered for optimization is applicable to the next phase of this work as prediction based on the long-term horizon can be applied to the battery state of charge and a fuzzy-based approach can be used in the short-term horizon. Also, electric vehicle charging facilities, planned for installation when additional funding is secured, can be utilized as a DER and be dispatched in the optimizer layer to participate in grid ancillary services, as an advanced version of a setup tested in a nearby facility [81]. Finally, utilizing other software such as Distributed Energy Resources-Customer Adoption Model (DER-CAM) [82] for solving the optimal dispatch problem is an area that can be investigated using this control platform. Although DER-CAM has been integrated off-line with a number of microcontrollers [15,61], integrating it with the microgrid controller in real-time, similar to the setup introduced in [83] is a potential new direction for this project as additional funding becomes available as a more extensive software option compared to HOMER for integration with this behind-the-meter building-scale microgrid controller. Last but not least is the installation and configuration of high-accuracy meters in addition to the built-in meters in the DERS so that the controller receives energy generation and consumption from multiple points and can factor those into computing the overall efficiency of the system.

Author Contributions: Conceptualization, P.D. and J.D.F.; methodology, P.D.; resources, J.D.F.; writing—original draft preparation, P.D.; writing—review and editing, P.D.; visualization, P.D.; funding acquisition, J.D.F. Funding: This work was supported by the U.S. Department of Energy, Office of Energy Efficiency and Renewable Energy (EERE), under the State Energy Program Award Number DE-EE0006994 through a grant from the Pennsylvania Department of Environmental Protection. Conflicts of Interest: The authors declare no conflict on interest.

References

1. Microgrid Definitions. Available online: https://building-microgrid.lbl.gov/microgrid-definitions (accessed on 16 August 2019). 2. Fu, Q.; Nasiri, A.; Solanki, A.; Bani-Ahmed, A.; Weber, L.; Bhavaraju, V. Microgrids: Architectures, Controls, Protection, and Demonstration. Electr. Power Compon. Syst. 2015, 43, 1453–1465. [CrossRef] 3. U.S. Solar Photovoltaic System Cost Benchmark: Q1 2018. Available online: https://www.nrel.gov/docs/ fy19osti/72399.pdf (accessed on 16 August 2019). 4. Riley, D.; Wright, K.; Mohammadpour, A.; Delgoshaei, P. Challenges and Opportunities in Microgrid and Solar Photovoltaic System Construction. In Proceedings of the Construction Research Congress 2016, San Juan, PR, USA, 31 May–2 June 2016; pp. 1092–1101. 5. Schittekatte, T.; Stadler, M.; Cardoso, G.; Mashayekh, S.; Sankar, N. The Impact of Short-Term Stochastic Variability in Solar Irradiance on Optimal Microgrid Design. IEEE Trans. Smart Grid 2018, 9, 1647–1656. [CrossRef] 6. Wan, C.; Zhao, J.; Song, Y.; Xu, Z.; Lin, J.; Hu, Z. Photovoltaic and solar power forecasting for smart grid energy management. CSEE J. Power Energy Syst. 2015, 1, 38–46. [CrossRef] 7. Bracale, A.; Caramia, P.; Carpinelli, G.; Di Fazio, A.; Ferruzzi, G. A Bayesian Method for Short-Term Probabilistic Forecasting of Photovoltaic Generation in Smart Grid Operation and Control. Energies 2013, 6, 733–747. [CrossRef] 8. Chu, Y.; Pedro, H.T.C.; Li, M.; Coimbra, C.F.M. Real-time forecasting of solar irradiance ramps with smart image processing. Sol. Energy 2015, 114, 91–104. [CrossRef] 9. Zhu, H.; Li, X.; Sun, Q.; Nie, L.; Yao, J.; Zhao, G. A Power Prediction Method for Photovoltaic Power Plant Based on Wavelet Decomposition and Artificial Neural Networks. Energies 2016, 9, 11. [CrossRef] 10. Kaun, B.; Chen, S.; Willard, S. Integration Guidelines for Energy Storage: A Project Management Handbook for Distribution-Connected Energy Storage Developed by the Energy Storage Integration Council; Electric Power Research Institute: Palo Alto, CA, USA, 2015. 11. Maitra, A. Microgrid Controller Requirements and Relationship with DMS; National Renewable Energy Laboratory: Golden, CO, USA, 2015. Energies 2019, 12, 4202 27 of 30

12. Riley, D.; Delgoshaei, P. Energy storage and microgrid construction. In Proceedings of the 6th CSCE-CRC International Construction Specialty Conference 2017—Held as Part of the Canadian Society for Civil Engineering Annual Conference and General Meeting 2017, Vancouver, BC, Canada, 31 May–1 June 2017; pp. 572–581. 13. Hampson, A.; Rackley, J. Guide to Using Combined Heat and Power for Enhancing Reliability and Resiliency in Buildings; ICF International: Vairfax, VA, USA, 2013. 14. Mid-Atlantic CHP Technical Assistance Partnership. Available online: https://betterbuildingssolutioncenter. energy.gov/chp/mid-atlantic-chp-technical-assistance-partnership (accessed on 16 August 2019). 15. Alegria, E.; Ma, A.; Idrees, O. Certs Microgrid Demonstration with Largescale Energy Storage and Renewables at Santa Rita Jail; Chevron Energy Services Company: San Ramon, CA, USA, 2014. 16. Bose, S. Smart Microgrid Energy Management Controls for Improved Energy Eﬃciency and Renewables Integration at DoD Installations; GE Global Research: Niskayuna, NY, USA, 2013. 17. Bose, S.; Krok, M. Microgrid Design, Development and Demonstration; GE Global Research Center: Niskayuna, NY, USA, 2011. 18. Lasseter, R.H.; Eto, J.H.; Schenkman, B.; Stevens, J.; Vollkommer, H.; Klapp, D.; Linton, E.; Hurtado, H.; Roy, J. CERTS Microgrid Laboratory Test Bed. IEEE Trans. Power Deliv. 2011, 26, 325–332. [CrossRef] 19. Panora, R.; Gehret, J.E.; Furse, M.M.; Lasseter, R.H. Real-World Performance of a CERTS Microgrid in Manhattan. IEEE Trans. Sustain. Energy 2014, 5, 1356–1360. [CrossRef] 20. Paquette, A.; Harley, R.; Bhavaraju, V.; Krstic, S.; Theisen, P. Design of the Fort Sill Microgrid. In Proceedings of the 2014 IEEE Energy Conversion Congress and Exposition (ECCE), Pittsburgh, PA, USA, 14–18 September 2014. 21. Mienski, R.; Pawelek, R.; Wasiak, I.; Gburczyk, P. Monitoring and Control Systems for Testing Microgrids Operation on the Example of Laboratory of Distributed Generation at the Technical University of Lodz. In Proceedings of the 2009 10th International Conference on Electrical Power Quality and Utilisation, Lodz, Poland, 15–17 September 2009. 22. Pawelek, R.; Wasiak, I.; Gburczyk, P.; Mienski, R. Study on Operation of Energy Storage in Electrical Power Microgrid—Modeling and Simulation. In Proceedings of the 14th International Conference on Harmonics and Quality of Power—ICHQP 2010, Bergamo, Italy, 26–29 September 2010. 23. Torres, E.; Larragueta, J.M.; Eguía, P.; Mazón, J.; San Martín, J.I.; Zamora, I. Dynamic performance of a microturbine connected to a low voltage network. Renew. Energy Power Qual. J. 2008, 1, 468–473. [CrossRef] 24. Wasiak, I.; Pawelek, R.; Mienski, R.; Gburczyk, P. Using Energy Storage for Energy Management and Load Compensation in LV Microgrids. In Proceedings of the 2012 IEEE 15th International Conference on Harmonics and Quality of Power, Hong Kong, China, 17–20 June 2012. 25. Engler, A.; Osika, O.; Barnes, M.; Jenkins, N.; Arulampalam, A. Large Scale Integration of Micro-Generation to Low Voltage Grids; University of Kassel: Kassel, Germany, 2004. 26. Mahmoud, M. Microgrid; Elsevier: Oxford, UK, 2016. 27. Agile Digital Substations 2.0 Handbook; Alstom: Saint-Ouen, France, 2015. 28. DS Agile A301 Substation Gateway; Alstom: Saint-Ouen, France, 2014. 29. Duke Energy Builds Microgrid Control System with Standard Power System Components; Schweitzer Engineering Laboratories: Pullman, WA, USA, 2017. 30. Li, Y.; Nejabatkhah, F. Overview of control, integration and energy management of microgrids. J. Mod. Power Syst. Clean Energy 2014, 2, 212–222. [CrossRef] 31. Olivares, D.E.; Mehrizi-Sani, A.; Etemadi, A.H.; Canizares, C.A.; Iravani, R.; Kazerani, M.; Hajimiragha, A.H.; Gomis-Bellmunt, O.; Saeedifard, M.; Palma-Behnke, R.; et al. Trends in Microgrid Control. IEEE Trans. Smart Grid 2014, 5, 1905–1919. [CrossRef] 32. Bidram, A.; Davoudi, A. Hierarchical Structure of Microgrids Control System. IEEE Trans. Smart Grid 2012, 3, 1963–1976. [CrossRef] 33. Du, W.; Jiang, Q.; Erickson, M.J.; Lasseter, R.H. Voltage-Source Control of PV Inverter in a CERTS Microgrid. IEEE Trans. Power Deliv. 2014, 29, 1726–1734. [CrossRef] 34. Hoogenboezem, J. Distributed Network Protocols—The Old and the New DNP3, IEC 60870-5 and IEC 61850. Available online: https://pdfs.semanticscholar.org/dd4b/7f2f9a3b1203137ec8dd039430d1389fac39.pdf (accessed on 20 September 2019). Energies 2019, 12, 4202 28 of 30

35. EtherNet/IP to IEC60870-5-104 Client Communication Gateway. Available online: https://www.prosoft-technology. com/Products/Gateways/EtherNet-IP/EtherNet-IP-to-IEC60870-5-104-Client-Communication-Gateway (accessed on 20 September 2019). 36. Maitra, A.; Simmins, J. Grid Interactive Microgrid Controllers and the Management of Aggregated Distributed Energy Resources (DER); Electric Power Research Institute: Palo Alto, CA, USA, 2015. 37. Wang, J.; Lu, X.; Chen, C. Guidlines for Implementing Advanced Distribution Management Systems: Requirements for DMS Integration with DERMS and Microgrids; U.S. Department of Energy: Washington, DC, USA, 2015. 38. Mlynek, P.;Misurec, J.; Fujdiak, R.; Kolka, Z.; Pospichal, L. Heterogeneous networks for smart metering–power line and radio communication. Elektronika Ir Elektrotechnika 2015, 21, 85–91. [CrossRef] 39. Li, Y.; Cheng, X.; Cao, Y.; Wang, D.; Yang, L. Smart Choice for the Smart Grid: Narrowband Internet of Things (NB-IoT). IEEE Internet Things 2018, 5, 1505–1515. [CrossRef] 40. The IEC and Cyber Security. Available online: https://blog.iec.ch/2018/09/the-iec-and-cyber-security- management-structure (accessed on 20 September 2019). 41. Fujdiak, R.; Mlynek, P.; Blazek, P.; Barabas, M.; Mrnustik, P. Seeking the relation between performance and security in modern systems: Metrics and measures. In Proceedings of the 41st International Conference on Telecommunications and Signal Processing (TSP), Athens, Greece, 4–6 July 2018. 42. Sohn, J.-M.; Yun, S.-Y. Software Functional Requirements and Architectures of Microgrid Energy Management System. Kepco J. Electr. Power Energy 2016, 2, 269–272. [CrossRef] 43. The Center for Building Energy Science (Building 661)—CBEI Headquarters. Available online: http://cbei.psu. edu/the-center-for-building-energy-science-building-661-cbei-headquarters/ (accessed on 16 August 2019). 44. Consortium for Building Energy Innovation—CBEI: Solving Regional Energy Innovation. Available online: https://www.pidcphila.com/impact/success-stories/success-stories-detail/consortium-for-building- energy-innovation-cbei (accessed on 16 August 2019). 45. Pennsylvania State University GridSTAR Center: Smart Grid Training and Application Resource Center. Available online: https://www.smartgrid.gov/project/pennsylvania_state_university.html (accessed on 16 August 2019). 46. Combined Heat and Power (CHP)-Enabled Renewable, Distributed Energy Technology in a Micro-Grid via Leveraged Use of Pennsylvania Marcellus Shale Gas. Available online: https://www.dep.pa.gov/Business/ Energy/OfficeofPollutionPrevention/State-Energy-Plan/Pages/CHP_and_Micro-Grid_Technology.aspx (accessed on 16 August 2019). 47. How can we demonstrate best practices for energy efficiency that can be affordably replicated in buildings across the country? Available online: https://kierantimberlake.com/pages/view/196/consortium-for-building- energy-innovation (accessed on 20 September 2019). 48. ISOFTON Monocrystalline Module Isf-260. Available online: https://www.altestore.com/static/datafiles/ Others/Isofoton%20ISF-250%20Mono%20Datasheet%20USA.pdf (accessed on 16 August 2019). 49. What Is a Combiner Box? Available online: https://www.solarpowerworldonline.com/2015/06/what-is-a- combiner-box/ (accessed on 16 August 2019). 50. NexSys Technical Manual. Available online: https://www.enersys.com/NexSys/ (accessed on 16 August 2019). 51. 38 kWh Li-ion Battery System Technical Datasheet. Available online: http://www.allcelltech.com/index.php/ products/battery-packs (accessed on 16 August 2019). 52. Cellwatch Battery Monitoring System Installation Manual. Available online: http://www.cellwatch.com/ manuals/manual.pdf?utm_source=newsletter (accessed on 16 August 2019). 53. 50 KW MULTI PORT PCS MANUAL Model Number: EMS-7-M00561VNF. Available online: https://www. dynapower.com/products/energy-storage-solutions/dc-coupled-utility-scale-solar-plus-storage (accessed on 16 August 2019). 54. Combined Heat and Power Technology Fact Sheet Series. Available online: https://www.energy.gov/sites/ prod/files/2016/09/f33/CHP-Microturbines_0.pdf (accessed on 16 August 2019). 55. Capstone MicroTurbine Model C65 User’s Manual. Available online: https://www.capstoneturbine.com/ products/c65 (accessed on 16 August 2019). 56. Product Specification Model C65—Capstone MicroTurbine. Available online: https://www.capstoneturbine. com/products/c65 (accessed on 16 August 2019). 57. Capstone Technical Reference Dual Mode System Controller (DMSC). Available online: https://www. capstoneturbine.com/products/c65 (accessed on 16 August 2019). Energies 2019, 12, 4202 29 of 30

58. Bialek, T. Borrego Springs Microgrid Demonstration Project; California Energy Commission: San Diego, CA, USA, 2013. 59. Heckmann, W. Overview of Microgrids in Europe. In Proceedings of the Niagara 2016 Symposium on Microgrids, Niagara-on-the-Lake, ON, Canada, 20–21 October 2016. 60. Microgrids for Critical Facility Resiliency in New York State; New York State Energy Research and Development Authority: Albany, NY, USA, 2014. 61. Wiegman, H. Microgrid Plant Control Design and Development; GE Global Research: Niskayuna, NY, USA, 2017. 62. Zadeh, M.R.D.; Hajimiragha, A.; Adamiak, M.; Palizban, A.; Allan, S. New Control and Automation System for an Islanded Microgrid with Energy Storage Systems. In Proceedings of the 2011 IEEE PES Conference on Innovative Smart Grid Technologies—Middle East, Jeddah, Saudi Arabia, 17–20 December 2011. 63. ProtoGen Distributed Energy Consulting, Training, and Technical Services. Available online: https: //protogenenergy.com/ (accessed on 16 August 2019). 64. Bushby, S.T. Back to the Basics about BACnet; National Institute of Standards and Technology: Gaithersburg, MD, USA, 1993. 65. Energy IQ Advanced Energy Solutions. Available online: https://energyiq.com/en/ (accessed on 16 August 2019). 66. Niagara Framework Compatible Drivers and Applications. Available online: https://www.tridium.com/ ~{}/media/tridium/library/documents/collateral/brochures/niagara%20compatible%20drivers%20and% 20applications_jan%202019.ashx?la=en (accessed on 16 August 2019). 67. Microturbine Interface Module (mTIM). Available online: http://www.e-finity.com/products/brochures/ ControlSystem_ERMS_lowres.pdf (accessed on 16 August 2019). 68. Eaton Circuit Protection & Control Products. Available online: https://www.eaton.com/ecm/groups/public/ @pub/@seasia/@elec/documents/content/pct_1199597.pdf (accessed on 16 August 2019). 69. Operation and Maintenance Manual, Automatic Transfer Switch Controller, ATC-900. Available online: https://www.eaton.com/content/dam/eaton/products/low-voltage-power-distribution-controls-systems/ ats/resources/atsc9-ib140012en.pdf (accessed on 16 August 2019). 70. SEL-547 Distributed Generator Interconnection Relay. Available online: https://selinc.com/products/547/ (accessed on 16 August 2019). 71. Delgoshaei, P.; Heidarinejad, M.; Xu, K.; Wentz, J.R.; Delgoshaei, P.; Srebric, J. Impacts of building operational schedules and occupants on the lighting energy consumption patterns of an office space. Build. Simul. 2017, 10, 447–458. [CrossRef] 72. Manwell, J.; McGowan, J.; Baring-Gould, E.; Leotta, A. Evaluation of battery models for wind/hybrid powersystem simulation. In Proceedings of the fifth European Wind Energy Association Conference (EWEC ’94), Thessaloniki, Greece, 10–14 October 1994; pp. 1182–1187. 73. HOMER Grid 1.5 Manual. Available online: https://www.homerenergy.com/products/grid/docs/latest/index. html (accessed on 16 August 2019). 74. HOMER Pro 3.13 Manual. Available online: https://www.homerenergy.com/products/pro/docs/latest/index. html (accessed on 16 August 2019). 75. Lambert, T.; Gilman, P.; Lilienthal, P. Micropower System Modeling with HOMER. In Integration of Alternative Sources of Energy; Farret, F.A., Simoes, M.G., Eds.; John Wiley & Sons: Hoboken, NJ, USA, 2006; pp. 379–418. 76. GENABILITY Tariff APIs. Available online: https://developer.genability.com/api-reference/tariff-api/ (accessed on 16 August 2019). 77. Pavlak, G.S.; Henze, G.P.; Cushing, V.J. Optimizing commercial building participation in energy and ancillary service markets. Energy Build. 2014, 81, 115–126. [CrossRef] 78. Jiang, Q.; Xue, M.; Geng, G. Energy Management of Microgrid in Grid-Connected and Stand-Alone Modes. IEEE Trans. Power Syst. 2013, 28, 3380–3389. [CrossRef] 79. Hooshmand, A.; Asghari, B.; Sharma, R.K. Experimental Demonstration of a Tiered Power Management System for Economic Operation of Grid-Tied Microgrids. IEEE Trans. Sustain. Energy 2014, 5, 1319–1327. [CrossRef] 80. Jafari, M.; Malekjamshidi, Z.; Zhu, J.; Khooban, M.H. A Novel Predictive Fuzzy Logic-Based Energy Management System for Grid-connected and Off-grid Operation of Residential Smart Micro-grids. IEEE J. Emerg. Sel. Top. Power Electron. 2018.[CrossRef] Energies 2019, 12, 4202 30 of 30

81. Kydd, P.H.; Martin, C.A.; Komara, K.J.; Delgoshaei, P.; Riley, D. Vehicle-Solar-Grid Integration II: Results in Simulated School Bus Operation. IEEE Power Energy Technol. Syst. J. 2016, 3, 198–206. [CrossRef] 82. Marnay, C.; Venkataramanan, G.; Stadler, M.; Siddiqui, A.S.; Firestone, R.; Chandran, B. Optimal Technology Selection and Operation of Commercial-Building Microgrids. IEEE Trans. Power Syst. 2008, 23, 975–982. [CrossRef] 83. Stadler, M.; Marnay, C.; DeForest, N.; Eto, J.; Cardoso, G.; Klapp, D.; Lai, J. Web-Based Economic and Environmental Optimization of Microgrids. In Proceedings of the 2012 IEEE PES Innovative Smart Grid Technologies (ISGT), Washington, DC, USA, 16–18 January 2012.

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).