A Stochastic Approach to “Dynamic-Demand” Refrigerator Control David Angeli, Senior Member, IEEE, and Panagiotis-Aristidis Kountouriotis

IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 20, NO. 3, MAY 2012 581 A Stochastic Approach to “Dynamic-Demand” Refrigerator Control David Angeli, Senior Member, IEEE, and Panagiotis-Aristidis Kountouriotis

Abstract—Dynamic demand management is a very promising In order for such (supply) regulation to be possible, however, research direction for improving power system resilience. This it is required that “frequency response services”, as well as suffi- paper considers the problem of managing power consumption by cient reserves, are included in the system.1 This is essential not means of “smart” thermostatic control of domestic refrigerators. In this approach, the operating temperature of these appliances only for instantaneous frequency balancing, but, more impor- and thus their energy consumption, is modified dynamically, tantly, for the ability to respond to sudden power plant failures, within a safe range, in response to mains frequency fluctuations. which would otherwise lead to severe blackouts. Previous research has highlighted the potential of this idea for From an economic perspective, frequency response services responding to sudden power plant outages. However, deterministic and reserve power are costly and any method which manages control schemes have proved inadequate as individual appliances tend to “synchronize” with each other, leading to unacceptable to reduce the magnitude of these services, without sacrificing levels of overshoot in energy demand, when they “recover” their system stability, is of significant importance [16]. In recent steady-state operating cycles. In this paper we design decentral- years, research has been initiated on the possibility of using ized random controllers that are able to respond to sudden plant frequency responsive loads, commonly referred to as “dynamic outages and which avoid the instability phenomena associated demand control”, so as to reduce the amount of frequency with other feedback strategies. Stochasticity is used to achieve desynchronization of individual refrigerators while keeping response and reserve services that are required, potentially overall power consumption tightly regulated. leading to significant reductions in overall system costs [6]. The principal idea of these methods is to shift part of the Index Terms—Dynamic demand management, jump linear systems, load balancing, power system stability, refrigerators. regulation burden to the consumer side, by employing the use of intelligent domestic appliances that can defer their energy consumption in such a way so as not to apply excessive stress I. INTRODUCTION on the grid when the need arises. Such ideas have been ex- plored, for example, in [5], in the context of domestic air conditioning devices and [7] for water heater load control. There are YNAMIC demand management (also known as demand- two principal alternatives that present themselves for the con- D side management) is a very promising research direction trol task, which differ in their complexity and, therefore, im- for improving power system resilience [5], [6]. In a power grid, plementation costs: the loads can either react to changes in the the system frequency (mains frequency) is an indicator of the supply in a completely autonomous (i.e., decentralized) fashion, balance between demand (load) and supply (generation), with by (individually) monitoring the overall system frequency, or the nominal frequency of 50 Hz corresponding to perfect bal- they can respond to external requests made, for example, by ance between the two. When demand levels exceed the available the grid operator [18]. An intermediate solution is for the con- supply, the frequency drops below 50 Hz, while in the case of sumer appliances to respond to (dynamic) price information excess (with respect to system load) generation, the frequency originating from the utility company and decide on their op- rises above 50 Hz. As a result, system frequency continuously erating schedule accordingly; this approach, however, still re- fluctuates around the nominal level and the system operator en- quires the availability of a communications infrastructure, as sures that the balance between demand and supply is continu- does the method of externally supervised control which was dis- ously maintained, stabilizing the frequency within narrow bands cussed above. around 50 Hz, by regulating the available supply. In this paper, we consider the problem of managing power demand by means of “smart” thermostatic control of domestic re- Manuscript received November 25, 2010; accepted April 01, 2011. Manu- frigerators. In this approach, the operating temperature of these script received in final form April 06, 2011. Date of publication May 10, 2011; appliances and thus their energy consumption, is modified dy- date of current version April 11, 2012. Recommended by Associate Editor Z. namically, within a safe range, in response to mains frequency Wang. This work was supported by the EPSRC Grant “Control For Energy and Sustainability”, Grant reference EP/G066477/1. fluctuations. Previous research [2], [9], [16] has shown that this D. Angeli is with the Department of Electrical and Electronic Engineering, is an effective way to respond to sudden power plant outages, Imperial College, London SW7 2AZ, U.K., and also with the Department of reducing the cost of reserve power required to deal with such Systems and Information, University of Florence, 50139 Florence, Italy (e-mail: [email protected]). events. The feasibility of the approach for demand management P.-A. Kountouriotis is with the Department of Electrical and Electronic En- stems from the large number of domestic refrigerators that are gineering, Imperial College, London SW7 2AZ, U.K. (e-mail: pk201@imperial.ac.uk). 1Frequency response services are provided by synchronized generators, run- Color versions of one or more of the figures in this paper are available online ning only part-loaded (and hence not at maximum efficiency), as well as from at http://ieeexplore.ieee.org. industrial customers [2]. Reserve power is identified with slower, part-loaded Digital Object Identifier 10.1109/TCST.2011.2141994 plants and generation units that can start producing at short notice.

1063-6536/$26.00 © 2011 IEEE 582 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 20, NO. 3, MAY 2012 in use (around 40 million appliances are estimated to operate vices and so each device has to act in an autonomous setting. in the U.K. [2]). Similar control schemes can also be employed While this is a severe constraint and complicates the problem, for any other types of appliances, both domestic and industrial, we note that the quantity of interest is the temperature distri- that exhibit energy storage in the form of heat, such as freezers, bution of the whole population of appliances at each particular water heaters, etc. [16], greatly expanding the potential applica- time point. We therefore pose the problem in a probabilistic tions. framework, in which we try to find control schemes that steer In particular, [9] and [16] investigate the potential of de- the probability densities involved towards desired distributions. centralized dynamic demand control of domestic refrigerators, The advantage of this approach is that it greatly reduces the di- when the thermostat’s temperature thresholds (and, thus, the mensionality of the original problem, while it allows for simple, duty cycle of appliances) are varied as linear functions of yet successful solutions. mains frequency deviation from its nominal value, while they A viable control scheme in this setting, is the replacement also perform an assessment of the control method in scenarios of classical hysterisis-based controllers with controls that ran- with significant supply variability, due to power generated by domly jump between the “on” and “off” states of the appliances. wind turbines. In both cases, their results demonstrate that Careful selection of the jump propensities allows for the de- the amount of standing reserve required by the power system centralized control of individual appliances’ duty cycles (and, can be safely reduced. A similar approach is followed in [2], therefore, power consumption), while, during “recovery”, the where the economic impacts of such control strategies are also “population” of refrigerators is sufficiently diversified (mixed) quantified depending on the types of generation units in the with respect to temperature, thereby avoiding undesirable over- system (nuclear plants, coal plants, combined cycle gas turbine shoot phenomena. plants, etc.). The probabilistic description of the problem allows for the The problem of dynamic demand management of refriger- derivation of closed-form expressions for the first two moments ator appliances is also addressed in [18] and [19], in the context of the temperature distributions involved, in terms of the afore- of (centralized) model predictive control (MPC). In this case, mentioned jump propensities, so that the latter can be selected the appliances are assumed to be connected to a communica- in order to control these quantities of interest. The resulting tions network and are able to receive and execute commands that closed-loop system can be shown to exhibit properties of local are generated by a central processing node. The approach is ap- asymptotic stability, regardless of parameter values, as well as plied to problems in which there is considerable supply variation boundedness of solutions for all initial conditions. due to the significant employment of alternative energy sources The performance of the proposed controllers is assessed via (e.g., wind and photovoltaic). Non-anticipated events such as simulations, when coupled with a simple model of the power generator failures are not explicitly addressed. As expected, the grid. Initial results verify the theoretical underpinnings of our closed-loop behavior is far superior to that corresponding to the approach and clearly illustrate the robustness of the method simpler schemes of [16] and [9], but the prospects of immediate when compared to earlier approaches. utilization of such ideas are not enhanced, not least by the extra The rest of this paper is organized as follows. Section II costs that would be required for widespread implementation. elaborates the mathematical results enabling the proposed Simple decentralized feedback schemes, however, such as solution. A model of refrigerators is outlined in Section II-A, those employed in [16] and [9], in which the operating tempera- with the corresponding analysis shown in Section II-B. The in- ture is varied in a linear fashion with respect to mains frequency terconnected system and the random controller are described in deviations, can prove inadequate in achieving desired perfor- Section III, while Section IV proceeds with a stability analysis mance, as individual appliances tend to “synchronize” with each of the closed-loop system. Results of simulation studies are other, leading to unacceptable levels of overshoot in energy de- shown in Section V. Conclusions can be found in Section VI, mand, when they “recover” their steady-state operating temper- while the instability of deterministic control schemes is dis- atures. The appearance of such phenomena can be slow, but they cussed in the Appendix. do ultimately lead to unstable oscillations in the frequency of the overall system. II. STOCHASTIC APPROACH TO REFRIGERATOR CONTROL The synchronization phenomenon described above was, in fact, anticipated by [16] and was also recognized in [19] and [8], even in the case where a communications infrastructure between A. Refrigerator Modeling power utilities and domestic appliances is available. [8] pro- Refrigerator appliances operate in two states—they are either poses two desynchronization strategies, which are related to the ON or OFF. Transitions between these two states take place ac- centralized control schemes derived in [18] and [19] (in which cording to a hysteretic relay, driven by temperature. Given the the grid operator has the ability to send control signals to the state of an appliance, its temperature is typically assumed individual households) and reports on their success in removing to evolve according to a first-order, affine ordinary differential undesirable oscillations. This is, to the best of our knowledge, equation [3], [18] of the form the first contribution in which some form of randomization is identified as an effective strategy to counteract synchronization when ON phenomena. In contrast to [8] and [18], however, we adopt a framework in which there is no communication between the controlled de- when OFF (1) ANGELI AND KOUNTOURIOTIS: STOCHASTIC APPROACH TO “DYNAMIC-DEMAND” REFRIGERATOR CONTROL 583

Due to ergodicity of the underlying Markov Chain, the vector , for each given pair , converges to a unique stationary distribution , where

(4)

We note that also represents the average duty cycle of a Fig. 1. Markov chain illustration. single appliance, so that the average duty cycle of an appliance is entirely determined by the transition rates and . Due to the low-dimensionality of the Markov chain, explicit expressions In the above equations, the scalar parameters , , , and for the transient probability distributions can be computed as are used to denote the thermal insulation, the thermal mass, the coefficient of performance and the power rating of the device respectively. denotes the ambient temperature, which is assumed constant. For the purposes of deriving a random control algorithm, we In particular, we note that both of the above expressions are extend the model (1), so that refrigerators are, instead, modelled monotone functions of time. Therefore, if parameters and as Markov-jump linear systems [4], [13], or, to be more pre- undergo a step change (as will be the case later), the cor- cise, jump-affine systems. Roughly speaking, these are switched responding probabilities and will evolve monotoni- affine systems whose driving signal is the stochastic process as- cally to their new steady-state value. Since the total power ab- sociated with a finite Markov chain. This particular type of ran- sorption of a population of appliances is proportional to the frac- domization is chosen because it leads to simple population-level tion of them that are in the ON state, this monotonicity of equations, which can then be used for synthesis of controllers is a first important indication of the absence of undesired over- that achieve population-level specifications (this will be clari- shoots in power absorption. fied in later Sections). In particular, we consider Markov chains with two states only, B. Open-Loop Behavior: Analytical Results an OFF and an ON state and transition probability rates between them which are denoted by and , respectively.2 It is cus- As our goal is to regulate the overall behavior of a popula- tomary to graphically represent such systems as in Fig. 1. Let- tion of refrigerators, it is convenient to derive formulae that de- ting denote the temperature of a single appliance at time , scribe the time-evolution of the probability distribution of tem- its evolution in each of the two states is described by an affine, peratures of a single appliance and expressions for the associ- first-order ordinary differential equation, as ated first two moments. These formulae will be useful for the derivation of a control strategy in Section III. when ON To this end, let and denote the unnormal- when OFF (2) ized pdf of the temperature of a device in the ON and OFF state respectively, at time . In particular note that In (2), and denote, respectively, the ambient temperature and the steady-state temperature reached by a refrigerator which is always ON, while the positive coefficient can be re- garded as a thermal dispersion coefficient. These parameters are related to the model (1), by identifying so that

We also use and to denote the probability These (unnormalized) temperature distributions satisfy a form of a single refrigerator being in the ON and OFF state, respec- of Kolmogorov’s forward equation, such that tively. Obviously, for all times . Then, according to our notation, the equations governing the time evolution of these probabilities are given as (5)

(3) Even though (5) admit no closed-form solution, it is possible, due to the afﬁne nature of the underlying dynamics, to 2That is, we depart from the deterministic ON/OFF switching operation of appliances and consider devices that randomly switch between their two operating obtain ODEs that describe the evolution of the ﬁrst two mo- states. ments associated with these distributions, as well as asymptotic 584 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 20, NO. 3, MAY 2012

(steady-state) values for these moments. To this end, we deﬁne and as

(6)

(7) so that . Differentiating the previous formula with respect to time and using (5), we obtain differential Fig. 2. Schematic simpliﬁed model of the power grid. equations for the time evolution of these quantities as follows:

(8) Due to the cascaded structure of this system and linearity of (9) its diagonal terms, it is easy to see that the system is globally asymptotically convergent. It then follows that the expected temperature satisfies the differential equation III. THE INTERCONNECTED SYSTEM (10) In this section, we consider a population of refrigerators that Taking into account the steady-state values of and is connected to a power grid as shown schematically in Fig. 2. given in (4), we deduce that the expected value of converges As mentioned in Section II-A, standard refrigerator con- asymptotically to trollers operate on a hysteretic basis, in which two temperature levels and trigger the motor ON and OFF, respectively. Initial attempts at dynamic demand refrigerator control [2], [16] focused on dynamically adjusting these threshold (11) levels, by imposing a linear dependence on mains frequency deviations ,as Now consider the variance of . This is given as (15) where is a constant of proportionality. Even though such strategies are effective in the short-term (i.e., immediately after a plant failure occurs), evidence is provided in the Appendix and Simulation Sections that they eventually lead to unstable overall behavior of the closed-loop system. Differentiating the above with respect to and using (5), yields This phenomenon takes the following two different forms: • long-term phase synchronization of refrigerators: even nonidentical refrigerators, with duty cycles of comparable (12) duration, will tend to asymptotically synchronize their oscillations, giving rise to the so called phase-locking from which we deduce the asymptotic value for the variance, , phenomenon (see, for instance, [14]); as • uncontrolled modifications of the population’s temperature distribution: the uniform-in-phase distribution (which (13) one expects of a population of devices that were initially switched on at random times) gets unpredictably modified Overall, we note that the first and second moments of the by the occurrence of frequency (or load) disturbances [via population of appliances are governed by the following block- (15)] and leads to significant oscillations in power demand triangular set of ODEs: even in the medium term. In the rest of this section, we describe a set of alternative control strategies that avoid such instabilities.

A. Random Control Strategy Noting that the transition rates and , deﬁned in Section II, determine both the average temperature and the average power consumption of a collection of appliances, we propose that fully decentralized control of the individual appli- (14) ances can be achieved by (each appliance locally) selecting the ANGELI AND KOUNTOURIOTIS: STOCHASTIC APPROACH TO “DYNAMIC-DEMAND” REFRIGERATOR CONTROL 585 transition rates and as functions of the grid frequency 2) Start evaluating the integral , deviation . where is computed from (16), (20), and (22). In what follows, it is constructive to deﬁne the control vari- 3) Switch to OFF at time , for which . ables and as • When device switches to OFF mode: 1) Set and . (16) 2) Start evaluating the integral , where is computed from (16), (20), and (22). Now consider (11) and (13). In terms of the new variables, 3) Switch to OFF at time , for which . these can be rewritten as In the above, RND denotes a random number, uniformly distributed in the interval . Note that the scheme is computa- tionally simple, in that it only involves a random number gener- (17) ator and a standard quadrature routine.

(18) B. Algorithm Variations Based on the random control strategy of Section III-A, several Note also that (4), which determines the duty cycle of each “hybrid” algorithms can be constructed. We describe two such appliance as a function of the transition rates, can be rewritten alternatives below. as 1) “Constrained” Random Algorithm: Even though the random controller of the previous Section regulates the mean (19) and variance of a refrigerator’s temperature, it might be desirable in practical applications to introduce safety thresholds The idea is, then, to ﬁx a desired value for the variance in op- and , which would serve to prohibit temperature ex- erating temperatures, say , in (18) and determine the transi- cursions outside “safe” levels. This leads to a random controller tion rates and (or, equivalently, and ) by postulating with temperature constraints, according to which, if any of the a desired average temperature, , or a desired average duty or thresholds is exceeded, the appliance forcibly cycle, . In particular, if the latter choice is adopted, the fol- switches ON (or, respectively, OFF), overriding the random lowing expressions are obtained by inverting the previous for- control action. mulas: 2) Random Algorithm With Variable Constraints: Initial simulations indicated that, even though the proposed random controller performs overall better than simpler linear feedback schemes, the latter tend to respond faster at the onset of a failure, where the initial frequency drop is very sharp (see, for (20) example, Fig. 8 for an illustration). The ability of an algorithm to respond fast to a sudden plant outage is highly desirable, as the initial frequency drop has to be contained as soon as Our decentralized control strategy is, therefore, to vary either possible. It is possible to modify the constrained version of or as linear functions of the frequency deviation the algorithm and provide it with faster response capabilities, by introducing a lower temperature safety level, , that is (21) frequency dependent as follows: (22) where and are proportionality constants and and are the nominal (target) values of the average temperature In the above, is a (positive) constant of proportionality. and the (corresponding) duty cycle when there is no frequency The implementation of these algorithms differs only slightly deviation in the grid . from the basic version outlined in the previous Section—for ex- In the simulations shown in Section V, was chosen as ample, when the device is ON, Step 3 would have to be replaced the reference variable, as it led to faster responses. by the following statement. The control scheme described above results in a time-inho- 1) Switch to OFF at time , where is such that mogeneous Markov chain for the (ON/OFF) switching schedule and is such that ( in of each appliance, with rate functions and . These the case of the non-variable constrained algorithm). time varying functions and can be computed as functions of the instantaneous mains frequency , simply by com- IV. STABILITY ANALYSIS posing (20) and (22). The implementation of an inhomogeneous Markov chain is relatively simple [1], [15], leading to the fol- We now proceed to the stability analysis of a large population lowing algorithm (to be run on each individual appliance). of refrigerators, governed by the random algorithm described in • When device switches to ON mode: Section III-A, when connected to a power supply network. Our 1) Set and . main result in this respect is stated as follows. 586 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 20, NO. 3, MAY 2012

Claim: Assuming that frequency reacts to load variations as a passive operator,3 the interconnection of a large population of identical refrigerators, regulated according to the random algorithm described above, yields a locally asymptotically stable closed-loop system, regardless of parameters values and control gains. To substantiate our claim, we ﬁrst derive a closed-loop mathematical model of this large scale system. To this end, it is useful to recall that (5) carry both a stochastic and a deterministic interpretation. From the stochastic point of view, they describe the evolution of probability distributions of a continuous-time random process (a single, randomly-regulated refrigerator). From the deterministic point of view, the quantity can be seen as the fraction of refrigerators in Fig. 3. Closed-loop system. state that, at any given time , have temperature in the interval , provided that there is a large population of identical refrigerators, each of which updates its state according to the The nonlinear switching propensities and are designed previously illustrated algorithm. It is worth pointing out that such that (5) remain valid also if and are time-varying inputs. The deterministic interpretation described above, allows us unique to derive the closed-loop system simply by juxtaposition of (5) with the power-network model Moreover, for all in , the following inequality holds:

(23) which implies global asymptotic stability of the equilibrium , for the constant input . where is used to denote the transfer function from varia- Linearization around such an equilibrium yields the equations tions of power supply to the frequency . The input is used of a first-order, asymptotically stable linear system, of positive to denote the nominal supply to the system, while represents DC gain. This is a strictly passive system (see for instance [12]). the load of the refrigerators. Notice that the load resulting from Due to passivity of and the passivity theorem, the equilib- the refrigerators is proportional (through the gain ) to the frac- rium point of (25) is locally asymptotically stable, regardless of tion of fridges which are in the ON state . parameters values. The same is also true for the equations in- While (23) are infinite-dimensional, their momenta evolve ac- volving higher-order momenta, due to their cascaded structure. cording to a finite-dimensional nonlinear system of equations. While global asymptotic stability of the closed-loop system In particular appears difficult to prove, boundedness of (always in ) and BIBO stability of allow us to conclude boundedness of solutions of the closed-loop system (25), for all initial conditions. By a standard analysis based on input-to-state stability of cascades (see for instance [17]), it is possible to show bounded- (24) ness of first and second momenta of for all initial conditions. Notice that the overall system exhibits a cascaded structure, with the - feedback loop forcing the remaining variables , V. S IMULATION RESULTS and (see Fig. 3). We may exploit the conservation law In this section, we present preliminary results on the perfor- in order to write the subsystem as a scalar nonlinear system mance of the random controller and its variations, outlined in Sections III-A and III-B and compare with the (decentralized) deterministic controller of [16], when these algorithms are em- (25) ployed to control a population of 40 million refrigerators (90% of which is assumed to be of the “dynamic demand” type), con- It is convenient to introduce the function nected to a power supply network as shown in Section III. For the purposes of the simulations, we use a simple third-order linear system [10] with a state vector to represent the power grid as 3The passivity assumption is valid for simple models, such as the one employed in Section V and can also be expected to hold for more elaborate models ([11], [20]). (26) ANGELI AND KOUNTOURIOTIS: STOCHASTIC APPROACH TO “DYNAMIC-DEMAND” REFRIGERATOR CONTROL 587

TABLE II CONTROLLER PARAMETER VALUES

Fig. 4. Model of power grid.

TABLE I GRID PARAMETER VALUES

Fig. 6. Power loss.

(consumed by all the appliances that are connected to the network) from their nominal consumption level. The additional variable is used so as to simulate a sudden loss of power in the system4. Fig. 5. Fridge cycle. The nominal parameters in the refrigerator model (2), namely , , and , were set to , 38.3 and 20, respectively, corresponding to the cycle shown in where Fig. 5 and a nominal duty cycle of approximately 25%. Both “dynamic demand” and “non-dynamic demand” (i.e., conven- tional) appliances were simulated, with 1000 appliances in each set, while the total power consumption of the two sets was scaled up so as to approximate the total of 40 million appliances. The state and temperature of each simulated refrigerator were randomly initialized, while model parameters for each appliance were also randomly chosen from a uniform window around the nominal values previously stated. The parameters (controller gains, etc.) employed in the various control algorithms are collected5 in Table II. corresponding to the block diagram shown in Fig. 4. (Notice The performance of the various algorithms was assessed in that (26) is expressed in a “per-unit” basis [2], so that the case of a sudden loss of 1.32 GW of power from the system, ). which was imposed by introducing a step increase in in (26). The parameters of the model (26) were set to the values used The duration of the loss was 15 min, after which the power re- in [2] and are given in Table I. Two different scenarios were con- covered to the original levels in a ramp fashion, with the re- sidered, differing in the amount of total power being supplied by covery period lasting 10 min (see Fig. 6). the network, so as to highlight the effects of the refrigerators in 4Note that a sudden power loss is equivalent to a sudden increase in demand, the closed-loop system. of the same magnitude. The appliances load the grid at the summing junction via the 5To ensure H % I, a ﬂoor and ceiling value were introduced when variable , which represents the deviation in overall power calculating % according to (22) in the random algorithms. 588 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 20, NO. 3, MAY 2012

Fig. 7. Scenario I and II results. (a) Scenario I @ a 55 GWA (b) Scenario II @ a 25 GWA.

Fig. 7(a) and (b) show the system frequency deviation , deterministic control method of [16]; the closed loop is stable, the overall power consumed by refrigerators and the average while it also has the desired transient properties (see Fig. 8(a) temperature across the appliances as functions of time, for the and (b) for a magniﬁed version of the transient response). two scenarios considered. The results demonstrate the superi- The instability phenomena associated with the deterministic ority of the proposed control algorithm when compared to the method (and which are elaborated upon in the appendix) are ANGELI AND KOUNTOURIOTIS: STOCHASTIC APPROACH TO “DYNAMIC-DEMAND” REFRIGERATOR CONTROL 589

Fig. 8. Scenario I and II results (magnified). (a) Scenario I @ a 55 GWA. (b) Scenario II @ a 25 GWA. highlighted in Fig. 7, especially in cases where the total power smaller ) than the deterministic algorithm, without leading consumed by the ’adaptive’ refrigerators constitutes a signifi- to undesirable overshoot in power consumption during the re- cant fraction of the overall power demand in the system [Sce- covery phase, as is the case for the latter. A tradeoff is identi- nario II-Fig. 7(b)]. The random strategy does not suffer from fied between the absence of overshoot of the consumed power such drawbacks. and the time required for the average temperature to recover its In particular, the random controller (in all its variations) man- steady-state value, which is (significantly) longer for the random ages to stabilize the system’s frequency at higher levels (i.e., controller. The proposed scheme does not allow for the control 590 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 20, NO. 3, MAY 2012 of the recovery time, as the time constant in expression (10) for is equal to , which is a device constant and cannot be altered. The “variable constrained” algorithm described in Section III-B2 responds as fast as the deterministic algorithm, while the introduction of “safety” temperature thresholds does not adversely affect the closed-loop performance.

VI. CONCLUSION A new algorithm for dynamic-demand control of refrigerator appliances has been presented and theoretically justified. The proposed algorithm adopts a probabilistic description of the problem, resulting in a relatively simple control scheme. Appli- cation of this control strategy ensures sufficient diversification Fig. 9. Relay nonlinearity. (mixing) of the temperature across the controlled appliances and does not lead to overshoot or instability phenomena asso- Indeed, assuming , we have, for ciated with simpler deterministic schemes. The control scheme can also be applied to other kinds of devices that exhibit energy storage in the form of heat, such as freezers, water heaters, etc. and, for Similar ideas may also be exploited in other areas where the issue of node “synchronization” potentially leads to problem- atic behavior, such as in internet congestion studies. Initial simulation results verify the theoretical underpinning When a population of identical refrigerators is operated in par- of the proposed approach. The random controller is capable of allel, the power absorption of the devices asymptotically tends maintaining the power system’s frequency for a longer period of to a periodic function of period . Moreover, if the same popula- time, when compared to the deterministic scheme and results in tion is connected to a model of a power network, it will induce a faster recovery at the end. Contrary to deterministic feedback, periodic fluctuation of mains frequency of the same period. For which breaks down in cases where the total average power con- a uniform initial distribution of phases, however, the number sumed by refrigerators is large relative to the overall system de- of refrigerators in the ON state is approximately constant (and mand, the random controller was shown to perform robustly. exactly constant in the limiting case). This implies that even a Future work will focus on testing the approach in several sce- large population of fridges uniformly distributed with respect to narios of practical importance in the power systems industry, phase, only induces a negligible fluctuation of mains frequency. involving more complex models of power networks (consisting We now analyze the effect of periodically forcing (27), by of several different generators), while an assessment of the eco- defining the hysteresis threshold values and to be nomic impact of the proposed method on the U.K. power net- periodic functions of time. We assume the following, for all work will also be undertaken.

APPENDIX PHASE-LOCKING IN DETERMINISTICALLY (29) REGULATED APPLIANCES for some positive number . In addition, we may deﬁne The standard approach to temperature control in refrigerators is to set the ON/OFF state of the motor by means of a relay system. In this case, the equations governing the temperature evolution (30) read as Under these assumptions and with the notation introduced (27) above, rough lower and upper-bounds to the duration of temperature oscillations for a forced refrigerator may be derived as where is the hysteresis nonlinearity shown in Fig. 9. follows: If and are held constant and , then the temperature converges in ﬁnite time to a periodic solution which has maximum and minimum at and , respectively. This asymptotic solution has a period which can be explicitly computed by solving the following two equations:

(28) ANGELI AND KOUNTOURIOTIS: STOCHASTIC APPROACH TO “DYNAMIC-DEMAND” REFRIGERATOR CONTROL 591

Tighter bounds can instead be computed, by solving

subject to

(31) and

subject to

(32) Fig. 10. Stable and unstable fixed points of (39). It is useful for the subsequent analysis to keep track of the th maximum (and possibly minimum) point of (that is, since the initial time ) by defining the variables and Notice also that the functions , with to be the time instants at which maximum and minimum and , fulfill the following property: temperature values are reached for the th time, respectively. In particular, there exists a function , such that (36)

Similarly, for the inverse functions we have

Notice that, due to continuity of and with respect (37) to and continuity of solutions, is also continuous. To obtain a more explicit expression for , we may deﬁne the following Exploiting the properties in (36) and (37), we rewrite the recur- functions: sion (35) in terms of the new variable , which is deﬁned as

(38)

The variable keeps track of the phase difference (expressed in units of time) between temperature peaks and the forcing periodic signals and . Straightforward manipu- lations show that, letting (33)

Now assume that . By explicit integration of (27) we obtain leads to the following equation:

(34)

Notice that, whenever

Three distinct scenarios are possible for the recursion the functions and are monotone with respect to , (39) so that their inverse is well defined and one may define the recursion 1) The first possibility is that for all . In this case, is an increasing sequence (with positive average slope). This implies that temperature oscillations (35) are slower than those of the forcing signal. 592 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 20, NO. 3, MAY 2012

2) The second possibility is that for all . [7] G. C. Heffner, C. A. Goldman, and M. M. Moezzi, “Innovative ap- In this case, is a decreasing sequence (with negative proaches to verifying demand response of water heater load control,” IEEE Trans. Power Del., vol. 21, no. 1, pp. 388–397, Jan. 2006. average slope) and temperature oscillations are faster than [8] C. Hinrichs, U. Vogel, and M. Sonnenschein, “Modelling and evalua- those of the forcing signal. tion of desynchronization strategies for controllable cooling devices,” 3) The last possibility is that there exist fixed points of the map in Proc. 6th Vienna Int. Conf. Math. Model., 2009, pp. 220–225. [9] D. G. Infield, J. Short, C. Horne, and L. L. Freris, “Potential for do- (in particular if is a fixed point than also mestic dynamic demand-side management in the UK,” in Proc. IEEE is a fixed point for all ). This happens whenever Power Eng. Soc. General Meet., 2007, pp. 1–6. [10] D. I. Jones, “Dynamic system parameters for the national grid,” IEE Proc. Generation Transmission Distrib., vol. 152, no. 1, pp. 53–60, Jan. (40) 2005. [11] A. Khodabakhshian, “Enhancement of power system performance by The stability of fixed points can be easily determined, by LFC analysis of hydro power plants using QFT,” Euro. Trans. Elect. Power, vol. 19, pp. 323–338, 2009. checking the relative slopes of the map with the [12] R. Lozano, B. Brogliato, O. Egeland, and B. Maschke, Dissipative Sys- diagonal line at intersection points (see Fig. 10). tems Analysis and Control. London, U.K.: Springer-Verlag, 2000. If the function crosses the diagonal only twice [13] M. Mariton, Jump Linear Systems in Automatic Control. New York: Marcel Decker, 1990. on each interval of length , then only two fixed points [14] A. Pikovsky, M. Rosenblum, and J. Kurths, Synchronization: A Uni- will exist (modulo equivalence ), the one being al- versal Concept in Nonlinear Sciences. Cambridge, U.K.: Cambridge most globally asymptotically stable and the other one being Univ. Press, 2003. [15] R. Y. Rubinstein and D. P. Kroese, Simulation and the Monte Carlo unstable. From a physical point of view, this situation cor- method. New York: Wiley, 2008. responds to a scenario in which phase synchronization oc- [16] J. A. Short, D. G. Infield, and L. L. Freris, “Stabilization of grid fre- curs and temperature oscillations take place at the same quency through dynamic demand control,” IEEE Trans. Power Syst., vol. 22, no. 3, pp. 1284–1293, Aug. 2007. frequency as that of the forcing signal. [17] E. D. Sontag, “Input to state stability: Basic concepts and results,” in Notice that, in a closed-loop scenario in which and Nonlinear and Optimal Control Theory, P. Nistri and G. Stefani, Eds. are determined as a function of the global network Berlin, Germany: Springer-Verlag, 2007, pp. 163–220. [18] M. Stadler, W. Krause, M. Sonnenschein, and U. Vogel, “Modelling frequency and frequency fluctuations are, in turn, determined by and evaluation of control schemes for enhancing load shift of electricity variations in the absorbed power, which is periodic with period demand for cooling devices,” Elsevier Environmental Model. Softw., , we necessarily have . That is, scenarios 1 and 2 are not vol. 24, pp. 285–295, 2009. [19] M. Stadler, W. Krause, M. Sonnenschein, and U. Vogel, “The Adap- possible in a periodic regime in which deterministic hysteretic tive Fridge—comparing different control schemes for enhancing load regulation is adopted. The above analysis suggests that the pres- shifting of electricity demand,” in Proc. 21st Conf. Inform. for Envi- ence of small periodic ripples in power system frequency will ronmental Protection (Enviroinfo), 2007, pp. 199–206. [20] A. M. Stankovic, G. Tadmor, and T. A. Sakharuk, “On robust control gradually entrain oscillations of refrigerators that have similar analysis and design for load frequency regulation,” IEEE Trans. Power frequencies of oscillation, thus reinforcing the frequency ripple Syst., vol. 13, no. 2, pp. 449–455, May 1998. and eventually leading to an even larger number of entrained David Angeli (SM’08) was born in Siena, Italy, in refrigerators. Simulations do indeed confirm the intrinsic risks 1971. He received the B.S. degree in computer sci- of such a regulation approach, as catastrophic oscillations can ence engineering and the Ph.D. in control theory from eventually develop in the network. University of Florence, Florence, Italy, in 1996 and 2000, respectively. He was an Assistant and Associate Professor with the Department of Systems and Computer Science, REFERENCES University of Florence. He was a visiting Professor with I.N.R.I.A de Rocquencourt, Paris, France, [1] S. Asmussen and P. W. Glynn, Stochastic Simulation: Algorithms and in 2007. Since 2008, he is a Senior Lecturer in Analysis. New York: Springer, 2007. control systems within the Department of Electrical [2] M. Aunedi, J. E. O. Calderon, V. Silva, P. Mitcheson, and G. Strbac, and Electronic Engineering, Imperial College, London, U.K. His research “Economic and environmental impact of dynamic demand,” Centre interests include stability of nonlinear systems, dynamical systems, control of for Sustainable Electricity and Distributed Generation, 2008. [Online]. constrained systems, and chemical reaction networks. Available: http://www.supergen-networks.org.uk/filebyid/50/file.pdf [3] P. Constantopoulos, F. C. Schweppe, and R. C. Larson, “ESTIA: A real-time consumer control scheme for space conditioning usage under spot electricity pricing,” Comput. Oper. Res., vol. 18, no. 8, pp. 751–765, 1991. Panagiotis-Aristidis Kountouriotis received the [4] O. L. do Valle Costa, M. D. Fragoso, and R. P. Marques, Discrete-Time M.Eng. degree (First Class) in electrical engineering Markov Jump Linear Systems. London, U.K.: Springer-Verlag, 2005. and the Ph.D. degree from Imperial College, London, [5] J. H. Eto, J. Nelson-Hoffman, C. Torres, S. Hirth, B. Yinger, J. Kueck, U.K., in 2005 and 2009, respectively. B. Kirby, C. Bernier, R. Wright, A. Barat, and D. S. Watson, “Demand He is a Research Associate with the Control and Response Spinning Reserve Demonstration,” Energy Analysis Dept., Power Group, Department of Electrical and Elec- Ernest Orlando Lawrence Berkeley Nat. Lab., Berkeley, CA, LBNL- tronic Engineering, Imperial College. His research 62761, 2007. interests focus on theoretical and computational [6] C. Gellings and J. Chamberlin, Demand-Side Management: Concepts aspects of stochastic systems and control. and Methods. Lilburn, GA: The Fairmont Press, 1988.