Toward a Sustainable Modern Electricity Grid: The Effects of Smart Metering and Program Investments on Demand-Side Management Performance in the US Electricity Sector 2009-2012

Jacqueline Corbett Université Laval

Kathrine Wardle Canadian Tire Corp Ltd

Chialin Chen National Taiwan University [email protected]

Published: IEEE Transactions on Engineering Management

To cite please consult published version: Corbett, J., Wardle, K. & Chen, C. (2018). Toward a Sustainable Modern Electricity Grid: The Effects of Smart Metering and Program Investments on Demand-Side Management Performance in the US Electricity Sector 2009-2012. IEEE Transactions on Engineering Management, 65(2), 252-263.

Acknowledgement: This research was funded in part by Natural Sciences and Engineering Research Council (NSERC) of Canada, Grant RGPGP/328942-2013

1

Toward a Sustainable Modern Electricity Grid: The Effects of Smart Metering and Program Investments on Demand-Side Management Performance in the US Electricity Sector 2009-2012

Abstract Building a sustainable modern electricity sector is a key part of the United Nations’ 2030 Sustainable Development Goals. Smart meters are expected to contribute to the achievement of this goal by providing new demand-side management opportunities for utilities. Attempting to address existing gaps in the extant literature, this paper examines the effects of metering technologies and demand-side management program investments using panel data from 87 American electricity utilities for the period 2009-2012. Our model provides strong explanatory power with respect to energy efficiency, and shows that automated meter reading (AMR) devices, advanced metering infrastructure (AMI) devices, direct program costs and incentive costs all have a positive influence on energy efficiency effects. In contrast, the results for are largely insignificant, with the exception of incentive costs which has a small, but significant, negative impact on peak load reductions. We discuss the implications and potential research directions arising from these findings, in particular the need to consider interactions between different demand-side programs and the influence of different types of information technologies that are emerging as part of the .

Managerial Relevance Statement Panel data analysis is performed to investigate the impacts of smart metering and program financial investments on energy demand management by the US electricity sector. Unlike most existing studies on smart grid implementation which utilize cross-sectional data, the use of panel data in this research allow for the identification of impacts of the smart grid technology by observing the performances of the same utilities over time. Decision makers can then input their specific characteristics, such as numbers of different meters types and their level of investment, to understand and determine what benefits could be expected given a certain level of technology implementation or financial investment. Our research also leads to a number of analytical results which are different from those in the existing literature. In particular, we show that automated meter reading devices, advanced metering infrastructure devices, direct program costs and incentive costs all have a positive influence on energy efficiency effects, but not on load reductions. We also discuss the importance for decision makers to consider the interactions between different types of demand-side initiatives in conjunction with transformations occurring on the supply-side as well as the continuing innovation in smart grid technologies in use by intermediary organizations and consumers.

Key Words: Smart Grid, Green IT, Management, Demand-Side Management, Sustainability

2

1. Introduction

Much like clean air to breath and water to drink, energy has become an essential resource of our modern society, so much so that the United Nations identified “access to affordable, reliable, sustainable and modern energy for all” as one of the 17 Sustainable Development Goals (SDGs) for 2030 [1]. Electricity is one of the most important energy types and it continues to be the fast- growing form of end-use . Global demand for electricity in the residential sector is projected to increase 1.4% annually to 2040 [2]. Beyond simply meeting increasing demands, the electricity sector has much work to do in order to achieve the SDGs, particularly in relation to sustainability and climate change. In 2014, 30% of the ’ came from the electricity sector as about 67% of its electricity production was generated primarily from fossil fuels, such as and [3]. Attaining the goal of sustainable electricity depends in part on the achievement a modernized electricity sector, or

‘smart’ grid. The smart grid is the next generation electricity grid that “enables bidirectional flows of energy and uses two-way communication and control capabilities that will lead to an array of new functionalities and applications” [4]. It is estimated that the electricity industry spent $18 billion on deploying smart grid technology in the United States from 2010 to 2013 [5], and the development of the smart grid has raised expectations that the transformation needed to achieve national energy security goals in conjunction with global sustainability goals is possible.

Smart grid strategies to improve the sustainability of the electricity sector can target both supply-side (e.g., generation) and demand-side (e.g., consumption) activities, thus allowing the smart grid to play a significant role in a low-carbon future [6]. In regards to the former, the smart grid can support the integration of and more variable energy sources, such as wind and solar, while maintaining system reliability. In the case of the latter, information and

3 communications technologies associated with the smart grid can facilitate changes in energy consumption. Recent developments of the smart grid allow demand-side management (DSM) techniques to be implemented more effectively and permit the use of new strategies [7]. These techniques encourage the reduction of the total electricity demand and contribute to smoothing peak load curves, both of which will contribute to lower carbon emissions [8].

Among the different information and communications technologies involved in bringing the smart grid to fruition, smart meters have received a significant amount of attention [e.g., 9,

10-12] as they represent one of the first and most tangible signs of the smart grid. There were

58.5 million smart meters in place in the United States in 2014 [13], and globally, it is estimated that 800 million smart meters will be installed by 2020 [14], making them an important preoccupation for the industry. The relationship between smart meters and improved DSM is complex and multifaceted. In general, the goal is to use smart meters to reduce electricity consumption at an overall level, as well as during critical peak load periods [15], without significantly reducing the service experienced by customers. To these ends, utilities may use a range of DSM strategies, from conservation and energy efficiency programs to more active initiatives. Smart meters are important enabling devices particularly when integrated into broader energy efficiency or DSM programs and policies [16, 17]. For example, smart meters can support utilities’ efforts by providing the technological integration, two-way communications, and analytical and automated decision-making, so-called ‘technology solutions’[15]. In addition, smart meters can also provide more detailed information to consumers regarding their consumption of electricity and allow for more informed decisions.

These latter functionalities have been referred to as ‘technology-enabled behavioral solutions’

[15]. With technology-enabled behavioral solutions, the influence of smart meters complements

4 and interacts with other DSM programs put in place by utilities such as education, financial incentives and variable pricing regimes.

Despite the perceived benefits of smart meters with respect to DSM, research findings to date are inconclusive. A study based on cross-sectional data from U.S. utilities reveals the presence of both positive and negative relationships between different metering technologies and energy efficiency effects [9]. Lang [15] suggests that technology solutions may provide the largest reductions in peak load and overall consumption because consumers only have one decision to make (e.g., whether or not to participate), rather than having to engage in a series of decisions over time; however, behavioral reactions to technology solutions may offset some of the benefits

[15]. Further, Dyson et al. [10] suggest that due to significant customer heterogeneity, demand response programs should be targeted rather than implemented on mass to maximize cost effectiveness. Given this uncertainty and the importance of building a sustainable, modern electricity sector and the equivocality in the extant literature, additional research is warranted.

The goal of this paper is to contribute to the continued development of the smart grid by examining the impact of smart metering technologies on utilities’ DSM performance, in relation to both energy efficiency and load management programs. We use panel data econometrics to perform this analysis over the years 2009 to 2012. As compared to cross-sectional analyses, panel data allows for the identification of impacts that may arise over time as the technology is adopted and appropriated by utilities and their customers. In addition, our analyses, which also take into account utilities’ investments in DSM programs, provide a more comprehensive view of the relationships between smart grid technologies, DSM program characteristics and energy savings. For instance, our analytical results will allow a utility to determine the amount of energy efficiency savings that may be achieved by investing an additional dollar into energy efficiency

5 programs, all other factors being equal. As new technologies become available, it will be essential for utilities to understand these relationships so that they can effectively design and tailor their DSM programs to achieve specific performance goals. Our findings also contribute to academia by highlighting areas in which the benefits of smart grid technologies are present and where they remain to be developed. In this respect, our work provides direction to engineering and information systems scholars who will work collaboratively with the electricity sector to achieve of an affordable, reliable, sustainable and modern energy system for all.

This paper is structured as follows. Section 2 reviews the relevant literature as a foundation for the study. Section 3 presents the research methodology. Section 4 describes the results of the study. Section 5 discusses the findings and their implications. To conclude, Section presents the contributions, limitations and future research directions arising from this work.

2. Literature Review

2.1 Overview of Demand-Side Management in Electricity Sector

Demand-side management is a collection of strategies and techniques aimed at altering

“customer’s electricity consumption patterns to produce the desired changes in the load shapes of power distribution systems” [18, p. 1245]. Passive DSM involves attempts to permanently reduce the overall demand for electricity, without considering short-term fluctuations in demand, while active DSM techniques are more proactive as they seek to alter short-term demand in anticipation of changes in peak loads. Passive DSM strategies typically focus on conservation and energy efficiency. These strategies encourage consumers to reduce their total electricity consumption through education, or incentives such as tax rebates. Consumers may also be encouraged to to more energy efficient appliances (e.g., refrigerators, furnaces) offering the same services while demanding less electricity. Conservation and energy efficiency programs

6 have been used for decades by utilities and are relatively inexpensive, with reports suggesting that the cost of energy efficiency programs is around $0.03/kWh [19].

Alternatively, innovations in the smart grid are permitting more active DSM approaches, referred to as load management, or demand response programs. Demand response implies short- term changes in the pattern of end-use electricity consumption in response to grid conditions

[10]. Demand response, such as controlling use of air conditioning equipment, can provide significant flexibility to the grid by reducing demand during periods and shifting it to another time [10]. In this type of program, a utility monitors aggregate electricity demand using real-time data from automated meters and determines when demand is reaching the limits of their supply. Then, a signal is sent to consumers to request a reduction in consumption, such as by shutting off equipment (e.g., appliances) or postponing their use to a lower demand (off- peak) period. Consumers may grant utilities permission to control systems or appliances (e.g., hot water tanks, pool pumps) directly, or they may decide on a case-by-case basis whether to respond to a particular request by their utility. Utilities may encourage their consumers to participate by implementing time of use (or real-time use) pricing or other financial incentives.

Active DSM strategies, such as load management, rely more strongly on information technologies because effectively balancing supply and demands requires not only finer grain and real-time data, but also some form of automation [20].

Although energy efficiency and demand response approaches represent two distinct demand-side management strategies, research suggests they are complementary [21] and could offer utilities additional benefits when used in conjunction with each other. For example, although most demand response programs are not designed to focus on total energy consumption, many of these programs also deliver at least small energy conservations effects

7

[22]. These effects occur in part due to the feedback mechanisms associated with demand response programs: with proactive strategies, customers receive additional direct usage information which increases their energy awareness and leads to permanent changes and reductions in energy consumption [22]. Passive and active strategies can be mutually reinforcing: on one hand, energy efficiency improvements help to reduce demand, including at peak times and provide more flexibility for load shifting strategies [20]; alternatively, load shifting can help to reduce overall demand [20] because some consumption activities targeted by load management are not simply shifted, but eliminated entirely. Despite the complementary nature of energy efficiency and load management DSM strategies, program-level coordination of passive and active DSM strategies is still not prevalent [23].

One potential reason for the apparent lack of coordination may relate to the different motivating factors with respect to each type of program. For utilities, the adoption of energy efficiency is influenced more by regulatory obligations and while the adoption of demand response is driven more by capacity needs and market conditions [21]. In addition, there exists a large number of different and innovative options with respect to enabled services and measurement, not all of which are applicable for all types of consumers [10, 16]. For example, despite the potential of more active and automated DSM programs there remains significant resistance and low adoption from residential consumers with respect to direct load control practices [17, 20]. However, adoption of automated approaches could be improved when appropriate complementary information and services are put in place [16, 17] and consumers perceive that such approaches reflect an appropriate balance between passive and active , provide environmental benefits to the greater society, and present opportunities for personal financial advantages [12]. Further, by coupling energy efficiency approaches with

8 demand response, it could be possible to increase adoption of demand response by reducing customer discomfort and inconvenience, and by increasing the social acceptability of either manual or automated load-shifting [20].

2.2 Demand-Side Management and Smart Grid Technologies

Investment in smart grid technology has increased significantly over the past decade. In 2009, a

$9 billion public-private investment in smart grid projects was committed under the American

Recovery and Reinvestment Act. This increased investment allowed utilities to initiate a variety of projects and also stimulated the interest of researchers, seeking to understand the implications of smart grid technologies on utilities’ DSM activities. Initial results suggest certain utilities have achieved 25-37% reductions in peak demand from their pilot projects under this program [24].

Further analytical studies have sought to quantify the potential of the smart grid or to determine models for optimizing energy savings. For example, Logenthiran et al. [18] propose a demand- side management strategy, which uses a load shifting model formulated as a minimization problem. A heuristic-based algorithm is used to solve this problem, which allows for the management of a smart grid with a large number of different types of devices.

Other studies have approached the subject from the customer perspective. Khan et al.

[25] develop a generic model for residential customers to use demand-side management while reducing their total energy costs and appliance waiting times. Conejo et al. [26] present an optimization model that adjusts the hourly load of residential customers in response to hourly electricity prices in order to maximize the utility to the consumer, while Saravanan [27] proposes an algorithm that considers interactions between the utility and consumers with the objectives of reducing the consumer costs and the average peak demand. Despite the potential for automated routines and algorithms, Zheng et al. [28], through an agent-based simulation approach, suggest

9 that to achieve greater energy efficiency and cost savings, customers should become more experienced smart meter users through behavioral learning, especially in the case when the use of such technologies are mandated.

While the extant research approaches the question of DSM from different perspectives, a few common points can be retained. First, the research suggests there is a significant potential to use smart grid technologies to improve DSM. Second, it appears that there is some variation in the potential impacts: there is no ‘one size fits all’ solution. Instead, utilities must be somewhat discriminatory in their deployment of smart meters in order to maximize value. Third, the proposed models largely focus on one category of DSM - demand response - rather than the full range of DSM approaches.

In contrast to the range of analytical research from engineering and technology researchers, we find fewer empirical studies examining the impact of financial and technological investments on utilities’ DSM performance. Among the studies, Loughran and Kulick’s [29] report on a panel data study of the effects of DSM expenditures and number of customers on energy efficiency, using retail sales as a proxy measure for energy efficiency. They find that from 1989 to 1999, DSM expenditures reduced electricity sales, which suggests that DSM investments can lead to increased energy efficiency savings [29]. A similar study by Rivers and

Jaccard [30] concludes, not without controversy, that from 1990 to 2005, DSM programs did not have an impact on overall electricity consumption in Canada. Using a similar time period, Gupta

[31] finds that from 1992 to 2008 there was a positive effect from the introduction of load management programs, but that the overall amount of peak load reduction was very small, and that there was substantial regional variability. Finally, Bouhou et al. [32] use a mixed regression model including many additional factors in order to model the impact of a consumer’s

10 technology on the performance of DSM interventions. The results of this last study suggest that a household’s relative technological state (e.g., presence of a programmable thermostat) has an influence on DSM performance [32].

Collectively, the literature suggests that DSM program expenditures and available technologies choices may contribute to energy efficiency and load management savings.

However, the extent of this impact may be limited or variable depending on other factors.

Further, many of these studies do not directly measure the impact of smart metering technologies. In one exception, Corbett [9] uses cross-sectional data to examine the relationship between metering technology and energy efficiency effects of DSM programs. In this study, both positive and negative effects are found, depending on the type of metering technology in use

[33]. Thus, although some research has been done, there remains a substantial gap in our understanding of the relationship between smart metering technologies, both energy efficiency and load management types of DSM programs, and other characteristics of the DSM program, such as program investments and expenditures.

2.3 Research Questions

Through this research, we seek to address gaps identified in the literature and extend scholarship in this important area. Specifically, we are interested in empirically assessing the impact of investment levels and metering technology on utilities’ DSM performance for residential customers. Our investigation is guided by two questions: 1) what is the impact of metering technologies on utilities’ DSM performance; and 2) does greater investment in energy efficiency programs and load management programs lead to greater energy efficiency effects and greater peak load reductions respectively? We believe that answers to these questions will help in building an affordable, modern and sustainable electricity system.

11

3. Methodology

A panel study using the data from the same utilities over the four-year period from 2009 to 2012 is used for this research. Panel data, where the same units of observation are examined over several periods of time, has several advantages over cross-sectional and time series data. By observing the data from the same utilities, panel analysis allows for the control of unobservables which are correlated with the regressors and are specific to certain individuals, but which are time-invariant, or do not change over time. It also allows for the control of unobservables which are correlated with the regressors and common to all of the individuals. Finally, panel data also allows for the study of heterogeneous choices, and the dynamics of individual choices, without making assumptions about aggregation. We chose to use econometric analyses because this method has the ability to quantify relationships “on the basis of available data and using statistical techniques, and to interpret, use and exploit the resulting outcomes appropriately” [25].

This ability to determine a quantitative relationship between variables, such as the number of smart meters installed and the savings from an energy-efficiency program, is necessary in order to allow utilities make decisions regarding their programs.

An example of a panel data model is shown below:

(1)

In this model, is a vector of independent regressors that explain , the dependent variable.

The subscript i represents the different individuals in the data, while the subscript t represents the different years of data. In addition, and are independent unobservables with zero mean which make up the error term of the model. The unobservable is called the individual effect, or the time-invariant unobservable. This is a factor which is correlated with the regressors and is specific to certain individuals, but which is time-invariant, or does not change over time. The

12 unobservable is called the transitory shock or the time-variant unobservable. This factor is correlated with the regressors, but changes over time.

The above model allows us to study the effects of demand-side management on energy efficiency and load management (as dependent variables) as a result of different forms of metering technology implementation and financial investments (as independent variables). With the above model, we can also control for unobservable factors (i.e., data omitted or unreported in the EIA database used in this study), such as the specific demographic characteristics of customers or geographic characteristics of regions served by different utilities, by observing changes in the dependent variable over time. While it is possible to use ordinary regression techniques on panel data, they may not be optimal. The estimates of coefficients derived from regression may be subject to omitted variable bias – a problem that arises when there is some unknown variable or variables that cannot be controlled for that affect the dependent variable

[34]. (Refer to Section 3.3 for the specific models used in this research.)

In our study, the fixed effects model is used to perform panel data analysis. It is noted, however, that there are other alternative methods, such as simple linear regression and random effects model, which can be considered for analyzing panel data. The fixed effects model generally assumes that unobserved heterogeneity will bias the estimates (and thus we need to control for this), while the random effects model assumes otherwise. The conventional linear regression with OLS generally omits time/space variables and unobserved heterogeneity [35]. To gain some insights to the right approach for our study, a simple regression with pooled OLS was also performed, and it shows that the fixed effects model (using AReg routine) leads to higher values of adjusted R-square than the simple regression. (Refer to the results from pooled OLS

13 analysis given in Appendix A as an online resource). The choice between the fixed effects model and random effects model is discussed further in Section 3.2.3 where endogeneity is analyzed.

3.1 Data Description

Consistent with previous studies [e.g., 9], data to create the panel was sourced from the United

States Energy Information Administration (EIA) reports. Specifically, the data comes from the

EIA-861, the Annual Industry Report (www.eia.gov/electricity/data/eia861/).

Form EIA-861 data are collected from approximately 2,800 utilities annually and have been reported since 1990. Although utilities may engage in DSM across their range of customers

(residential, commercial and industrial), we chose to focus on the residential sector. In this way, our research is consistent with and builds on existing research [e.g., 17, 20, 26, 32, 36]. It has also been suggested that residential customers have more varied and complex reactions than industrial or commercial customers [37]. Finally, as a matter of convenience, there is more data available regarding residential customers than any of the other types of customers.

Data were extracted for the period of 2009 to 20121. In 2012, 789 utilities provided data regarding DSM performance. Out of these utilities, 87 were identified as having the necessary amount of complete data over four years regarding their DSM (energy efficiency and load management) programs: these utilities implemented a DSM program with at least one customer during the four-year periods. With data for each of these four years, a sample of 348 observations2 was achieved.

1 Although more recent data is available, the structure of the reports was changed substantially in 2013 making it more difficult to ensure consistency of data across the period of study. Accordingly, we decided to use 2012 as the terminal year. 2 Similar datasets were extracted from the EIA reports for the years 2007-2012 and 2008-2012 and examined. Although these datasets had more observations in total, they also had more missing data, creating challenges for the analysis. Thus the 2009-2012 dataset was selected.

14

Relevant data points were extracted for these utilities from the EIA-861 annual reports, including: number of customers by metering technology, the effects of energy efficiency and load management programs, and the costs for DSM programs. In addition, descriptive data for each of the utilities was extracted, including: utility location, total number of customers, and ownership type.

3.1.1 Dependent Variables

For dependent variables, we sought measures that best represent a utility’s performance in relation to the two main types of DSM programs. The EIA-861 report provides ten different measures regarding this performance. Some of these measures include only new customers, while some include existing programs, and others use an incremental time horizon versus an annual time horizon. For this research we chose measures for annual and actual results, as these seem to be the most applicable to actual performance: Annual Energy Efficiency Effects

(measured in terms of megawatt hours) for energy efficiency programs, and Annual Load

Management Actual Peak Reduction (measured in terms of megawatts) for load management programs.

3.1.2 Independent Variables

As the type of metering technology used by utilities is in a state of transition, the EIA-861 report includes data regarding three types of metering technologies: AMR (automated meter reading devices), AMI (advanced metering infrastructure devices, or smart meters) and net meters. In examining the data, we noted that there was a relatively small number of net meters deployed by utilities as compared to AMR and AMI, and that the correlation between AMI and net meters was very high (r = .93, p < .001). As a result, we decided to use two metering types – AMR and

AMI as independent variables in this analysis. AMR devices are meters that collect data for

15 billing and information purposes and transmit this data one way, from the customer to the distribution utility. Technologically, AMI devices, or smart meters, are more advanced than

AMR. AMI meters measure and transmit usage data more frequently (e.g., 15 minute intervals) and allow for two-way communication between the utility and the device.

In order to assess whether the level of financial investment impacts the performance of

DSM programs, the costs of providing energy efficiency and load management programs are included as independent variables. The direct cost of both energy efficiency programs and load management programs are included, which measure the actual cost of implementing these programs. The costs are reported in thousands of dollars. These costs could involve the cost of replacing equipment and meters, labour costs, and other costs of implementation. The incentive costs of energy efficiency and load management programs are also included. These costs involve the payments made to residential customers in order to create an incentive for them to participate in these programs. These could be one-time lump sum payments or could involve a calculation of an incentive per megawatt of electricity saved by the consumer. Finally, we include the combined indirect cost of these two programs. Indirect costs are costs associated with either type of DSM programs, but that are not directly relevant to the implementation of the program. These could involve administration, marketing, or monitoring and evaluation costs. It is assumed that the direct, incentive and indirect costs noted above represent the utility’s decision regarding the amount of investment in these two DSM programs.

3.1.3 Control Variables

Three control factors - utility size, ownership type, and location - were included in the analysis as they could have an influence on DSM performance. For utility size, the total number of customers was used because it is easily understandable and relates to the independent variables

16 of number of meters. For ownership type, we followed Corbett [9] and regrouped the ten different ownership type categories from the EIA report into two categories and created a binary control variable to represent whether the utility is investor-owned (=1) or non-investor-owned

(=0). A final ANOVA showed a significant effect of ownership type on energy efficiency F

(1,348) =26.97, p<0.001 and load management F(1,348) = 19.88, p <.001. The third control variable was location because different challenges and opportunities may exist for utilities across regions due to regulatory environment and climatic conditions. A binary control variable was created for utility location. Using the 2012 State Energy Efficiency Scorecard rankings, the utilities in the top ten energy efficient states were coded as 1, and the utilities in all the other states were coded as 0.3 The ANOVA comparing the means between these two groups indicated a significant difference in the effect on energy efficiency F(1,348) = 11.88, p < 0.001. However, there does not appear to be a significant difference in the effect on load management F(1,348) =

1.46, p = 0.2276.

3.2 Quality Assessment of Data

As the EIA-861 report data has been used in previous studies [9], we had a reasonably high degree of confidence in the quality of the data. Still, we conducted appropriate tests to verify the characteristics and suitability of the data.

3.2.1 Normality and Outliers

Normality was assessed through frequency statistics, histograms, and skew and kurtosis tests for normality in the statistical package Stata. These tests showed statistically significant (p < 0.001)

3 The American Council for an Energy-Efficiency Economy (ACEEE) has created a ranking program called the State Energy Efficiency Scorecard, and has been ranking states in terms of energy efficiency since 2006. These rankings encompass six policy areas, and states are ranked using 30 individual metrics to determine the most energy efficient states. The top ten states were chosen because the ACEEE splits the states into tiers of ten states each, and thus selecting only the first tier of ten states appears to be a method of selecting the most highly favorable states for demand-side management programs.

17 positive skew and kurtosis for all the variables as reported in Table 1. These results are not surprising given the composition of the U.S. electricity sector: there is a large number of relatively small utilities and a smaller number of very large utilities. Although log transformations of data can be useful as remedies for outliers and failures of normality, they are not universally recommended and may have limited benefits when variables have similar distributions [38]. In addition, data transformations may result in unintended results [39] and make the interpretability of results more difficult [40]. In light of these concerns and lacking a theoretical basis for transforming the data, we retained the untransformed values in the analyses.

Table 1: Descriptive Statistics

s.e. s.d. s.e. s.e. Variable Min Max Mean Skew Kurtosis (mean) (mean) (Skew) (Kurtosis) EFF Effects 0 6304223 219413.00 35761.93 667130.10 7.0555 0.131 57,325 0.261 LM Peak Reduction 0 808 22.97 3.69 68.84 7.0316 0.131 64,485 0.261 AMR Metering 0 1513773 180880.20 17918.09 334257.70 2.1214 0.131 3,595 0.261 AMI Metering 0 4230772 80792.09 21812.36 406904.30 8.0414 0.131 72,484 0.261 Direct Cost- EFF 0 286320 8076.95 1485.69 27715.12 8.1896 0.131 75,289 0.261 Incentive Cost- EFF 0 162995 9864.57 110159 20549.94 3.9424 0.131 20,218 0.261 Direct Cost- LM 0 51849 2411.58 328.97 6136.84 4.8471 0.131 27,920 0.261 Incentive Cost- LM 0 82146 2954.83 48636 9072.97 5.1536 0.131 33,969 0.261 Indirect Costs 0 199954 1711.57 592.03 11044.25 16.7618 0.131 301,479 0.261 Number Customers 5514 4599078 431784.70 37324.15 696272.90 3.4513 0.131 15,573 0.261 n=348

Cook’s distance was used to identify and evaluate the influence of outliers in the data. This test measures “the aggregate change in the estimated coefficients when each observation is left out of the estimation” [41, p. 6] which shows whether a large change would occur if the influential cases were omitted from the regression model. Typically, data with a Cook’s distance greater than 1.0 are considered potentially influential and worthy of further investigation [38, 42] . For both Energy Efficiency and Load Management, two observations had a Cook’s distances greater than 1. Upon further analyses, we decided to retain these observations in the dataset.

18

3.2.2 Collinearity

Collinearity diagnostics for an Energy Efficiency model showed that the VIF (Variance Inflation

Factor) was greater than 10 in the case of three variables, namely AMI Metering, and Direct Cost of Energy Efficiency programs. Variables with VIFs greater than 10 may merit further investigation, because this suggests the variable could be considered a linear combination of other independent variables [43]. Thus, there may be a threat of multicollinearity when the variables AMI Metering, Net Metering, and Direct Cost of Energy Efficiency programs are included in the model. All models created should be examined for multicollinearity, and these variables should be considered as candidates for exclusion from Energy Efficiency models. To reduce the potential risk associated with multicollinarily, we decided to remove the variable Net

Metering (as discussed above). However, we decided to retain the AMI Metering and Direct Cost of Energy Efficiency Programs variables as these variables are core to answering our research questions. The correlation matrix for the variables is shown in Table 2.

Table 2: Correlation Matrix

1 2 3 4 5 6 7 8 9 10 11 12 1 EFF Effects 1.00 2 LM Peak Reduction 0.07 1.00 3 AMR Metering 0.15 0.35 1.00 4 AMI Metering 0.87 0.00 -0.10 1.00 5 Direct Cost- EFF 0.91 0.07 0.07 0.91 1.00 6 Incentive Cost- EFF 0.77 0.11 0.27 0.61 0.75 1.00 7 Direct Cost- LM 0.15 0.18 0.38 0.13 0.14 0.26 1.00 8 Incentive Cost- LM 0.17 0.51 0.50 0.03 0.14 0.29 0.42 1.00 9 Indirect Costs 0.36 0.05 0.07 0.25 0.24 0.37 0.36 0.24 1.00 10 Number Customers 0.71 0.25 0.35 0.64 0.77 0.75 0.40 0.36 0.37 1.00 11 Ownership Type 0.27 0.23 0.44 0.17 0.27 0.40 0.37 0.30 0.14 0.52 1.00 12 Location ID 0.18 -0.06 0.03 0.12 0.19 0.21 0.03 0.05 0.08 0.05 -0.09 1.00 Notes: N = 348, EFF = Energy Efficiency, LM = Load Management

19

3.2.3 Endogeneity

Endogeneity is present when a regressor is correlated with the error term and can cause biased parameter estimates. Endogeneity can be caused by omitted variables, measurement error, and simultaneity [44]. Methods of controlling for endogeneity, such as the Fixed Effects model and

Random Effects model, can be used with panel data when there is the possibility of omitted variables in the model. To choose between the two models, we first conducted the standard

Hausman test, but the difference of the two covariance matrices were not positive definite, thus the statistic may not be accurate. To deal with this issue, we used Stata to conduct Hausman test with the options Sigmamore/Sigmaless for both Energy Efficiency and Load Management. The option Sigmamore specifies that the covariance matrices are based on the estimate of disturbance variance from the efficient estimator, while the Sigmaless option specifies that the matrices are based on the estimate from the consistent estimator. When both of these options are used, the differences are no longer positive definite and the results are very similar. The Sigmamore and

Sigmaless test results for Energy Efficiency and Load Management are given in Appendix B (as an online resource). From these tables, because the p-value is less than 0.05, we reject the null hypothesis that the difference between the estimators is small and that the Random Effects estimator is consistent. Thus, the Fixed Effects model should be selected, since the Random

Effects model is inconsistent and the individual effects appear to be correlated with the regressors. Therefore, it is on this basis that we report our findings in this paper.

3.3 Model Descriptions

In order to test for the effects of different independent variables, hierarchical regressions are performed. Specifically, four models are created for both Energy Efficiency Savings and Load

Management Peak Reduction as shown in Table 3. The first model includes only the control

20 variables, while in the next two models the number of customers with AMR Metering and AMI

Metering are added. This is done to help isolate the impact of different metering technologies on

DSM program performance. A positive beta indicates that adding more of the certain type of meter improves the performance of the DSM program, while a negative beta indicates a drop in performance if more meters are added. The fourth model adds the Direct, Incentive, and Indirect costs for DSM programs.

Table 3: Regression Models

4. Results

4.1 Energy Efficiency Effects

As shown in Table 4, Model 4 is statistically significant and the variation in this model’s independent variables explains 98.67% of the variation in energy efficiency savings (F=83.84,

21 p<0.001). All of the other variables are individually statistically significant. With respect to metering technologies, the beta coefficients for both types are positive and significant.

Accordingly, this model suggests that additional AMR and AMI meters will have a small positive impact on energy efficiency savings. For example, AMI metering’s beta of 0.649 suggests adding one additional smart metering customer will increase the utility’s energy efficiency savings by 0.694 MWh. Similarly, the betas for direct cost and incentive costs are positive, with direct costs having the greatest impact. The results suggest that an additional thousand dollars of investment directly into energy efficiency programs has the largest impact, with an increase of 4.52 MWh of savings whereas an additional thousand dollars of incentive payments leads to an increase of 2.50 MWh of savings.

Table 4: Results of Fixed Effects Regression Analysis for Energy Efficiency

Variables Model 1 Model 2 Model 3 Model 4 Constant 368456.7*** 394532.9*** 208136.8*** 150310.1*** AMR Metering -0.1480476 0.5407673*** 0.478438*** AMI Metering 0.7795841*** 0.6494712*** Direct Cost- EFF 4.522667*** Incentive Cost- EFF 2.500524*** Indirect Costs 1.46878 F 32.92*** 17.82*** 131.61*** 83.84*** R^2 0.9648 0.9652 0.9843 0.9867 Adjusted R^2 0.9531 0.9533 0.9789 0.9819 Notes: *p<0.05, **p<0.01, ***p<0.001

4.2 Load Management Performance

The results for load management are presented in Table 5. With respect to load management peak reduction, the variation in the independent variables in Model 4 explains 69.47% of the variation in the dependent variable (F=3.28, p<0.01). Model 4 also shows very small positive, but statistically insignificant betas, for the variables AMR metering and AMI metering, which

22 suggests that increasing the number of meters has almost no influence on the performance of load management programs; however, no firm conclusions can be drawn. Unlike the results for energy efficiency effects, incentive cost is the only investment variable that is statistically significant in Model 4. The negative beta coefficient of the incentive cost measure suggests that increasing the amount of incentives given to participants in load management programs will marginally reduce program performance.

Table 5: Results of Fixed Effects Regression Analysis for Load Management

Variables Model 1 Model 2 Model 3 Model 4 Constant 20.55271* 18.01342 16.58091 19.11243 AMR Metering .0000144 0.0000197 .0000106 AMI Metering 0.00000599 7.61e-07 Direct Cost- LM -.0010206 Incentive Cost- LM -.0027917*** Indirect Costs .0006466 F 0.09 0.16 0.14 3.28** R^2 0.6712 0.6716 0.6717 0.6947 Adjusted R^2 0.5612 0.5600 0.5584 0.5845 Notes: *p<0.05, **p<0.01, ***p<0.001

5. Discussion and Implications

Our goal in presenting this research is to contribute to the development of a modern and sustainable ‘smart’ grid by examining the relationship between metering technologies, financial investments in DSM programs and DSM performance. In this respect, a number of observations are worthy of further discussion.

5.1 Impact of Metering Technologies

5.1.1 Energy Efficiency

Our first research question asked what the impact of metering technologies was on utilities DSM performance. In the case of energy efficiency programs, the answer seems clear: the number of

23 meters – whether AMR or AMI – has a positive influence on the energy efficiency effects associated with DSM programs. Taking a managerial perspective, perhaps this result is simply a reflection of Peter Drucker’s well-known adage ‘you can’t manage what you can’t measure’. In the case of energy efficiency, meters provide valuable information to utilities and electricity consumers regarding consumption levels and the impacts of different energy efficiency investments, allowing them to make better decisions. For utilities this may allow better decisions with respect to the design and promotion of energy efficiency programs, whereas for consumers, the increased information may enable them to become more engaged in utility-sponsored programs.

That being said, the results suggest there is an additional value associated with more advanced technologies and builds on the idea that the technological state of the home may influence consumers’ responses to various DSM initiatives [32]. As compared to AMR, the influence of AMI on energy efficient savings is stronger. In this respect, our results are consistent with previous research suggesting positive relationships between smart metering and energy efficiency [e.g., 9, 16]. To achieve these savings, both consumers and utilities play a part.

The rollout of AMI, with their advanced informational and communication capabilities, has permitted utilities and third-parties to develop ancillary smart grid enabled services [12], such as in-home and mobile electricity displays, as well as offer different informational resources, such as interactive websites, informative billing, and consumption feedback [16]. Through these services and informational resources, consumers gain access to detailed consumption details and regular feedback that permit consumers to better understand their electricity use, make more informed decisions about participation in utility programs, and adopt appropriate behavioral changes [16, 45]. From the utilities’ perspective, more advanced metering technologies, such as

24 smart meters may provide additional organizational information processing capacities (e.g., data collection, storage and diffusion) [9], to better measure the performance of various programs to see which one work better in practice [16].

Finally, influence of AMI metering on DSM performance may have something to do with the observation that, in response to utilities’ requests to reduce consumption, households may seek more energy independence by reducing their reliance of the main electricity grid [46]. They may do so by implementing more energy efficient appliances, consistent with the goals of the

DSM program and also by investing in their own small-scale generation capabilities such as photovoltaic systems. Given our decision not to test the influence of net metering separate from

AMI metering, we cannot say for certain if this is in fact the case; however, the results point to the need for further investigation into whether households engaging in distributed generation

(and having net metering devices) are more likely to also engage in energy efficiency programs.

5.1.2 Load Management Effects

While the answer to our first research question is quite clear for energy efficiency effects, the results are not conclusive with respect to the relationship between metering technologies and

Peak Load reductions associated with load management DSM programs. This result was unexpected and contrasts with previous research suggesting that the use of smart meters to enable demand response can result in substantial electricity savings [e.g., 16]. Nevertheless, the extant literature may also provide some clues to help make sense of these results.

First, low adoption of load management programs by utilities and their consumers may have made it difficult to find statistical and stable relationships between the dependent variables and load management performance. As compared to energy efficiency programs, fewer utilities offer residential load management programs and average investments are lower. In addition,

25 these more active strategies, being relatively new, have experienced a degree of resistance from consumers [17]. For example, research suggests that while consumers may be generally receptive to smart grid technologies and demand shifting [17] or automation [12], many barriers stand in the way of actual adoption. With respect to direct load management by utilities, consumers perceive threats from smart meters in terms of loss of control and autonomy (by allowing their utility to control various devices within their home), mistrust the energy providers’ motives behind such programs, and worry about threats to personal privacy [12]. Other factors, such as market regulations, household practices and usage behaviors, and the informational and technical aspects of smart meters and load management programs also influence consumers’ decisions regarding participation in load management programs [17].

Even after consumers decide to partake in load management programs, utilities face substantial challenges in keeping them engaged and deriving benefits from the program. For example, technology-enabled behavioral solutions (e.g., demand response) that repeatedly ask consumers to alter their behaviors may be less effective [15] and harder to sustain [46] than other programs requiring a single decision or intervention. This is in part due to the learning that must take place for consumers to develop new behaviors [28]. In addition, when faced with the prospect of automated demand response or the possibility of needing to reduce consumption at specific periods, consumers may adopt alternative behaviors [15, 46]. In effect, in the face of load management programs, consumers may choose instead to reduce their energy consumption by replacing their inefficient appliances with new ones (captured in energy efficiency effects), rather than participating actively in load shifting. When consumers do not switch their timing of usage or do not allow utilities to adjust their appliance usage during peak periods, there would be

26 little impact on load management performance even with increased investment in these programs by utilities.

Beyond considerations with respect to adoption and engagement, an alternative explanation of our results is that our model fails to capture other important factors such as customers’ willingness to shift their energy use from peak to off-peak periods. For example, a more environmentally-conscious or cost-conscious consumer might be more likely to engage in a load management program and truly change their behaviour by shifting their energy consumption to off-peak periods. Further, in response to specific DSM programs, consumers may make strategic decisions which could offset potential benefits [15] or lead them to different behaviors.

Although consumers are generally welcoming of smart grid technologies, they have stronger preferences for energy savings and energy efficiency [6, 17], which may reduce their likelihood of actively participating in other programs. As our objective was to examine factors within the utilities’ control (e.g., metering technology and investments), we did not include factors related to customers in our models, yet this could be a valuable extension to our work as we describe further below.

Continuing with the idea that other factors may be at play, a final explanation could be that smart meters on their own are not be sufficient to produce significant load management benefits such as peak load reductions. In this regard, smart meters could be considered necessary, but not sufficient enabling devices for achieving significant peak load reductions. On one hand, residential consumers may require additional information systems and technologies, such as in- home, always-on displays [45], home area networks [47] or mobile applications that provide detailed consumption feedback allowing them to more easily control and shift their consumption loads. Just as for energy efficiency programs, the degree of technology adoption within a home

27 can influence the households’ response to DSM initiatives [32], but this is more pronounced for load management programs because it requires that consumers make repeated decisions about whether to accept the utilities’ request for a change in consumption [15]. In the second case, utilities may implement, either directly or indirectly, direct load control programs. For these purer technology solutions, consumers are not required to make consumption decisions once they enter the program. Instead, the utility can directly send signals to specific appliances (e.g., air conditioners, water tanks, pool pumps) to shut off or reduce their electricity demand during critical peak periods. In such an arrangement, referred to as direct-load control, the meter is effectively by-passed because the communications occurs directly between the utility and the appliance. Thus, the metering device is less relevant as a factor of success, particularly if the direct load program is managed by a third party intermediary that combine and manage the load of multiple consumers [47].

5.2 Influence of Program Investments

Our second research question concerned the impact of utilities’ financial investments on their

DSM performance. With respect to program investment costs, our findings show a positive relationship between financial investments and DSM program success. Our research goes further by parsing the different types of costs (i.e., direct, incentive and indirect) incurred by utilities, showing that investments in direct costs have the greatest impact, followed by inventive costs.

This means that customers’ energy saving habits are not influenced as much by actual dollar incentives they could receive, but are motivated by such things as advertising or awareness campaigns. Therefore, if a utility has additional investment funds available, it may be more effective to direct these dollars towards the direct cost of the energy efficiency program rather than increasing the amount of incentives given to customers.

28

With respect to load management performance, the results are less encouraging. Here, direct and indirect costs are statistically insignificant and incentive costs have a small, but statistically significant, negative influence on peak load reductions. We propose two potential explanations for this result. First, there may be spillover effects between different types of DSM programs. It is likely that residential customers who sign up for load management programs are likely those who are most concerned about their electricity bills. Therefore, in order to save money, they also tend to conserve their energy usage overall and partake in their utility’s energy efficiency programs. In this respective, costs associated with promoting and incentivizing participation in load management programs may actually have some spillover effects onto the energy efficiency programs without it being captured as part of peak load reduction.

Second, utilities may have developed an over-capacity with respect to peak load shifting.

Two common incentive plans are sign-up payments (e.g., a stand-by fee) and flat-rate incentive payments for participation in a program (e.g.; customers receive $5 per month for installing a control device on a hot water heater). Utilities pay these incentives regardless of whether the customer is called upon to shift loads during critical peak periods. Most of the incentives are likely provided to customers with AMR or AMI meters, but according to our results (Table 5), none of the device types has any significant effect on load management. As a result, demand fluctuations from year to year could have caused the negative relationship between incentive cost and load management since the metering technologies intended for load management were underutilized. In other words, it is possible that some customers may not be called upon to reduce their load.

In terms of over-capacity, it is also important to consider the interactions between passive and active DSM strategies. Both energy efficiency programs and demand response can be used

29 by utilities to support capacity adequacy as a cost effective alternative to new [21]. However, as discussed previously, each of these programs responds to different motivations and provides different benefits to utilities [20]. Being a dispatchable resource with relatively short-term (non-permanent) benefits, demand response provides utilities great flexibility, but they cannot ignore the effects of energy efficiency programs in their capacity planning. More specifically, utilities must be attentive to the permanent reductions in peak load that result from energy efficiency programs in order to avoid excess capacity procurement [20,

21]. In addition, utilities are advised to adopt measurement and verification procedures that consider the possibility that consumers may engage in both types of programs [21]. Practically, the actual peak load savings from energy efficiency programs may relatively small, but under certain conditions, it could create over-capacity situations and confound the evaluation of load management program performance.

5.3 Managerial and Policy Implications

A number of important managerial implications derive from this research. First, given the explanatory power of the energy efficiency model, this model could be used to support decision- making with respect to DSM programs. For instance, a utility could input their specific characteristics, such as number of customers, numbers of different meters types, their location, and their level of investment, and then determine what their energy efficiency savings would be on average. Such a decision-support tool could be beneficial as a starting point for determining the level of savings the utility could expect at a certain level of investment, and could help utilities to determine whether any adjustments might need to be made in order to meet their desired program savings. In addition, the estimates could provide a benchmark against which the utility could track and improve its performance.

30

Second, it will be important for utilities and policy makers to consider the interactions between different types of DSM initiatives in conjunction with transformations occurring on the supply-side. As discussed, it is possible that the implementation of demand response programs may indirectly contribute to energy efficiency effects associated with these types of more passive

DSM strategies as consumers make trade-offs and choices in their behaviors responses. In practice, this type of coordination is still not widespread [23]. Thus, rather than treating DSM programs discreetly, utilities and policymakers may be well-advised to consider indirect effects and common factors influencing the effectiveness and efficiency of these programs. We also agree with other researchers [10] who have suggested that utilities may want to be more selective in their targeting of customers for different DSM programs. Finally, it will also be important going forward for utilities to consider changes in demand-side activities in conjunction with supply-side activities. Increasingly, there is a link between these two sides of utility operations

[46] and a failure to take into account the influence of changing one on the other may create problems for utilities in the future.

Third, it will be important for utilities and policy-makers to be aware of the continuing innovation in smart grid technologies, not just in use by the utilities themselves, but also by intermediary organizations and consumers. As the electricity sector continues to modernize, new information technologies will inevitably be introduced by third parties that could mediate the relationships between utilities and their customers. The advent and adoption of smart appliances and integrated home management systems will also change the landscape of how utilities communicate with their customers and their ability to effect desired changes in electricity consumption behaviors.

31

6. Conclusion

6.1 Contributions

The world has set out an ambitious target of making affordable, reliable and available to all by 2030. Achieving this goal will require the rapid development of a modern

‘smart’ grid. In presenting our research, we hope to contribute to ongoing advancement in this area. In particular, our research makes two notable contributions. First, we provide an empirical examination of the relationship between metering technologies, financial investments and DSM performance in terms of both energy efficiency and load management. In addition, our research uses panel data over a four year period that provides greater insight than cross-sectional studies.

To date, empirical studies have been lacking in this area and the previous is fragmented, considering only parts of these questions. Through our work, we provide a more comprehensive view into the interactions between these factors and their implications for success. A second important contribution of this research relates to the managerial and policy implications that derive from our findings. Specifically, we highlight the fact that demand-side and supply side activities must be considered and managed more holistically by utilities, taking into account the impact of one on the other. We also highlight the importance of being aware of the emerging technologies. While the promise of the smart grid is to permit the development of “new functionalities and applications” [4], this may also be one of its greatest challenges.

6.2 Limitations and Future Research

The most important limitation of this research concerns the data sample which is restricted to 78 utilities in the United States over the period from 2009 to 2012. The extent to which this data is representative of all utilities around the world affects the generalizability of the results. Without a doubt, different jurisdictions have different regulatory regimes, energy concerns, national

32 cultures and sustainability preoccupations that could lead to different outcomes. By considering relevant research conducted in other countries, we believe there is a reasonably broad applicability to our findings. That being said, future research that attempts to replicate this study in other jurisdictions would be welcome and could provide interesting insights into the development of smart grid on a global scale.

Second, the statistically insignificant results related to load management limit our ability to draw any firm conclusions with respect to the relationships between technology, program investments and peak load reductions. The results obtained for load management do not necessarily mean that such programs are not effective, but rather that the critical explanatory factors, whether consumers’ attitudes, household responses, or other technologies, may not have been captured in the EIA database and our model. This is a particularly important area for future research because the ability to manage demand in a more dynamic and real-time fashion has been touted as one of the main benefits of smart meters. If nothing else, our research raises questions about that potential. We encourage research that seeks to identify the technological, organizational, individual factors that contribute to reductions in peak loads as part of load management DSM programs. Such research could consider both pure technology solutions that involve direct control and intelligent systems, as well as technology-enabled behavioral solutions in which technology and consumers interact in order to achieve desired changes in consumption patterns. This type of research will be essential for turning the promise into reality and will aide utilities and policy-makers in the design and development of effective programs.

6.3 A Final Word

The United Nation’s Sustainable Development Goal of ensuring universal access to affordable, reliable, sustainable and modern energy is an ambitious target that will require the engagement

33 of all stakeholders and collaboration between practice and research. Information technologies associated with the smart grid will play a key role. Given the urgency of the goals and increasing constraints on resources it is essential that we seek to minimize waste of all types – of money, time, information and technologies. We hope that our research has helped to build our collective knowledge in this area and provide concrete directions for moving forward. The job now is to ensure that the smart grid and its related technologies and programs are mindfully developed and implemented so as to provide positive, sustainable outcomes for all.

References

[1] United Nations. (2015, December 8). Sustainable Development Knowledge Platform.

Available: https://sustainabledevelopment.un.org/sdgs

[2] EIA. (2016, June 13). International Energy Outlook 2016. Available:

http://www.eia.gov/forecasts/ieo/exec_summ.cfm

[3] EPA. (2016, June 13). Sources of Greenhouse Gas Emissions. Available:

https://www3.epa.gov/climatechange/ghgemissions/sources.html

[4] NIST. (2016). Smart Grid. Available: www.nist.gov/smartgrid/

[5] United States Department of Energy. (2014). 2014 Smart Grid System Report: Report to

Congress. Available: http://energy.gov/sites/prod/files/2014/08/f18/SmartGrid-

SystemReport2014.pdf

[6] D. Ngar-yin Mah, J. Marius van der Vleuten, J. C. Ip, and P. R. Hills, "Governing the

transition of socio-technical systems: A case study of the development of smart grids in

Korea," Energy Policy, vol. 45, pp. 133-141, 2012.

[7] S. D. Ramchurn, P. Vytelingum, A. Rogers, and N. R. Jennings, "Putting the 'Smarts' into

the Smart Grid," Communications of the ACM, vol. 55, pp. 86-97, 2012.

34

[8] S.-I. Inage. (2010, July 11). Modeling Load shifting Using Electric Vehicles in a Smart

Grid Environment. Available:

https://www.iea.org/publications/freepublications/publication/load_shifting.pdf

[9] J. Corbett, "Using information systems to improve energy efficiency: Do smart meters

make a difference?," Information Systems Frontiers, vol. 15, pp. 747-760, 2013.

[10] M. E. H. Dyson, S. D. Borgeson, M. D. Tabone, and D. S. Callaway, "Using smart meter

data to estimate demand response potential, with application to solar energy integration,"

Energy Policy, vol. 73, pp. 607-619, 2014.

[11] R. Hartway, S. Price, and C. K. Woo, "Smart meter, customer choice and profitable time-

of-use rate option," Energy, vol. 24, pp. 895-903, 1999.

[12] K. Buchanan, N. Banks, I. Preston, and R. Russo, "The British public's perception of the

UK smart metering initiative: Threats and opportunities," Energy Policy, vol. 91, pp. 87-

97, 2016.

[13] EIA. (2016, September 28). How many smart meters are installed in the United States,

and who has them? Available: http://www.eia.gov/tools/faqs/faq.cfm?id=108&t=3

[14] Telefonica. (2014, September 28). m2m: 800 million electric smart meters to be installed

globally by 2020. Available: https://iot.telefonica.com/press/m2m-800-million-electric-

smart-meters-to-be-installed-globally-by-2020

[15] C. Lang and E. Okwelum, "The mitigating effect of strategic behavior on the net benefits

of a direct load control program," Energy Economics, vol. 49, pp. 141-148, 2015.

[16] A. Podgornick, B. Sucic, P. Bevk, and D. Stanicic, "The Impact of Smart Metering on

Energy Efficiency in Low-Income Housing in Mediterranean," in Climate-Smart

Technologies, W. L. e. a. Filho, Ed., ed Berlin: Springer-Verlag, 2013.

35

[17] M. A. R. Lopes, C. H. Antunes, K. B. Janda, P. Peixoto, and N. Martins, "The potential

of energy behaviours in a smart(er) grid: Policy implications from a Portuguese

exploratory study," Energy Policy, vol. 90, pp. 233-245, 2016.

[18] T. Logenthiran, D. Srinivasan, and T. Z. Shun, "Demand Side Management in Smart Grid

Using Heuristic Optimization," IEEE Transactions on Smart Grid, vol. 3, pp. 1244-1252,

2012.

[19] M. Curtis and A. Khare, " in electric utilities: An opportunity for

restorative economics at SaskPower," Technovation, vol. 24, pp. 395-402, 2004.

[20] S. J. Darby and E. McKenna, "Social implications of residential demand response in cool

temperate climates," Energy Policy, vol. 49, pp. 759-769, 2012.

[21] Y. Liu, "Demand response and energy efficiency in the capacity resource procurement:

Case studies of forward capacity markets in ISO New England, PJM and Great Britain,"

Energy Policy, vol. 100, pp. 271-282, 2017.

[22] C. King and D. Delurey, "Energy and Demand Response: Twins, Siblings or Cousins?,"

Public Utilities Fortnightly, vol. March 2005, 2005.

[23] C. Goldman, M. Reid, R. Levy, and A. Slilverstein, Coordination of Energy Efficiency

and Demand Response: Environmental Energy Technologies Division, Ernest Orlando

Lawrence Berkeley National Laboratory, 2010.

[24] DOE. (2012). Demand Reductions from the Application of Advanced Metering

Infrastructure, Pricing Programs, and Customer Based Systems - Initial Results, Smart

Grid Investment Grant Program. Available:

https://www.smartgrid.gov/files/peak_demand_report_final_12-13-2012.pdf

36

[25] M. A. Khan, J. Nadeem, A. Mahmood, Z. A. Khan, and N. Alrajeh, "A Generic

demand‐side Management Model for Smart Grid," International Journal of Energy

Research, vol. 39, pp. 954-964, 2015.

[26] A. J. Conejo, J. M. Morales, and L. Baringo, "Real-time Demand Response Model,"

IEEE Transactions on Smart Grid, vol. 1, pp. 236-242, 2010.

[27] B. Saravanan, "DSM in an area consisting of residential, commercial and industrial load

in smart grid," Frontiers in Energy, vol. 9, pp. 211-216, 2015.

[28] T. Zhang, P.-O. Siebers, and U. Aickelin, "Simulating user learning in authoritative

technology adoption: An agend based model for council-led smart meter deployment

planning in the UK," Technological Forecasting & Social Change, vol. 106, pp. 74-84,

2016.

[29] D. S. Loughran and J. Kulick, "Demand-Side Management and Energy Efficiency in the

United States," The Energy Journal, vol. 25, pp. 19-43, 2004.

[30] N. Rivers and M. Jaccard, "Electric Utility Demand Side Management in Canada," The

Energy Journal, vol. 32, pp. 93-116, 2011.

[31] P. Gupta, "Impact of Demand Side Management Programs on Peak Load Electricity

Demand in North America, 1992 to 2008," International Association for Energy

Economics, vol. 1st Quarter, pp. 34-37, 2013.

[32] N.-E. I. Bouhou, M. F. Blackhurst, and P. Torres, "An empirical analysis of joint

residential electricity efficiency gains within and across end uses: implications for

demand-side management," Ecological Economics, vol. 110, pp. 61-70, 2015.

37

[33] J. Corbett, "Designing and Using Carbon Management Systems to Promote Ecologically

Responsible Behaviors," Journal of the Association for Information Systems, vol. 14, pp.

39-378, 2013.

[34] B. H. Baltagi, Econometric Analysis of Panel Data. Chichester: John Wiley & Sons,

2013.

[35] F. Hayashi, Econometrics. Princeton, NJ: Princeton University Press, 2000.

[36] K. Herter, "Residential implementation of critical-peak pricing of electricity," Energy

Policy, vol. 35, pp. 2121-2130, 2007.

[37] K. Herter, P. McAuliffe, and A. Rosenfeld, "An exploratory analysis of

residential customer response to critical peak pricing of electricity," Energy, vol. 32, pp.

25-34, 2007.

[38] B. G. Tabachnick and L. S. Fidell, Using Multivariate Statistics, Fifth Edition ed. Boston:

Pearson Education, Inc., 2007.

[39] L. A. Polgreen and J. M. Brooks, "Estimating Incremental Costs with Skew," Applied

Health Economics and Health Policy, vol. 10, pp. 319-329, 2012.

[40] C. Feng, H. Wang, N. Lu, T. Chen, H. He, Y. Lu, and X. M. Tu, "Log-transformation and

its implications for data analysis," Shanghai Archives of Psychiatry, vol. 26, pp. 105-109,

2014.

[41] R. Williams. (2016, July 12). Outliers. Available:

https://www3.nd.edu/~rwilliam/stats2/l24.pdf

[42] A. Field, Discovering Statistics Using SPSS. London: Sage Publications Ltd, 2005.

38

[43] X. Chen, P. B. Ender, M. Mitchell, and C. Wells. (2003). Chapter 2 - Regression

Diagnostics. Available:

http://www.ats.ucla.edu/stat/stata/webbooks/reg/chapter2/statareg2.htm

[44] W. H. Green, Econometric Analysis, Seventh ed. Upper Saddle River: Prentice Hall -

Pearson Education, 2012.

[45] Y. Riche, J. Dodge, and R. A. Metoyer, "Studying Always-On Electricity Feedback in the

Home," in SIGCHI Conference on Human Factors in Computing Systems (CHI'10),

2010, pp. 1995-1998.

[46] T. Nakada, K. Shin, and S. Managi, "The effect of demand response on purchase

intention of distributed generation: Evidence from Japan," Energy Policy, vol. 94, pp.

307-316, 2016.

[47] S. Feuerriegel, P. Bodenbenner, and D. Neumann, "Value and granularity of ICT and

smart meter data in demand response systems," Energy Economics, vol. 54, pp. 1-10,

2016.

39