Building : Ten Challenges

Tianzhen Hong*, Jared Langevin, Kaiyu Building and Urban Systems Division Lawrence Berkeley National Berkeley, California 94720, USA

*Corresponding author. T. Hong. [email protected], 1(510)486-7082

Abstract

Buildings consume more than one-third of the world’s primary energy. Reducing energy use and greenhouse-gas emissions in the buildings sector through energy conservation and efficiency improvements constitutes a key strategy for achieving global energy and environmental goals. Building performance simulation has been increasingly used as a tool for designing, operating and retrofitting buildings to save energy and utility costs. However, opportunities remain for researchers, software developers, practitioners and policymakers to maximize the value of building performance simulation in the design and operation of low energy buildings and communities that leverage interdisciplinary approaches to integrate humans, buildings, and the power grid at a large scale. This paper presents ten challenges that highlight some of the most important issues in building performance simulation, covering the full building life cycle and a wide range of modeling scales. The formulation and discussion of each challenge aims to provide insights into the state-of-the-art and future research opportunities for each topic, and to inspire new questions from young researchers in this field.

Keywords

Building energy use, energy efficiency, building performance simulation, , building life cycle, zero-net-energy buildings

Introduction

The buildings sector consumes about 40% of primary energy in the United States and European countries and about 25%-30% in developing countries like China. In the United States, federal, state and local governments set stringent energy goals for new and existing buildings. For example, in the 2016 multi-year program plan (U.S. Department of Energy 2016), the U.S. Department of Energy’s Building Office set a goal to reduce the energy use intensity (EUI) of buildings by 30% by 2030 and 50% over the long-term. At the state level, California’s long-term energy efficiency strategic plan (California Public Utilities Commission 2008) stipulates that all new residential buildings must be zero-net-energy (ZNE) by 2020, all new commercial buildings must be ZNE by 2030, and 50% of existing commercial buildings must be retrofitted to ZNE by 2030. At the city level, the City of San Francisco requires all new municipal construction projects of 5,000 square feet or larger to be LEED Gold certified; several other U.S. cities have similar new construction requirements.

Building performance simulation (BPS) – also known as building simulation, building energy modeling, or energy simulation - has played a growing role in the design and operation of low energy, high-performance buildings and development of policies that drive the achievement of the aforementioned energy goals. BPS is defined as the use of computational mathematical models to represent the physical characteristics, expected or actual operation, and control strategies of a building (or buildings) and its (their) energy systems. BPS calculations include building energy flows, air flows, energy use, thermal comfort and other indoor environmental quality indexes (e.g., glare).1

BPS has a decades-long history of development, beginning with the replacement of manual procedures with computing tools to determine HVAC loads in the 1960s (Clark, 2001 and 2015; Hensen and Roberto, 2011; BEMBook 2018). Several review articles (e.g., Hong et al. 2000; Li and Wen, 2014; Clarke 2015; Clarke and Hensen 2015; Wang and Zhai 2016; Østergård et al. 2016; Harish and Kumar 2016) survey key developments, applications and challenges for the BPS field across its history.

BPS development has been particularly pronounced in the past ten years, as demonstrated by the founding of two new journals in 2008, Building Performance Simulation and Building Simulation, as well as the growth of the International Building Performance Simulation Association (IBPSA), which was formed in 1987. In parallel, several international research efforts under the International Energy Agency (IEA)’s Energy in Buildings and Communities (EBC) Programme have advanced the application of BPS to support the design of buildings and communities (Hong 2018). Key BPS- related IEA EBC projects include: Annex 1, which developed algorithms to determine load and energy of existing buidings; Annex 10, which focused on building HVAC system simulation; Annex 21 and 43, which developed standard methods and test cases to validate and benchmark BPS programs; Annex 30, which provided best practices to integrate simulation in various phases of building design; Annex 53, which used BPS to analyze impact of six influencing factors of real building performance (Yoshino et al. 2017); Annex 58, which studied methods and collected data

1 For a more detailed overview of BPS, see the U.S. Department of Energy’s “101” articles (Roth 2017) on building energy modeling and major use cases including architectural design, HVAC design and operation, building performance rating, and building stock analysis. for full-scale empirical validation of detailed BPS Programs; Annex 60, which developed a Modelica-based library of building component models (Wetter et al. 2015); Annex 66 (Yan et al. 2017), which developed new data, methods, modeling tools and case studies to understand, model and quantify occupant behavior in buildings; and Annex 22, 51 and 73, which studied energy efficient communities.

From a practical perspective, BPS is commonly used to: (1) perform load calculations in support of HVAC equipment selection and sizing, (2) demonstrate the code compliance of a building by comparing the energy performance of the proposed design with the code baseline, and (3) evaluate and compare design scenarios. For further details, the book Building Performance Simulation for Design and Operation (Hensen and Lamberts 2011) provides a comprehensive overview of how building performance simulation is used in the complete building life-cycle from conception to demolition. Note that although the use of BPS in the building design process is widespread, its use in the operation, control and retrofit of existing buildings remains limited.

Figure 1 outlines a theoretical BPS ecosystem in which successful applications integrate users (i.e., energy modelers), programs, data and resource support. These three components are discussed further below.

Figure 1 A theoretical Building Performance Simulation ecosystem integrates users, programs, data and resource support.

Regarding BPS users, the Rocky Mountain Institute has developed the concept of ‘black belt energy modeling’ (Rocky Mountain Institute 2010) to set forth BPS user expectations, training materials, and professional development paths. Under this concept, becoming a master user requires a depth and breadth of knowledge about , building science and BPS programs. Additionally, ASHRAE’s Fundamentals Handbook Chapter 19, Energy Estimating and Modeling Methods, covers fundamental concepts for energy modelers, particularly those new to the field. ASHRAE’s building energy modeling professional certification further ensures that BPS users have the training needed to develop and perform successful energy .

Regarding BPS programs, though many are available,2 no program is perfect in terms of accuracy and ease-of-use (Zhu et al. 2013; Zhou et al. 2014). Moreover, available BPS programs are used to answer a wide range of questions for architects, and other stakeholders, and it is important to select a BPS program that is appropriate for the particular application of interest – indeed, this notion is the basis of the fit-for-purpose modeling concept (Gaetani et al. 2016). Stretching a BPS program beyond its intended scope of use should be avoided; this practice may lead to modeling errors and at minimum requires a deep understanding of the BPS program in question.

Finally, regarding data and resource support, inadequate efforts to collect supporting data for BPS underpin the “Garbage In, Garbage Out” aphorism. When modeling new buildings, for example, users must anticipate how the building will be used and accurately specify design performance goals. When modeling existing buildings, on-site inspections and energy audits can be used to establish reliable input data for energy models. Sound data collection is not replaced by parallel efforts, e.g., model calibration that attempts to fine-tune key model input parameters. As data for many parameters are needed to build detailed energy models using BPS programs such as EnergyPlus (U.S. Department of Energy 2018a), user experience is needed to focus data collection around the most important model input parameters.

Looking ahead, several studies have surveyed state-of-the-art in BPS research and highlighted key challenges and research items for future BPS development. For example, Hong et al. (2000) presented seven use categories of BPS and predicted continued BPS development in five areas: (1) integrating BPS with knowledge-based systems to support decision making, (2) using BPS in early design stage, (3) integrating information monitoring and diagnostic systems (Piette et al. 2001) with BPS for building energy management and control, (4) integrating multiple BPS programs in the building life cycle, and (5) using virtual reality technology to enable digital building design and operation experience. Despite some advances in these five areas over the last 20 years, each remains a significant challenge.

In another study, Clarke (2015) developed 16 propositions for IBPSA to advance the BPS field, emphasizing the need for high-integrity emulations of building performance through BPS while acknowledging the need for accordant changes to company work practices, user-interfaces, support, and accreditation, tool screening, scientific communication, and process management. Clarke and Hensen (2015) further summarized the state-of-the-art in building performance simulation, outlining issues relating to a high integrity representation of building performance, identifying emerging challenges that will dictate new directions for BPS development, and characterizing barriers to collaborative development in the field.

Wang and Zhai (2016) provide an overview of BPS advancements and trends for development and application achieved between 1987 and 2014, focusing on six different topics including ventilation performance , whole building energy and thermal load simulation, lighting and daylighting modeling, building information modeling, indoor acoustic simulation, and life cycle analysis of buildings.

2 The building energy software tools directory, https://www.buildingenergysoftwaretools.com/, lists hundreds of BPS programs at various levels of fidelity, accuracy, and ease of use. Building on these forward-looking studies, this paper aims to pinpoint and discuss the ten most important challenges currently facing the BPS research area, discussing potential solutions to each challenge while acknowledging its technical complexity and significance to a variety of stakeholders. Table 1 lists the selected challenges and outlines the practical significance of addressing each. As seen in the Table, the challenges cover several existing or emerging areas of BPS research and application, including: (1) understanding the gap between expected and actual building performance to achieve targeted design performance goals, (2) understanding and quantifying human-building interactions, (3) modeling existing buildings and their large energy use contributions to the building sector, (4) supporting the design and operation of ZNE and grid- responsive buildings, (5) large-scale building technology adoption, evaluation, and modeling to inform energy policy making in city, state and federal governments, and (6) integrating BPS across the building life cycle.

The challenges were selected based on the preceding literature review and reflect recent advances in the technologies and software capabilities that support BPS, as well as broader advances in the BPS field and building simulation community as a whole. These challenges also reflect the wide range of potential BPS applications, spanning multiple stages in the building life cycle from design to operation and retrofit, and multiple scales of analysis from individual buildings and building occupants to cities, utility regions, and the national building stock (Figure 2). The paper’s broad coverage of potential BPS tasks aims to highlight common research needs surrounding data collection, standardization, and integration; model development and selection; and the development of modeling workflows that are of practical use.

Table 1 Ten challenges of building performance simulation Challenge Significance  Addressing the building performance gap BPS supports verification of building performance goals and ratings/certifications  Modeling human-building interactions BPS incorporates models of environmentally adaptive occupant behavior, which has significant impacts on building energy performance  Energy model calibration BPS supports operational improvements and  Modeling operation, controls and retrofits energy efficiency improvements/retrofits in  Modeling operational faults in buildings existing buildings  Zero-net-energy (ZNE) and grid-responsive BPS supports the design of ZNE buildings buildings and representation of building energy load dynamics needed to deploy building efficiency as a grid resource  Urban-scale building energy modeling BPS supports city-scale building energy efficiency measures needed to achieve energy/environmental goals  Evaluating the energy-saving potential of BPS supports government decision making building technologies at national or regional on building efficiency research, technology scales development and assessment  Modeling energy efficient technology adoption  Integrated BPS supports decision making across the entire building life cycle

Figure 2 The ten BPS challenges described in this paper span multiple stages in the building life cycle and scales of focus, which range from the individual building level to the national building stock level.

We provide an overview of each challenge, discuss why it is important, highlight recent advances in addressing it, and propose potential future research directions. Note that herein, the scope of the term building ‘performance’ is limited to energy performance simulation, though the authors acknowledge that important challenges exist in modeling other types of performance metrics such as thermal comfort, indoor air quality and CFD in the built environment.

Ten Challenges 1. Addressing the building performance gap

With the increasing demand for more energy efficient buildings, the buildings industry is faced with the challenge of ensuring that the energy performance predicted during the design stage is achieved once a building is in use. However, previous studies have identified a significant ‘performance gap’ between designed and actual energy performance of both commercial and residential buildings (Frei et al. 2017; Calì et al. 2016; Van Dronkelaar et al. 2016; Meng 2016; De Wilde 2014), also known as the ‘credibility gap’ (Bordass et al. 2004; Dasgupta et al. 2012). The performance gap between designed and measured energy use is best illustrated by Figure Error! Reference source not found.3, from a study by the New Buildings Institute (Turner and Frankel 2008). Here, it is clear that while measured and design energy use intensities (EUI) are correlated, they often differ substantially from one another in absolute terms. Energy performance gaps may originate in all stages of the building development process, from design to construction to operation. For example, factors such as miscommunication between designers, engineers, and contractors, and inadequate quality control during construction may contribute to observed performance gaps.

Three types of performance gap are identified by Wilde (De Wilde 2014) from the energy calculations perspective: (1) a mismatch between ‘first principle’ energy models and measurements undertaken on actual buildings, (2) a mismatch between data-driven empirical approaches and measurements from real buildings, (3) a mismatch between the energy ratings provided by compliance test methods and energy display certificates as enshrined in regulation. The performance gap is not only limited to energy and has recently been extended to other metrics such as embodied emissions (Pomponi and Moncaster 2018). In this paper, however, the focus is placed on the energy performance gap.

Figure 3 Measured versus Design EUIs (Turner and Frankel 2008).

When faced with an energy performance gap, building owners may suggest the designers mis- specified their energy model, while designers might argue that the building is used in unexpected ways and/or improperly operated and managed (Bordass et al. 2004). The performance gap erodes the credibility of the design and engineering sectors of the building industry; in turn, this leads to skepticism about high-performance building concepts, undermining public confidence in the role of building energy efficiency in national carbon reduction efforts (De Wilde 2014). Indeed, bridging the energy performance gap is essential if designers and engineers are to influence the delivery of high-performance buildings that meet ambitious targets such as zero-net-energy (ZNE, covered in a subsequent section). Bridging this gap will also improve the ability of buildings to adapt to changing use conditions by ‘occupant proofing’ or ‘climate change proofing’ (De Wilde 2014).

Multiple factors are potentially responsible for the energy performance gap. According to IEA Annex 53’s findings, building energy consumption is mainly influenced by six factors: (1) climate (Cui et al. 2017; Hong et al. 2013), (2) building envelope (Fang et al. 2014), (3) building services and energy systems, (4) building operation and maintenance (Lin and Hong 2013), (5) occupant activities and behavior (D’Oca et al. 2018) and (6) indoor environmental quality provided. The latter three factors, related to human behavior, can have an influence as great as or greater than the former three factors, which are building-related (Yoshino et al. 2017).

Indeed, Li studied 51 high-performance office buildings in the US, Europe, China and other parts of Asia, and discovered that climate, building size, or technology do not determine energy use alone; occupant behavior, building operation and maintenance also significantly influence realized energy savings (Li et al. 2014). In particular, occupant behavior has been identified as a major factor contributing to the discrepancy between simulation and real energy use (Yan et al. 2017; Ahn et al. 2017). User-related factors are and have been found to vary substantially from design values in buildings; accordingly, new scientific approaches are needed to describe and quantify the influence of occupant behavior and account for this influence in the building simulation process (see next section). Moreover, the existence of ‘rebound’ and ‘prebound’ effects can further lead to the over- or under- estimation of the effects of occupant behaviors on energy use (Haas and Biermayr 2000; Sorrell et al. 2009; Hens et al. 2010; Galvin 2014; Sunikka-Blank and Galvin 2012).

In recent years, several research efforts have focused on reducing the energy performance gap. Eschewing traditional, deterministic energy simulation, for example, Sun proposed a probabilistic framework of predicting energy consumption using computation-based uncertainty quantification, which shows improvement in model prediction capabilities and reduces prediction errors for case study buildings (Sun 2014). IEA Annex 66 proposed scientific approaches to reduce the energy performance gap by representing occupant behaviors in a standardized quantitative way, going further by integrating simulated behaviors with current BPS programs (Yan et al. 2017). Regarding behavior modeling approaches, Markov Chain (Wang et al. 2011), probabilistic (Sun et al. 2014), and random walk (Ahn et al. 2017) models of occupancy have been proposed.

Post-occupancy evaluation (POE) has also proven essential in understanding the energy performance gap and can potentially be used to inform better predictions, improving the input assumptions used in detailed energy modeling and closing the building performance feedback loop (Van Dronkelaar et al. 2016; Menezes et al. 2012; Choi et al. 2012). POE has been embedded into Building Information Modeling (BIM) to engage different stakeholders in the collaborative effort of continuous building performance improvement (Göçer et al. 2015).

Efforts to bridge the performance gap span the building design stage, construction and operational stage (De Wilde 2014; Jones et al. 2015; Burman et al. 2014; Dasgupta et al. 2012). Regarding the design stage, design guidance and reports have been developed to raise awareness amongst clients and design teams, ensuring that design intent and responsibilities are communicated and leaving no room for error during building construction. Regarding the construction stage, efforts such as Building with Care attempt to increase the quality of the construction delivery process (Tofield 2012). Finally, regarding the operational stage, standardized data collection and monitoring techniques have been used to reduce uncertainty in collected operational data. The handover between the construction and operation stages is also being improved by new programs such as ‘Soft Landings’ (BSRIA and UBT 2014), which was developed in the UK to keep designers and constructors involved in verifying the performance of buildings beyond completion. In order to continue bridging the energy performance gap caused by miscommunication and misalignment of different roles within a building development process, future work should focus on developing integrated methods of building design, construction, operation, and commissioning. Such methods will enable a more thorough and accurate exchange of information across the building life cycle. As stated in the Zero Carbon Hub report, this effort will involve fundamental changes to the traditional building industry (Zero Carbon Hub 2014). At the same , more work is needed to address the issue of occupant behavior, among the strongest influences on the performance gap. Improved representation of occupant behavior in detailed energy modeling requires a better understanding of the nature of occupants’ interactions with different types of buildings, including how occupants use energy and respond to socio/technical energy saving initiatives. This topic is the subject of the next section.

2. Modeling the human-building interaction for occupant-centric building design and operation

Building occupants interact with indoor environments and control systems through their presence in a space and the adaptive actions they take to maintain personal environmental satisfaction. These human-building interactions (HBIs) affect both energy use and occupant comfort outcomes and are therefore central to building design and operation (D’Oca et al., 2018). Occupants’ behavioral interactions with buildings are of a wide variety, including the passive exchange of heat with space; opening and closing doors and windows; adjustment of thermostat settings, light settings, blinds and shades, or clothing levels; use of personal heating and cooling devices; and consumption of warm or cold drinks. Each interaction may be motivated by a number of factors ranging from the physical environment and availability of control options to occupants’ personal preferences and environmental attitudes, social interactions, and broader cultural context (Fabi et al. 2012; Gunay et al. 2013; Langevin et al. 2015).

Modeling capabilities that accurately simulate HBI are needed for both design-stage tools for BPS and controls system software that enables more efficient operation and management of building energy services. In the building design stage, the ability to represent expected occupant behaviors and their effects on simulated energy flows can support design strategies that are robust to these behaviors. In the building operation stage, models of building occupants, their comfort and behavior can support model-predictive control (MPC) and human-in-the-loop (HIL) control schemes that minimize building energy use while maximizing occupant comfort.

Efforts to model the human-building interaction face the scientific challenge of accurately representing behavioral diversity and its potential determinants as well as the practical challenge of implementing such models in widely used building design and operation software. Observed behaviors tend to vary widely across and within occupant populations and can even vary within an individual upon repeated observation (Yan et al. 2015). Moreover, the set of variables that best explains the observed variation in behaviors depends strongly on the particular building context of interest, characteristics of the occupant population, and the type of behavior(s) being studied. Top- down, -based modeling frameworks (Haldi and Robinson 2009, 2010), which are the most tractable to implement as part of BPS program, cannot explicitly represent the causal structures that yield behavioral diversity across individuals and populations. Agent-based models (ABMs) (Azar and Menassa 2012; Langevin et al. 2015; Lee and Malkawi 2014b; Putra et al. 2017), which can simulate individual-level decision-making processes for multiple behaviors at once and social interactions, offer greater flexibility in exploring causality; however, these models require more resources to develop and implement in BPS programs (e.g., more data, modeler time, computing power).

Efforts to put data behind HBI models bring their own challenges (Wagner et al. 2018; Sun and Hong 2017b). Certain occupant behaviors may be difficult or impossible to measure without the use of self-report surveys, which introduce potential recall bias and limit the frequency and duration of measurement campaigns. Moreover, given the wide array of potential occupant behaviors and the heterogeneity in occupant characteristics and building contexts, in situ occupant data collection efforts must target a large sample of occupants to yield broadly representative insights about observed behaviors. In practice, resources may not allow such large-scale sampling of occupants in real building settings. Additionally, cross-sectional (point-in-time) field studies may fail to capture a full range of behavior outcomes or detect statistically significant relationships between these outcomes and other measured variables (e.g., environmental conditions). While longitudinal field studies are better suited to capturing time-resolved variation in behaviors, these studies are expensive to implement and are often constrained to smaller sample sizes. Laboratory experiments offer the most control over occupant sample selection and exposure to environmental conditions, but may omit important aspects of a field setting (e.g., availability of natural light, social interactions).

HBI models and datasets that overcome these challenges will improve the accuracy of building energy modeling and support occupant-centric building control schemes with large energy savings potential. Previous research has established occupant behavior as one of six influencing variables on real building energy use (Yoshino et al. 2017) and a key source of uncertainty in predicting energy use; multiple studies report the sensitivity of simulated energy use outcomes to changes in occupant behavior parameters (up to 150%) (Clevenger and Haymaker 2006; Hong and Lin 2013). Behavior-related energy model inputs also strongly affect simulated indoor environmental conditions and thermal comfort performance (Langevin et al. 2016). Regarding building controls, HBI models can serve as proxies for direct occupant measurements in control schemes that require occupant feedback, such as occupant-based MPC (Mirakhorli and Dong, 2016) and indirect or hybrid human-in-the-loop controls (Munir et al. 2013). Importantly, modeled HBI proxies reduce occupant reporting and measurement burdens, a key barrier to the long-term implementation of such control schemes. Previous studies report the potential for these schemes to yield from 10-40% HVAC and lighting energy savings while maintaining or improving comfort (Ghahramani et al. 2014; Nagy et al. 2015; A. Williams A et al. 2012).

Recent progress in HBI modeling and data collection has been strongly supported by the IEA EBC Annex 66: Definition and Simulation of Occupant Behavior, which has driven rapid growth in the number of studies concerning building occupant behavior (Yan et al. 2017). Progress can be grouped into three categories: fundamental model development, data collection methods, and model integration with building design and operation software. Model development in the Annex has found that equation-based discrete-time or discrete-event Markov and survival models can accurately describe the adjustment of lights, blinds, windows, and the use of plug-in equipment. Agent-based models, though not a focus of the Annex, have continued to grow in use throughout the occupant behavior modeling community. Most ABMs have been used in an exploratory fashion without validation efforts, e.g., (Papadopoulos and Azar, 2016; Putra et al., 2017); attempts at ABM validation have shown promising predictive capabilities, but for limited occupant samples (Langevin et al. 2015).

HBI data collection advances supported through the Annex are categorized as in situ, laboratory and surveys. In situ experiments have benefited from advances in occupant sensing technologies, which include continuous logging of occupant presence and movement, control state (e.g., window position, thermostat setting, light level), and plug loads. Sensors that transmit occupant data wirelessly are now widely available, reducing maintenance burden for longer-term experiments. Nevertheless, up front sensor costs remain high (U.S. Department of Energy 2015b). Online surveys have been used as lower cost substitutes for sensor measurements that can also directly explore the social and psychological determinants of behavior (D’Oca et al. 2017). Emerging approaches include mixed method data collection (Creswell, 2006), which uses both qualitative and quantitative measurement techniques, and immersive virtual environments (IVEs) (Heydarian and Becerik-Gerber 2017), which blend aspects of laboratory and field studies.

State-of-the-art integration of HBI models with widely used building energy simulation programs leverages the Functional Mockup Interface (FMI) standard (Otter et al. 2011) for co-simulation of behavior and energy use. An occupant behavior Functional Mockup Unit (FMU) (obFMU, Hong et al. 2016c) was developed to support the exchange of behavior data in a standardized XML format (obXML, Hong et al. 2015b, 2015c) with building energy simulation programs like EnergyPlus and ESP-r. Here, energy models simulate environmental conditions to use as inputs to the behavior model, while the behavior model provides the energy model with updated control states. Additionally, some studies have attempted to integrate occupant models with building controls systems. For example, the use of predictive occupancy models in MPC schemes has been explored (Mirakhorli and Dong 2016), and Bayesian schemes for learning personalized environmental preference profiles that tune HVAC operation have also been developed (Lee et al. 2017).

Going forward, multiple areas of HBI research development are needed. First, meta-analyses of existing HBI modeling studies should quantitatively compare existing equation- and non-equation- based models using a consistent set of metrics. Metrics might include measures of model accuracy, parsimony, and uncertainty across the range of behaviors typically studied in the residential and commercial building sectors. Ideally, model comparison and validation would be performed by those outside the research team that developed each model, with data that were not used to develop the models (Yan et al. 2017).

Data for meta-analyses should be compiled from the large number of existing field and laboratory HBI studies, a second area of focus for future work. Indeed, while the number of HBI measurement studies has grown dramatically in recent years, a single, easily accessible repository for HBI data does not yet exist. Such repositories have spurred research advancements in related fields – see, for example, the ASHRAE RP-884 Database (de Dear 1998), which supported the development of an adaptive thermal comfort standard and continues to drive progress in thermal comfort modeling. Third, future work should seek further dissemination of HBI modeling capabilities in widely used BPS programs. While methods for co-simulation of behavior and energy models have been successfully demonstrated by tools like obFMU, dynamic HBI modeling capabilities are not yet offered with new builds of major energy simulation engines. Additional work is needed to add HBI modules to standard releases of these engines and associated interfaces like OpenStudio, DesignBuilder, and Sefaira. This work can continue to build on the FMI standard, which allows HBI models to be executed by any software that adheres to the same. Given the large numbers of behavior models that may be selected, HBI menus should encourage a fit-for-purpose approach to model selection (Gaetani et al. 2016).

Finally, future work must re-examine the typical building- or zone-level scale of HBI model application in the face of growing interest in occupant-centric building operations (U.S. Department of Energy 2018b) and in interactions between buildings and the utility grid (Nemtzow 2017). At the occupant scale, advances in both human-in-the-loop control approaches and technologies for localized environmental conditioning (U.S. Department of Energy Advanced Research Projects Agency 2014) demand the representation of individual preferences, actions, and their impacts across a zone or building-level occupant population. At the grid scale, accurate models of hourly energy load shapes (Electric Power Research Institute 2018) that consider HBI are needed to design efficiency programs that shift these loads away from peak periods of energy use while maintaining occupant comfort. Agent-based models, which can generate aggregate-level HBI predictions for building-to-grid operations from individual-level representations of occupant and operator decisions, warrant further consideration for these new scales of model application.

3. Model calibration

As the first challenge section described, previous studies have indicated significant discrepancies between simulated energy use from building energy models and actual measured data (Balaras et al. 2016; Yoshino et al. 2017; Yin et al. 2014; Karlsson et al. 2007; La Fleur et al. 2017; Maile et al. 2012). Again, this undermines confidence in model predictions and curtails adoption of building energy performance tools during design, commissioning, operation (Samuelson et al. 2016; Coakley et al. 2014) and retrofit (Johnson 2017; Heo et al. 2012). To address this issue, building energy models must be improved to closely represent the actual performance of modeled buildings. This can be achieved through model calibration, the process of using an existing BPS program and “tuning” or calibrating various inputs to the program so that observed energy use matches closely with that predicted by the simulation program (Reddy 2006).

Calibration can significantly improve the validity of and confidence in energy models while being used to: (1) compare the cost-effectiveness of ECMs in the design stage, (2) assess various performance optimization measures during the operational stage (Coakley et al. 2014), (3) implement continuous commissioning or fault detection measures to identify equipment malfunction (Reddy 2006; Zibin et al. 2016), and (4) support decision making in existing building retrofits, assessing the benefits and uncertainties associated with each (Reddy 2006; Heo et al. 2012).

While the simulation accuracy of building energy models is determined by thousands of parameters, there are usually limited measured data available as calibration inputs. This makes calibration a highly under-determined and over-parameterized problem without a unique solution. Moreover, the collection of detailed sub-metered data may entail considerable time and cost. Currently, building energy simulation models are considered ‘calibrated’ if they meet the internationally-accepted criteria defined by ASHRAE Guideline 14 (ANSI/ASHRAE 2014). However, there are numerous models that meet these criteria and may be considered “calibrated” for the same building; non-unique solutions therefore remain a key issue with model calibration. In addition, it should be noted that current calibration criteria relate solely to predicted energy consumption and do not account for input parameter uncertainty or inaccuracy, the accuracy of BPS program, or the accuracy of the simulated environment (e.g., profiles) (Coakley et al. 2014).

Manual and automated are the two main categories of model calibration. As manual calibration is a user-driven process of trial and error, it not only requires professional engineering expertise and experience, but is also very time-consuming and cost-ineffective. Therefore, researchers have expended significant effort to improve existing manual calibration procedures to make them more systematic and efficient. In parallel, a number of innovative automated calibration methods have been developed. Recent progress in this area includes:

 Researchers are breaking down the traditional whole building calibration problem into sub- pieces that are easier to solve. For example, Cacabelos et al. divides the entire building into different sub-models and calibrates them separately according to output , delivery energy or power consumption, varying the most influential parameters during different periods of the year (Cacabelos et al. 2017). The results show that the new multi-stage procedure can achieve better accuracy than those obtained with a global calibration. Mihai and Zmeureanu proposed a bottom-up calibration technique based on Building Automation System trend data, which starts with zone level calibration with supply air flow rate to each zone, indoor air temperature and cooling load, followed by AHU level calibration. The results show that the AHU model was calibrated naturally on top of most calibrated zones, which avoids any additional tuning through the trial-and-error method (Mihai and Zmeureanu 2017).  Occupancy patterns and internal loads contribute significantly to the discrepancy between the predicted and actual energy consumption and have thus formed a key focus for input calibration (Sun et al. 2014; Hong et al. 2017a; Liang et al. 2016; Yan et al. 2017; Sun and Hong 2017a). Kim et al. applied the occupancy and plug-load schedules derived from metered electric use data to building energy model calibration, substantially improving the accuracy of building energy modeling results (Kim et al. 2017). Similarly, Lam et al. adopted occupant behavior data mining techniques to generate occupancy schedules, using them in the model calibration process and achieving better calibration accuracy (Lam et al. 2014). Sun et al. proposed a stochastic model to describe overtime occupancy, and used the model to generate overtime occupancy schedules, which were applied to energy model calibration and improved model accuracy (Sun et al. 2014).  Advanced optimization techniques are being used to improve the performance of optimization- based automated calibration methods. For instance, Hong et al. developed an automatic calibration model using the (GA) with the optimization objective of approaching the minimum CV(RMSE) (Hong et al. 2017b). The CV(RMSE) was reduced from 18.10% to 12.62%. Multiple other studies have used the GA algorithm for auto-calibration (Ramos Ruiz et al. 2016) (Andrade-Cabrera et al. 2016; Andrade-Cabrera et al. 2017). The Autotune project developed by Oak Ridge National Laboratory leverages supercomputing, large simulation ensembles, and big data mining with multiple machine learning algorithms to allow auto-calibration of energy simulations (Garrett and New 2015).  Several studies address the computational cost of auto-calibration, which has slowed the adoption of such techniques. Specifically, the computational cost can be reduced by fitting a statistical emulator or meta-model, to replace the physical model. A typical application example is Bayesian calibration, where meta-models are often adopted in combination with Bayesian calibration approaches (Kristensen et al. 2017; Kim and Park 2016). Manfren et al. successfully integrated model-driven and data-driven procedures by training a Gaussian process meta-model with simulation data and using it in a Bayesian calibration process, reducing computational cost without sacrificing much accuracy (Manfren et al. 2013). Lim and Zhai evaluated the performance of five types of meta-models and their effects on the Bayesian calibration based on computing time and calibration accuracy. It was found that all five meta-models significantly reduced the computing time compared with the original process without meta-models; a Gaussian process emulator was found to be the most accurate but most time-consuming approach, while a multiple linear regression model was the fastest approach but showed the worst performance (Lim and Zhai 2017a). Li addressed the issue of high computing time for a standard Gaussian process emulator by introducing a lightweight approach with the linear regression emulator. The regression emulator calibrates more quickly while maintaining similar performance compared to the standard Gaussian process emulator (Li et al. 2016b). Lastly, for optimization-based auto-calibration approaches, feature selection and sampling is a very important step; emerging methods include Latin Hypercube Sampling (Kim and Park 2016; Yun and Song 2017), Markov chain Monte Carlo (MCMC) (Garrett and New 2015) and No-U-Turn-Sampler MCMC (Chong et al. 2017), etc.  Optimization is based on mathematical methods and typically lacks critical inputs from and engineering perspectives, thus sometimes leading to unreasonable calibrated results. Sun et al. addressed this issue by combining the strengths of both manual and automated calibration based on pattern recognition, encompassing more engineering insights and experience than purely mathematical optimization-based methods for auto-calibration (Sun et al. 2016).  An automatic assisted calibration tool was developed that couples building automation system trend data with building commissioning tasks. This tool reduces the considerable time required to manually process and analyze large sets of trend data for use in calibrated simulation (Zibin et al. 2016).

In general, while calibration techniques have improved greatly in recent years, current calibration criteria from ASHRAE guideline 14 may not be sufficient for comprehensively assessing the accuracy of calibration results. Such criteria specify broad ranges of allowable error in the total predicted energy consumption of a building but do not specifically address uncertainty or inaccuracy in input parameters or zone-level environmental data. More comprehensive criteria are needed to accommodate different levels and purposes of model calibration.

Looking ahead, one important area for advances in BPS calibration is Urban Building Energy Modeling (UBEM), which is increasingly used to explore energy efficiency solutions at the urban or district scales (see subsequent section on this topic). As there are at least hundreds of buildings involved in UBEM, it is extremely time-consuming to collect detailed information and calibrate the buildings one-by-one to guarantee the accuracy. Future work must, therefore, shift attention from single building calibration to urban-scale calibration, supporting the growing interest in urban-scale modeling.

The most common approach for formulating a UBEM involves segmenting a building stock into archetypes, characterizing each type, and validating the model by comparing its output to aggregated measured energy consumption. Calibration is needed to define unknown or uncertain parameters in the face of incomplete information about the buildings being modeled. For example, Julia et al. developed a Bayesian methodology to calibrate the parameters of building archetypes using measured energy data. Probabilistic representation was used for parameters with limited or no information, and distributions were updated to a posterior joint distribution that was more representative of the district (Julia et al. 2017). While computational cost might be a concern for such applications of Bayesian calibration to UBEM, other studies mentioned above show a successful reduction of this cost by integration of Bayesian calibration with simplified meta- models. Such advances make Bayesian calibration a promising approach for urban-scale modeling.

4. Modeling building operations, controls, and retrofits

From a life cycle point of view, most of a building’s energy is consumed during its operational phase; thus, it is crucial that building simulation tools be applied during this phase to identify and evaluate impactful energy-saving technologies and strategies. Moreover, in developed countries the buildings sector is dominated by existing buildings. Accordingly, improvement of existing building operation and controls and existing building retrofits are key strategies for reducing the overall energy use of the buildings sector. Three important use cases of building simulation in the building operation, control and retrofit phases are presented as follows.

Energy retrofit analysis

Detailed energy models created using BPS programs can be used to explore and evaluate energy conservation measures (ECMs) for energy retrofit projects. Usually, the basecase model is calibrated using monthly utility bill data before being used to simulate and analyze ECMs. Recent developments in this area include web-based platforms or toolkits that enable easy-to-use energy retrofitting analysis, which in turn informs ECM selection.

For example, in the commercial sector CBES (Hong et al. 2015a) is a web-based energy retrofit analysis toolkit for small-to-medium-sized commercial buildings in California. The tool provides energy benchmarking and three levels of retrofit analysis considering the project goal, data availability, and user experience. The three levels of retrofit analysis are: (1) smart meter data analytics to derive and benchmark electric load to identify no or low-cost operation improvements (Luo et al. 2017), (2) a lookup table style query of ECMs using building high-level information and a pre-simulated large database (Lee et al. 2015), and (3) detailed energy modeling and a pattern-based calibration method (Sun et al. 2016) to evaluate retrofit ECMs. CBES currently offers 82 ECMs for lighting, envelope, plug-in equipment, HVAC, and service hot water retrofit upgrades, using OpenStudio and EnergyPlus to create and run energy models. An extended version of the tool, CBESPro, covers all U.S. climate zones. Regnier et al. (2018) demonstrated the use of EnergyPlus models to assess ECMs for retrofitting a building in Hawaii. Emphasis is on the integrated systems approach to consider all related energy systems and their integrative effects for deep energy retrofit of buildings

On the residential side, Home Energy Saver™ (HES) empowers homeowners and renters to save money by reducing energy use in their homes. HES recommends energy-saving upgrades that are appropriate to each home and make economic sense given the home's climate and local energy prices. HES also estimates the home's carbon footprint and shows how much this footprint may be reduced by energy-saving upgrades. For the urban scale energy retrofit analysis, more tools are emerging; for example, CityBES (Hong et al. 2016a) is a web-based data and computing platform for large-scale energy retrofit analysis of hundreds or thousands of buildings in a city district or entire city (Chen et al. 2017a).

Model-based retro-commissioning

Most buildings do not perform as well in practice as intended by design, as their energy performance levels deteriorate over time. Reasons for this deterioration in performance include faulty construction, malfunctioning equipment, incorrectly configured control systems and inappropriate operating procedures. One approach to addressing this problem is to compare the predictions of an energy simulation model of the building to the measured performance and analyze significant differences to infer the presence and location of faults, a topic that is discussed further in the next section. Model-based retro-commissioning refers to this use of building energy models to help identify and evaluate operation problems in buildings as part of a retro- commissioning process. Calibrated energy models can be a good tool in assisting the measurement and verification (M&V) of a retro-commissioning project. As an example, Marmaras (2014) discussed how building energy models can be used in the retro-commissioning process of an under-performing LEED Gold-level certified police station.

Real-time optimization, control, and fault detection and diagnosis

Building control systems are critical to ensuring efficient operations and occupant comfort. To support building control, BPS is being coupled in real time with building energy monitoring and control systems (EMCS) and sensors, where it is used to predict thermal loads in buildings and provide guidance on energy- and comfort-optimal control strategies (e.g., set point adjustments, charging and discharging of energy storage, demand response strategies).

Typically, real-time building operation data (equipment and systems), predictive weather data, and occupant data are fed to energy models that simulate and evaluate various control strategies across a future time horizon, identifying the control strategy with the best predicted energy and comfort outcomes. This type of model predictive control (MPC) is an advanced method of process control that has been in use in chemical plants and oil refineries since the 1980s, only recently being appropriated for power system balancing models and building controls (Morari and Lee 1999; Salakij et al. 2016). Model predictive controllers rely on dynamic models of the process, most often linear empirical models obtained by system identification. The main advantage of MPC is the fact that it allows the current operation timeslot to be optimized while simultaneously accounting for future timeslots. In an early example, Cumali and Sezgen (1989) introduced a control optimization approach in a high rise office complex, solving nonlinear that represent the building environmental system in real time to identify optimal control strategies, which were subsequently implemented through an EMCS.

More recently, a research project (Piette et al. 2016; Blum and Wetter 2017) under the joint U.S.- China Clean Energy Research Center (CERC) is developing and demonstrating a hierarchical, occupancy-responsive MPC framework that optimizes the operation of buildings and campuses by controlling lighting levels, HVAC operation, indoor air temperature and humidity, indoor environmental quality, window opening, and shading devices. The framework takes into account anticipated weather, occupancy, price signals from the electrical grid or district heating/cooling networks and active and passive measures to store energy and reduce peak loads. The proposed occupancy-responsive MPC technology seamlessly integrates building technologies, controls, and human behavior - a substantial need for zero-net-energy and grid-responsive buildings.

Real-time BPS can also be used to detect and diagnose faults (FDD) in buildings, a topic that is covered in more detail in the next section. For example, Pang et al. (2012) introduced a framework for simulation-based real-time whole building performance assessment. The framework allows comparison of actual and expected building performance in real time using EnergyPlus, the Building Controls Virtual Test Bed (BCVTB) and an Energy Management and Control System (EMCS). Here, an EnergyPlus model determines and reports the expected performance of a building in real time; the BCVTB provides the software platform for acquiring relevant inputs from the EMCS through a BACnet interface; and these inputs are sent to EnergyPlus as well as a database for archiving. Pang et al. (2016) updated the framework to use the open functional mockup interface (FMI) standard. In a separate study, Bonvini et al. (2014) introduced a robust online FDD for HVAC components based on nonlinear state estimation techniques.

Going forward, key challenges to the use of building simulation in improving building operation, control, and retrofit decisions include: (1) the need to create a detailed physics-based energy model of a building in cases with limited data availability (e.g., when as-built drawings and specifications or records of building changes are not available or are in a form that can be easily used), (2) the lack of training data for data-driven or reduced-order models, (3) the need to execute simulation models in real-time, requiring computationally fast processes for data collection, communication, and model execution, and (3) the lack of BPS expertise and/or technical resources among building operation staff or energy managers. Regarding the latter challenge, the use of BPS in model-based retro-commissioning or real-time control requires specialized skills that are still new to the building simulation community - particularly practitioners. Improved practitioner education and pilot demonstration projects are needed to promote and scale up such applications in real-world settings.

5. Modeling operational faults in existing buildings Operational faults are common in existing buildings, leading to decreased energy efficiency and occupant discomfort. It is estimated that poorly maintained and improperly controlled HVAC equipment is responsible for 15% to 30% of energy consumption in commercial buildings. Most buildings, especially those with complex building energy systems, have various degrees and types of operational problems. Mills et al. (2005) analyzed 85 retro-commissioning projects of existing buildings and found a total of 3,500 deficiencies, 11 per building. Correcting these deficiencies through retro-commissioning proved to be cost effective. It is reported that the number of maintenance requests for building energy systems has increased exponentially throughout the past decades, indicating an increase in building operational faults (Cotts et al. 2010). Typical operational faults may come from improper installation, equipment degradation, sensor offset or failures, or control logic problems. Such faults can be grouped into several categories, including: (1) control faults, (2) sensor offset, (3) equipment performance degradation, (4) fouling faults, (5) stuck faults, and (6) others (Cheung and Braun 2015). Figure Error! Reference source not found.Error! Reference source not found.4 (Zhang and Hong 2017) illustrates some common faults in a typical variable air volume (VAV) system with a central plant.

Figure 4 Potential operational faults in a typical VAV system with a central plant (Zhang and Hong 2017).

HVAC operational faults may lead to a considerable discrepancy between actual HVAC operation performance and design expectations (Djuric and Novakovic 2009; Karaguzel et al. 2014; Wang and Cui 2005). A series of questionnaire surveys and interviews conducted by Au-Yong et al. (2014) show the significant influence of poor HVAC operation on occupant comfort, and some maintenance factors are identified that are significantly correlated with occupant satisfaction.

Simulating HVAC operational faults allows for an estimation of the severity of common faults and thus supports decision making about timely fault corrections, which can then enable efficient system operation, improve occupant thermal comfort, reduce equipment downtime, and prolong equipment service life (Comstock et al. 2002; Wang et al. 2013). Such modeling can also support commissioning efforts by providing estimates for potential energy/cost savings that could be achieved by fixing the faults during retro-commissioning.

Quantified information on the impacts and priorities of various coexisting operational faults can be provided to the commissioners or the building management system, resulting in more reasonable and reliable commissioning decisions, especially when budget and staff resources are limited. Moreover, modeling operational faults is critical to achieving more reliable energy model calibrations, as most energy models for existing buildings assume ideal conditions without any operational problems. Specifically, this ability to estimate the severity of common faults is expected to improve the accuracy and transparency of the calibrated model, thereby increasing the analysis accuracy of different retrofit measures (Lam et al. 2014; Hong et al. 2015a).

Various FDD methods have been developed for HVAC operational faults at the component or subsystem level. Cheung and Braun (2015, 2016) developed fault models for a variety of typical building energy system equipment with three modeling techniques: empirical modeling, semi- empirical modeling, and physical modeling. Radhakrishnana et al. (2016) investigated the various constraints of HVAC scheduling and proposed a novel, token-based distributed control/scheduling approach that can account for varying indoor environment and occupant conditions. Zhao et al. (2013) proposed a pattern recognition-based method to detect and diagnose faults in chiller operations, using a one-class classification algorithm. Li et al. (2016a) also investigated chiller operational problems, but with a two-stage, data-driven approach based on linear discriminant analysis. Cai et al. (2014) developed a novel method to analyze the faults of a ground-source heat pump. Cai’s model achieves multi-source information, fusion-based fault diagnosis by deriving Bayesian networks from sensor data. Han et al. (2012) proposed an automated fault detection and diagnosis strategy for vapor-compression refrigeration systems, combining principle component analysis feature extraction and a multiclass support vector machine classification algorithm. The operational faults of several other major HVAC components have also been investigated, such as air handling units (Du and Jin 2008; Gao et al. 2016; Najafi et al. 2012), heat exchangers (Palmer et al. 2016), and fan coil units (Lauro et al. 2014).

Existing methods for fault detection and diagnosis generally fall short of holistically predicting the overall impacts of faults at the building level—an approach that addresses the coupling between various operational components, the synchronized effect between simultaneous faults, and the dynamic nature of fault severity. One recent advance in this area is the addition of new features to EnergyPlus to model HVAC operational faults and simulate their impact on energy use and occupant comfort in buildings (Zhang and Hong 2017). With this addition, EnergyPlus can now represent the whole building impacts of sensor faults (temperature, humidity, pressure, and enthalpy), faults in thermostat and humidistat offsets, economizer damper faults, and the fouling of air filters, coils, cooling towers, chillers, boilers and evaporative coolers.

Modeling operational faults remains a challenge due to an inadequate understanding of the complexity and dynamic nature of faults and limitations in the measured operational data from real building systems and equipment. Going forward, state-of-the-art experimental facilities such as LBNL’s FLEXLAB (Lawrence Berkeley National Laboratory 2018) can be leveraged to generate new data to develop, test and benchmark fault modeling and simulation capabilities in tools like EnergyPlus. Key technical challenges include the development of simple yet robust fault models, the collection of high-quality data to represent fault characteristics, and the integration of physics- based and data-driven methods or models. As larger volumes of data from sensors, meters, energy monitoring and control systems and IoT devices in buildings become available, advanced data analytics, machine learning, and hybrid modeling techniques can be used to extract valuable information for the development and application of novel fault modeling and simulation approaches in BPS programs.

6. Zero-net-energy buildings and grid-responsive buildings

Zero-net-energy buildings

Zero-net-energy (ZNE) buildings, also named net-zero-energy buildings, refer to buildings that are self-sufficient with on-site energy production meeting their energy consumption needs on an annual basis. The definition of ZNE buildings may vary depending on what energy performance metrics are used (U.S. Department of Energy 2015a), for example, annual site (final) energy use and annual source (primary) energy use. In the U.S. and Canada, the number of ZNE buildings is on the rise. New Building Institute’s Getting to Zero database (New Building Institute 2018) lists nearly 500 certified, verified and emerging ZNE buildings projects, reflecting a steep curve upward with the count increasing over 700% since 2012.

There are a number of long-term advantages of moving toward ZNE buildings, including lower environmental impacts, lower operating and maintenance costs, better resiliency to power outages and natural disasters, and improved energy security. Reducing building energy consumption in new building construction or renovation can be accomplished through various means, including integrated design, energy efficiency retrofits, reduced plug loads and energy conservation programs. Reduced energy consumption makes it simpler and less expensive to meet the building’s energy needs with renewable sources of energy.

In its long-term energy efficiency strategic plan, California targets ZNE buildings for all new residential construction by 2020 and all new commercial construction by 2030. The ZNE goals will play a significant role in regulatory agency and utility efforts to promote the achievement of the state’s greenhouse gas reductions goals. ZNE buildings usually adopt energy efficiency technologies and advanced operation and controls to reduce energy demand as much as possible first, then generate on-site renewable power with solar PV or wind turbines. The Road to ZNE (Heschong Mahone Group 2012), lists loading order or ‘steps to ZNE buildings’ including: (1) minimizing building loads, (2) optimizing system efficiency based on equipment efficiency and use, (3) using highest efficiency appliances, (4) optimizing building operations to better meet occupant and energy efficiency needs, (5) improved occupant interactions with the building, and (6) renewable power generation when feasible.

Integrated design approaches and dynamic controls (Arup 2012) are usually adopted for ZNE buildings that optimize energy performance at the system and whole-building levels considering interactions between all energy end-use systems including envelope, HVAC systems, lighting, plug-loads, and domestic/service hot water. In a recent book, Eley (2016) identifies the building types and climates where meeting the ZNE goal will be a challenge and offers solutions for these special cases. One critical challenge is to balance the operation of energy efficiency technologies on the demand side and renewable generation on the supply side. Evaluation and optimization of ZNE design and operation strategies varies for each ZNE project and cannot be done using general rules-of-thumb. Supporting such case-by-case analysis, BPS provides a quantitative evaluation of design alternatives with various levels of complexity to inform decision making. Modeling passive and advanced interactive control strategies (Shen and Hong 2009; Hong and Fisk 2010) for ZNE buildings remains a challenge for BPS programs and users, e.g., natural ventilation, effective use of thermal mass, precooling, phase change materials, radiant cooling/heating systems, dynamic facades integrating needs of daylighting and shading, zonal HVAC systems enabling individual zone on-demand controls (e.g., VRF systems, Hong et al. 2016b), and smart devices managing plug-loads based on occupancy and use.

Grid-responsive buildings

Until recently, buildings have been regarded as pure energy consumers of grid electricity. However, with on-site electricity generation from solar PV and other renewable sources, buildings are now able to produce more electricity than they consume and can feed surplus energy to the grid – e.g., buildings are becoming prosumers rather than consumers. Accordingly, from the perspective of electricity flows, the building-to-grid relationship is moving in two directions. Additionally, with increased deployment of intermittent distributed energy resources like solar PV and wind turbines grid capacity is becoming more variable and uncertain.

Grid-responsive buildings are those that can adjust electricity demand and on-site energy generation based on the dynamic needs of the grid. The ways and means of such grid- responsiveness are found in increased deployment of IoT devices and equipment and human-in- the-loop feedback control strategies. The resulting flexibility in building electricity demands help to avert system stress, enhancing the reliability of the entire power grid.

The design and operation of grid-responsive buildings is challenging because energy cost may be valued differently depending on fast-changing grid conditions. Use of various types of energy and electricity storage is key to serving critical loads in such buildings, which must also be able to operate in partial services modes both in time and space. Rapidly coordinating demand and supply from groups of buildings is necessary for smooth grid operation and security. Viewed from the perspective of BPS tool development, a key challenge is coupling traditional building energy simulation with simulation of renewable energy generation and the utility grid, where the temporal and spatial fidelity of such models can be dramatically different.

Recently, the U.S. Department of Energy’s (DOE) Building Technologies’ Office (BTO) launched a grid-interactive efficient buildings (GEB) initiative to promote research on technologies and policies that enable buildings to be responsive and dispatchable in response to grid needs (Nemtzow 2018). The BTO GEB initiative works closely with DOE’s broader Grid Modernization Initiative (GMI), a comprehensive effort with public and private partners across different DOE offices and national to help shape the future of our nation’s grid.

Energy-positive buildings and zero-net-emission buildings

As buildings become more grid-responsive, the potential for energy-positive and carbon-neutral buildings is also emerging. Such buildings employ advanced technologies, including: building- integrated PV, direct-current driven appliances and HVAC equipment, electric batteries, thermal energy storage, smart thermostats, occupant-based controls, and electric space heating, hot water heating, cooking and drying (e.g., via heat pumps). Modeling these technologies interactively poses a particular challenge for BPS applications.

In these advanced cases, accurately representing both the technologies and behavioral opportunities may be beyond the native modeling capabilities of BPS programs. Accordingly, BPS programs must be flexible to the addition of expanded modeling capabilities - e.g., by coupling with other simulation programs. BPS programs like EnergyPlus already allow users to write custom computer code using an Energy Management System feature, which can enable new control models and/or overwrite existing algorithms for the program. However, such use requires advanced user experience and deep knowledge of a particular BPS program. Coupling BPS with Modelica-based equation-type modeling tools (Wetter 2009, 2015) is another possibility that evidences greater modularity and flexibility in meeting such advanced modeling needs.

7. Urban building energy modeling

In cities, buildings are responsible for up to 70% of total primary energy use. Energy conservation and efficiency improvements constitute a key strategy for achieving cities’ energy and climate goals. To support such improvements, cities and their consultants need urban building energy modeling and analysis tools that combine measured data, physics-based and data-driven models to inform urban energy planning as well as to guide building retrofits at scale.

While there is currently no common definition of urban building energy modeling (UBEM), UBEM usually refers to computational simulation of the performance of a group of buildings in an urban context (from a city block to a district to an entire city) to account for the dynamics of individual buildings and, more importantly, inter-building effects that are coupled with the urban microclimate, providing quantitative insights for urban planning and energy policy making. The concept of urban building performance includes individual building energy performance, occupant comfort, district energy systems, as well as building on-site or community-scale renewable power generation and storage systems.

Accurately representing the urban microclimate constitutes a key challenge for UBEM. The urban microclimate is determined by: (1) local air velocity, temperature and humidity; (2) solar irradiation and specular and diffuse reflections; and (3) surface temperatures of buildings, the ground and the sky, with the respective long-wave radiant exchange between surfaces. While the urban environment has strong influences on building thermal loads, operation strategies (e.g., natural ventilation), on-site renewable power generation (e.g., solar PV), and energy and occupant comfort, buildings also influence the urban environment (for example, buildings emit air and heat to the surrounding urban context). UBEM captures these interactions between buildings and the urban microclimate, and can represent on-site renewable energy generation as well as district energy systems that serve a group of buildings, taking advantage of their thermal load diversity and the potential for heat recovery between buildings. Considering urban buildings as part of whole urban systems (a system of systems) enables greater performance improvements than would be possible given independent consideration of individual buildings. Increasingly, UBEM tools are becoming available with diverse fidelity and requirements of computational resources and user inputs. Recent examples (Keirstead et al. 2012; Reinhart and Davila 2016) include the Urban Building Energy Models (UBEM) (Reinhart and Davila 2016; MIT Sustainable Design Group 2016), the Urban Modeling Interface (UMI) (Reinhart et al. 2013), CitySim (Emmanuel and Jerome 2015), UrbanOpt (National Renewable Energy Laboratory 2018), and the City Building Energy Saver (CityBES) (Hong et al. 2016a, Chen et al. 2017a). Each tool is further described here.

UBEM estimates citywide hourly energy demand from energy simulations of individual buildings in a city, supporting city policy makers to evaluate strategies on urban building energy efficiency. UMI is a Rhino-based design environment for architects and urban planners interested in modeling the environmental performance of neighborhoods and cities with respect to operational and embodied energy use, walkability and daylighting potential. UMI creates EnergyPlus models using simplified zoning and HVAC systems. CitySim uses its own XML schema to represent building information and a reduced order energy models assuming simplified zoning and HVAC systems. UrbanOpt is an analytics platform for high-performance buildings and energy systems within one geographically cohesive area in a city. UrbanOpt uses Openstudio and EnergyPlus to model and evaluate city district planning scenarios. Note that such tools are limited to specific applications, and do not use open data standards, which are key to sharing and exchanging information across a wide array of urban modeling tools.

Another approach by KU Leuven and 3E uses the Modelica-based framework developed for open Integrated District Energy Assessment by Simulation (OpenIDEAS). This approach employs building load profiles to optimize district energy, leveraging Modelica libraries (Fuchs et al. 2015; Wetter et al. 2015) and integrating physics-based modules of systems in a larger context such as district heating/cooling or shared energy infrastructures (Baetens et al. 2012; Baetens et al. 2015).

CityBES is an open web platform for simulating city building energy efficiency. It provides: (1) a GIS-based building performance visualization, (2) portfolio scale building energy benchmarking, and (3) urban scale building energy retrofit modeling, simulation and analysis. CityBES builds upon open city datasets compiled in CityGML which is an international OGC standard for representation and exchange of 3D city models.

While UBEM programs such as CitySim and OpenIDEAS employ reduced order energy models, others use physics-based detailed energy models - e.g., UMI, UrbanOpt and CityBES use EnergyPlus.

Recent advances in UBEM include new features added to EnergyPlus version 8.8 to improve its use for UBEM, including: (1) enabling import and export of external shading results, (2) explicitly considering the long-wave radiant exchange between buildings to address the urban canyon effect, and (3) using urban microclimate conditions to address the urban heat island effect.

Additionally, an exascale computing project (U.S. Department of Energy 2018), Multiscale Coupled Urban Systems, is currently developing a data and computing framework to couple building energy models (EnergyPlus), urban climate models (WRF and NEK5000, National Center for Atmospheric Research 2018; Argonne National Laboratory 2018) and transportation models (Transportation Utility Management System 2018), and to quantify their interdependencies to inform urban planning.

As hundreds or more buildings are involved in a typical urban building energy modeling application, automatic integration of data and simulation tools in a seamless workflow with high- performance computing capabilities remains a challenge for users. Specific issues include:

a. Big data: as large amounts of operational data (at the terabyte scale) become available from buildings and cities, significant effort is needed to quality control the data and integrate them into models and standards that support interoperability across diverse urban analysis tools and applications. b. Modeling and simulation: the interdependencies of city sectors must be further studied by coupling urban system models at various spatial and temporal resolutions, encompassing buildings, the urban microclimate and transportation. c. Computing: UBEM may constitute an exascale computing problem that requires next- generation supercomputers. For example, consider the computing that would be required to run millions of building energy models representing the City of New York in a reasonable time frame (say, up to one hour). d. Workflow: GIS-based visualization of UBEM results is needed to ensure that stakeholder easily understand key takeaways, such that UBEM models can meaningfully inform decision making in a seamless workflow.

8. Modeling the national or regional impacts of building energy efficiency

Moving beyond the urban scale, BPS is relevant to regional and national modeling efforts as well. Indeed, federal and state policy efforts to drive long-term reductions in energy use and CO2 emissions through building energy efficiency require quantitative representations of the national or regional building stock and its energy use under future scenarios of technology deployment. Such building efficiency impact models integrate three classes of information: (1) national building and technology stocks and their change over time; (2) the energy use intensities of installed building equipment, envelope components, and operational routines; and (3) the likelihood of consumer or organization choices to adopt new technologies or operational strategies, or to replace or retrofit existing technologies.

Existing models of the building stock segment buildings by geographic location and physical characteristics (e.g., size, vintage, program type) and apply functions for annual additions and demolitions in each stock segment in order to make projections (Energy and Environmental (E3) 2016; U.S. Department of Energy Building Technologies Office (BTO) 2017; U.S. Energy Information Administration 2017a). Installed bases of equipment in each segment are concurrently represented along with flows into and out of equipment stocks over time as technologies are replaced upon burnout or retrofitted. The overall energy use intensity of a particular segment depends on the rate of turnover in its installed equipment base, the unit-level energy performance level of installed equipment, and physical improvements to the building structure. Here, building-level energy use may be represented implicitly or explicitly by statistical or engineering models, discussed further below. New, more efficient equipment or building components penetrate the installed base over time based on technology choice assumptions, which are often driven by economic considerations about the cost of purchasing and operating the new technologies over the course of their useful lifetimes (e.g., (Wilkerson et al. 2013) and see next section on technology adoption modeling).

National and regional-scale impact models are challenged by the large scale of the modeled phenomena over time and space, which leads to difficulties in collecting and updating the data needed to specify a model with a high degree of geographic granularity and in developing model validation methods for outcomes that span several decades into the future. Regarding the data collection challenge: in the United States there are 125 million homes and 6 million commercial buildings, each of which has a unique equipment inventory, construction characteristics, occupant population, and energy use intensity (U.S. Energy Information Administration 2016, 2017d). Collecting data that are sufficiently representative of this heterogeneous building population and comprehensive enough to inform national-scale stock and energy models requires a robust measurement protocol and substantial buy-in from the building owners who will be asked to provide these data.

From a modeling perspective, efforts to validate the outputs of national or regional impact assessments confront the impossibility of evaluating predictions that extend decades into the future. As a result, key dynamics in the model, such as rates of equipment turnover, technology market penetration rates, and changes in energy prices, are extrapolated from historically-derived relationships (Koomey 2000). When historical data are not available, such assumptions may be based on educated guesses by the modeler and/or expert elicitation. Absent well-established protocols for reporting and validating these assumptions, models are limited to exploratory rather than explanatory and/or predictive use cases. Nevertheless, these model validation issues are rarely highlighted, leading to the incorrect treatment of simulated results as predictions about the future rather than rough indicators of impactful strategies for long-term energy and CO2 curtailment.

The importance of developing high-quality building efficiency impact models lies in the use of such models to frame high-profile energy policy decisions. Indeed, models of national scale energy demand have been used to evaluate participation in international climate agreements (U.S. Energy Information Administration 1998), develop and assess energy use and emissions reduction targets (Williams et al. 2012), and craft technology R&D strategies that are likely to yield long-term energy savings cost-effectively (Farese et al. 2012). Moreover, such models may be useful outside the policy context – for example, for utilities designing building efficiency program measures and incentives, or for businesses seeking to anticipate future trends in the building efficiency market.

Recent progress in building efficiency impact modeling can be grouped into top-down and bottom- up studies (Lim and Zhai 2017b). In top-down studies, historical relationships are derived between aggregate-level energy use and macro-economic indicators (e.g., gross domestic product, price indices), climatic conditions, appliance ownership, and housing stock turnover rates. Top-down approaches benefit from their simplicity and reliance on widely available historical data; however, energy use scenario projections are strongly dependent on historical trends, and the lack of end use- or technology-level energy use disaggregation precludes the assessment of impacts for specific ECMs. Example top-down models include the Global Climate Change Assessment Model (GCAM) (Joint Global Change Research Institute 2017).

By contrast, bottom-up modeling studies use statistical or engineering models to explicitly represent energy end uses at the building level along with key determinants of energy use (e.g., climate, equipment, occupancy, building shell characteristics). Sector-level energy use projections are then developed from stock- or floor area-weighted combinations of the energy use calculated for multiple building types. Given their greater degree of energy use disaggregation, bottom-up models allow the direct assessment of ECMs; however, they require more data to develop than top-down models, and may also be more complex. Example bottom-up models include the EIA National Energy Modeling System (NEMS) (U.S. Energy Information Administration 2017b, 2017c) Scout (U.S. Department of Energy Building Technologies Office (BTO) 2017), ResStock (National Renewable Energy Laboratory 2017), and EnergyPATHWAYS (Energy and Environmental Economics (E3) 2016).

Ongoing building stock and energy data collection efforts include the U.S. EIA Residential Energy Consumption Survey (RECS) (U.S. Energy Information Administration 2017d) and Commercial Building Energy Consumption Survey (CBECS) (U.S. Energy Information Administration 2016), which have been conducted on a nationally representative sample of residential and commercial buildings roughly every four years since the late 1970s. RECS and CBECS data collection includes building characteristics, appliances and equipment, demographics, and energy use. The U.S. Department of Energy’s Building Performance Database (U.S. Department of Energy 2018d), which contains records on the energy-related characteristics of over one million buildings, provides another source of large-scale building energy use data, though these data are not yet nationally representative. In the European Union, a Building Stock Observatory (European Commission - Energy 2018) was recently launched that aggregates national-level studies of the building stock from 20 member countries and establishes a plan for continuous data updating in the future; the data track a similar set of variables to RECS and CBECS.

Future work should explore innovative methods for large-scale stock and energy data collection. By pairing machine learning techniques with GIS data, for example, the physical characteristics of a national-scale building stock can be determined without the need to conduct costly in-person assessments, as is currently done for RECS and CBECS. Moreover, the burden of building owner surveys can be reduced by conducting the surveys online and pairing responses with readings from advanced metering infrastructure (U.S. Energy Information Administration 2015). In parallel with these advances in data collection methods, advances are needed in methods for organizing and sharing available data. For example, existing platforms like the U.S. Department of Energy’s Building Energy Data Specification (BEDES) (U.S. Department of Energy 2018e), which serves as a buildings data dictionary, should be explored as common standards for building stock and energy data exchange.

On the modeling side, protocols must be developed to improve the transparency of impact model elements, development, and validation. While recent modeling progress mostly concerns national- scale analyses, certain tools claim flexibility in extending to regional or state-level analyses, given regionally- or state-specific input data. Without a clear description of model elements, however, development of custom input datasets that are compatible with the model is a substantial burden for utility or state energy analysts. Indeed, available building efficiency impact models range in their geographical scale of applicability, input variable types, and implementation. Description guidelines akin to the ODD protocol (Grimm et al. 2010), which is used to compare agent-based models across disciplines, will help structure a comparison of these disparate impact modeling options. Such protocols can also improve the understanding of approaches to model validation and uncertainty quantification, which are currently not widely published.

Recently, a new IEA EBC Annex was launched (International Energy Agency (IEA) Energy in Buildings and Communities Programme 2018) that seeks to support many of these future research tasks. Specifically, Annex 70 proposes the epidemiological study of large-scale energy demand, which will inform models that estimate changes in this demand due to energy efficiency and occupant behavior measures. The Annex places a particular focus on cataloging existing datasets and models and establishing best practices for new data collection and modeling efforts.

9. Modeling the adoption of energy efficient technologies

As the previous section suggests, forecasts of the regional or national energy and CO2 emissions reduction potential of building efficiency technologies depend on the assumed rates at which the technologies diffuse into targeted segments of building energy use. These rates stem on one hand from technology stock-and-flow dynamics – rates of new construction, retrofits, and replacement, for example – and on the other hand from the behavioral dynamics of consumer or organization technology adoption decisions. Yet, little research has been devoted to developing building technology adoption models and studying their application to forecasts of future building energy use.

Efforts to model building technology adoption decisions are challenged by the broad array of potential adoption drivers and constraints, which may vary by adoption decision type (U.S. Energy Information Administration; 2017b), adopter type (Rogers 1995), and technology type (Jaccard and Dennis 2006). Examples of such variables include: adopter preferences, perceptions of technology attributes, the availability of capital and expertise to implement the technology, social influences, demographic, political, and economic trends, and external constraints on technology installation.

Several modeling frameworks may be used to explain and/or predict technology adoption outcomes (Gilshannon and Brown 1996; P.S. Raju and A.P.S. Teotia Energy 1985; Packey 1993). Indeed, adoption model types range from simple historical analogy approaches, where the market penetration of a new technology is mapped to the historical shares of a similar, existing technology, to agent-based approaches, where adoption decisions are explicitly represented at the level of individual adopters. Nevertheless, little guidance exists on which modeling framework should be chosen for a particular use case.

Additionally, all of these modeling approaches require some degree of supporting data on historical market sales, technology characteristics, adopter characteristics, and/or societal trends; yet, relevant datasets are sparsely organized and differ in their degree of relevance, representativeness, recurrence, and richness (Ratcliffe et al. 2007). Even if these existing data were made more widely accessible and comprehensive, clear trends in technology adoption might take several years or even decades to emerge, and past trends in adoption may not hold for new or emerging technologies that depart substantially from the features of historical precedents. The importance of addressing such challenges through future research is underscored by the clear influence that building technology adoption assumptions have on the outcomes of national-scale energy use projections. In the U.S. Energy Information’s 2014 Annual Energy Outlook (AEO) projections (U.S. Energy Information Administration 2017a), for example, a scenario that assumes adoption of only the best available technologies yields a 20% reduction in Reference Case building energy use by 2040. A study of an earlier AEO version (Wilkerson et al. 2013) similarly explored the effects of both more and less efficient technology choice assumptions on Reference Case outcomes, finding +11%/-14% sensitivities in projected energy use outcomes by 2035. Similar analyses by IEA (International Energy Agency 2017) further evidence the influence of technology choice assumptions on projected energy use and CO2 outcomes, finding that these outcomes are more strongly tied to modeled technology choices than to modeled technology performance improvements.

Recent progress in modeling energy technology adoption mostly concerns the areas of transportation and renewable energy; however, the fewer buildings-focused studies that do exist represent a wide range of modeling approaches. On the simpler end of the spectrum, multiple Technical Support Documents (TSDs) from DOE’s Appliance Standards Program rely on time series and historical analogy models, which project future equipment shipments based on average historical market saturations for the technology in question or - if these historical data are not available – on the saturations for a similar technology with available historical data (Navigant Consulting and Lawrence Berkeley National Laboratory 2011; Navigant Consulting and Pacific Northwest National Laboratory 2014, 2016).

Other studies have relied on diffusion modeling approaches, assuming that the spread of a new technology is driven by a process of innovation (“external influence”) and/or imitation (“internal influence) (Buskirk 2014; Elliott et al. 2004; Farese et al. 2012); cost models, where technology market shares are projected based on tangible costs (e.g., capital cost, operating costs) and intangible costs (e.g., perceived changes to comfort and system responsiveness) (Jaccard and Dennis 2006; U.S. Energy Information Administration 2017b; Weiss et al. 2010), econometric and discrete choice models , where a functional relationship is developed between technology market share and one or more influencing variables (Andrews and Krogmann 2009; Higgins et al. 2014; Kok et al. 2012; Li 2011; Navigant Consulting and Lawrence Berkeley National Laboratory 2014; Noonan et al. 2013; U.S. Energy Information Administration 2017c); and and agent-based models where causal mechanisms behind adoption behavior are explicitly represented at the aggregate or individual adopter level (Lee et al. 2014a; Moglia et al. 2017; Muehleisen et al. 2016; Müller 2013; Nachtrieb 2013; Navigant Consulting 2013; Sopha et al. 2013; Zhang and Nuttall 2007).

The data requirements of the above modeling frameworks range from historical market shares (time series, historical analogy, and diffusion models) to perceived technology attributes, individual-level adoption preferences and decision weights, and contextual factors (discrete choice, system dynamics, and agent-based models). Market share data are available from DOE Appliance Standards TSDs (Navigant Consulting 2017; U.S. Department of Energy Appliance Standards Program 2016) EIA consumption surveys (U.S. Energy Information Administration 2016, 2017d), ENERGY STAR (ENERGY STAR 2017), and several industry associations such as the Consumer Technology Association (CTA) (Consumer Technology Association 2018), American Heating and Refrigeration Institute (AHRI) (American Heating and Refrigeration Institute (AHRI) 2017), and the National Electrical Manufacturers Association (NEMA) (National Electrical Manufacturers Association (NEMA) 2018). DOE, EIA, ENERGY STAR, and AHRI also offer publicly available datasets on technology performance, cost, and/or lifetime characteristics (American Heating and Refrigeration Institute (AHRI) 2018; ENERGY STAR 2018; Navigant Consulting 2017, 2016; U.S. Department of Energy Appliance Standards Program 2016). Consumer and/or organization data are collected through surveys, most notably the Johnson Controls Energy Efficiency Indicator (Institute for Building Efficiency, 2016) and ENERGY STAR Awareness Survey (EPA Office of Air and Radiation 2017). Data on consumer demographics and larger social, economic, and political trends may be obtained from the U.S. Census (U.S. Census Bureau 2017a, 2017b), U.S. Bureau of Economic Analysis (BEA) (U.S. Department of Commerce 2017), and American Council for an Energy Efficient Economy (ACEEE) (ACEEE 2017).

Examined independently, existing building technology adoption models and datasets exhibit a narrow focus on one or a few technology types, predictor variables of interest, or areas of application within the buildings sector; indeed, each of these modeling approaches and datasets has unique strengths and drawbacks. Near term research efforts must accordingly focus on using the strengths of one model type or dataset to mitigate the weaknesses of another. Parallel, long term research efforts could then be dedicated to filling the gaps that are most likely to remain after existing models and data are merged.

Regarding models, areas for potential integration include: using historical analogy models to select diffusion model parameter coefficients for new technologies with little data; using cost models and/or econometric models to provide long range market share potential estimates for diffusion models; and incorporating bottom-up agent adoption dynamics into top-down system dynamics or equation-based models. Some of these model combinations are already observed in the buildings literature, albeit for a limited set of technology types (e.g., see (Farese et al. 2012; Higgins et al. 2012; Jaccard and Dennis 2006; Navigant Consulting and Lawrence Berkeley National Laboratory 2014)).

Similar opportunities for data integration are observed. For example, in the residential sector, ENERGY STAR historical shipments data may be cross referenced with concurrent versions of the ENERGY STAR Awareness Survey, the ENERGY STAR products database, Census demographics data, and ACEEE energy policy environment data. In the commercial sector, Johnson Controls EEI data on efficient measure adoption, adoption barriers, and/or payback preferences may be cross referenced with point-in-time shipments and saturation data (e.g., from TSDs, CBECS, ENERGY STAR). Emerging data sources such as Google Trends and Correlate (Choi and Varian 2009; Google Labs 2011, 2018), Amazon Mechanical Turk (Amazon Inc. 2017), and the Twitter API should be explored for their ability to supplement these traditionally referenced databases.

Looking further ahead, new data collection efforts must anticipate the gaps in key variables that will remain after existing datasets are merged. Here, data on consumer or organization decision preferences is expected to serve as an important area of focus; these data can be generated through large-scale discrete choice experiments (Henser et al. 2015) that elicit parameter weights for explanatory models of technology adoption. Finally, protocols must be developed to guide the selection, verification and validation, and communication of building technology adoption models. Model selection should be determined by the model’s use case, the simulated time horizon, the scope of modeled technologies, and the level of resources available for model development. Model verification and validation must address the difficulty in acquiring long-term technology market share data to validate modeled outcomes against and emphasize the importance of ground-truthing key input assumptions and variable relationships (Koomey 2000). Model communication efforts should seek to clearly describe model inputs, outputs, and key relationships; the data sources used for model development and validation; and the limitations inherent to the modeling approach and data sources (for example, see (Sopha et al. 2013)). Given inevitable gaps in the data that are available for building technology adoption model development and validation, uncertainties in modeled outcomes should be communicated through scenario analysis and quantified where possible using formal statistical techniques.

10. Integrated building performance simulation

The preceding sections suggest a wide field of potential applications for BPS. Ensuring the future flexibility and robustness of BPS across these varied use cases will require greater focus on integration activities across four dimensions: (1) data, (2) domain, (3) tool, and (4) workflow. These opportunities for BPS integration are described further below.

Data integration

During the building life cycle, BPS is used in various ways from early design to detailed design to commissioning, operation and controls to retrofit. Data from all available sources should be integrated under the building information modeling (BIM) framework, which enables the application of one model across multiple simulation cases (Hong et al. 1997). Specifically, an energy model developed to inform early design decisions can be refined as more data are made available in the detailed design or operation phases, allowing the model to inform decisions later on in the building life cycle. In practice, such efforts are hindered by the lack of regulations or policies that require new building projects to submit BIM or energy models of the buildings. By result, most BPS models are not standardized or shared among key stakeholders. This leads to the time-consuming, error-prone, and wasteful effort to recreate multiple models of the same building for different purposes by different users.

Although BIM started decades ago to represent building geometry data and simplified data of thermal loads in buildings, it is still limited in representing HVAC systems, occupant behavior or operational and control data. This leads to problems with storing, managing and integrating these other data sources. Current BIM is also limited in representing simulation results at various levels of spatial and temporal resolutions.

Domain integration

As BPS moves from its application to individual building design and operation to the simulation of grid-responsiveness, communities, and regional or national energy use, multiple technical domains must be integrated such that their interactive effects may be quantified, yielding more holistic assessments across a diverse set of stakeholder needs. Technical domains to be integrated include: (1) energy efficiency of buildings, (2) occupant behavior of energy use and human- building interactions, (3) energy storage, (4) building operation controls, (5) renewable energy, on-site or at the community scale, (6) demand response and grid-responsive strategies, (7) indoor environmental quality including thermal comfort, visual comfort and indoor air quality, and (8) water use in buildings.

Simulation tool integration

Modeling and simulation efforts that span multiple technical domains usually require the use of several different simulation tools, which may cover building energy flows (e.g., EnergyPlus), distributed energy resources (e.g., DER-CAM), CFD (e.g., FLUENT), grid conditions (e.g., the Integrated Grid Modeling System IGMS), and human behavior (e.g., agent-based modeling tool AnyLogic, obXML and obFMU (Hong et al. 2015b, 2015c), Occupancy Simulator (Chen et al. 2017b)). Various approaches have been developed (Trcka et al. 2009; Wetter 2011) to couple cross-domain tools through co-simulation. Co-simulation using the functional mockup interface and functional mockup units shows particular promise: here, two simulators solve coupled differential-algebraic systems of equations and exchange data that couples these equations during the time integration. Additionally, visualization of the co-simulation process and results across simulation tools is important for supporting design decision making. Chen et al. (2017b) provide an example of simulating and visualizing occupant behavior and its impact on building performance.

Workflow integration

Within the buildings industry, companies and consultants each use their own workflows and suite of tools to support decision making on buildings projects across the building life cycle. Integrating new BPS programs with these existing workflows and tools (e.g., CRM, finance tools, databases) is a challenge from a business perspective. A particular issue for new BPS programs is the need for data exchange and interoperability with existing tools, such that no duplicate data need be collected or re-entered for existing BPS applications. Integrating BPS across stakeholders from multiple firms (architects, engineers, energy consultants and building owners) brings the additional challenge of data privacy and IP ownership concerns. In this area, web-based tools and the integration of web services for businesses are becoming popular.

Summary and Future Perspectives

Over the past decade, building performance simulation (BPS) has emerged as a crucial tool for the design and operation of low energy buildings and communities. The selected ten challenges aim to highlight some of the most important technical needs currently facing BPS, covering the full building life cycle and a wide range of modeling scales of focus. The formulation and discussion of each challenge aims to provide insights into the state-of-the-art for the given topic and future research directions, and to inspire new questions from young researchers in the field. In addition to these research-level needs, several practical barriers to BPS adoption and implementation warrant further discussion here. An overarching practical issue that most energy modelers face is the time and effort required to collect adequate data and develop reliable energy models. Detailed energy modeling using today’s BPS programs requires many inputs, and modelers may not have full knowledge of each input’s relative importance to simulation outcomes, level of uncertainty, and the appropriate default values to use (if not already specified). This issue is exacerbated when actual or realistic data (e.g., occupancy, operational schedules, infiltration) are not available and the use of a typical input value or assumption is not appropriate for the application case. The issue can be addressed by developing new standards for collecting and sharing input data for energy models – for example, ASHRAE SPC 205, Standard Representation of Performance Simulation Data for HVAC&R and Other Facility Equipment.

Additionally, while BPS is often beneficial to use for building design and operation, this is not always the case. For example: when a project lacks the time, budget or expertise to develop sound energy models; when a project does not have the buy-in or support from key stakeholders (e.g., building owners, architects and engineers); or when rules-of-thumb and recent experiences are sufficient for conventional design needs.

Finally, BPS programs that originate from sophisticated research problems are only valuable in the long term if they are of interest to a broad set of users. Regarding this point, the aforementioned building performance gap is important to address going forward, as it affects the perceived credibility of BPS and weakens the justification for its widespread use by building practitioners. This problem must be addressed through a dedicated, interdisciplinary effort that engages stakeholders spanning research, academia and industry. In parallel, BPS value propositions must be communicated amongst these stakeholders and reflected through building codes and standards, rating schemes, policy and regulations. Furthermore, best practices, education and training, and professional certification programs for BPS practitioners should be enhanced to highlight the quality and value of BPS among its potential user base.

BPS is presently entering a new era of research and application, given more affordable and powerful computing resources and the rapid development of IoT, big data, machine learning and artificial intelligence. In the future, we believe BPS will provide unprecedented value to the design and operation of low energy buildings and communities that address timely issues of resource efficiency, environmental sustainability, and resiliency in the built environment. Under this vision, every new building will be virtually designed and tested using building information modeling, computational modeling and simulation, and virtual reality technologies, and will be operated using augmented reality and machine learning-driven predictive controls to achieve ambitious energy performance goals.

Acknowledgment

This work was supported by the Assistant Secretary for Energy Efficiency and Renewable Energy of the U.S. DOE under Contract No. DE-AC02-05CH11231.

References ACEEE. (2017). The State Energy Efficiency Scorecard. Retrieved August 17, 2017, from http://aceee.org/state-policy/scorecard Ahn KU, Kim DW, Park CS, de Wilde P (2017). Predictability of occupant presence and performance gap in building energy simulation. Applied Energy. http://doi.org/10.1016/j.apenergy.2017.04.083 Amazon Inc. (2017). Amazon Mechanical Turk. Retrieved August 17, 2017, from https://www.mturk.com/mturk/welcome American Heating and Refrigeration Institute (AHRI). (2017). Historical Data. Retrieved August 15, 2017, from http://www.ahrinet.org/Resources/Statistics/Historical-Data.aspx American Heating and Refrigeration Institute (AHRI). (2018). AHRI Certification Directory. Retrieved August 15, 2017, from https://www.ahridirectory.org/ahridirectory/pages/home.aspx Andrews CJ, Krogmann U (2009). Explaining the adoption of energy-efficient technologies in U.S. commercial buildings. Energy and Buildings, 41(3), 287–294. https://doi.org/10.1016/j.enbuild.2008.09.009 Andrade-Cabrera C, Burke D, Turner WJN, Finn DP (2017). Ensemble Calibration of lumped parameter retrofit building models using Particle Swarm Optimization. Energy and Buildings, 155, 513–532. http://doi.org/10.1016/j.enbuild.2017.09.035 Andrade-Cabrera C, Turner WJN, Burke D, Neu O, Finn DP (2016). Lumped Parameter Building Model Calibration using Particle Swarm Optimization. 3rd Asia Conference of International Building Performance Simulation Association (ASIM 2016). ANSI/ASHRAE (2014). ASHRAE Guideline 14-2014 Measurement of Energy, Demand, and Water Savings. American Society of Heating, Refrigerating and Air-Conditioning Engineers. Argonne National Laboratory (2018). NEK5000. https://nek5000.mcs.anl.gov. Accessed January 14, 2018. Arup (2012). The Technical Feasibility of Zero Net Energy Buildings in California. http://www.energydataweb.com/cpucFiles/pdaDocs/904/California_ZNE_Technical_Feasibility_Re port_Final.pdf. Accessed January 28, 2018. Au-Yong CP, Ali AS, Ahmad F (2014). Improving occupants’ satisfaction with effective maintenance management of HVAC system in office buildings, Autom. Constr. 43 (2014) 31–37. doi:10.1016/j.autcon.2014.03.013. Azar E, Menassa CC (2012). Agent-Based Modeling of Occupants and Their Impact on Energy Use in Commercial Buildings. Journal of Computing in Civil Engineering, 26(4), 506–518. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000158 Baetens D, De Coninck R, Jorissen R, Picard F, Helsen D, Saelens L (2015). OpenIDEAS - An Open Framework for Integrated District Energy Simulations. International Buildings Performance Simulation Association. Baetens R, De Coninck R, Van Roy J, Verbruggen B, Driesen J, Helsen L, Saelens D (2012). Assessing electrical bottlenecks at feeder level for residential net zero-energy buildings by integrated system simulation. Applied Energy, 96:74–83. Balaras CA, Dascalaki EG, Droutsa KG, Kontoyiannidis S (2016). Empirical assessment of calculated and actual heating energy use in Hellenic residential buildings. Applied Energy, 164, 115–132. http://doi.org/10.1016/j.apenergy.2015.11.027 BEMbook (2018). History of Building Energy Modeling. bembook.ibpsa.us. Retrieved January 20, 2018. Blum D, Wetter M (2017). MPCPy: An open-source software platform for model predictive control in buildings. Proceedings of the 15th Conference of International Building Performance Simulation, Aug 7 – 9, 2017. San Francisco, CA. Bonvini M, Sohn MD, Granderson J, Wetter M, Piette MA (2014). Robust on-line fault detection diagnosis for HVAC components based on nonlinear state estimation techniques. Applied Energy, 124(1):156- 166, 2014. Bordass B, Associates WB, Cohen R (2004). Energy Performance of Non-Domestic Buildings: Closing the Credibility Gap. 8th International Conference on Improving Energy Efficiency in Commercial Buildings, 1–10. BSRIA, UBT (2014). the Soft Landings Framework. Burman E, Mumovic D, Kimpian J (2014). Towards measurement and verification of energy performance under the framework of the European directive for energy performance of buildings. Energy, 77, 153– 163. http://doi.org/10.1016/j.energy.2014.05.102 Buskirk RV (2014). Estimating Energy Efficiency Technology Adoption Curve Elasticity with Respect to Government and Utility Development Program Indicators. Berkeley, CA. https://doi.org/10.1016/0301-4215(93)90208-W Cacabelos A, Eguía P, Febrero L, Granada E. (2017). Development of a new multi-stage building energy model calibration methodology and validation in a public library. Energy and Buildings, 146, 182– 199. http://doi.org/10.1016/j.enbuild.2017.04.071 Cai B, Liu Y, Fan Q, Zhang Y, Liu Z, Yu S, Ji R (2014). Multi-source information fusion based fault diagnosis of ground-source heat pump using Bayesian network, Appl. Energy. 114 (2014) 1–9. doi:10.1016/j.apenergy.2013.09.043. Calì D, Osterhage T, Streblow R, Müller D (2016). Energy performance gap in refurbished German dwellings: Lesson learned from a field test. Energy and Buildings, 127, 1146–1158. http://doi.org/10.1016/j.enbuild.2016.05.020 California Public Utilities Commission (2008). California long term energy efficiency strategic plan. http://www.cpuc.ca.gov/WorkArea/DownloadAsset.aspx?id=5305. Accessed January 28, 2018. Chen Y, Hong T, Piette MA (2017a). Automatic Generation and Simulation of Urban Building Energy Models Based on City Datasets for City-Scale Building Retrofit Analysis. Applied Energy, 205:323- 335. Chen Y, Hong T, Luo X (2017b). An agent-based stochastic occupancy simulator. Building Simulation, 2017. Cheung H, Braun JE (2016). Empirical modeling of the impacts of faults on water-cooled chiller power consumption for use in building simulation programs, Appl. Therm. Eng. 99 (2016) 756–764. doi:10.1016/j.applthermaleng.2016.01.119. Cheung H, Braun JE (2015). Development of Fault Models for Hybrid Fault Detection and Diagnostics Algorithm, National Renewable Energy Laboratory, Boulder, CO. Choi JH, Loftness V, Aziz A (2012). Post-occupancy evaluation of 20 office buildings as basis for future IEQ standards and guidelines. Energy and Buildings, 46, 167–175. http://doi.org/10.1016/j.enbuild.2011.08.009 Choi H, Varian H (2009). Predicting the Present with Google Trends. Retrieved from https://static.googleusercontent.com/media/www.google.com/en//googleblogs/pdfs/google_predictin g_the_present.pdf Chong A, Lam KP, Pozzi M, Yang J (2017). Bayesian calibration of building energy models with large datasets. Energy and Buildings, 154, 343–355. http://doi.org/10.1016/j.enbuild.2017.08.069 Clarke JA (2001). Energy simulation in building design. Oxford: Butterworth-Heinemann. ISBN 0750650826. OCLC 46693334. Clarke JA, Hensen J (2015). Integrated building performance simulation: progress, prospects and requirements. Building and Environment, 91: 294-306. Clarke JA (2015). A vision for building performance simulation: a position paper prepared on behalf of the IBPSA Board. Building Performance Simulation, 8(2): 39-43. Clevenger CM, Haymaker J (2006). The Impact Of The Building Occupant On Energy Modeling Simulations. In Joint International Conference on Computing and Decision Making in Civil and Building Engineering, 1–10. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000323 Coakley D, Raftery P, Keane M (2014). A review of methods to match building energy simulation models to measured data. Renewable and Sustainable Energy Reviews, 37, 123–141. http://doi.org/10.1016/j.rser.2014.05.007 Comstock MC, Braun JE, Groll EA (2002). A survey of common faults for chillers, ASHRAE Trans. 108 (2002) 819–825. Cotts D, Roperathy K, Payant R (2010). The Facility Management Handbook, 3rd ed., American Management Association. Consumer Technology Association. (2018). Historical Sales Data. Retrieved August 16, 2017, from https://www.cta.tech/Research-Standards/Historical-Sales-Data.aspx Creswell JW (2006). The Foundations of Mixed Methods Research. In Designing and Conducting Mixed Methods Research (pp. 19–52). Cui Y, Yan D, Hong T, Xiao C, Luo X, Zhang Q. Comparison of typical and multiyear building simulations using a 55-year actual weather data set from China. Applied Energy 195 (2017) 890–904. Cumali Z, Sezgen O (1989). Simulation and optimization in the real time control of building environmental systems. Proceedings of IBPSA Building Simulation Conference. Dasgupta A, Prodromou A, Mumovic D (2012). Operational versus designed performance of low carbon schools in England: Bridging a credibility gap. HVAC and R Research, 18(1–2), 37–50. http://doi.org/10.1080/10789669.2011.614318 De Wilde P (2014). The gap between predicted and measured energy performance of buildings: A framework for investigation. Automation in Construction, 41, 40–49. http://doi.org/10.1016/j.autcon.2014.02.009 de Dear RJ (1998). Global database of thermal comfort field experiments. In ASHRAE Transactions (Vol. 104, pp. 1141–1152). Djuric N, Novakovic V (2009). Review of possibilities and necessities for building lifetime commissioning, Renew. Sustain. Energy Rev. 13 (2009) 486–492. doi:10.1016/j.rser.2007.11.007. D’Oca S, Chen CF, Hong T, Belafi Z (2017). Synthesizing building physics with social : An interdisciplinary framework for context and occupant behavior in office buildings. Energy Research and , 34, 240–251. https://doi.org/10.1016/j.erss.2017.08.002 D’Oca, S., Hong, T., Langevin J. (2018). The Human Dimensions of Energy Use in Buildings: A Review. Renewable and Sustainable Energy Reviews, 81, 732-742. Du Z, Jin X (2008). Multiple faults diagnosis for sensors in air handling unit using Fisher discriminant analysis, Energy Convers. Manag. 49 (2008) 3654–3665. doi:10.1016/j.enconman.2008.06.032. Eley C (2016). Design Professional's Guide to Zero Net Energy Buildings. ISBN: 9781610917636. Island Press. Electric Power Research Institute. (2018). Load Shape Library. Retrieved January 21, 2018, from http://loadshape.epri.com/ Elliott DB, Anderson DM, Belzer DB, Dirks JA (2004). Methodological Framework for Analysis of Buildings-Related Programs : The GPRA Metrics Effort. Richland, WA. Emmanuel W, Jérôme K (2015). A verification of CitySim results using the BESTEST and monitored consumption values. Building Simulation Applications Conference. Energy and Environmental Economics (E3). (2016). Introduction — energyPATHWAYS documentation. Retrieved January 21, 2018, from http://energypathways.readthedocs.io/en/latest/introduction.html ENERGY STAR. (2017). Unit Shipment Data. Retrieved August 15, 2017, from https://www.energystar.gov/index.cfm?c=partners.unit_shipment_data ENERGY STAR. (2018). ENERGY STAR Certified Products. Retrieved August 15, 2017, from https://data.energystar.gov/ EPA Office of Air and Radiation. (2017). National Awareness of ENERGY STAR for 2016: Analysis of 2016 CEE Household Survey. Washington, D.C. European Commission - Energy. (2018). EU Building Stock Observatory. Retrieved January 21, 2018, from https://ec.europa.eu/energy/en/eubuildings Fabi V, Andersen RV, Corgnati S, Olesen BW (2012). Occupants’ window opening behaviour: A literature review of factors influencing occupant behaviour and models. Building and Environment, 58, 188– 198. https://doi.org/10.1016/j.buildenv.2012.07.009 Fang Z, Li N, Li B, Luo G, Huang Y (2014). The effect of building envelope insulation on cooling energy consumption in summer. Energy and Buildings, 77, 197–205. Farese P, Gelman R, Hendron R (2012). A Tool to Prioritize Energy Efficiency Investments. National Renewable Energy Laboratory, (August), 140. Retrieved from www.nrel.gov/docs/fy12osti/54799.pdf? Frei B, Sagerschnig C, Gyalistras D (2017). Performance gaps in Swiss buildings : an analysis of conflicting objectives and mitigation strategies. Energy Procedia, 122, 421–426. http://doi.org/10.1016/j.egypro.2017.07.425 Fuchs M, Teichmann J, Lauster M, Streblow R, Müller D (2015). Approach for simulation-based scenario analyses of district energy systems. 28th International Conference on Efficiency, Cost, Optimization, Simulation and Environmental Impact of Energy Systems. Gaetani I, Hoes PJ, Hensen JLM (2016). Occupant behavior in building energy simulation: Towards a fit- for-purpose modeling strategy. Energy and Buildings, 121, 188–204. https://doi.org/10.1016/j.enbuild.2016.03.038 Galvin R (2014). Making the “rebound effect” more useful for performance evaluation of thermal retrofits of existing homes: Defining the “energy savings deficit” and the “energy performance gap.” Energy and Buildings, 69, 515–524. http://doi.org/10.1016/j.enbuild.2013.11.004 Garrett A, New J (2015). Scalable tuning of building models to hourly data. Energy, 84, 493–502. http://doi.org/10.1016/j.energy.2015.03.014 Gao D, Wang S, Shan K, Yan C (2016). A system-level fault detection and diagnosis method for low delta- T syndrome in the complex HVAC systems, Appl. Energy. 164 (2016) 1028–1038. doi:10.1016/j.apenergy.2015.02.025. Ghahramani A, Jazizadeh F, Becerik-Gerber B (2014). A knowledge based approach for selecting energy- aware and comfort-driven HVAC temperature set points. Energy and Buildings, 85, 536–548. https://doi.org/10.1016/j.enbuild.2014.09.055 Gilshannon ST, Brown DR (1996). Review of methods for forecasting the market penetration of new technologies. U.S. Department of Energy, Pacific Northwest Laboratory, PNNL-11428, 1–63. https://doi.org/10.2172/432867 Göçer Ö, Hua Y, Göçer K (2015). Completing the missing link in building design process: Enhancing post- occupancy evaluation method for effective feedback for building performance. Building and Environment, 89, 14–27. http://doi.org/10.1016/j.buildenv.2015.02.011 Google Labs. (2011). Google Correlate. Retrieved August 17, 2017, from https://www.google.com/trends/correlate Google Labs. (2018). Google Trends. Retrieved August 17, 2017, from https://trends.google.com/trends/ Grimm V, Berger U, DeAngelis DL, Polhill JG, Giske J, Railsback SF (2010). The ODD protocol: A review and first update. Ecological Modelling, 221(23), 2760–2768. https://doi.org/10.1016/j.ecolmodel.2010.08.019 Gunay HB, O’Brien W, Beausoleil-Morrison I (2013). A critical review of observation studies, modeling, and simulation of adaptive occupant behaviors in offices. Building and Environment. https://doi.org/10.1016/j.buildenv.2013.07.020 Haldi F, Robinson D (2009). Interactions with window openings by office occupants. Building and Environment, 44(12), 2378–2395. https://doi.org/10.1016/j.buildenv.2009.03.025 Haldi F, Robinson D (2010). Adaptive actions on shading devices in response to local visual stimuli. Journal of Building Performance Simulation, 3(2), 135–153. https://doi.org/10.1080/19401490903580759 Han H, Cao Z, Gu B, Ren N (2012). PCA-SVM-Based Automated Fault Detection and Diagnosis (AFDD) for Vapor-Compression Refrigeration Systems, HVAC&R Res. 16 (2012) 295–313. doi:10.1080/10789669.2010.10390906. Harish VSKV, Kumar A (2016). A review on modeling and simulation of building energy systems. Renewable and Sustainable Energy Reviews. 56:1272-1292. Haas R, Biermayr P (2000). The rebound effect for space heating empirical evidence from Austria. Energy Policy, 28(6–7), 403–410. http://doi.org/10.1016/S0301-4215(00)00023-9 Hens H, Parijs W, Deurinck M (2010). Energy consumption for heating and rebound effects. Energy and Buildings, 42(1), 105–110. http://doi.org/10.1016/j.enbuild.2009.07.017 Hensen J, Lamberts R (2011). Building Performance Simulation for Design and Operation. ISBN 978-0- 415-47414-6. Spon Press. Henser DA, Rose JM, Greene WH (2015). Applied Choice Analysis: A Primer (2nd ed.). Cambridge, UK: Cambridge University Press. Heo Y, Choudhary R, Augenbroe GA (2012). Calibration of building energy models for retrofit analysis under uncertainty. Energy and Buildings, 47, 550–560. http://doi.org/10.1016/j.enbuild.2011.12.029 Heschong Mahone Group (2012). Road to ZNE: Mapping Pathways to ZNE Buildings in California. https://energydesignresources.com/media/18918368/hmg-road-to-zne-final- report_withappendices.pdf. Accessed January 28, 2018. Heydarian A, Becerik-Gerber B (2017). Use of immersive virtual environments for occupant behaviour monitoring and data collection. Journal of Building Performance Simulation, 10(5–6), 484–498. https://doi.org/10.1080/19401493.2016.1267801 Higgins A, Paevere P, Gardner J, Quezada G (2012). Combining choice modelling and multi-criteria analysis for technology diffusion: An application to the uptake of electric . Technological Forecasting and Social Change, 79(8), 1399–1412. https://doi.org/10.1016/j.techfore.2012.04.008 Higgins A, Syme M, McGregor J, Marquez L, Seo S (2014). Forecasting uptake of retrofit packages in office building stock under government incentives. Energy Policy, 65, 501–511. https://doi.org/10.1016/j.enpol.2013.10.041 Hong (2018). IEA EBC annexes advance technologies and strategies to reduce energy use and GHG emissions in buildings and communities, Energy and Buildings 158 (2018) 147–149. (editorial). Hong T, Yan D, D’Oca S, Chen CF (2017a). Ten questions concerning occupant behavior in buildings: The big picture. Building and Environment, 114, 518–530. http://doi.org/10.1016/j.buildenv.2016.12.006 Hong T, Kim J, Jeong J, Lee M, Ji C (2017b). Automatic calibration model of a building energy simulation using optimization algorithm. Energy Procedia, 105, 3698–3704. http://doi.org/10.1016/j.egypro.2017.03.855 Hong T, Chen Y, Lee S, Piette MA (2016a). CityBES: A Web-based Platform to Support City-Scale Building Energy Efficiency, Urban Computing 2016. Hong T, Sun K, Zhang R, Hinokuma R, Kasahara S, Yura Y (2016b). Development and Validation of a New Variable Refrigerant Flow System Model in EnergyPlus. Energy and Buildings, 117 (2016) 399– 411. Hong T, Sun H, Chen Y, Taylor-Lange SC, Yan D (2016c). An occupant behavior modeling tool for co- simulation. Energy and Buildings, 117, 272–281. https://doi.org/10.1016/j.enbuild.2015.10.033 Hong T, Piette MA, Chen Y, Lee SH, Taylor-Lange SC, Zhang R, Sun K, Price P (2015a). Commercial Building Energy Saver: An energy retrofit analysis toolkit, Appl. Energy. 159 (2015) 298–309. doi:10.1016/j.apenergy.2015.09.002. Hong T, D'Oca S, Taylor-Lange SC, Turner W, Chen Y, Corgnati S (2015b). An ontology to represent energy-related occupant behavior in buildings. Part II: Implementation of the DNAS framework using an XML schema. Building and Environment 94 (2015) 196-205. Hong T, D'Oca S, Turner W, Taylor-Lange SC (2015c). An ontology to represent energy-related occupant behavior in buildings. Part I: Introduction to the DNAs Framework. Building and Environment 92 (2015) 764-777. Hong T, Lin HW (2013). Occupant Behavior: Impact on Energy Use of Private Offices. ASim 2012 - 1st Asia Conference of International Building Performance Simulation Association., 12. Hong T, Chang W, Lin H (2013). A Fresh Look at Weather Impact on Peak Electricity Demand and Energy Use of Buildings Using 30-Year Actual Weather Data, Applied Energy 111 (2013) 333–350. Hong T, Fisk W (2010). Assessment of energy savings potential from the use of demand controlled ventilation in general office spaces in California, Building Simulation, 2010. Hong T, Chou SK, Bong TY (2000). Building simulation: an overview of development and information sources, Building and Environment, 35(4): 347-361. Hong T, Zhang J, Jiang Y (1997). IISABRE: An integrated building simulation environment, Building and Environment, No.3, 1997. Institute for Building Efficiency. (2016). 2016 Energy Efficiency Indicator Survey. Milwaukee, WI. International Energy Agency. (2017). Energy Technology Perspectives 2017: Catalysing Energy Technology Transformations. https://doi.org/10.1787/energy_tech-2017-en International Energy Agency (IEA) Energy in Buildings and Communities Programme. (2018). IEA-EBC Annex 70 – Building Energy Epidemiology: Analysis of Real Building Energy Use at Scale. Retrieved September 21, 2017, from https://energyepidemiology.org/ Jaccard M, Dennis M (2006). Estimating home energy decision parameters for a hybrid energy-economy policy model. Environmental Modeling and Assessment, 11(2), 91–100. https://doi.org/10.1007/s10666-005-9036-0 Johnson N (2017). Building Energy Model Calibration for Retrofit Decision Making by Nicolas R . Johnson A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in Mechanical Engineering Thesis Committee : Sung Yi , Chair Elliott Ga. Joint Global Change Research Institute. (2017). Global Change Assessment Model. Retrieved January 21, 2018, from http://www.globalchange.umd.edu/gcam/ Jones RV, Fuertes A, Wilde P de. (2015). THE GAP BETWEEN SIMULATED AND MEASURED ENERGY PERFORMANCE : A CASE STUDY ACROSS SIX IDENTICAL NEW-BUILD FLATS IN THE UK Rory V . Jones , Alba Fuertes , Pieter de Wilde Building Performance Analysis Group , School of Architecture , Design and Environment ,. International Conference of the International Building Performance Simulation Association. Julia S, Carlos CD, Christoph F. R (2017). Validation of a Bayesian-based method for defining residential archetypes in urban building energy models. Energy and Buildings, 134, 11–24. http://doi.org/10.1016/j.enbuild.2016.10.050 Karaguzel OT, Zhang R, Lam KP (2014). Coupling of whole-building energy simulation and multi- dimensional numerical optimization for minimizing the life cycle costs of office buildings, Build. Simul. 7 (2014) 111–121. doi:10.1007/s12273-013-0128-5. Karlsson F, Rohdin P, Persson ML (2007). Measured and predicted energy demand of a low energy building: important aspects when using Building Energy Simulation. Building Services Engineering Research and Technology, 28(3), 223–235. http://doi.org/10.1177/0143624407077393 Keirstead J, Jennings M, Sivakumar A (2012). A review of urban energy system models: Approaches, challenges and opportunities. Renewable and Sustainable Energy Reviews, 16(6):3847–3866. Kim YS, Heidarinejad M, Dahlhausen M, Srebric J (2017). Building energy model calibration with schedules derived from electricity use data. Applied Energy, 190, 997–1007. http://doi.org/10.1016/j.apenergy.2016.12.167 Kim YJ, Park CS (2016). Stepwise deterministic and stochastic calibration of an energy simulation model for an existing building. Energy and Buildings, 133, 455–468. http://doi.org/10.1016/j.enbuild.2016.10.009 Kok N, McGraw M, Quigley JM (2012). The diffusion over time and space of energy efficiency in building. Annals of Regional Science, 48(2), 541–564. https://doi.org/10.1007/s00168-011-0494-9 Koomey J (2000). Avoiding “the big mistake” in forecasting technology adoption. Proceedings of the Energex Conference, 69, 511–518. https://doi.org/10.1016/S0040-1625(01)00153-6 Kristensen MH, Choudhary R, Petersen S (2017). Bayesian calibration of building energy models: Comparison of predictive accuracy using metered utility data of different temporal resolution. Energy Procedia, 122, 277–282. http://doi.org/10.1016/j.egypro.2017.07.322 La Fleur L, Moshfegh B, Rohdin P (2017). Measured and predicted energy use and indoor climate before and after a major renovation of an apartment building in Sweden. Energy and Buildings, 146, 98–110. http://doi.org/10.1016/j.enbuild.2017.04.042 Lam KP, Zhao J, Ydstie EB, Wirick J, Qi M, Park J (2014). An EnergyPlus whole building energy model calibration method for office buildings using occupant behavior data mining and empirical data, in: 2014 ASHRAE/IBPSA-USA Build. Simul. Conf., 2014: pp. 160–167. Langevin J, Gurian PL, Wen J (2015). Tracking the human-building interaction: A longitudinal field study of occupant behavior in air-conditioned offices. Journal of Environmental Psychology, 42. https://doi.org/10.1016/j.jenvp.2015.01.007 Langevin J, Wen J, Gurian PL (2015). Simulating the human-building interaction: Development and validation of an agent-based model of office occupant behaviors. Building and Environment, 88. https://doi.org/10.1016/j.buildenv.2014.11.037 Langevin J, Wen J, Gurian PL (2016). Quantifying the human-building interaction: Considering the active, adaptive occupant in building performance simulation. Energy and Buildings, 117. https://doi.org/10.1016/j.enbuild.2015.09.026 Lara RA, Naboni E, Pernigotto G, Cappelletti F, Zhang Y, Barzon F, Romagnoni P (2017). Optimization Tools for Building Energy Model Calibration. Energy Procedia, 111(September 2016), 1060–1069. http://doi.org/10.1016/j.egypro.2017.03.269 Lauro F, Moretti F, Capozzoli A, Khan I, Pizzuti S, Macas M, Panzieri S (2014). Building Fan Coil Electric Consumption Analysis with Fuzzy Approaches for Fault Detection and Diagnosis, Energy Procedia. 62 (2014) 411–420. doi:10.1016/j.egypro.2014.12.403. Lawrence Berkeley National Laboratory (2018). FLEXLAB: Facility for Low-Energy eXperiments in Buildings. https://Flexlab.lbl.gov, accessed January 14, 2018. Lee S, Bilionis I, Karava P, Tzempelikos A (2017a). A Bayesian approach for probabilistic classification and inference of occupant thermal preferences in office buildings. Building and Environment, 118, 323–343. https://doi.org/10.1016/j.buildenv.2017.03.009 Lee SH, Hong T (2017b). Leveraging Zone Air Temperature Data to Improve Physics-Based Energy Simulation of Existing Buildings. IBPSA Building Simulation Conference, August 2017, San Francisco. Lee SH, Hong T, Piette MA, Sawaya G, Chen Y, Taylor-Lange SC (2015). Accelerating the energy retrofit of commercial buildings using a database of energy efficiency performance. Energy 90 (2015) 738- 747. Lee T, Yao R, Coker P (2014a). An analysis of UK policies for domestic energy reduction using an agent based tool. Energy Policy. https://doi.org/10.1016/j.enpol.2013.11.004 Lee YS, Malkawi AM (2014b). Simulating multiple occupant behaviors in buildings: An agent-based modeling approach. Energy and Buildings, 69, 407–416. https://doi.org/10.1016/j.enbuild.2013.11.020 Li C, Hong T, Yan D (2014). An insight into actual energy use and its drivers in high-performance buildings, Applied Energy, 131:394-410. Li D, Hu G, Spanos CJ (2016a). A Data-driven Strategy for Detection and Diagnosis of Building Chiller Faults Using Linear Discriminant Analysis, Energy Build. 128: 519–529. doi:10.1016/j.enbuild.2016.07.014. Li J (2011). Modeling Household Energy Consumption and Adoption of Energy-Efficient Technology Using Recent Micro-Data. In USAEE/IAEE North American Conference, Redefining the Energy Economy: Changing Roles of Industry, Government, and Research. Washington, D.C. Retrieved from http://www.mendeley.com/research/no-title-avail/ Li Q, Augenbroe G, Brown J (2016b). “Assessment of Linear Emulators in Lightweight Bayesian Calibration of Dynamic Building Energy Models for Parameter Estimation and Performance Prediction.” Energy and Buildings 124: 194–202. doi:10.1016/j.enbuild.2016.04.025. Li X, Wen J (2014). Review of building energy modeling for building control and operation. Renewable and Sustainable Energy Reviews, 37:517-537. Liang X, Hong T, Shen GQ (2016). Improving the accuracy of energy baseline models for commercial buildings with occupancy data. Applied Energy, 179, 247–260. http://doi.org/10.1016/j.apenergy.2016.06.141 Lim H, Zhai ZJ (2017a). Comprehensive evaluation of the influence of meta-models on Bayesian calibration. Energy and Buildings, 155, 66–75. http://doi.org/10.1016/j.enbuild.2017.09.009 Lim H, Zhai ZJ (2017b). Review on stochastic modeling methods for building stock energy prediction. Building Simulation. https://doi.org/10.1007/s12273-017-0383-y Lin H, Hong T (2013). On Variations of Space-heating Energy Use in Office Buildings, Applied Energy 111 (2013) 515–528. Luo X, Hong T, Chen Y, Piette MA (2017). Electric Load Shape Benchmarking for Small- and Medium- Sized Commercial Buildings. Applied Energy 204 (2017) 715–725. Morari M and Lee JH. 1999. Model predictive control: past, present and future. & Chemical Engineering, 23(4–5): 667-682. Maile T, Bazjanac V, Fischer M (2012). A method to compare simulated and measured data to assess building energy performance. Building and Environment, 56, 241–251. http://doi.org/10.1016/j.buildenv.2012.03.012 Manfren M, Aste N, Moshksar R (2013). Calibration and uncertainty analysis for computer models – A meta-model based approach for integrated building energy simulation. Applied Energy, 103, 627–641. http://doi.org/10.1016/j.apenergy.2012.10.031 Marmaras JM (2014). Measurement and Verification - Retro-Commissioning of a LEED Gold Rated Building Through Means of an Energy Model: Are Aggressive Energy Simulation Models Reliable? Masters Theses available at http://scholarworks.umass.edu/masters_theses_2/31. Menezes AC, Cripps A, Bouchlaghem D, Buswell R (2012). Predicted vs. actual energy performance of non-domestic buildings: Using post-occupancy evaluation data to reduce the performance gap. Applied Energy, 97, 355–364. http://doi.org/10.1016/j.apenergy.2011.11.075 Meng JFR (2016). Performance gap analysis case study of a non-domestic building, 169. Mihai A, Zmeureanu R (2017). Bottom-up evidence-based calibration of the HVAC air-side loop of a building energy model. Journal of Building Performance Simulation, 10(1), 105–123. http://doi.org/10.1080/19401493.2016.1152302 Mills E, Bourassa N, Piette MA (2005). The Cost-Effectiveness of Commissioning New and Existing Commercial Buildings: Lessons from 224 Buildings. Proceedings of the National Conference on Building Commissioning: May 4-6, 2005. MIT Sustainable Design Group (2016). Boston Citywide Energy Model. Available: http://web.mit.edu/sustainabledesignlab/projects/BostonEnergyModel/. [Accessed: 15-May-2016]. Mirakhorli A, Dong B (2016). Occupancy behavior based model predictive control for building indoor climate—A critical review. Energy and Buildings. https://doi.org/10.1016/j.enbuild.2016.07.036 Moglia M, Cook S, McGregor J (2017). A review of Agent-Based Modelling of technology diffusion with special reference to residential energy efficiency. Sustainable Cities and Society, 31, 173–182. https://doi.org/10.1016/j.scs.2017.03.006 Muehleisen RT, Bergerson J, Collier N, Graziano DJ, Tatara E (2016). Agent Based Technology Adoption Model for Program Planning and Design, 1–12. Müller M (2013). Diffusion dynamics of energy-efficient renovations: Causalities and policy recommendations. Lecture notes in energy. https://doi.org/10.1007/978-3-642-37175-2 Munir S, Stankovic JA, Liang CJM, Lin S (2013). Cyber Physical System Challenges for Human-in-the- Loop Control. The 8th International Workshop on Feedback Computing. Najafi M, Auslander DM, Bartlett PL, Haves P, Sohn MD (2012). Application of machine learning in the fault diagnostics of air handling units, Appl. Energy. 96 (2012) 347–358. doi:10.1016/j.apenergy.2012.02.049. National Center for Atmospheric Research (2018). Weather Research and Forecasting Model. www.mmm.ucar.edu/weather-research-and-forecasting-model. Accessed January 14, 2018. National Renewable Energy Laboratory (2018). UrbanOpt, https://www.nrel.gov/buildings/urbanopt.html, accessed January 8, 2018. Nachtrieb R (2013). System Dynamics Model of Residential and Commercial Lighting Markets. 31st International Conference of the System Dynamics Society. Nagy Z, Yong FY, Frei M, Schlueter A (2015). Occupant centered lighting control for comfort and energy efficient building operation. Energy and Buildings, 94, 100–108. https://doi.org/10.1016/j.enbuild.2015.02.053 National Electrical Manufacturers Association (NEMA). (2018). Lamp Indices. Retrieved September 11, 2017, from http://www.nema.org/Intelligence/Pages/Lamp-Indices.aspx National Renewable Energy Laboratory. (2017). ResStock Analysis Tool. Retrieved January 21, 2018, from https://www.nrel.gov/buildings/resstock.html Navigant Consulting. (2013). Demand-Side Resource Potential Study Report. Kansas City, MO. Navigant Consulting. (2017). Residential End Uses : Historical Efficiency Data and Incremental Installed Costs for Efficiency Upgrades. Washington, D.C. Navigant Consulting, I. (2016). Updated Buildings Sector Appliance and Equipment Costs and Efficiency. Washington, D.C. Retrieved from https://www.eia.gov/analysis/studies/buildings/equipcosts/pdf/full.pdf Navigant Consulting, Lawrence Berkeley National Laboratory. (2011). Technical Support Document: Energy Efficiency Program for Consumer Products and Commercial and Industrial Equipment: Residential Clothes Dryers and Room Air Conditioners. Washington, D.C. Navigant Consulting, Lawrence Berkeley National Laboratory. (2014). Technical Support Document: Energy Efficiency Program for Consumer Products and Commercial and Industrial Equipment: General Service Lamps. Washington, D.C. Retrieved from https://www.regulations.gov/#!documentDetail;D=EERE-2010-BT-STD-0037-0136 Navigant Consulting, Pacific Northwest National Laboratory. (2014). Technical Support Document : Energy Efficiency Program for Consumer Products and Commercial and Industrial Equipment : Commercial Packaged Boilers. Washington, D.C. Retrieved from https://www.regulations.gov/#!documentDetail;D=EERE-2010-BT-STD-0037-0136 Navigant Consulting, Pacific Northwest National Laboratory. (2016). Technical Support Document: Energy Efficiency Program for Consumer Products and Commercial and Industrial Equipment: Commercial Water Heating Equipment. Washington, D.C. Nemtzow D (2017). Buildings and the Grid 101: Why Does it Matter for Energy Efficiency? Retrieved January 21, 2018, from https://energy.gov/eere/buildings/articles/buildings-and-grid-101-why-does- it-matter-energy-efficiency Nemtzow D (2018). Buildings and the Grid 101: Why Does it Matter for Energy Efficiency? Accessed January 27, 2018. New Buildings Institute (2018). Getting to Zero database. https://newbuildings.org/resource/getting-to- zero-database/. Accessed January 28, 2018. Noonan DS, Hsieh LHC, Matisoff D (2013). Spatial Effects in Energy-Efficient Residential HVAC Technology Adoption. Environment and Behavior, 45(4), 476–503. https://doi.org/10.1177/0013916511421664 Østergård T, Jensen RL, Maagaad SE (2016). Building simulations supporting decision making in early design – A review. Renewable and Sustainable Energy Reviews. 61:187-201. Otter M, Elmqvist H, Blochwitz T, Mauss J, Junghanns A, Olsson H (2011). Functional Mockup Interface - Overview. Retrieved from http://synchronics.inria.fr/lib/exe/fetch.php/modelica-fmi-elmqvist.pdf Palmer KA, Hale WT, Such KD, Shea BR, Bollas GM (2016). Optimal design of tests for heat exchanger fouling identification, Appl. Therm. Eng. 95 (2016) 382–393. doi:10.1016/j.applthermaleng.2015.11.043. Packey DJ (1993). Market Penetration of New Energy Technologies, (February 1993). Pang X, Wetter M, Bhattacharya P, Haves P (2012). A framework for simulation-based real-time whole building performance assessment, Building and Environment 54, 100-108. Pang X, Nouidui TS, Wetter M, Fuller D, Liao A, Haves P (2016). Building energy simulation in real time through an open standard interface, Energy and Buildings, 117: 282-289. Papadopoulos S, Azar E (2016). Integrating building performance simulation in agent-based modeling using regression surrogate models: A novel human-in-the-loop energy modeling approach. Energy and Buildings, 128, 214–223. https://doi.org/10.1016/j.enbuild.2016.06.079 Piette MA, Kinney SK, Haves P (2001). Analysis of an information monitoring and diagnostic system to improve building operations. Energy and Buildings 33(8): 783-791. Piette MA, Hong T, Wetter M, et al. (2016). Hierarchical Occupancy Responsive Model Predictive Control at Room, Building and Campus Levels. A proposal of the joint U.S.-China CERC-BEE program. Pomponi F, Moncaster A (2018). Scrutinising embodied carbon in buildings : The next performance gap made manifest. Renewable and Sustainable Energy Reviews, 81(May 2017), 2431–2442. http://doi.org/10.1016/j.rser.2017.06.049 Raju PS and Teotia APS. (1985). An Evaluation of Market Penetration Forecasting Methodologies for New Residential and Commercial Energy Technologies. U.S. DEPARTMENT OF ENERGY Office of Minority Economic Impact Research and Education Division. Putra HC, Andrews CJ, Senick JA (2017). An agent-based model of building occupant behavior during load shedding. Building Simulation, 10(6), 845–859. https://doi.org/10.1007/s12273-017-0384-x Radhakrishnan N, Su Y, Su R, Poolla K (2016). Token based scheduling for energy management in building HVAC systems, Appl. Energy. 173 (2016) 67–79. doi:10.1016/j.apenergy.2016.04.023. Ramos Ruiz G, Fernández Bandera C, Gómez-Acebo Temes T, Sánchez-Ostiz Gutierrez A (2016). Genetic algorithm for building envelope calibration. Applied Energy, 168, 691–705. http://doi.org/10.1016/j.apenergy.2016.01.075 Ratcliffe C, Chen H, Shanks TRW, Nam Y, Schreiner M, Zhan M, Sherraden M (2007). Assessing Asset Data on Low-Income Households: Current Availability and Options for Improvement. Washington, D.C. Retrieved from http://psidonline.isr.umich.edu/Guide/Quality/AssessingAssetData.pdf%5Cnpapers2://publication/u uid/B1A68A8F-A5A3-4F78-8D89-721D9AA90DFC Reddy TA (2006). Literature Review on Calibration of Building Energy Simulation Programs. ASHRAE Transactions, 112(1), 226–240. http://doi.org/Article Regnier C, Sun K, Hong T, Piette MA (2018). Quantifying the benefits of a building retrofit using an integrated system approach: A case study, Energy and Buildings, 2018. Reinhart CF, Davila CC (2016). Urban Building Energy Modeling – A Review of a Nascent Field. Building and Environment, 97:196-202. Reinhart CF, Dogan T, Jakubiec JA, Rakha T, Sang A (2013). Umi - an Urban Simulation Environment for Building Energy Use, Daylighting and Walkability. Proceedings of BS 2013: 13th Conference of International Building Performance Simulation Association. Rocky Mountain Institute (2010). https://d231jw5ce53gcq.cloudfront.net/wp- content/uploads/2017/04/Pathways-to-Zero_Black-Belt-Modeling_Matrix_2010.pdf. Accessed January 28, 2018. Rogers EM (1995). . Macmillian Publishing Co. https://doi.org/citeulike-article- id:126680 Roth A (2017). Building Energy Modeling 101: What Is It and What Is DOE’s Role? https://energy.gov/eere/buildings/articles/building-energy-modeling-101-what-it-and-what-doe-s- role. Accessed January 28, 2018. Salakij S, Yu N, Paolucci S, Antsaklis P (2016). Model-Based Predictive Control for Building Energy Management 1: Thermal Modeling and Optimal Control. Energy and Buildings, 133:345-358. Samuelson HW, Ghorayshi A, Reinhart CF (2016). Analysis of a simplified calibration procedure for 18 design-phase building energy models. Journal of Building Performance Simulation, 9(1), 17–29. http://doi.org/10.1080/19401493.2014.988752 Shen E, Hong T (2009). Simulation-based assessment of the energy savings benefits of integrated control in office buildings, Building Simulation, No.4, 2009. Sopha BM, Klöckner CA, Hertwich EG (2013). Adoption and diffusion of heating systems in Norway: Coupling agent-based modeling with empirical research. Environmental Innovation and Societal Transitions. https://doi.org/10.1016/j.eist.2013.06.001 Sorrell S, Dimitropoulos J, Sommerville M (2009). Empirical estimates of the direct rebound effect: A review. Energy Policy, 37(4), 1356–1371. http://doi.org/10.1016/j.enpol.2008.11.026 Sun K, Hong T (2017a). A Framework for Quantifying the Impact of Occupant Behavior on Energy Savings of Energy Conservation Measures. Energy and Buildings 146 (2017) 383–396. Sun K, Hong T (2017b). A Simulation Approach to Estimate Energy Savings Potential of Occupant Behavior Measures, Energy and Buildings 136 (2017) 43–62. Sun K, Hong T, Taylor-Lange SC, Piette MA (2016). A Pattern-based Automated Approach to Building Energy Model Calibration. Applied Energy 165 (2016) 214–224. Sun K, Yan D, Hong T, Guo S (2014). Stochastic modeling of overtime occupancy and its application in building energy simulation and calibration. Building and Environment, 79, 1–12. http://doi.org/10.1016/j.buildenv.2014.04.030 Sun Y (2014). Closing the building energy performance gap by improving our predictions, (August). Sunikka-Blank M, Galvin R (2012). Introducing the prebound effect: The gap between performance and actual energy consumption. Building Research and Information, 40(3), 260–273. http://doi.org/10.1080/09613218.2012.690952 Tofield B (2012). Delivering a low-energy low energy building: Making quality commonplace. Build with CaRe report, University of East Anglia, Norwich. Transportation Utility Management System. http://wisconsindot.gov/Pages/doing-bus/eng- consultants/cnslt-rsrces/util/tums.aspx. Accessed January 14, 2018. Trcka M, Hensen J, Wetter M (2009). Co-simulation of innovative integrated HVAC systems in buildings. Building Performance Simulation, 209-230. Turner C, Frankel M (2008). Energy Performance of LEED ® for New Construction Buildings. New Buildings Institute, 1–46. U.S. Census Bureau. (2017a). American Community Survey (ACS). Retrieved August 21, 2017, from https://www.census.gov/programs-surveys/acs/ U.S. Census Bureau. (2017b). American Housing Survey (AHS). Retrieved August 21, 2017, from https://www.census.gov/programs-surveys/ahs.html U.S. Department of Commerce. (2017). Data Archive. Retrieved August 17, 2017, from https://www.bea.gov/histdata/ U.S. Department of Energy Advanced Research Projects Agency. (2014). DELTA Program Overview. Washington, D.C. U.S. Department of Energy (2015a). A common definition of zero energy buildings. https://energy.gov/sites/prod/files/2015/09/f26/bto_common_definition_zero_energy_buildings_093 015.pdf. Accessed January 28, 2018. U.S. Department of Energy (2015b). Wireless Occupancy Sensors for Lighting Controls : An Applications Guide for Federal Facility Managers. Washington, D.C. U.S. Department of Energy. Building Technologies Office (2016). Multi-year program plan. https://energy.gov/sites/prod/files/2016/02/f29/BTO_MYPP_2016.pdf. Accessed January 28, 2018. U.S. Department of Energy Appliance Standards Program (2016). Commercial Packaged Boiler National Impact Analysis (NIA). Retrieved August 15, 2017, from https://www.regulations.gov/document?D=EERE-2013-BT-STD-0030-0047 U.S. Department of Energy Building Technologies Office (BTO). (2017). Scout. Retrieved August 17, 2017, from https://energy.gov/eere/buildings/scout U.S. Department of Energy (2018a). EnergyPlus. EnergyPlus.net. Accessed January 14, 2018. U.S. Department of Energy (2018b). Sensors and Controls R&D. Retrieved January 21, 2018, from https://energy.gov/eere/buildings/sensors-and-controls-rd-0 U.S. Department of Energy (2018c). Exascale Computing Project. www.exascaleproject.org. Accessed January 14, 2018. U.S. Department of Energy (2018d). Building Performance Database. Retrieved January 21, 2018, from https://energy.gov/eere/buildings/building-performance-database U.S. Department of Energy (2018e). Building Energy Data Exchange Specification (BEDES). Retrieved January 21, 2018, from https://energy.gov/eere/buildings/building-energy-data-exchange- specification-bedes U.S. Energy Information Administration. (1998). Impacts of the Kyoto Protocol on US Energy Markets and Economic Activity. Energy Information Administration Report. Washington, D.C. Retrieved from papers3://publication/uuid/25820077-B3E8-466C-AB36-DC2113CDE1B2 U.S. Energy Information Administration. (2015). An Assessment of Interval Data and Their Potential Application to Residential Electricity End- Use Modeling. Washington. U.S. Energy Information Administration. (2016). Commercial Buildings Energy Consumption Survey (CBECS). Retrieved August 15, 2017, from https://www.eia.gov/consumption/commercial/ U.S. Energy Information Administration. (2017a). Annual Energy Outlook 2017. Retrieved August 15, 2017, from https://www.eia.gov/outlooks/aeo/ U.S. Energy Information Administration. (2017b). Commercial Demand Module of the National Energy Modeling System : Model Documentation. Washington, D.C. U.S. Energy Information Administration. (2017c). Residential Demand Module of the National Energy Modeling System : Model Documentation. Washington, D.C. U.S. Energy Information Administration. (2017d). Residential Energy Consumption Survey (RECS). Retrieved August 15, 2017, from https://www.eia.gov/consumption/residential/ Van Dronkelaar C, Dowson M, Spataru C, Mumovic D (2016). A Review of the Energy Performance Gap and Its Underlying Causes in Non-domestic Buildings. Frontiers in Mechanical Engineering, 1(January), 1–14. http://doi.org/10.3389/fmech.2015.00017 Wagner A, O’Brien L, Dong B (2018). Exploring Occupant Behavior in Buildings: Methods and Challenges. Cham, Switzerland: Springer International. Wang C, Yan D, Jiang Y (2011). A novel approach for building occupancy simulation. Building Simulation, 4(2), 149–167. http://doi.org/10.1007/s12273-011-0044-5 Wang H, Zhai Z (2016). Advances in Building Simulation and Computational Techniques: A Review between 1987 and 2014. Energy and Buildings, 128: 319–335. Wang L, Greenberg S, Fiegel J, Rubalcava A, Earni S, Pang X, Yin R, Woodworth S, Hernandez- Maldonado J (2013). Monitoring-based HVAC commissioning of an existing office building for energy efficiency, Appl. Energy. 102 (2013) 1382–1390. doi:10.1016/j.apenergy.2012.09.005. Wang S, Cui J (2005). Sensor-fault detection, diagnosis and estimation for centrifugal chiller systems using principal-component analysis method, Appl. Energy. 82 (2005) 197–213. doi:10.1016/j.apenergy.2004.11.002. Weiss M, Junginger M, Patel MK, Blok K (2010). A review of experience curve analyses for energy demand technologies. Technological Forecasting and Social Change, 77(3), 411–428. https://doi.org/10.1016/j.techfore.2009.10.009 Wetter M, Fuchs M, Grozman M, Helsen P, Jorissen L, Müller F, Nytsch-Geusen D, Picard C, Sahlin D, Thorade P (2015). IEA EBC Annex 60 Modelica Library - An international collaboration to develop a free open-source model library for buildings and community energy systems. International Buildings Performance Simulation Association. Wetter M (2011). Co-simulation of building energy and control systems with the Building Controls Virtual Test Bed, Building Performance Simulation, 4 (3), 185-203. Wetter M, Bonvini M, Nouidui TS (2015). Equation-based languages - A new paradigm for building energy modeling, simulation and optimization. Energy and Buildings. Wetter (2009). Modelica-based modelling and simulation to support research and development in building energy and control systems, Building Performance Simulation, 2 (2), 143-161. Wilkerson JT, Cullenward D, Davidian D, Weyant JP (2013). End use technology choice in the national energy modeling system (NEMS): An analysis of the residential and commercial building sectors. Energy Economics, 40, 773–784. https://doi.org/10.1016/j.eneco.2013.09.023 Williams A, Atkinson B, Garbesi K, Page E, Rubinstein F (2012). Lighting controls in commercial buildings. LEUKOS - Journal of Illuminating Engineering Society of North America, 8(3), 161–180. https://doi.org/10.1582/LEUKOS.2012.08.03.001 Williams JH, Debenedictis A, Ghanadan R, Mahone A, Moore J, Iii WRM, … Torn MS (2012). 2050 : The Pivotal Role of Electricity. Science (New York, N.Y.), 335(April), 53–60. https://doi.org/10.1126/science.1208365 Yan D, Hong T, Dong B, Mahdavi A, D’Oca S, Gaetani I, Feng X (2017). IEA EBC Annex 66: Definition and simulation of occupant behavior in buildings. Energy and Buildings, 156, 258–270. https://doi.org/10.1016/j.enbuild.2017.09.084 Yan D, O’Brien W, Hong T, Feng X, Burak Gunay H, Tahmasebi F, Mahdavi A (2015). Occupant behavior modeling for building performance simulation: Current state and future challenges. Energy and Buildings, 107, 264–278. https://doi.org/10.1016/j.enbuild.2015.08.032 Yin R, Kiliccote S, Piette MA (2014). Linking measurements and models in commercial buildings : A case study for model calibration and demand response strategy evaluation. Yoshino H, Hong T, Nord N (2017). IEA EBC annex 53: Total energy use in buildings—Analysis and evaluation methods. Energy and Buildings, 152, 124–136. https://doi.org/10.1016/j.enbuild.2017.07.038 Yun GY, Song K (2017). Development of an automatic calibration method of a VRF energy model for the design of energy efficient buildings. Energy and Buildings, 135, 156–165. http://doi.org/10.1016/j.enbuild.2016.11.060 Zero Carbon Hub (2014). Closing the Gap Between Design & As-built Performance. Zhang R, Hong T (2017). Modeling of HVAC operational faults in building performance simulation. Applied Energy, 2017. Zhang T, Nuttall WJ (2007). An agent based simulation of smart metering technology adoption. International Journal of Agent Technologies and Systems (IJATS). https://doi.org/10.4018/jats.2012010102 Zhao Y, Wang S, Xiao F (2013). Pattern recognition-based chillers fault detection method using Support Vector Data Description (SVDD), Appl. Energy. 112 (2013) 1041–1048. doi:10.1016/j.apenergy.2012.12.043. Zhou X, Hong T, Yan D (2014). Comparison of HVAC System Modeling in EnergyPlus, DeST and DOE- 2.1E, Building Simulation, 2014, 7(1). Zhu D, Hong T, Yan D, Wang C (2013). A Detailed Comparison of Three Building Energy Modeling Programs: EnergyPlus, DeST, and DOE-2.1E. Building Simulation, 2013. Zibin N, Zmeureanu R, Love J (2016). Automatic assisted calibration tool for coupling building automation system trend data with commissioning. Automation in Construction, 61, 124–133. http://doi.org/10.1016/j.autcon.2015.10.004

Disclaimer This document was prepared as an account of work sponsored by the United States Government. While this document is believed to contain correct information, neither the United States Government nor any agency thereof, nor The Regents of the University of California, nor any of their employees, makes any warranty, express or implied, or assumes any legal responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by its trade , trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof, or The Regents of the University of California. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof or The Regents of the University of California.