Energy Monitoring and Targeting as diagnosis; Applying work analysis to adapt a statistical change detection strategy using representation aiding by Antony Hilliard A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy, Industrial Engineering Department of Mechanical & Industrial Engineering University of Toronto © Copyright by Antony Hilliard 2015 ii Energy Monitoring and Targeting as diagnosis; Applying work analysis to adapt a statistical change detection strategy using representation aiding Antony Hilliard Doctor of Philosophy, Industrial Engineering Department of Mechanical & Industrial Engineering University of Toronto 2015 Abstract Energy Monitoring and Targeting is a well-established business process that develops information about utility energy consumption in a business or institution. While M&T has persisted as a worthwhile energy conservation support activity, it has not been widely adopted. This dissertation explains M&T challenges in terms of diagnosing and controlling energy consumption, informed by a naturalistic field study of M&T work. A Cognitive Work Analysis of M&T identifies structures that diagnosis can search, information flows un-supported in canonical support tools, and opportunities to extend the most popular tool for M&T: Cumulative Sum of Residuals (CUSUM) charts. A design application outlines how CUSUM charts were augmented with a more contemporary statistical change detection strategy, Recursive Parameter Estimates, modified to better suit the M&T task using Representation Aiding principles. The design was experimentally evaluated in a controlled M&T synthetic task, and was shown to significantly improve diagnosis performance. iii iv Acknowledgments Above all, thanks to Linda for her patience and support. Thanks to Greg Jamieson for supporting this work in an unusual and difficult domain, and to my committee members for their guidance and thoughtful critiques. This work could not have been completed without the contributions of field study participants, including management and staff at Energent Inc. Experimental participants would not have been available without support from energy technology professors at Humber and Seneca College. Students at the Cognitive Engineering as well as Human Factors and Applied Statistics Labs provided design, experiment, pilot testing, data validation, and writing feedback. This research was funded by the Association of Major Power Consumers in Ontario, the Ontario Centres of Excellence, Energent Inc., the Natural Science and Engineering Council of Canada, The Federal Economic Development Agency for Southern Ontario, and departmental support. v vi Table of Contents CHAPTER 1 ENERGY MONITORING AND WORK ANALYSIS ..................................................................... 1 1.1 FOCUS: PRACTICAL ENERGY M&T CHALLENGES ............................................................................................ 2 1.1.1. ‘Work that EMs do’, and differences between jobs ............................................................................... 2 1.1.2. Why focus on M&T for EMs? ................................................................................................................ 4 1.2 SOCIETAL IMPLICATIONS OF M&T ................................................................................................................... 5 1.2.1. Efficiency and energy gaps .................................................................................................................... 5 1.2.2. Cognitive cost of Energy Efficiency ....................................................................................................... 6 1.2.3. Cognitive engineering to tilt equilibrium ............................................................................................... 6 1.3 THEORETICAL WORK ANALYSIS & DESIGN CHALLENGES ............................................................................... 7 1.3.1. Work Support Approaches ..................................................................................................................... 7 1.3.2. Invariant constraints for formative design ............................................................................................ 8 1.3.3. Ecological problem-solving structures .................................................................................................. 8 1.3.4. Tractable cost-effectiveness ................................................................................................................... 8 1.4 STRUCTURE AND CONTRIBUTIONS OF THE DISSERTATION ................................................................................ 9 1.4.1. Contribution 1: Describing the M&T task in the context of Energy Management .............................. 10 1.4.2. Contribution 2: Characterize M&T task in CWA framework .............................................................. 10 1.4.3. Contribution 3: Apply CWA to M&T tool design ................................................................................ 11 1.4.4. Contribution 4: Controlled evaluation of M&T tool ........................................................................... 12 1.5 SIGNIFICANCE OF THE CONTRIBUTIONS .......................................................................................................... 12 CHAPTER 2 DESCRIBING M&T WORK AS REPORTED AND OBSERVED .............................................. 13 2.1 MOTIVATION: COMPARING M&T NORMS WITH A DESCRIPTIVE ACCOUNT ..................................................... 13 2.1.1. Residential field observations informing policy and design ................................................................ 13 2.1.2. Methods used to investigate M&T work .............................................................................................. 14 2.1.2.1. Interviews with Industrial Energy Managers .............................................................................................. 14 2.1.2.2. M&T Participant Observation .................................................................................................................... 15 2.2 M&T PRACTICE, PAST AND PRESENT ............................................................................................................. 15 2.2.1. Management Systems ........................................................................................................................... 16 2.2.2. Model-comparative monitoring ........................................................................................................... 17 2.2.3. Good Housekeeping ............................................................................................................................. 17 2.2.4. Real-time utility meter monitoring ....................................................................................................... 17 2.2.5. Automated control or decision support ................................................................................................ 18 2.2.6. First principles energy efficiency analysis .......................................................................................... 19 2.2.7. Discussion: Tools for which approach? .............................................................................................. 19 2.3 M&T WITH RECURSIVE CUMULATIVE SUM OF RESIDUALS (CUSUM) .......................................................... 20 2.3.1. Origin of CUSUM method ................................................................................................................... 20 2.3.2. Method Outline .................................................................................................................................... 21 2.3.3. Model Building .................................................................................................................................... 22 2.3.4. Model Application and Calculations ................................................................................................... 23 2.3.5. Interpretation ....................................................................................................................................... 24 2.3.6. Problems with CUSUM Interpretation ................................................................................................ 24 2.3.7. CUSUM Conclusions ........................................................................................................................... 26 2.4 OBSERVATIONAL STUDY ................................................................................................................................ 26 2.4.1. Environments Observed ....................................................................................................................... 26 2.4.2. Workers ................................................................................................................................................ 27 2.4.3. Methods ............................................................................................................................................... 28 2.4.4. Limitations ........................................................................................................................................... 29 2.5 RESULTS OF M&T OBSERVATIONS ................................................................................................................ 29 2.5.1. Participant discoveries and learning ..................................................................................................