Data-Efficient Analytics for Optimal Human-Cyber-Physical Systems
Total Page:16
File Type:pdf, Size:1020Kb
Data-efficient Analytics for Optimal Human-Cyber-Physical Systems Ming Jin Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-2017-228 http://www2.eecs.berkeley.edu/Pubs/TechRpts/2017/EECS-2017-228.html December 15, 2017 Copyright © 2017, by the author(s). All rights reserved. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission. Data-efficient Analytics for Optimal Human-Cyber-Physical Systems by Ming Jin Adissertationsubmittedinpartialsatisfactionofthe requirements for the degree of Doctor of Philosophy in Engineering – Electrical Engineering and Computer Sciences and the Designated Emphasis in Communication, Computation and Statistics in the Graduate Division of the University of California, Berkeley Committee in charge: Professor Costas J. Spanos, Chair Professor Pieter Abbeel Professor Stefano Schiavon Professor Alexandra von Meier Fall 2017 Data-efficient Analytics for Optimal Human-Cyber-Physical Systems Copyright 2017 by Ming Jin 1 Abstract Data-efficient Analytics for Optimal Human-Cyber-Physical Systems by Ming Jin Doctor of Philosophy in Engineering – Electrical Engineering and Computer Sciences and the Designated Emphasis in Communication, Computation and Statistics University of California, Berkeley Professor Costas J. Spanos, Chair The goal of this research is to enable optimal human-cyber-physical systems (h-CPS) by data- efficient analytics. The capacities of societal-scale infrastructures such as smart buildings and power grids are rapidly increasing, becoming physical systems capable of cyber computation that can deliver human-centric services while enhancing efficiency and resilience. Because people are central to h-CPS, the first part of this thesis is dedicated to learning about the human factors, including both human behaviors and preferences. To address the central challenge of data scarcity, we propose physics-inspired sensing by proxy and a framework of “weak supervision” to leverage high-level heuristics from domain knowledge. To infer human preferences, our key insight is to learn a functional abstraction that can rationalize people’s behaviors. Drawing on this insight, we develop an inverse game theory framework that determines people’s utility functions by observing how they interact with one another in a social game to conserve energy. We further propose deep Bayesian inverse reinforcement learning, which simultaneously learns a motivator representation to expand the capacity of modeling complex rewards and rationalizes an agent’s sequence of actions to infer its long-term goals. Enabled by this contextual awareness of the human, cyber, and physical states, we in- troduce methods to analyze and enhance system-level efficiency and resilience. We propose an energy retail model that enables distributed energy resource utilization and that exploits demand-side flexibility. The synergy that naturally emerges from integrated optimization of thermal and electrical energy provision substantially improves efficiency and economy. While data empowers the aforementioned h-CPS learning and control, malicious attacks can pose major security threats. The cyber resilience of power system state estimation is analyzed. The envisioning process naturally leads to a power grid resilience metric to guide “grid hard- ening.” While the methods introduced in the thesis can be applied to many h-CPS systems, this thesis focuses primarily on the implications for smart buildings and smart grid. i Contents Contents i 1 Introduction 1 1.1 Thesis approach .................................. 3 1.2 Contributions ................................... 6 1.3 Thesis overview .................................. 11 I Learning about the human factor 17 2 Sensing by proxy 18 2.1 System model with distributed sensor delays .................. 18 2.2 Sensing by proxy methodology .......................... 19 2.3 Application: occupancy detection in buildings ................. 21 2.4 Chapter summary ................................. 27 3 Learning under weak supervision 28 3.1 Overview of weak supervision .......................... 29 3.2 Multi-view iterative training ........................... 30 3.3 Learning with surrogate loss ........................... 33 3.4 Application: smart meter data analytics .................... 35 3.5 Chapter summary ................................. 41 4 Gamification meets inverse game theory 42 4.1 Overview of gamification ............................. 43 4.2 Game-theoretic formulation ........................... 44 4.3 Reverse Stackelberg game – incentive design .................. 45 4.4 Inverse game theory framework ......................... 46 4.5 Energy efficiency via gamification ........................ 47 4.6 Chapter summary ................................. 53 5 Deep Bayesian inverse reinforcement learning 54 5.1 Introduction of inverse reinforcement learning ................. 55 ii 5.2 Deep GP for inverse reinforcement learning ................... 56 5.3 Experiments on benchmarks ........................... 61 5.4 Chapter summary ................................. 64 II System-level efficiency and resilience 65 6 Enabling optimal energy retail in a microgrid 66 6.1 Microgrid and optimal energy retail ....................... 66 6.2 Integrated energy retail model .......................... 69 6.3 Optimal rate design and operation strategy ................... 74 6.4 Scenario analysis and case study ......................... 77 6.5 Chapter summary ................................. 88 7 Cyber resilience of power grid state estimation 89 7.1 Power grid resilience and state estimation ................... 89 7.2 Vulnerability of AC-based state estimation ................... 95 7.3 SDP convexification of the FDIA problem ................... 97 7.4 Experiments on IEEE standard systems .................... 103 7.5 Chapter summary ................................. 107 8 Conclusion and future directions 108 8.1 Challenges and opportunities in h-CPS ..................... 109 8.2 Closing thoughts ................................. 114 A Main proofs and derivations 115 List of Figures 137 List of Tables 142 Bibliography 144 iii Acknowledgments I could not ask for a better advisor than Costas Spanos. His enthusiasm and patience, in- sight and vision on research, academia and life have been a perpetuating motivation through- out my Ph.D., and will continue to empower me in my future career and life. I am deeply grateful that Costas took a chance on me after my “elevator pitch” during his visit to the Hong Kong University of Science and Technology – I would not have been where I am today without his invaluable support and guidance. I would like to express my gratitude to the other committee members – Pieter Abbeel, Stefano Schiavon, and Sascha von Meier – for bringing insightful and interdisciplinary per- spectives on my research. Pieter Abbeel for introducing me to the fascinating world of robotic research. Stefano Schiavon for sharing his deep insights on human-centric design in smart buildings. Sascha von Meier for the fruitful and constructive discussions on power system research. Working at the Lawrence Berkeley National Lab has been an amazing experience, and Iwouldliketothankmycolleaguesandcollaborators,WeiFeng,ChrisMarnay,BoShen, Nan Zhou, Jiang Lin, Lynn Price, Michel Foure, and Tianzhen Hong, for o↵ering the unique opportunity to gain perspectives on conducting impactful research to solve societal-scale energy problems. Many of my recent work has been in close collaboration with Javad Lavaei. Javad has given me invaluable advice and shared his vast knowledge on optimization and control theory. It is truly empowering to collaborate with Javad on high-impact research problems. Iamverygratefulforthefacultywhohelpedshapemyresearchduringmytimeat Berkeley: Alex Bayen for his vital support and guidance, and productive collaboration on the work “sensing by proxy” that led to my first best paper award. Peter Bartlett for his insightful discussions on statistical learning theory. Claire Tomlin and Kameshwar Poolla for the advice and constructive discussions. Murat Arcak, Shankar Sastry, and Laurent El Ghaoui for introducing me to the world of control and optimization. Sergey Levine for his inspiring class on deep reinforcement learning. Martin Wainwright, Bin Yu, Jitendra Malik, and Michael Jordan for the introduction to machine learning, computer vision, and theoretical statistics, all of which underpin the various parts of my research. I have also been so fortunate to have many other collaborators and mentors outside Berkeley: Lin Zhang for sharing his profound knowledge in mobile sensing and information theory, and for his valuable support and guidance throughout my Ph.D. Kalle Johansson for the thoughtful discussions and productive collaboration on cyber security. Jinyue Yan for his world-class insights on energy transformation and passion for promoting young scholars to conduct clean technology research and entrepreneurial endeavor. Lillian Ratli↵for the fruitful collaboration on game theory and her unique perspectives on academia and research. Nikos Bekiaris-Liberis and Andreas Damianou for the in-depth discussions on control theory and deep