Detection and Optimization Algorithms for Cyber-Physical Systems by Pedro Ivo Bastos Hespanhol A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Engineering- Industrial Engineering and Operations Research in the Graduate Division of the University of California, Berkeley Committee in charge: Professor Anil Jayanti Aswani, Chair Professor Zuo-Jun (Max) Shen Professor Prasad Raghavendra Spring 2020 Detection and Optimization Algorithms for Cyber-Physical Systems Copyright 2020 by Pedro Ivo Bastos Hespanhol 1 Abstract Detection and Optimization Algorithms for Cyber-Physical Systems by Pedro Ivo Bastos Hespanhol Doctor of Philosophy in Engineering- Industrial Engineering and Operations Research University of California, Berkeley Professor Anil Jayanti Aswani, Chair Cyber-Physical Systems (CPS) play an ubiquitous role in operation and control in many different domains: power systems, finance, robotics, and automation. The complex interplay between cyber components such as software, communication protocols, computer servers and physical components, such as sensors and pieces of dedicated hardware, requires advanced and sophisticated methods and algorithms that ensure safe and efficient operation. In this thesis we tackle both safety and efficiency: We develop novel detection algorithms that are able to identify malicious attacks, sensor corruption and faulty measurements. Our detection mechanisms have provable guarantees based on rigorous asymptotic and non-asymptotic statistical analysis and can be readily implemented in CPS, such as robotic systems and autonomous vehicles. In addition, we developed collusion detection mechanisms that can be used to identify whether two or more CPS are colluding or not. We also design a mechanism that is able to induce selfish systems/agents to behave cooperatively. We showcase the performance of our algorithms with several different case studies. In our analyses, we place emphasis on algorithms that can be implemented in real-time, that is can be used while the system is under operation in the real-world. On the efficiency side, we developed real-time non-linear Model Predictive Control (MPC) Methods that can provide optimal solutions to the Optimal Control problem faced by the CPS during operation. Our algorithm exploits the control structure and is tailored for implementation in embedded hardware and can operate both with memory and computation time constraints. We showcase the performance of our algorithm with a C/C++ implementation and we compare to several current state- of-the-art Optimal Control solvers. We also extend our methodology to be used together with Pseudo-spectral Methods and Hybrid Systems, developing an integrated Mixed-Integer MPC algorithm that can handle complex non-linear dynamics and both continuous and discrete variables. With this thesis, our goal is to provide real-time practical algorithms that have provable guarantees in performance both in the detection task and in the optimal control task. Our algorithms are based on rigorous theoretical analysis and display very good performance and can be readily implemented in practical Cyber-Physical Systems. i To my parents, Luiza and Jorge ii Contents Contents ii List of Figures iv List of Tables v 1 Introduction 1 2 Dynamic Watermarking in Cyber-Physical Systems 7 2.1 Dynamic Watermarking in MIMO LTI Systems . 9 2.2 Simulations: Dynamic Watermarkig for Autonomous Vehicle . 16 2.3 Statistical Watermarking for Networked Control Systems . 19 2.4 Simulation: Autonomous Vehicle Platooning . 29 3 Switching in Cyber-Physical Systems: Finite-time consistency tests and estimation 31 3.1 Sensor Switching Control Under Attacks Detectable by Finite Sample Dy- namic Watermarking Tests . 33 3.2 Experimental Results: Finite-time switching with Autonomous vehicles . 54 3.3 Statistical Consistency of Set-Membership Estimator for Linear Systems . 58 3.4 Numerical Experiments: Set-membership estimator . 65 4 Detection Algorithm in Competitive Environments 71 4.1 Hypothesis Testing Approach to Detecting Collusion in Competitive Environ- ments . 72 4.2 Computational Experiments for Inverse Hyposthesis Testing . 79 4.3 Surrogate Optimal Control for Strategic Multi-Agent Systems . 83 4.4 HVAC Control Case Study . 93 5 Real-time MPC for Cyber-Physical Systems 96 5.1 Adjoint-based SQP Method with Block-wise quasi-Newton Jacobian Updates for Nonlinear Optimal Control . 97 5.2 Convergence Results for Block-wise TR1-based SQP Method . 104 iii 5.3 Lifted Collocation Algorithm with Block-TR1 Jacobian Updates . 114 5.4 Numerical Case Studies of Nonlinear Model Predictive Control . 118 6 Advanced applications of Real-time MPC 124 6.1 Quasi-Newton Jacobian and Hessian Updates for Pseudospectral based NMPC 125 6.2 NMPC Case Study: Chain of Masses . 133 6.3 A Structure Exploiting Branch-and-Bound Algorithm for Mixed-Integer Model Predictive Control . 137 6.4 Mixed-Integer MPC Algorithm . 145 6.5 Case Studies: Mixed-Inter MPC . 149 7 Conclusion and Outlook 153 Bibliography 156 iv List of Figures 2.1 Deviation of (2.11) in Simulation of Autonomous Vehicle . 17 2.2 Deviation of (2.12) in Simulation of Autonomous Vehicle . 18 2.3 Value of (2.29) for Simulation of Autonomous Vehicle, with a Negative Log- Likelihood Threshold for α = 0:05 False Detection Error Rate . 19 2.4 Simulation of Vehicle Platoon . 30 3.1 Schematic representation of the LTI system with switching . 39 3.2 Average Time to Detect Replay Attack . 55 3.3 Switching Decision Values . 56 3.4 Performance Comparison of Simulated Autonomous Vehicle Lane Keeping . 57 3.5 Estimation Error From Trajectory . 66 3.6 Estimation Error . 69 3.7 Arm p Chosen by Algorithm . 69 3.8 Norm of System State . 70 4.1 Comparing CDF's of Residuals For Scenario 1 . 82 4.2 Comparing CDF's of Residuals For Scenario 2 . 84 4.3 Room Configuration with Heat Exchange Vectors highlighted . 93 4.4 Closed-Loop State Trajectories for P-MPC, M-MPC, and A-MPC . 94 4.5 MPC Aggregated Stage Cost with Agents' True Utility Functions . 95 5.1 Local convergence analysis . 120 5.2 Comparison of the average preparation and feedback computation times . 121 5.3 Closed-loop NMPC performance of two double lane changes . 122 6.1 Average computation time per RTI step . 135 6.2 Illustration of the branch-and-bound method . 139 6.3 Illustration of the tree propagation technique . 146 6.4 Computational results for closed-loop mixed-integer MPC . 150 6.5 MPC state evolution for satellite station keeping . 152 6.6 Closed-loop results of mixed-integer MPC for satellite station keeping . 152 v List of Tables 4.1 Numerical Results for Scenario 1 (Competing) . 82 4.2 Numerical Results for Scenario 2 (Colluding) . 83 5.1 Average computation times . 121 5.2 Average computation times (in ms) for vehicle control . 123 6.1 Average timing results . 136 6.2 Timing results . 151 vi Acknowledgments In this thesis, I present the work I have develop at UC Berkeley in the last five years in the Industrial Engineering and Operations Research (IEOR) Department. It was a long journey, often times arduous, but now I am sure that it was worth it and, most importantly, it was satisfying and it made me feel confident, not only as a researcher, but also as a human being. The experiences and the knowledge I have gained in those years are irreplaceable and they make what I am today. I would like to express my gratitude to my advisor Professor Anil Aswani. Professor Aswani was one of the first people I ever had contact with at Berkeley and he was the one who sparked on me the desire to pursue a Ph.D. degree in IEOR. Ever since that summer back in 2014 where he hired me as a research assistant during my time as an exchange student, he has been a great source of knowledge and motivation, never failing to support me and offering advice. Our discussions, throughout the entire Ph.D. program at Berkeley, were fundamental in my development as a researcher and as a person. I will always be grateful for your help and for the cooperation we had during those five years. I would like to thank Professor Ram Vasudevan and Matthew Porter from University of Michigan, with whom I had many productive collaborations. The research materials in Chapter 2-4 were developed with their help: Our joint effort across teams were very productive and we were able to develop new and interesting ideas. I would like to thank you both for your comments, suggestions, and discussions across the several papers we wrote together. I would like to thank Researchers Rien Quirynen and Stefano Di Cairano and the entire Mistubishi Electric Research Laboratories (MERL) team for their collaboration with the research materials in Chapters 5-6: the experience I had working with MERL was invaluable and I would like to acknowledge their support and partnership with my research projects. The research environment of MERL is of utmost excellency and I felt confortable expressing my ideas and proposing new solutions and algorithms with the research team. I would like to thank Professor Max Shen and Professor Prasad Raghavendra for being in my dissertation committee and for being always open and helpful to discussions and questions. I would like to thank Professor Alper Atamt¨urk,for giving me very good advice, support and for allowing me to be the teaching instructor of IEOR 262A (Math Programming I), which was one of my personal goals ever since I sat down on Etcheverry Hall to learning Linear Programming and Farkas's Lemma. I also would like to thank professor Max Shen, and by extension the entire IEOR de- partment staff and faculty for allowing me to teach a course (IEOR 265, Learning and Optimization) on my last semester, as a Ph.D. student. Lecturing a course was challenging, but a very rewarding experience.