Enforcing Abstract Immutability by Jonathan Eyolfson A thesis presented to the University of Waterloo in fulfillment of the thesis requirement for the degree of Doctor of Philosophy in Electrical and Computer Engineering Waterloo, Ontario, Canada, 2018 © Jonathan Eyolfson 2018 Examining Committee Membership The following served on the Examining Committee for this thesis. The decision of the Examining Committee is by majority vote. External Examiner Ana Milanova Associate Professor Rensselaer Polytechnic Institute Supervisor Patrick Lam Associate Professor University of Waterloo Internal Member Lin Tan Associate Professor University of Waterloo Internal Member Werner Dietl Assistant Professor University of Waterloo Internal-external Member Gregor Richards Assistant Professor University of Waterloo ii I hereby declare that I am the sole author of this thesis. This is a true copy of the thesis, including any required final revisions, as accepted by my examiners. I understand that my thesis may be made electronically available to the public. iii Abstract Researchers have recently proposed a number of systems for expressing, verifying, and inferring immutability declarations. These systems are often rigid, and do not support “abstract immutability”. An abstractly immutable object is an object o which is immutable from the point of view of any external methods. The C++ programming language is not rigid—it allows developers to express intent by adding immutability declarations to methods. Abstract immutability allows for performance improvements such as caching, even in the presence of writes to object fields. This dissertation presents a system to enforce abstract immutability. First, we explore abstract immutability in real-world systems. We found that developers often incorrectly use abstract immutability, perhaps because no programming language helps developers correctly implement abstract immutability. We believe that this omission leads to incorrect usages. Specifically, we wrote a dynamic analysis that reports any writes through immutability declarations. To our knowledge, this work was the first to explore how objects implement abstract immutability (or fail to implement it). Our novel study found three uses of abstract immutability: caching, delayed initialization, and unit testing. Unit testing was a surprising application of abstract immutability, and we believe that the ability to modify state is needed for proper unit testing. Next, we explore developers’ revealed needs for immutability in the source code. We found that the majority of classes contain a mix of immutable and mutable methods, with a majority of the overall methods being immutable. Immutability systems with only im- mutable or all-mutating classes are insufficient: developers need immutability declarations at method granularity. Our study then combined developer immutability declarations with results from a static analysis to estimate the true number of immutable methods. The static analysis checked that no transitive writes to a receiver object occurred. Our results indicated the need for a sophisticated analysis to check that these apparently abstractly immutable methods were indeed abstractly immutable. Finally, we created a novel static analysis which checks that developers follow abstract immutability. Specifically, we define abstract immutability to mean that a class’s setof immutable methods is collectively free of writes to exposed fields. Our analysis found incorrect usages of abstract immutability, such as incorrect caching. This analysis is par- ticularly valuable in the context of code evolution, whereby subsequent programmers may make changes that break previously-correct cache implementations, for instance. Our work allows developers to trust that their code is abstractly immutable. iv Acknowledgements First, I need to thank my parents. Especially my dad, who has supported me to follow my passion no matter what. Whether I’m at my highest highs or lowest lows, he has guided me through them all. Thank you Grandma Cook, you’ve always ensured I was taken care of and I couldn’t have done it without you. I would also like to thank Tracy for all her love and support, and putting up with horrible puns. I would be remiss if I didn’t thank my teachers at Frontenac High School: Mr. Kotecha, Mr. Wills, Mr. Lee, and Mr. Grew. I would not have found my passion if it wasn’t for them opening my eyes, and setting an excellent example. They encouraged the pursuit of knowledge, and curiosity I didn’t know I had. I owe a deep gratitude to my supervisor Patrick Lam, who has been guiding me ever since we first met at Curry Original in Kingston. Without his knowledge and expertiseI would not have completed my masters, much less a doctorate. He has been patient and understanding through my graduate studies, and always allowed me to explore problems that interested me. Thank you to my committee members. I appreciate the time you’ve taken out of your busy schedules to add your valuable insight to this work. Finally, I would like to thank friends that I’ve been able to bounce ideas off of during my research. There are too many to name, you know who you are, thank you. For the purpose of this thesis in particular: Jean-Christophe Petkovich, you’ve been invaluable, as both a set of open ears and a friend dating back from grade 3. v Dedication In memory of Sisters Kathryn and Mary Joan LaFleur. vi Table of Contents List of Figures x Listings xii List of Tables xv 1 Introduction 1 1.1 Motivation .................................... 2 1.2 Case Study: Clang ............................... 4 1.3 Exploratory Study ............................... 4 1.4 Outline ..................................... 6 2 Background 7 2.1 Immutability Terminology ........................... 7 2.1.1 Transitive and Non-Transitive ..................... 7 2.1.2 Concrete and Abstract ......................... 8 2.1.3 Object and Reference Immutability .................. 8 2.1.4 Relation to Purity ........................... 9 2.2 Immutability in C++ .............................. 9 2.2.1 const keyword ............................. 10 2.2.2 mutable keyword ............................ 11 2.2.3 Bitwise const .............................. 12 2.2.4 Logically const ............................. 12 2.2.5 Additional Loopholes .......................... 12 2.3 Immutability in Java .............................. 15 2.3.1 final Keyword ............................. 17 2.3.2 Javari .................................. 17 2.4 Static Analysis ................................. 17 2.4.1 Dataflow Analysis ........................... 17 2.4.2 Pointer Analysis ............................ 18 3 Dynamic Observations of Writes-Through-Immutability 19 3.1 Motivation .................................... 20 3.2 Technique .................................... 23 3.3 Classification .................................. 29 vii 3.4 Results ...................................... 32 3.4.1 Protobuf ................................. 33 3.4.2 LevelDB ................................. 35 3.4.3 fish shell ................................. 38 3.4.4 Mosh (mobile shell) .......................... 38 3.4.5 LLVM TableGen ............................ 39 3.4.6 Tesseract ................................ 40 3.4.7 Ninja .................................. 40 3.4.8 Weston ................................. 40 3.4.9 Summary ................................ 42 4 Static Observations of Immutability 43 4.1 Motivation .................................... 44 4.2 Technique .................................... 45 4.3 Results ...................................... 48 4.3.1 LLVM .................................. 50 4.3.2 OpenCV ................................. 51 4.3.3 Protobuf ................................. 53 4.3.4 fish shell ................................. 54 4.3.5 libsequence ............................... 56 4.3.6 Mosh .................................. 57 4.3.7 Ninja .................................. 58 4.3.8 Summary ................................ 60 5 Abstract Immutability Analysis 63 5.1 Motivating Example .............................. 64 5.2 Technique .................................... 68 5.2.1 Formalization .............................. 69 5.2.2 Analysis Operation ........................... 71 5.2.3 Abstraction ............................... 71 5.2.4 Running Example ........................... 74 5.2.5 Transfer Functions ........................... 77 5.2.6 Memory Alias Sets ........................... 79 5.2.7 Merge Operation ............................ 83 5.2.8 Assumptions .............................. 86 5.3 Results ...................................... 87 5.3.1 libsequence ............................... 87 5.3.2 fish shell ................................. 90 5.3.3 Mosh .................................. 92 5.3.4 Ninja .................................. 92 5.3.5 Summary ................................ 93 viii 6 Related Work 95 6.1 Type Systems .................................. 96 6.1.1 Javari .................................. 96 6.1.2 ReIm .................................. 97 6.1.3 Glacier ................................. 98 6.2 Type Inference ................................. 98 6.2.1 Javarifier ................................ 99 6.2.2 ReImInfer ................................ 99 6.2.3 Inferring const for C programs .................... 100 6.3 Language Features ............................... 100 6.3.1 Usability of Language Features .................... 100 6.3.2 Frozen Objects ............................. 101 6.3.3