Vulnerability-Tolerant Secure Architectures (Invited Paper) Todd Austin 1, Valeria Bertacco 1, Baris Kasikci1, Sharad Malik2, and Mohit Tiwari3 1University of Michigan 2Princeton University 3University of Texas at Austin Corresponding Email: [email protected] Abstract • Even for a specific vulnerability and definition of system Today, secure systems are built by identifying potential vul- security, the resulting verification problems are intractable nerabilities and then adding protections to thwart the associ- and tend not to scale to practical systems. ated attacks. Unfortunately, the complexity of today's systems • It is not possible to know all possible threats and vulner- makes it impossible to prove that all attacks are stopped, so abilities a priori. This lack of knowledge is exactly what clever attackers find a way around even the most carefully attackers exploit. designed protections. In this article, we take a sobering look at the state of secure system design, and ask ourselves why the While much progress has been made in formal verification, "security arms race" never ends? The answer lies in our inabil- success is limited in type and scale of the systems addressed ity to develop adequate security verification technologies. We (e.g., SLAM [3]) or scope of the analysis (e.g., Whoop [4]). then examine an advanced defensive system in nature -the hu- Thus given the complexity of today's systems, the goal of man immune system -and we discover that it does not remove securing general software and hardware remains unreachable vulnerabilities, rather it adds offensive measures to protect through security proofs. the body when its vulnerabilities are penetrated. We close The incompleteness and intractability of formalized secu- the article with brief speculation on how the human immune rity verification techniques is further exacerhated by the fast- system could inspire more capable secure system designs. growing complexity of modern systems. Fueled by the com- 1.Introduction putational and memory resources made available by Moore's Law, designers are continually building systems with increased It seems that one inescapable truth in computer security is that, complexity. A recent study of lines of code for avionic systems regardless of the system, attackers always find vulnerabilities found that the code size in Boeing and Airbus commercial jets to exploit, and if those vulnerabilities are fixed, they will soon is growing at a rate of 2x every four years [5]. Another study find ways around those protections. 1bis "security arms race," for Ford vehicles found a !Ox increase in the size of on-board as it is called, persists because, despite advances in security software over a 3 year period [6]. Similar trends can be seen verification [l, 2], the complexity ofreal systems precludes in hardware design complexity. A recent study of Apple SoCs the possibility of proving that a design cannot be attacked. found that the number of accelerators grew from to in 9 37 less than decade, with no sign of slowing [7]. 1.1. Why the Security Arms Race Never Ends In an ideal world, hardware and software designers could 1.2. Functional Verification Isn't Scaling to the Challenge harness powerful security analysis tools that would implement Functional verification works to find system design errors, proofs for each vulnerability of concern, in an effort to show many of which are vulnerabilities that could be exploited to that for a particular system: penetrate security. During functional testing, which is the V (programs, inputs, vulnerabilities ) (system is secure) workhorse of verification technologies, engineers use hand- made tests, random testing, and formal analysis to exercise Unfortunately, the creation of truly secure systems is not possi- as much of the functionality of the design as possible, given ble for arbitrary systems running arbitrary software, because: engineering resources [8]. Again, due to the complexity of • Itis not possible to easily define what it means for the system modern systems, there is no hope to fully test the system, so to be secure. Specific properties such as confidentiality and verification engineers direct tests toward likely runtime states. integrity are useful, but are incomplete notions of security. For functional verification, it is important to test the most Permission to make digital or hard copies of all or part of this work for personal or likely states to be visited while the system is in operation. classroom use is granted without fee provided that copies are not made or distributed Thus, "coverage metrics" are used to assess to what extent for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM these likely states have been exantined [9]. Studies have shown must be honored. Abstracting with credit is permitted To copy othe.rwi.se, or republish, that while a design possesses a multitude of reachable states, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. normal operation only touches a tiny fraction of these states ICCAD '18,November 5--8, 2018. San Diego, CA, USA [10, ll], leading to verification of likely states being a very © 2018 Association for Computing Machinery. effective functional verification approach. ACM ISBN 978-1-4503-5950-4/1 8111... $15.00 https://doi.org/10.1145/3240765.3273057 While "best effort" serves functional verification well, it falls way short of what is necessary for high-quality security holder of the bounty and verified, the attacker will typically verification. Functional verification wants to find the bugs be paid a bounty in proportion to the vulnerability'• severity. that are likely to be encountered with real-world software, Severity is often scored using the industry standard Common thus, it is possible to find most of them by running real-world Vulnerability Scoring System (CVSS) [17]. Bug bounties have software on test systems. Security verification, on the other been quickly growing in number and amounts in the last few hand, is a very different endeavor. Attackers do not seek out years because it has been shown to be an effective method to likely execution scenarios, as these are quite unlikely to have entice attackers to disclose a uew vulnerability, rather than vulnerabilities. Instead, attackers seek out the most unlikely use it to harm the users of a vulnerable system. For example, configurations, because these have not been well tested. For ex- Intel instituted its security bug bounty program in March 2017, ample, if an attacker wants to exploit a program asking which and expanded its payouts to up to $250,000 in Fehruary 2018 day of the month you were born, their inclination is to enter a [18]. Yet, even with lucrative bug bounties, attackers will negative number, to see how the program responds to the unex- often choose to retain vulnerabilities for their own use. pected. Much research supports this observation that attackers seek unlikely execution scenarios [12, 13]. The implication 2. InSearch of Advanced Defense Systems -The for verification engineers is that testing the most likely execu- Human Immune System tion scenarios provides very little value inhardening a system To move heyond the lintitations of security verification, in against security attacks. Instead, security verification engi- this work we explore vulnerability-tolerant secure systems. neers have to explore the most unlikely execution scenarios, These systems, despite having exposed vulnerabilities, are still and if they want high coverage vulnerability coverage, they resistant to attack because they incorporate defenses that make must work toward verifying all possible system configurations them impractically difficult to penetrate. Surprisingly, there is - which of course is impractical for non-trivial systems. As a prime example of a "security" system of this form in nature such, verification methods, which already fall short for func- in the human immune system. Consequently, we have studied tional verification, fall even shorter for security verification. In the human immune system for inspiration on how to achieve the end, large complex designs are released to the public with the goal of a vulnerability-tolerant secure system. yet-to-be-found security vulnerabilities lying within them. Given the higher bar for security verification, two additional 2.1. Properties of the Hnman Immune System forms of verification have emerged: red teaming and bug The human immune system is a very effective analogy for sys- bounties. Red teaming engages a team of attackers to ethically tem security, where the "system"is the human body, the "secu- attack a system to discover its vulnerabilities [14]. 'fypi.cally, rity defenses" are the immunity mechanisms inthe body, and the team is hired externally from a white-hat attacker company, the "attackers" are all forms of disease. It is very instrnctive to although some larger organizations have in-house red teams. examine the human immune system, since its capabilities are Red teams employ the same tools that black-hat attackers use significantly more advanced that even today's most advanced to penetrate the system, ranging from prepackaged exploits, security technologies. Specifically, the human immune system to random test-generation tools, and hand-crafted attacks. If has the following highly desirable defense properties: the red team is very skilled, the approach can be unmatched Vulnerability Tolerant - The human immune system is not for the degree and quality of the vulnerabilities found, but focused on finding and fixing vulnerabilities, in fact, our ultimately the system will still possess vulnerabilities. bodies are constantly under attack from diseases in our en- In a similar vein, the white-hat attack research community, vironment. Instead, the human immune system focuses on composed primarily of acadentics, security research compa- thwarting the attacks of diseases once they enter the body.