ANDI WILSON, ROSS SCHULMAN, KEVIN BANKSTON, AND TREY HERR BUGS IN THE SYSTEM A Primer on the Software Vulnerability Ecosystem and its Policy Implications JULY 2016 About the Authors About New America New America is committed to renewing American politics, Andi Wilson is a policy analyst at New America’s Open prosperity, and purpose in the Digital Age. We generate big Technology Institute, where she researches and writes ideas, bridge the gap between technology and policy, and about the relationship between technology and policy. curate broad public conversation. We combine the best of With a specific focus on cybersecurity, Andi is currently a policy research institute, technology laboratory, public working on issues including encryption, vulnerabilities forum, media platform, and a venture capital fund for equities, surveillance, and internet freedom. ideas. We are a distinctive community of thinkers, writers, researchers, technologists, and community activists who Ross Schulman is a co-director of the Cybersecurity believe deeply in the possibility of American renewal. Initiative and senior policy counsel at New America’s Open Find out more at newamerica.org/our-story. Technology Institute, where he focuses on cybersecurity, encryption, surveillance, and Internet governance. Prior to joining OTI, Ross worked for Google in Mountain About the Cybersecurity Initiative View, California. Ross has also worked at the Computer The Internet has connected us. Yet the policies and and Communications Industry Association, the Center debates that surround the security of our networks are for Democracy and Technology, and on Capitol Hill for too often disconnected, disjointed, and stuck in an Senators Wyden and Feingold. unsuccessful status quo. This is what New America’s Cybersecurity Initiative is designed to address. Working across our International Security program and the Open Kevin Bankston is the Director of New America’s Open Technology Institute, we believe that it takes a wider Technology Institute, where he works in the public interest network to face the multitude of diverse security issues. to promote policy and regulatory reforms to strengthen We engage across organizations, issue areas, professional communities by supporting open communications fields, and business sectors. And through events, writing networks, platforms, and technologies. He previously and research, our aim is to help improve cybersecurity in served as OTI’s Policy Director. ways that work—for the countries, for companies and for individuals. is a fellow with the Belfer Center's Cyber Security Trey Herr Our work is made possible through the generous support Project at the Harvard Kennedy School. He focuses of the William and Flora Hewlett Foundation, the Arizona on trends in state developed malicious software, the State University, Microsoft Corporation, Symantec Inc., The structure of criminal markets for malware components, Home Depot, Endgame Inc., and Facebook. and the proliferation of malware. Trey is also a non- resident fellow with New America’s Cybersecurity Initiative. About the Open Technology Institute Acknowledgments The Open Technology Institute (OTI) works at the intersection of technology and policy to ensure that every The authors would like to thank Chris Riley, Joe Hall, Katie community has equitable access to digital technology Moussouris and our other external reviewers for their and its benefits. We promote universal access to input and comments on an earlier version of this paper. communications technologies that are both open and This paper does not necessarily reflect their views. We secure, using a multidisciplinary approach that brings would also like to thank Donna Wentworth for her many together advocates, researchers, organizers, and valuable contributions to the paper. As well, we appreciate innovators. the extensive help of New America’s staff and fellows, especially Ian Wallace, Jordan McCarthy, Liz Woolery, Robert Morgus, and Robyn Greene. Contents Executive Summary 2 Introduction 4 What Are Vulnerabilities? 5 Who Discovers Vulnerabilities? 7 What Are Exploits and How Are They Used? 9 How Are Vulnerabilities Disclosed So They Can Be Patched? 10 How Are Vulnerabilities Patched (or Not)? 12 Which Laws Discourage Security Research and Vulnerability Disclosure? 13 What is the Vulnerabilities Market? 15 Why Governments Do (or Don't) Disclose the Vulnerabilities They Find or Buy 19 Conclusion: What Policies Will Foster the Discovery, Disclosure, and Patching of 20 Vulnerabilities? Notes 25 EXECUTIVE SUMMARY In recent years, a seemingly endless string of fix or exploit them. These bug-hunters range from massive data breaches in both the private and independent researchers, to small academic teams public sectors have been front-page news. Whether or security firms, to large tech companies working the target is a company like Sony or a government to improve their products, or even governments— agency like OPM, such breaches are very often made including our own government, and other much less possible by a software vulnerability—a “bug” in the rights-respecting states—seeking to use these flaws system—that was unknown or left unaddressed by for law enforcement or intelligence investigations. the target or its software vendor. After finding a vulnerability, the discoverer has three basic options: not disclosing the vulnerability The existence of such vulnerabilities—or “vulns” to the public or the software vendor; fully disclosing for short—is unavoidable. Software is complex and the vuln to the public, which in some cases may humans are fallible, so vulnerabilities are bound be the best way to get it patched but in others to occur. Much of cybersecurity can be reduced to may leave users of the software dangerously a constant race between the software developers exposed; and partial or “responsible” disclosure and security experts trying to discover and patch to the vendor so that they can fix the bug before it vulnerabilities, and the attackers seeking to uncover becomes public. Partial disclosure is often preferred and exploit those vulnerabilities. The question for because it can sometimes take months for a vendor policymakers is, what can they do to help speed the to fix their product, and even longer for all the discovery and patching of vulnerabilities so that affected users to update their software to patch the our computer systems—and therefore our economic security hole. stability, our national security, and consumers’ privacy—are safer? This paper is intended to Confusing the issue of disclosure is the fact that be a primer on the vulnerability ecosystem for there is a range of laws—such as the Computer policymakers and advocates seeking to answer that Fraud and Abuse Act, the Digital Millennium question, describing what vulns are, who discovers Copyright Act, and the Electronic Communications them, who buys them, how and when they do (or Privacy Act—that by their broad and vague terms don’t) get patched, and why. arguably criminalize and create civil penalties for actions that security researchers routinely engage There is a wide range of actors seeking to discover in while conducting legitimate security research. security flaws in software, whether to fix them, Unless reformed, these laws will continue to chill exploit them, or sell them to someone else who will researchers’ disclosure of critical vulnerabilities, for 2 OPEN TECHNOLOGY INSTITUTE fear that they will be sued or even thrown in jail. use of them for a variety of purposes, from law enforcement to foreign intelligence surveillance, Another disincentive to researchers’ disclosure and the longer they are secret and unpatched, the of vulnerabilities so that they can be patched is longer they are useful. Governments have to weigh the existence of open markets for vulnerabilities, the security value of disclosure versus the benefit of where researchers can often get top dollar from stockpiling and using vulnerabilities for their own criminal networks or governments seeking to purposes. exploit those vulnerabilities, or from intermediary agents who buy from researchers and then resell to In conclusion, we offer five initial policy criminals and states. Companies have responded by recommendations to ensure that more creating innovative vulnerability reward programs vulnerabilities are discovered and patched sooner: (VRPs), including “bug bounty” programs where (1) The U.S. government should minimize its they pay rewards for bugs that are submitted. participation in the vulnerability market, since it Some of these programs also seek to reduce the is the largest buyer in a market that discourages legal chill on researchers by promising not to sue researchers from disclosing vulns to be patched; those who submit through these programs. It is (2) The U.S. government should establish strong, sometimes difficult for these programs to compete clear procedures for government disclosure of the with the much more lucrative open market, but vulnerabilities it buys or discovers, with a heavy they give researchers who want to help improve presumption toward disclosure; (3) Congress should cybersecurity—and perhaps get a little cash or establish clear rules of the road for government recognition for their discovery—a legitimate avenue hacking in order to better protect cybersecurity and to pursue. civil liberties; (4) Government and industry should support bug bounty programs as an alternative to Researchers often have a range of incentives to the vulnerabilities market and