DOCSLIB.ORG

On Managing Vulnerabilities in AI/ML Systems

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
On Managing Vulnerabilities in AI/ML Systems On managing vulnerabilities in AI/ML systems Jonathan M. Spring April Galyardt jspringATseidotcmudotedu Software Engineering Institute CERT® Coordination Center Carnegie Mellon University Software Engineering Institute Pittsburgh, PA Carnegie Mellon University Pittsburgh, PA Allen D. Householder Nathan VanHoudnos CERT® Coordination Center Software Engineering Institute Software Engineering Institute Carnegie Mellon University Carnegie Mellon University Pittsburgh, PA Pittsburgh, PA ABSTRACT 1 INTRODUCTION This paper explores how the current paradigm of vulnerability man- The topic of this paper is more “security for automated reasoning” agement might adapt to include machine learning systems through and less “automated reasoning for security.” We will introduce the a thought experiment: what if flaws in machine learning (ML) questions that need to be answered in order to adapt existing vul- were assigned Common Vulnerabilities and Exposures (CVE) iden- nerability management practices to support automated reasoning tifiers (CVE-IDs)? We consider both ML algorithms and model ob- systems. We suggest answers to some of the questions, but some are jects. The hypothetical scenario is structured around exploring the quite thorny questions that may require a new paradigm of either changes to the six areas of vulnerability management: discovery, re- vulnerability management, development of automated reasoning port intake, analysis, coordination, disclosure, and response. While systems, or both. algorithm flaws are well-known in the academic research commu- First, some definitions. We follow the ®CERT Coordination Cen- nity, there is no apparent clear line of communication between this ter (CERT/CC) definition of vulnerability: “a set of conditions or research community and the operational communities that deploy behaviors that allows the violation of an explicit or implicit secu- and manage systems that use ML. The thought experiments identify rity policy” [23, §1.2]. We will follow Spring et al. [51] and define some ways in which CVE-IDs may establish some useful lines of machine learning (ML) as “a set of statistical tools that analyze data communication between these two communities. In particular, it to infer relationships and patterns. Ideally, the relationships and would start to introduce the research community to operational patterns inferred by ML will lead to a useful model of the object security concepts, which appears to be a gap left by existing efforts. or phenomenon that the data describes,” and define artificial intel- ligence (AI) as “a software agent that takes actions based on its CCS CONCEPTS environment.” To be concrete, this paper will focus on vulnerability • Computing methodologies ! Machine learning algorithms; management for just ML-enabled systems. • Software and its engineering ! Maintaining software; • Secu- One practical way to think of security services for an ML system rity and privacy ! Vulnerability management. is via the set of services a Computer Security Incident Response Team (CSIRT) might provide, which is produced by Forum of In- KEYWORDS cident Response and Security Teams (FIRST) and documented by Benetis et al. [2]. A complete risk management and security perspec- vulnerability management, machine learning, CVE-ID, prioritiza- tive on ML would include more than the CSIRT services framework. tion arXiv:2101.10865v1 [cs.CR] 22 Jan 2021 However, we will work from the assertion that to manage the se- ACM Reference Format: curity of an ML-enabled system, all CSIRT services will need to be Jonathan M. Spring, April Galyardt, Allen D. Householder, and Nathan able to handle ML systems. VanHoudnos. 2020. On managing vulnerabilities in AI/ML systems. In New Specifically, of the CSIRT services, we are carving out just vul- Security Paradigms Workshop 2020 (NSPW ’20), October 26–29, 2020, Virtual nerability management for discussion. The other services, as well conference. ACM, New York, NY, USA, 16 pages. https://doi.org/10.1145/ as wider issues such as risk management, all have challenges as 3442167.3442177 well, but we leave them as future work. Vulnerability management includes six services [2, §7]: Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed • Vulnerability discovery / research for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. • Vulnerability report intake For all other uses, contact the owner/author(s). • Vulnerability analysis NSPW ’20, October 26–29, 2020, Virtual conference • Vulnerability coordination © 2020 Copyright held by the owner/author(s). ACM ISBN 978-1-4503-8995-2/20/10. • Vulnerability disclosure https://doi.org/10.1145/3442167.3442177 • Vulnerability response NSPW ’20, October 26–29, 2020, Virtual conference Spring, Galyardt, Householder, and VanHoudnos These areas cover a wide range. They span the interface between There are two common axes that help distinguish vulnerabilities software developers, software users, security teams, and people who in modern security practice. One is within vulnerability identifica- find flaws in software. There is national infrastructure in multiple tion. The second is level of abstraction of the vulnerable product. countries dedicated to support these services and facilitate com- Sections 2.1 and 2.2, respectively, discuss these levels. Section 2.3 munication. For example, both the United States and the People’s summarizes background the the CVE project. Republic of China have National Vulnerability Databases (NVDs). FIRST, a global body, provides one definition of how to score the 2.1 Vulnerability Identification severity of vulnerabilities, Common Vulnerability Scoring Sys- Vulnerability identification and classification spans from scanning tem (CVSS). The Common Vulnerabilities and Exposures (CVE) individual systems to organizing vulnerabilities into categories to scheme is designed to assist such cataloging and ranking efforts. facilitate better programming principles. A number of vulnerabil- This paper’s goal is to facilitate the creation of a trading zone be- ity identification methods exist. CVE is perhaps the most widely tween ML engineers, software architects, and security practitioners. known, but there are others. Any trading zone requires a shared language, whether it is a physi- In increasing broadness along the identification and classification cal or intellectual trading zone [15]. All participants in a trading axis, we have: zone want something of value they can take back to their respective • Instance of a vulnerable product communities. These items of value are often “boundary objects,” • Vulnerability in a product (e.g., CVE-ID, VU#) which mark boundaries by being recognizable and functional in • Category of which a vulnerability is an example (e.g., Com- both cultures in the trading zone. Rawls and Mann [41] states that mon Weakness Enumeration (CWE), Open Web Application the Mitre Corporation (MITRE) produces identifiers, such as CVE Security Project (OWASP)) identifiers (CVE-IDs), in part for their value to trading zones as boundary objects. This background makes CVE-IDs an attractive 2.1.1 Instances of Vulnerable Products. On the very specific end and useful focal point for our thought experiments. of this spectrum, we have instances of a vulnerable product. An We will organize our exploration of a new paradigm for ML se- instance is a specific computer or service that uses a vulnerable curity around one hypothetical – what if flaws in ML systems were product. When an organization scans the systems it owns to per- assigned CVE-IDs? Sections 4 and 6 do the main work on exploring form asset management, it will find instances of a vulnerability. the thought experiment. Before we can answer this question, we Instances are often tagged as the association of a host or system first lay some background on the current ways of identifying soft- identifier accompanied by the ID of the vulnerability of which they ware vulnerabilities in Section 2. Section 3 will provide background are an instance.1 Instances may also be called findings or sightings. on the current state of adversarial attacks on ML algorithms. Sec- tion 4 then steps through each of the services areas of vulnerability 2.1.2 Vulnerable Products. Moving up from instances, we find vul- management to explore the impact of giving ML algorithm flaws nerable products. The practitioner community’s expectation is that CVE-IDs. Section 5 explores the ML algorithm thought experiment a product is some sort of artificial information processing system. from the perspective of CVE Numbering Authorities (CNAs). Sec- This definition is vague because a vulnerable product is usually tion 6 steps through each of the service areas to explore the impact the thing that security practitioners say “has” the vulnerability, of giving ML model object flaws CVE-IDs. as defined above. Since a vulnerability may be introduced bya software defect, configuration decision, design decision, system interaction, or environmental mismatch, the “product” that has a vulnerability cannot be constrained much. Section 2.2 will discuss different categories of vulnerable products. 2 VULNERABILITY BACKGROUND CVE-IDs are most closely associated with products. Section 2.3 Expanding on the definition of vulnerability cited in Section 1, a will detail the CVE program.
Load more

Recommended publications
  • Mary Beth Chrissis, Mike Konrad, Sandra
    ptg Process Areas by Category Process Management OPD Organizational Process Definition OPF Organizational Process Focus OPM Organizational Performance Management OPP Organizational Process Performance OT Organizational Training Project Management IPM Integrated Project Management PMC Project Monitoring and Control PP Project Planning QPM Quantitative Project Management REQM Requirements Management RSKM Risk Management SAM Supplier Agreement Management Engineering PI Product Integration RD Requirements Development TS Technical Solution ptg VAL Validation VER Verification Support CAR Causal Analysis and Resolution CM Configuration Management DAR Decision Analysis and Resolution MA Measurement and Analysis PPQA Process and Product Quality Assurance Generic Goals and Practices GG1 Achieve Specific Goals GP 1.1 Perform Specific Practices GG2 Institutionalize a Managed Process GP2.1 Establish an Organizational Policy GP 2.2 Plan the Process GP 2.3 Provide Resources GP 2.4 Assign Responsibility GP 2.5 Train People GP 2.6 Control Work Products GP 2.7 Identify and Involve Relevant Stakeholders GP 2.8 Monitor and Control the Process GP 2.9 Objectively Evaluate Adherence GP 2.10 Review Status with Higher Level Management GG3 Institutionalize a Defined Process GP 3.1 Establish a Defined Process GP 3.2 Collect Process Related Experiences ptg CMMI® for Development Third Edition ptg Wow! eBook <WoweBook.Com> The SEI Series in Software Engineering Visit informit.com/sei for a complete list of available products. ptg he SEI Series in Software Engineering represents is a collaborative Tundertaking of the Carnegie Mellon Software Engineering Institute (SEI) and Addison-Wesley to develop and publish books on software engineering and related topics. The common goal of the SEI and Addison-Wesley is to provide the most current information on these topics in a form that is easily usable by practitioners and students.
    [Show full text]
  • NYU School of Law Outline: Trademarks, Barton Beebe
    NYU School of Law Outline: Trademarks, Barton Beebe Will Frank (Class of 2011) Fall Semester, 2009 Contents 1 Introduction to Trademark and Unfair Competition Law 3 1.1 Sources and Nature of Rights . 4 1.2 The Nature of Unfair Competition Law . 4 1.3 Purposes of Trademark Law . 4 1.4 The Lanham Act . 5 2 Distinctiveness 6 2.1 The Spectrum of Distinctiveness . 7 2.2 Descriptiveness and Secondary Meaning . 7 2.3 Generic Terms . 8 2.4 Distinctiveness of Nonverbal Identifiers (Logos, Packages, Prod- uct Design, Colors) . 9 2.4.1 Different Tests/Standards? . 9 2.4.2 Expanding the Types of Nonverbal Marks . 9 2.4.3 The Design/Packaging Distinction . 10 2.4.4 Trade Dress Protection After Wal-Mart . 10 2.5 The Edge of Protection: Subject Matter Exclusions? . 12 2.5.1 Exotic Source-Identifiers . 12 2.6 Review . 12 3 Functionality 13 3.1 The Concept . 14 3.2 The Scope of the Doctrine . 15 3.3 The Modern Approach . 15 3.4 Post-TrafFix Devices Applications . 17 4 Use 18 4.1 As a Jurisdictional Prerequisite . 18 4.2 As a Prerequisite for Acquiring Rights . 18 4.2.1 Actual Use . 18 4.2.2 Constructive Use . 19 1 4.3 \Surrogate" Uses . 20 4.3.1 By Affiliates . 20 4.4 The Public as Surrogate . 20 4.5 Loss of Rights . 21 4.5.1 Abandonment Through Non-Use . 21 4.5.2 Abandonment Through Failure to Control Use . 21 5 Registration 22 5.1 The Registration Process . 22 5.1.1 Overview .
    [Show full text]
  • Protecting Folklore Under Modern Intellectual Property Regimes
    American University Law Review Volume 48 | Issue 4 Article 2 1999 Protecting Folklore Under Modern Intellectual Property Regimes: A Reappraisal of the Tensions Between Individual and Communal Rights in Africa and the United States Paul Kuruk Follow this and additional works at: http://digitalcommons.wcl.american.edu/aulr Part of the Intellectual Property Commons, and the International Law Commons Recommended Citation Kuruk, Paul. “ Protecting Folklore Under Modern Intellectual Property Regimes: A Reappraisal of the Tensions Between Individual and Communal Rights in Africa and the United States.” American University Law Review 48, no.4 (April, 1999): 769-843. This Article is brought to you for free and open access by the Washington College of Law Journals & Law Reviews at Digital Commons @ American University Washington College of Law. It has been accepted for inclusion in American University Law Review by an authorized administrator of Digital Commons @ American University Washington College of Law. For more information, please contact [email protected]. Protecting Folklore Under Modern Intellectual Property Regimes: A Reappraisal of the Tensions Between Individual and Communal Rights in Africa and the United States Keywords Folklore, Intellectual Property Law, Regional Arrangements This article is available in American University Law Review: http://digitalcommons.wcl.american.edu/aulr/vol48/iss4/2 PROTECTING FOLKLORE UNDER MODERN INTELLECTUAL PROPERTY REGIMES: A REAPPRAISAL OF THE TENSIONS BETWEEN INDIVIDUAL AND COMMUNAL RIGHTS IN AFRICA AND THE UNITED STATES * PAUL KURUK TABLE OF CONTENTS Introduction.............................................................................................. I. Folklore Under Traditional Systems........................................ 776 A. Nature of Folklore ............................................................. 776 B. Protection Under Customary Law .................................... 780 1. Social groups and rights in folklore ...........................
    [Show full text]
  • Basics Track: Franchisor's Intellectual Property and How to Protect It
    International Franchise Association 52nd Annual Legal Symposium May 5-7, 2019 Washington, DC Basics Track: Franchisor’s Intellectual Property and How to Protect It Christopher Kelly Partner, Wiley Rein LLP Washington, D.C. Vincent Frantz Attorney, Cheng Cohen LLC Chicago, Illinois 1 Table of Contents I. The Four Primary Types of Intellectual Property ....................................................... 4 A. Trademarks ........................................................................................................... 4 1. Selecting a Protectable Mark ............................................................................. 5 2. Common Law Trademarks ................................................................................. 7 3. Domestic Trademark Registration ...................................................................... 8 4. International Trademark Protection .................................................................. 11 5. Proper Use of Trademarks ............................................................................... 12 6. Enforcement of Rights/Trademark Infringement ............................................... 13 B. Trade Secrets ...................................................................................................... 14 1. Elements of a Trade Secret.............................................................................. 15 2. Misappropriation of Trade Secrets ................................................................... 16 3. Federal Protection for Trade Secrets
    [Show full text]
  • Introduction to Trademark Law and Practice
    WORLD INTELLECTUAL PROPERTY ORGANIZATION INTRODUCTION TO TRADEMARK LAW & PRACTICE THE BASIC CONCEPTS A WIPO TRAINING MANUAL GENEVA 1993 (Second Edition) ( ( WIPO PUBLICATION No 653 (El ISBN 92-805-0167-4 WIPO 1993 PREFACE The present publication is the second edition of a volume of the same title that was published by the World Intellectual Property Organization (WIPO) in 1987 and reprinted in 1990. The first edition was written by Mr. Douglas Myall, former Assistant Registrar of Trade Marks, United Kingdom. The present revised edition of the publication has been prepared by Mr. Gerd Kunze, Vevey, Switzerland, and reflects his extensive expertise and experience in the administration of the trademark operations of a large international corporation, Nestle S. A., as well as his intensive involvement, as a leading representative of several international non-governmental organizations, in international meetings convened by WIPO. This publication is intended to provide a practical introduction to trademark administration for those with little or no experience of the subject but who may have to deal with it in an official or business capacity. Throughout the text, the reader is invited to answer questions relating to the text. Those questions are numbered to correspond to the answers that are given, with a short commentary, in Appendix I. Arpad Bogsch Director General World Intellectual Property Organization February 1993 ( ( LIST OF CONTENTS CHAPTER 1. TRADEMARKS AND OTHER SIGNS: A GENERAL SURVEY 7 1.1 Use of trademarks in commerce . 9 1.2 What is a trademark?. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 9 1.3 Need for legal protection .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 10 1.4 How can a trademark be protected? .
    [Show full text]
  • Letters to the Editor
    Articles Letters to the Editor Research Priorities for is a product of human intelligence; we puter scientists, innovators, entrepre - cannot predict what we might achieve neurs, statisti cians, journalists, engi - Robust and Beneficial when this intelligence is magnified by neers, authors, professors, teachers, stu - Artificial Intelligence: the tools AI may provide, but the eradi - dents, CEOs, economists, developers, An Open Letter cation of disease and poverty are not philosophers, artists, futurists, physi - unfathomable. Because of the great cists, filmmakers, health-care profes - rtificial intelligence (AI) research potential of AI, it is important to sionals, research analysts, and members Ahas explored a variety of problems research how to reap its benefits while of many other fields. The earliest signa - and approaches since its inception, but avoiding potential pitfalls. tories follow, reproduced in order and as for the last 20 years or so has been The progress in AI research makes it they signed. For the complete list, see focused on the problems surrounding timely to focus research not only on tinyurl.com/ailetter. - ed. the construction of intelligent agents making AI more capable, but also on Stuart Russell, Berkeley, Professor of Com - — systems that perceive and act in maximizing the societal benefit of AI. puter Science, director of the Center for some environment. In this context, Such considerations motivated the “intelligence” is related to statistical Intelligent Systems, and coauthor of the AAAI 2008–09 Presidential Panel on standard textbook Artificial Intelligence: a and economic notions of rationality — Long-Term AI Futures and other proj - Modern Approach colloquially, the ability to make good ects on AI impacts, and constitute a sig - Tom Dietterich, Oregon State, President of decisions, plans, or inferences.
    [Show full text]
  • ANNEX 2 Provisions of the Convention for the Protection of Industrial Property (1967) Referred to in the TRIPS Agreement* TABLE
    ANNEX 2 Provisions of the Convention for the Protection of Industrial Property (1967) referred to in the TRIPS Agreement TABLE OF CONTENTS Article 1 Establishment of the Union; Scope of Industrial Property Article 2 National Treatment for Nationals of Countries of the Union Article 3 Same Treatment for Certain Categories of Persons as for Nationals of Countries of the Union Article 4 A to I. Patents, Utility Models, Industrial Designs, Marks, Inventors’ Certificates: Right of Priority - G. Patents: Division of the Application Article 4bis Patents: Independence of Patents Obtained for the Same Invention in Different Countries Article 4ter Patents: Mention of the Inventor in the Patent Article 4quater Patents: Patentability in Case of Restrictions of Sale by Law Article 5 A. Patents: Importation of Articles; Failure to Work or Insufficient Working; Compulsory Licences. B. Industrial Designs: Failure to Work; Importation of Articles C. Marks: Failure to Use; Different Forms; Use by Co-proprietors D. Patents, Utility Models, Marks, Industrial Designs: Marking Article 5bis All Industrial Property Rights: Period of Grace for the Payment of Fees for the Maintenance of Rights; Patents: Restoration Article 5ter Patents: Patented Devices Forming Part of Vessels, Aircraft, or Land Vehicles [WTO Secretariat note] The provisions reproduced herein are referred to in Article 2.1 and in other provisions of the TRIPS Agreement. This volume uses the titles that the International Bureau of WIPO has given to them to facilitate their identification. The Table of Contents is added for the convenience of the reader. Neither the titles nor the Table of Contents appear in the signed (French) text.
    [Show full text]
  • The Future of AI: Opportunities and Challenges
    The Future of AI: Opportunities and Challenges Puerto Rico, January 2-5, 2015 ! Ajay Agrawal is the Peter Munk Professor of Entrepreneurship at the University of Toronto's Rotman School of Management, Research Associate at the National Bureau of Economic Research in Cambridge, MA, Founder of the Creative Destruction Lab, and Co-founder of The Next 36. His research is focused on the economics of science and innovation. He serves on the editorial boards of Management Science, the Journal of Urban Economics, and The Strategic Management Journal. & Anthony Aguirre has worked on a wide variety of topics in theoretical cosmology, ranging from intergalactic dust to galaxy formation to gravity physics to the large-scale structure of inflationary universes and the arrow of time. He also has strong interest in science outreach, and has appeared in numerous science documentaries. He is a co-founder of the Foundational Questions Institute and the Future of Life Institute. & Geoff Anders is the founder of Leverage Research, a research institute that studies psychology, cognitive enhancement, scientific methodology, and the impact of technology on society. He is also a member of the Effective Altruism movement, a movement dedicated to improving the world in the most effective ways. Like many of the members of the Effective Altruism movement, Geoff is deeply interested in the potential impact of new technologies, especially artificial intelligence. & Blaise Agüera y Arcas works on machine learning at Google. Previously a Distinguished Engineer at Microsoft, he has worked on augmented reality, mapping, wearable computing and natural user interfaces. He was the co-creator of Photosynth, software that assembles photos into 3D environments.
    [Show full text]
  • Beneficial AI 2017
    Beneficial AI 2017 Participants & Attendees 1 Anthony Aguirre is a Professor of Physics at the University of California, Santa Cruz. He has worked on a wide variety of topics in theoretical cosmology and fundamental physics, including inflation, black holes, quantum theory, and information theory. He also has strong interest in science outreach, and has appeared in numerous science documentaries. He is a co-founder of the Future of Life Institute, the Foundational Questions Institute, and Metaculus (http://www.metaculus.com/). Sam Altman is president of Y Combinator and was the cofounder of Loopt, a location-based social networking app. He also co-founded OpenAI with Elon Musk. Sam has invested in over 1,000 companies. Dario Amodei is the co-author of the recent paper Concrete Problems in AI Safety, which outlines a pragmatic and empirical approach to making AI systems safe. Dario is currently a research scientist at OpenAI, and prior to that worked at Google and Baidu. Dario also helped to lead the project that developed Deep Speech 2, which was named one of 10 “Breakthrough Technologies of 2016” by MIT Technology Review. Dario holds a PhD in physics from Princeton University, where he was awarded the Hertz Foundation doctoral thesis prize. Amara Angelica is Research Director for Ray Kurzweil, responsible for books, charts, and special projects. Amara’s background is in aerospace engineering, in electronic warfare, electronic intelligence, human factors, and computer systems analysis areas. A co-founder and initial Academic Model/Curriculum Lead for Singularity University, she was formerly on the board of directors of the National Space Society, is a member of the Space Development Steering Committee, and is a professional member of the Institute of Electrical and Electronics Engineers (IEEE).
    [Show full text]
  • F.3. the NEW POLITICS of ARTIFICIAL INTELLIGENCE [Preliminary Notes]
    F.3. THE NEW POLITICS OF ARTIFICIAL INTELLIGENCE [preliminary notes] MAIN MEMO pp 3-14 I. Introduction II. The Infrastructure: 13 key AI organizations III. Timeline: 2005-present IV. Key Leadership V. Open Letters VI. Media Coverage VII. Interests and Strategies VIII. Books and Other Media IX. Public Opinion X. Funders and Funding of AI Advocacy XI. The AI Advocacy Movement and the Techno-Eugenics Movement XII. The Socio-Cultural-Psychological Dimension XIII. Push-Back on the Feasibility of AI+ Superintelligence XIV. Provisional Concluding Comments ATTACHMENTS pp 15-78 ADDENDA pp 79-85 APPENDICES [not included in this pdf] ENDNOTES pp 86-88 REFERENCES pp 89-92 Richard Hayes July 2018 DRAFT: NOT FOR CIRCULATION OR CITATION F.3-1 ATTACHMENTS A. Definitions, usage, brief history and comments. B. Capsule information on the 13 key AI organizations. C. Concerns raised by key sets of the 13 AI organizations. D. Current development of AI by the mainstream tech industry E. Op-Ed: Transcending Complacency on Superintelligent Machines - 19 Apr 2014. F. Agenda for the invitational “Beneficial AI” conference - San Juan, Puerto Rico, Jan 2-5, 2015. G. An Open Letter on Maximizing the Societal Benefits of AI – 11 Jan 2015. H. Partnership on Artificial Intelligence to Benefit People and Society (PAI) – roster of partners. I. Influential mainstream policy-oriented initiatives on AI: Stanford (2016); White House (2016); AI NOW (2017). J. Agenda for the “Beneficial AI 2017” conference, Asilomar, CA, Jan 2-8, 2017. K. Participants at the 2015 and 2017 AI strategy conferences in Puerto Rico and Asilomar. L. Notes on participants at the Asilomar “Beneficial AI 2017” meeting.
    [Show full text]
  • 2010 CERT Research Report
    The CERT® Program’s research mission is to leverage our operational experience and access to state-of-the-art research to fill gaps for our customers, collaborate with the cyber security research community, and provide longer-range perspectives on cyber-security research, technology, and policy. Table of Contents CERT Research Vision .................................................... 2 Executive Summary ..................................................... 3 2010 Research Report Abstracts . 4 CERT in the News ....................................................... 7 Special Project: Recommending Cyber Security Research Topics ................. 9 Critical Code ............................................................11 Insider Threat .......................................................... 14 Insider Threat Vulnerability Assessment Measurement . 17 Modeling and Analysis of Insider Fraud . 19 Insider Threat Lab . 21 Insider Threat in the Financial Services Sector . 22 Preventing the Federal Government from Being the Victim of Identity Theft . 24 Secure Coding ......................................................... 25 Secure Coding Initiative . 28 Software Security Assurance ............................................. 34 Building Assured Systems Framework (BASF) . 37 Supply Chain Assurance . 39 Measuring Software Security Assurance . 42 Security Requirements Engineering . 45 Using Trusted Hardware as a Foundation for Cyber Security . 47 Analysis of Catastrophic Failures . 50 Complexity Modeling and Analysis . 52 Science of Cyber
    [Show full text]
  • Public Response to RFI on AI
    White House Office of Science and Technology Policy Request for Information on the Future of Artificial Intelligence Public Responses September 1, 2016 Respondent 1 Chris Nicholson, Skymind Inc. This submission will address topics 1, 2, 4 and 10 in the OSTP’s RFI: • the legal and governance implications of AI • the use of AI for public good • the social and economic implications of AI • the role of “market-shaping” approaches Governance, anomaly detection and urban systems The fundamental task in the governance of urban systems is to keep them running; that is, to maintain the fluid movement of people, goods, vehicles and information throughout the system, without which it ceases to function. Breakdowns in the functioning of these systems and their constituent parts are therefore of great interest, whether it be their energy, transport, security or information infrastructures. Those breakdowns may result from deteriorations in the physical plant, sudden and unanticipated overloads, natural disasters or adversarial behavior. In many cases, municipal governments possess historical data about those breakdowns and the events that precede them, in the form of activity and sensor logs, video, and internal or public communications. Where they don’t possess such data already, it can be gathered. Such datasets are a tremendous help when applying learning algorithms to predict breakdowns and system failures. With enough lead time, those predictions make pre- emptive action possible, action that would cost cities much less than recovery efforts in the wake of a disaster. Our choice is between an ounce of prevention or a pound of cure. Even in cases where we don’t have data covering past breakdowns, algorithms exist to identify anomalies in the data we begin gathering now.
    [Show full text]