Proceedings of Workshop on Software Security Assurance Tools, Techniques, and Metrics

Total Page:16

File Type:pdf, Size:1020Kb

Proceedings of Workshop on Software Security Assurance Tools, Techniques, and Metrics NIST Special Publication 500-265 Proceedings of Workshop on Software Security Assurance Tools, Techniques, and Metrics Paul E. Black (workshop Chair) Michael Kass (Co-chair) Elizabeth Fong (editor) Information Technology Laboratory National Institute of Standards & Technology Gaithersburg MD 20899 February 2006 U.S. Department of Commerce Carlos M. Gutierrez. Secretary National Institute of Standards and Technology William Jeffrey, Director 1 Disclaimer: Any commercial product mentioned is for information only; it does not imply recommendation or endorsement by NIST nor does it imply that the products mentioned are necessarily the best available for the purpose. Software Security Assurance Tools, Techniques, and Metrics, SSATTM’05 ISBN # 1-59593-307-7/05/0011 2 Proceedings of Workshop on Software Security Assurance Tools, Techniques, and Metrics Paul E. Black (workshop chair) Michael Kass (co-chair) Elizabeth Fong (editor) Information Technology Laboratory National Institute of Standards and Technology Gaithersburg, MD 20899 ABSTRACT This is the proceedings of a workshop held on November 7 and 8, 2005 in Long Beach, California, USA, hosted by the Software Diagnostics and Conformance Testing Division, Information Technology Laboratory, of the National Institute of Standards and Technology. The workshop, “Software Security Assurance Tools, Techniques, and Metrics,” is one of a series in the NIST Software Assurance Measurement and Tool Evaluation (SAMATE) project, which is partially funded by DHS to help identify and enhance software security assurance (SSA) tools. The goal of this workshop is to discuss and refine the taxonomy of flaws and the taxonomy of functions, come to a consensus on which SSA functions should first have specifications and standards tests developed, gather SSA tools suppliers for “target practice” on reference datasets of code, and identify gaps or research needs in SSA functions. Keywords: Software assessment tools; software assurance; software metrics; software security; target practice, reference dataset; vulnerability 3 Foreword The workshop on “Software Assurance Tools, Techniques, and Metrics” was held 7-8 November 2005 at the Long Beach, California, USA, co-located with the Automated Software Engineering Conference 2005. This workshop consisted of eleven paper presentations for the first day. The second day morning consisted of “target practice” and the review of the nature of the reference dataset. The Program Committee consisted of the following: Freeland Abbott Georgia Tech Paul Ammann George Mason U. Elizabeth Fong NIST Michael Hicks U. of Maryland Michael Koo NIST Richard Lippmann MIT Robert A. Martin MITRE Corp. W. Bradley Martin NSA Nachiappan Nagappan Microsoft Research Samuel Redwine James Madison U. Ravi Sandhu George Mason U. Larry D. Wagoner NSA These proceedings have five main parts: • Summary • Workshop Announcement • Workshop Agenda • Reference Dataset Target Practice, and • Papers We thank those who worked to organize this workshop, particularly Elizabeth Fong, who handled much of the correspondence and Debra A. Brodbeck, who provided conference support. We appreciate the program committee for their efforts in reviewing the papers. We are grateful to NIST, especially the Software Diagnostics and Conformance Testing Division, for providing the organizers' time. On behalf of the program committee and the whole SAMATE team, thanks to everyone for taking their time and resources to join us. Sincerely, Dr. Paul E. Black 4 Table of Contents Summary …………………………………………………………………………………6 Workshop CALL FOR PAPERS …………………………………………………………7 Workshop Program ……………………………………………………………………….9 SAMATE Reference Dataset “Target Practice” ………………………………………...10 Where do Software Security Assurance Tools Add Value ……………………………...14 David Jackson and David Cooper Metrics that Matter ………………………………………………………………………22 Brian Chess The Case for Common Flaw Enumeration ……………………………………………...29 Robert Martin, Steven Christey, and Joe Jarzombek Seven Pernicious Kingdoms: A Taxonomy of Software Security Errors ………………36 Katrina Tsipenyuk, Brian Chess, and Gary McGraw A Taxonomy of Buffer Overflows for Evaluating Static and Dynamic Software Testing Tools……………………………………………………………………………………..44 Kendra Kratkiewicz and Richard Lippmann ABM: A Prototype for Benchmarking Source Code Analyzers ………………………..52 Tim Newsham and Brian Chess A Benchmark Suite for Behavior-Based Security Mechanisms ………………………..60 Dong Ye, Micha Moffie, and David Kaeli Testing and Evaluation of Virus Detectors for Handheld Devices ……………………...67 JoseAndre Morales, Peter J. Clarke, and Yi Deng Eliminating Buffer Overflows Using the Compiler or a Standalone Tool …………...…75 Thomas Plum and DavidM. Keaton A Security Software Architecture Description Language ………………………………82 Jie Ren and Richard N. Taylor Prioritization of Threats Using the K/M Algebra ……………………………………….90 Supreeth Vendataraman and Warren Harrison 5 Summary This is the proceeding of the workshop on Software Security Assurance Tools, Techniques, and Metrics, held on November 7 and 8, 2005 at Long Beach, California, USA, co-located with the Automated Software Engineering Conference 2005. It was organized by the Software Diagnostics and Conformance Testing Division, Information Technology Laboratory, National Institute of Standards and Technology (NIST). Forty- two people attended, including people from government, universities, tool vendors and service providers, and research companies. The workshop is one of a series in the NIST Software Assurance Measurement and Tool Evaluation (SAMATE) project, http://samate.nist.gov/ A previous workshop was on Defining the State of the Art in Software Security Tools, held on August 10 and 11, 2005 at the NIST in Gaithersburg, MD, USA. The call for papers resulted in eleven accepted papers, which were presented on the first day of the workshop. The second day was devoted to the discussion of reference dataset and target practice with three SSA tool vendors, and included an invited presentation “Correctness by construction: The case for constructive static verification” by Rob Chapman. The material and papers for the workshop were distributed on USB drives to the participants. The content of the USB drives was: • Introduction, • Workshop call of papers, • Workshop agenda, • Reference dataset target practice, • Flaw taxonomies, and • Accepted papers. Here are summaries of the workshop conclusions: • Today’s SSA tool does not add much value to real, large software products. • How do we score (rate) the risk of a piece of code is still a challenging question. • There is a need to harmonize the different taxonomy of vulnerabilities. • Very substantive feedbacks were gathered on the shared reference dataset. See write-up on SAMATE Reference Dataset “Target Practice” in this document. • There were consensuses that the first SSA specification and standard tests will be the source code scanner tools. 6 Workshop CALL FOR PAPERS (SSATTM'05) ------------------------------------------------------------------------------------------- National Institute of Standards and Technology (NIST) workshop on Software Security Assurance Tools, Techniques, and Metrics 7-8 November 2005 Co-located with ASE 2005, Long Beach, California, USA ------------------------------------------------------------------------------------------- Funded in part by the Department of Homeland Security (DHS), the National Institute of Standards and Technology (NIST) started a long-term, ambitious project to improve software security assurance tools. Security is the ability of a system to maintain the confidentiality, integrity, and availability of information processed and stored by a computer. Software security assurance tools are those that help software be more secure by building security into software or determining how secure software is. Among the project's goals are: (1) develop a taxonomy of software security flaws and vulnerabilities, (2) develop a taxonomy of software security assurance (SSA) tool functions and techniques which detect or prevent flaws, and (3) develop testable specifications of SSA functions and explicit tests to evaluate how closely tools implement the functions. The test materials include reference sets of buggy code. These goals extend into all phases of the software life cycle from requirements capture through design and implementation to operation and auditing. The goal of the workshop is to convene researchers, developers, and government and industrial users of SSA tools to • discuss and refine the taxonomy of flaws and the taxonomy of functions, which are under development, • come to a consensus on which SSA functions should first have specifications and standard tests developed, • gather SSA tools suppliers for "target practice" on reference datasets of code, and • identify gaps or research needs in SSA functions. REFERENCE DATASET "TARGET PRACTICE" Sets of code with known flaws and vulnerabilities, with corresponding correct versions, can be references for tool testing to make research easier and to be a standard of evaluation. Working with others, we will bring reference datasets of many types of code, like Java, C, binaries, and bytecode. We welcome contributions of code you've used. To help validate the reference datasets, we solicit proposals not exceeding 2 pages to participate in SSA tool "target practice" on the datasets. Tools can range from university projects to commercial products. Participation is intended to demonstrate the state of the art in finding flaws, consequently
Recommended publications
  • LZ Based Compression Benchmark on PE Files Introduction LZ Based
    LZ based compression benchmark on PE files Zsombor Paróczi Abstract: The key element in runtime compression is the compression algorithm, that is used during processing. It has to be small in enough in decompression bytecode size to fit in the final executable, yet have to provide the best compression ratio. In our work we benchmark the top LZ based compression methods on Windows PE files (both exe and dll files), and present the results including the decompres- sion overhead and the compression rates. Keywords: lz based compression, compression benchmark, PE benchmark Introduction During runtime executable compression an already compiled executable is modified in ways, that it still retains the ability to execute, yet the transformation produces smaller file size. The transformations usually exists from multiple steps, changing the structure of the executable by removing unused bytes, adding a compression layer or modifying the code in itself. During the code modifications the actual bytecode can change, or remain the same depending on the modification itself. In the world of x86 (or even x86-64) PE compression there are only a few benchmarks, since the ever growing storage capacity makes this field less important. Yet in new fields, like IOT and wearable electronics every application uses some kind of compression, Android apk-s are always compressed by a simple gzip compression. There are two mayor benchmarks for PE compression available today, the Maximum Compression benchmark collection [1] includes two PE files, one DLL and one EXE, and the Pe Compression Test [2] has four exe files. We will use the exe files during our benchmark, referred as small corpus.
    [Show full text]
  • Systems Administrator
    SYSTEMS ADMINISTRATOR SUMMARY The System Administrator’s role is to manage in-house computer software systems, servers, storage devices and network connections to ensure high availability and security of the supported business applications. This individual also participates in the planning and implementation of policies and procedures to ensure system provisioning and maintenance that is consistent with company goals, industry best practices, and regulatory requirements. ESSENTIAL DUTIES AND RESPONSIBILITIES: The Authority’s Senior Systems Manager may designate various other activities. The following statements are intended to describe the general nature and level of work being performed. They are not intended to be construed, as an exhaustive list of all responsibilities, duties and skills required of personnel so classified. Nothing in this job description restricts management's right to assign or reassign duties and responsibilities to this job at any time for any reason, including reasonable accommodation. Operational Management Manage virtual and physical servers with Windows Server 2003 – 2012 R2 and RHEL operating systems Manage Active Directory, Microsoft Office 365, and server and workstation patching with SCCM Manage the physical and virtual environment (VMware) of 300 plus servers Have familiarity with MS SQL server, windows clustering, domain controller setup, and group policy Ensure the security of the server infrastructure by implementing industry best-practices regarding privacy, security, and regulatory compliance. Develop and maintain documentation about current environment setup, standard operating procedures, and best practices. Manage end user accounts, permissions, access rights, and storage allocations in accordance with best- practices Perform and test routine system backups and restores. Anticipate, mitigate, identify, troubleshoot, and correct hardware and software issues on servers, and workstations.
    [Show full text]
  • The Advanced Persistent Threat (Or InformaOnized Force OperaOns)
    The Advanced Persistent Threat (or Informaonized Force Operaons) Michael K. Daly November 4, 2009 What is meant by Advanced, Persistent Threat? . Increasingly sophiscated cyber aacks by hosle organizaons with the goal of: Gaining access to defense, financial and other targeted informaon from governments, corporaons and individuals. Maintaining a foothold in these environments to enable future use and control. Modifying data to disrupt performance in their targets. APT: People With Money Who Discovered That Computers Are Connected APT in the News A Broad Problem Affecng Many Naons and Industries Is this a big deal? Is it new? . Yes, this is a very big deal. If “it” is the broad noon of the, spying, social engineering and bad stuff, then No, it is definitely not new. However, it is new (~2003) that naon states are widely leveraging the Internet to operate agents across all crical infrastructures. APT acvity is leveraging the expansion of the greater system of systems I’m not in the military. Why do I care? “[APT] possess the targeting competence to identify specific users in a unit or organization based on job function or presumed access to information. [APT] can use this access for passive monitoring of network traffic for intelligence collection purposes. Instrumenting these machines in peacetime may enable attackers to prepare a reserve of compromised machines that can be used during a crisis. [APT] … possess the technical sophistication to craft and upload rootkit and covert remote access software, creating deep persistent access to the
    [Show full text]
  • An Encrypted Payload Protocol and Target-Side Scripting Engine
    An Encrypted Payload Protocol and Target-Side Scripting Engine Dino A. Dai Zovi [email protected] Abstract into the remote process and then triggering the vulnera- Modern exploit payloads in commercial and open-source bility (the injection vector) in order to cause the remote penetration testing frameworks have grown much more process to execute the injected code (the payload). Tra- advanced than the traditional shellcode they replaced. ditionally, these payloads have been written in processor- These payloads permit interactive access without launch- specific assembly language, assembled, and extracted by ing a shell, network proxying, and many other rich fea- hand into reusable machine code components. These tures. Available payload frameworks have several lim- payloads were typically small and executed an operating itations, however. They make little use of encryption to system shell, causing them to be commonly referred to secure delivery and communications, especially in earlier as shellcode. Common shellcode variants included func- stage payloads. In addition, their richer features require tionality such as restoring dropped privileges, breaking a constant network connection to the penetration tester, out of chroot environments, and attaching the shell to making them unsuitable against mobile clients, such as an inbound or outbound network connection. This style laptops, PDAs, and smart phones. of payload construction was both labor and skill inten- This work introduces a first-stage exploit payload that sive. is able to securely establish an encrypted channel using With the growth of commercial penetration testing ElGamal key agreement and the RC4 stream cipher. The services and especially commercial penetration test- key agreement implementation requires only modular ing products, exploit payloads have gotten considerably exponentiation and RC4 also lends itself to an implemen- more capable and complex.
    [Show full text]
  • Lifecycle of the Advanced Persistent Threat
    Lifecycle of the Advanced Persistent Threat Counter Threat Unit research Table of Contents Summary .............................................................. 2 Background: Who, not what ......................................... 3 Organized .................................................................... 3 Efficient ...................................................................... 3 Tenacious .................................................................... 4 APT Lifecycle ......................................................... 5 Preparation .................................................................. 6 Initial intrusion .............................................................. 6 Primary Objective .................................................... 9 Expansion .................................................................... 9 Persistence ................................................................. 11 Search and Exfiltration .................................................... 12 Cleanup ...................................................................... 14 Conclusion .......................................................... 15 Copyright © 2012 Dell SecureWorks Page 1 Summary Advanced Persistent Threats (APT) are a serious concern in the today’s threat landscape. They represent a threat to an organization’s intellectual property, financial assets and reputation. In some cases, these threats target critical infrastructure and government suppliers, thereby threatening the victim country’s national
    [Show full text]
  • Designing a Framework for End User Applications
    Lincoln University Digital Thesis Copyright Statement The digital copy of this thesis is protected by the Copyright Act 1994 (New Zealand). This thesis may be consulted by you, provided you comply with the provisions of the Act and the following conditions of use: you will use the copy only for the purposes of research or private study you will recognise the author's right to be identified as the author of the thesis and due acknowledgement will be made to the author where appropriate you will obtain the author's permission before publishing any material from the thesis. Designing a Framework for End User Applications A thesis submitted in partial fulfilment of the requirements for Degree of Doctor of Philosophy in Software and Information Technology at Lincoln University By Yanbo Deng Lincoln University 2013 Abstract of a thesis submitted in partial fulfilment of the requirements for the Degree of Doctor of Philosophy in Software and Information Technology Designing a Framework for End User Applications by Yanbo Deng End user developers (i.e. non-professional developers) often create database applications to meet their immediate needs. However, these applications can often be difficult to generalise or adapt when requirements inevitably change. As part of this thesis, we visited several research institutions to investigate the issues of end user developed databases. We found that different user groups in the same organisation might require similar, but different, data management applications. However, the very specific designs used in most of these systems meant it was difficult to adapt them for other similar uses. In this thesis we propose a set of guidelines for supporting end user developers to create more flexible and adaptable data management applications.
    [Show full text]
  • Computer Viruses and Related Threats
    NIST Special Publication 500-166 Allia3 imfiB? NATTL INST OF STANDARDS & TECH R.I.C. A11 103109837 Computer Viruses and Technolo^ Related Threats: U.S. DEPARTMENT OF COMMERCE National Institute of A Management Guide Standards and Technology John P. Wack Lisa J. Camahan NIST PUBLICATIONS HP Jli m m QC - — 100 U57 500-166 1989 C.2 rhe National Institute of Standards and Technology^ was established by an act of Congress on March 3, 1901. The Institute's overall goal is to strengthen and advance the Nation's science and technology and facilitate their effective application for public benefit. To this end, the Institute conducts research to assure interna- tional competitiveness and leadership of U.S. industry, science and technology. NIST work involves development and transfer of measurements, standards and related science and technology, in support of continually improving U.S. productivity, product quality and reliability, innovation and underlying science and engineering. The Institute's technical work is performed by the National Measurement Laboratory, the National Engineering Laboratory, the National Computer Systems Laboratory, and the Institute for Materials Science and Engineering. The National Measurement Laboratory Provides the national system of physical and chemical measurement; Basic Standards^ coordinates the system with measurement systems of other nations Radiation Research and furnishes essential services leading to accurate and imiform Chemical Physics physical and chemical measurement throughout the Nation's scientific Analytical Chemistry community, industry, and commerce; provides advisory and research services to other Government agencies; conducts physical and chemical research; develops, produces, and distributes Standard Reference Materials; provides calibration services; and manages the National Standard Reference Data System.
    [Show full text]
  • Data Thieves: the Motivations of Cyber Threat Actors and Their Use
    Testimony Data Thieves The Motivations of Cyber Threat Actors and Their Use and Monetization of Stolen Data Lillian Ablon CT-490 Testimony presented before the House Financial Services Committee, Subcommittee on Terrorism and Illicit Finance, on March 15, 2018. For more information on this publication, visit www.rand.org/pubs/testimonies/CT490.html Testimonies RAND testimonies record testimony presented or submitted by RAND associates to federal, state, or local legislative committees; government-appointed commissions and panels; and private review and oversight bodies. Published by the RAND Corporation, Santa Monica, Calif. © Copyright 2018 RAND Corporation is a registered trademark. Limited Print and Electronic Distribution Rights This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions.html. www.rand.org Data Thieves: The Motivations of Cyber Threat Actors and Their Use and Monetization of Stolen Data Testimony of Lillian Ablon1 The RAND Corporation2 Before the Committee on Financial Services Subcommittee on Terrorism and Illicit Finance United States House of Representatives March 15, 2018 ood afternoon, Chairman Pearce, Ranking Member Perlmutter, and distinguished members of the subcommittee. Thank you for the invitation to testify at this important G hearing, “After the Breach: the Monetization and Illicit Use of Stolen Data.” Cybersecurity is a constant and growing challenge.
    [Show full text]
  • Effect of Payload Virus As Threatening for Internet Security
    International Journal of Modern Communication Technologies & Research (IJMCTR) ISSN: 2321-0850, Volume-2, Issue-9, September 2014 Effect of Payload Virus as Threatening For Internet Security Sana Mahmoud Eltayeb Almansour, Dr.Amin Babekir Abd elnabi Mustafa can really bypass security borders , stay in the victim devise or Abstract– Security remains at the top concerns of internet spread in the network without being noticed . usage despite this continuous effort of security engineers to make the Internet as safe as possible, the level of threats II. PAYLOAD:- continues to grow. Hackers are getting better and quicker every day, while In order to define payload it is best explained through the security programming companies do their best to stay the definition of vulnerability and exploit. A particular server, winner of the race. It is an everlasting fight against hacker and website or application is vulnerable when it can be intruders who are trying to breach the safety and privacy of compromised or overtaken by A hacker. their victim whom could be someone using the Internet. Now that we know that a particular application is vulnerable, In this paper one of the current, dangerous most effective we need to find an exploit for that vulnerability, which might method will be discussed, explaining how what is a payload, how be the piece of code, to overtake the victim's application it works, exploits, and the intrusion process, most definitely security precision needs to be taken. without prior authorization. To go further I need a tool, virus or Trojan sent along with the exploit, to be executed on the Index Terms– Payload, Hacking, Internet Security, Victim.
    [Show full text]
  • Software Assurance: an Overview of Current Industry Best Practices
    Software Assurance: An Overview of Current Industry Best Practices February 2008 Executive Summary Software Assurance: An Overview of Current Industry Best Practices Software underpins the information infrastructure that govern- ments, critical infrastructure providers and businesses worldwide depend upon for daily operations and business processes. These organizations widely and increasingly use commercial off-the- shelf software (“COTS”) to automate processes with information technology. At the same time, cyber attacks are becoming more stealthy and sophisticated, creating a complex and dynamic risk environment for IT-based operations that users are working to better understand and manage. As such, users have become in- creasingly concerned about the integrity, security and reliability of commercial software. To address these concerns and meet customer requirements, vendors have undertaken significant efforts to reduce vulner- abilities, improve resistance to attack and protect the integrity of the products they sell. These efforts are often referred to as “software assurance.” Software assurance is especially impor- tant for organizations critical to public safety and economic and national security. These users require a high level of confidence that commercial software is as secure as possible, something only achieved when software is created using best practices for secure software development. This white paper provides an overview of how SAFECode mem- bers approach software assurance, and how the use of best practices for software
    [Show full text]
  • An Introduction to Computer Security: the NIST Handbook U.S
    HATl INST. OF STAND & TECH R.I.C. NIST PUBLICATIONS AlllOB SEDS3fl NIST Special Publication 800-12 An Introduction to Computer Security: The NIST Handbook U.S. DEPARTMENT OF COMMERCE Technology Administration National Institute of Standards Barbara Guttman and Edward A. Roback and Technology COMPUTER SECURITY Contingency Assurance User 1) Issues Planniii^ I&A Personnel Trairang f Access Risk Audit Planning ) Crypto \ Controls O Managen»nt U ^ J Support/-"^ Program Kiysfcal ~^Tiireats Policy & v_ Management Security Operations i QC 100 Nisr .U57 NO. 800-12 1995 The National Institute of Standards and Technology was established in 1988 by Congress to "assist industry in the development of technology . needed to improve product quality, to modernize manufacturing processes, to ensure product reliability . and to facilitate rapid commercialization ... of products based on new scientific discoveries." NIST, originally founded as the National Bureau of Standards in 1901, works to strengthen U.S. industry's competitiveness; advance science and engineering; and improve public health, safety, and the environment. One of the agency's basic functions is to develop, maintain, and retain custody of the national standards of measurement, and provide the means and methods for comparing standards used in science, engineering, manufacturing, commerce, industry, and education with the standards adopted or recognized by the Federal Government. As an agency of the U.S. Commerce Department's Technology Administration, NIST conducts basic and applied research in the physical sciences and engineering, and develops measurement techniques, test methods, standards, and related services. The Institute does generic and precompetitive work on new and advanced technologies. NIST's research facilities are located at Gaithersburg, MD 20899, and at Boulder, CO 80303.
    [Show full text]
  • End User Device Strategy: Security Framework & Controls
    End User Device Strategy: Security Framework & Controls v1.2 February 2013 End User Device Strategy: Security Framework & Controls This document presents the security framework for End User Devices working with OFFICIAL information, and defines the control for mobile laptops to be used for both OFFICIAL and OFFICIAL­SENSITIVE. The selected standards aim to optimise for technology and security assurance, low cost and complexity, minimal 3rd party software, good user experience, wider competition and choice, and improved alignment to the consumer and commodity IT market. This document defines standards for a mobile laptop with a “thick” operating system installed, such as Linux, Windows or MacOS X. Forthcoming guidance will cover thin client devices, smartphones and tablets but is not expected to vary significantly from the key principles of this guidance. The scope of this document is central government departments, their agencies and related bodies. Wider public sector organisations can use this security framework as part of their broader compliance with the Public Services Network (PSN) codes of connection. Security IT Reform strategic goals for a security framework are to: • make optimum use of native security functions, avoiding third party products wherever possible • make better use of controls around the data and services where they can often be more effective, rather than adding additional complexity to devices • allow greater user responsibility to reduce security complexity, maintaining user experience for the majority of responsible
    [Show full text]