DOCSLIB.ORG

Coverage Validator Help

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Coverage Validator by Software Verification Copyright Software Verify Limited (c) 2002-2015 Coverage Validator Source code coverage analysis for Win32 applications by Software Verification Welcome to the Coverage Validator software tool. Coverage Validator is a source code coverage analysis software tool. Using Coverage Validator you can identify unvisited functions and unvisited lines in your source code. This information can be used to inform your testing program to ensure that you test all of your software. Coverage Validator provides numerous features to allow you integrate Coverage Validator into your regression tests and unit tests. This allows you to monitor the progress of your software testing during your development programme. We hope you will find this document useful. Coverage Validator Help Copyright © 2002-2015 Software Verification Limited All rights reserved. No parts of this work may be reproduced in any form or by any means - graphic, electronic, or mechanical, including photocopying, recording, taping, or information storage and retrieval systems - without the written permission of the publisher. Products that are referred to in this document may be either trademarks and/or registered trademarks of the respective owners. The publisher and the author make no claim to these trademarks. While every precaution has been taken in the preparation of this document, the publisher and the author assume no responsibility for errors or omissions, or for damages resulting from the use of information contained in this document or from the use of programs and source code that may accompany it. In no event shall the publisher and the author be liable for any loss of profit or any other commercial damage caused or alleged to have been caused directly or indirectly by this document. Printed: March 2016 in United Kingdom. I Coverage Validator Help Table of Contents Foreword 1 Part I Overview 3 1 Notatio.n.. .u..s.e..d.. .i.n.. .t.h..i.s.. .h..e..l.p.. ..................................................................................................... 3 2 Introdu.c.i.n..g.. .C...o..v.e..r..a..g..e.. .V..a..l.i.d..a..t.o..r. ............................................................................................ 5 3 Why Co..v.e..r..a..g..e.. .V..a..l.i.d..a..t.o..r.?.. ..................................................................................................... 6 4 What d.o.. .y.o..u.. .n..e..e..d.. .t.o.. .r..u..n.. .C..o..v..e..r.a..g..e.. .V..a..l.i.d..a..t.o..r.?.. ..................................................................... 8 5 Buying .C...o..v.e..r..a..g..e.. .V..a..l.i.d..a..t.o..r. .a..n..d.. .s..u..p..p..o..r.t. ............................................................................... 9 6 How d.o..e..s.. .C..o..v..e..r.a..g..e.. .V..a..l.i.d..a..t.o..r. .w...o..r.k..?.. .................................................................................. 10 7 Suppo.r..t.e..d.. .C..o..m...p..i.l.e..r.s.. .......................................................................................................... 11 8 User P.e..r..m...i.s.s..i.o..n..s. ................................................................................................................. 13 Part II Getting Started 19 1 Enabli.n..g.. .D..e..b..u..g..g..i.n..g.. ............................................................................................................ 19 Microsoft Com...p...i.l.e..r..s.. ............................................................................................................................................ 19 Borland Comp..i.l.e..r..s.. .(..C..+..+.. .a..n...d.. .D..e...l.p..h..i.).. ................................................................................................................. 20 MinGW - gcc/g..+..+.. .................................................................................................................................................... 20 QtCreator .......................................................................................................................................................... 20 Other Compile...r.s... ................................................................................................................................................... 21 2 Quick .S..t.a..r..t. .......................................................................................................................... 21 Part III The User Interface 26 1 Menu .R..e..f.e..r.e..n..c..e.. .................................................................................................................. 26 File menu .......................................................................................................................................................... 27 Edit menu .......................................................................................................................................................... 28 Configure me.n...u.. .................................................................................................................................................... 28 Managers me.n...u.. .................................................................................................................................................... 29 Query menu .......................................................................................................................................................... 29 Tools menu .......................................................................................................................................................... 30 Data View s m.e..n...u.. .................................................................................................................................................. 30 Softw are Upd.a..t.e..s... .m...e...n..u.. ...................................................................................................................................... 31 Help menu .......................................................................................................................................................... 32 2 Toolba..r. .R...e..f.e..r.e..n..c..e.. .............................................................................................................. 32 3 The st.a..t.u..s. .b..a..r. ...................................................................................................................... 33 4 Keybo.a..r.d.. .S...h..o..r.t.c..u..t.s. ............................................................................................................. 35 5 Icons ................................................................................................................................... 36 6 The m.a..i.n.. .d..i.s..p..l.a..y. ................................................................................................................. 36 Summary .......................................................................................................................................................... 37 Coverage .......................................................................................................................................................... 39 Branch Cover.a..g..e.. ................................................................................................................................................... 47 Functions .......................................................................................................................................................... 56 Directories .......................................................................................................................................................... 65 DLLs .......................................................................................................................................................... 73 Copyright © 2002-2015 Software Verification Limited Contents II Unit Tests .......................................................................................................................................................... 80 Files and lines.. ........................................................................................................................................................ 85 Diagnostic .......................................................................................................................................................... 93 7 User In..t.e..r.f.a..c..e.. .M...o..d..e.. ............................................................................................................ 95 8 Setting..s.. ............................................................................................................................... 96 Data Collectio.n.. .S..e...t.t.i.n...g..s.. ....................................................................................................................................... 96 Data Displa..y....................................................................................................................................................... 99 Colours ......................................................................................................................................... 99 Data Display ......................................................................................................................................... 101 Code View ing ........................................................................................................................................
Recommended publications
  • Essional-Software-Testing-With-Visual

    Essional-Software-Testing-With-Visual

    Arnold ffirs.tex V4 - 08/02/2007 10:24am Page v Professional Software Testing with Visual Studio® 2005 Team System ToolsforSoftwareDevelopers andTestEngineers Tom Arnold Dominic Hopton Andy Leonard Mike Frost Wiley Publishing, Inc. Arnold frontcover1.tex V4 - 08/02/2007 10:23am Page ii Arnold frontcover1.tex V4 - 08/02/2007 10:23am Page i Professional Software Testing with Visual Studio® 2005 Team System Introduction . .xxiii Chapter 1: Introduction to VSTEST and VSTESD . 1 Chapter 2: A Quick Tour of VSTEST and VSTESD . 15 Chapter 3: Unit Testing with VSTEST and VSTESD . 41 Chapter 4: Testing the Database . 73 Chapter 5: Web Testing . 125 Chapter 6: Using Manual, Ordered, and Generic Test Types . 165 Chapter 7: Load Testing . 191 Chapter 8: Using Code Analysis and Dynamic Analysis. 243 Chapter 9: VSTEST and VSTESD within the Software Development LifeCycle.....................................................273 Appendix A: Installing Team Explorer . 305 Appendix B: Creating and Running a Web Test: A High-Level Walk-Through ...............................................313 Appendix C: Creating and Running a Unit Test: A High-Level Walk-Through ...............................................319 Appendix D: Creating and Running a Load Test: A High-Level Walk-Through ...............................................327 Appendix E: Creating and Running a Manual Test: A High-Level Walk-Through ...............................................339 Appendix F: Other Sources of Information . 347 Index....................................................................351 Arnold frontcover1.tex V4 - 08/02/2007 10:23am Page ii Arnold ffirs.tex V4 - 08/02/2007 10:24am Page iii Professional Software Testing with Visual Studio® 2005 Team System Arnold ffirs.tex V4 - 08/02/2007 10:24am Page iv Arnold ffirs.tex V4 - 08/02/2007 10:24am Page v Professional Software Testing with Visual Studio® 2005 Team System ToolsforSoftwareDevelopers andTestEngineers Tom Arnold Dominic Hopton Andy Leonard Mike Frost Wiley Publishing, Inc.
  • Codewarrior™ Development Studio for Freescale™ 56800/E Digital Signal Controllers: Mc56f8xxx/Dsp5685x Family Targeting Manual

    Codewarrior™ Development Studio for Freescale™ 56800/E Digital Signal Controllers: Mc56f8xxx/Dsp5685x Family Targeting Manual

    CodeWarrior™ Development Studio for Freescale™ 56800/E Digital Signal Controllers: MC56F8xxx/DSP5685x Family Targeting Manual Revised: 19 June 2006 Freescale, the Freescale logo, and CodeWarrior are trademarks or registered trademarks of Freescale Corporation in the United States and/or other countries. All other trade names and trademarks are the property of their respective owners. Copyright © 2006 by Freescale Semiconductor company. All rights reserved. No portion of this document may be reproduced or transmitted in any form or by any means, electronic or me- chanical, without prior written permission from Freescale. Use of this document and related materials is gov- erned by the license agreement that accompanied the product to which this manual pertains. This document may be printed for non-commercial personal use only in accordance with the aforementioned license agreement. If you do not have a copy of the license agreement, contact your Freescale representative or call 1-800-377-5416 (if outside the U.S., call +1-512-996-5300). Freescale reserves the right to make changes to any product described or referred to in this document without further notice. Freescale makes no warranty, representation or guarantee regarding the merchantability or fitness of its products for any particular purpose, nor does Freescale assume any liability arising out of the application or use of any product described herein and specifically disclaims any and all liability. Freescale software is not authorized for and has not been designed, tested, manufactured, or intended for use in developing applications where the failure, malfunc- tion, or any inaccuracy of the application carries a risk of death, serious bodily injury, or damage to tangible property, including, but not limited to, use in factory control systems, medical devices or facilities, nuclear facil- ities, aircraft navigation or communication, emergency systems, or other applications with a similar degree of potential hazard.
  • Did Your Tests Pass Or Fail? Answering with Automation

    Did Your Tests Pass Or Fail? Answering with Automation

    Presentation P R E S E N T A T I O N Bio T6 Thursday, March 8, 2001 11:30 AM DID YOUR TESTS PASS OR FAIL? ANSWERING WITH AUTOMATION Noel Nyman Microsoft International Conference On Software Test Automation March 5-8, 2001 San Jose, CA, USA Did Your Tests Pass or Fail? Using Self-Verifying Data with Hard-to-Automate Applications Noel Nyman Desktop Applications Automation Test Lead Microsoft Windows Systems Group Agenda § Self-Verifying Data review üHow to use SVD to tell that your tests pass § Automating problem applications üIdeas on how to automate apps when your automation tools can’t see parts of them § Demos § Resources for more information § Questions 2 Self-Verifying Data • Has codes embedded in the data that act as an oracle and can tell you if the data is… üLegal - is this data from the correct data set? üValid - is this data that should be here? üCorrect type – is this data the type we’re looking üFrom correct record – data from the record we asked for? üAccurate - are characters missing, munged? § SVD data is like a debug build of an app 3 Benefits of Using SVD § No separate oracle needed § Can be scaled to very large data sets § Can be used with rich data not easily verified by humans § Easy to verify with automated testing § Adding additional test data usually doesn’t require updating automated tests 4 SVD Example First name from a rich data set… First name with embedded SVD codes… SVD codes replaced with evocative tags… 5 Automating Problem Apps § Forget capture/replay – if it works well for you, you don’t have a “problem
  • Porting Codewarrior Projects to Xcode

    Porting Codewarrior Projects to Xcode

    Porting CodeWarrior Projects to Xcode 2006-09-05 Intel and Intel Core are registered Apple Computer, Inc. trademarks of Intel Corportation or its © 2003, 2006 Apple Computer, Inc. subsidiaries in the United States and other All rights reserved. countries. Java and all Java-based trademarks are No part of this publication may be trademarks or registered trademarks of Sun reproduced, stored in a retrieval system, or Microsystems, Inc. in the U.S. and other transmitted, in any form or by any means, countries. mechanical, electronic, photocopying, recording, or otherwise, without prior OpenGL is a registered trademark of Silicon written permission of Apple Computer, Inc., Graphics, Inc. with the following exceptions: Any person PowerPC and and the PowerPC logo are is hereby authorized to store documentation trademarks of International Business on a single computer for personal use only Machines Corporation, used under license and to print copies of documentation for therefrom. personal use provided that the Simultaneously published in the United documentation contains Apple’s copyright States and Canada. notice. Even though Apple has reviewed this document, The Apple logo is a trademark of Apple APPLE MAKES NO WARRANTY OR Computer, Inc. REPRESENTATION, EITHER EXPRESS OR IMPLIED, WITH RESPECT TO THIS Use of the “keyboard” Apple logo DOCUMENT, ITS QUALITY, ACCURACY, MERCHANTABILITY, OR FITNESS FOR A (Option-Shift-K) for commercial purposes PARTICULAR PURPOSE. AS A RESULT, THIS without the prior written consent of Apple DOCUMENT IS PROVIDED “AS IS,” AND YOU, THE READER, ARE ASSUMING THE may constitute trademark infringement and ENTIRE RISK AS TO ITS QUALITY AND unfair competition in violation of federal ACCURACY.
  • Opportunities and Open Problems for Static and Dynamic Program Analysis Mark Harman∗, Peter O’Hearn∗ ∗Facebook London and University College London, UK

    Opportunities and Open Problems for Static and Dynamic Program Analysis Mark Harman∗, Peter O’Hearn∗ ∗Facebook London and University College London, UK

    1 From Start-ups to Scale-ups: Opportunities and Open Problems for Static and Dynamic Program Analysis Mark Harman∗, Peter O’Hearn∗ ∗Facebook London and University College London, UK Abstract—This paper1 describes some of the challenges and research questions that target the most productive intersection opportunities when deploying static and dynamic analysis at we have yet witnessed: that between exciting, intellectually scale, drawing on the authors’ experience with the Infer and challenging science, and real-world deployment impact. Sapienz Technologies at Facebook, each of which started life as a research-led start-up that was subsequently deployed at scale, Many industrialists have perhaps tended to regard it unlikely impacting billions of people worldwide. that much academic work will prove relevant to their most The paper identifies open problems that have yet to receive pressing industrial concerns. On the other hand, it is not significant attention from the scientific community, yet which uncommon for academic and scientific researchers to believe have potential for profound real world impact, formulating these that most of the problems faced by industrialists are either as research questions that, we believe, are ripe for exploration and that would make excellent topics for research projects. boring, tedious or scientifically uninteresting. This sociological phenomenon has led to a great deal of miscommunication between the academic and industrial sectors. I. INTRODUCTION We hope that we can make a small contribution by focusing on the intersection of challenging and interesting scientific How do we transition research on static and dynamic problems with pressing industrial deployment needs. Our aim analysis techniques from the testing and verification research is to move the debate beyond relatively unhelpful observations communities to industrial practice? Many have asked this we have typically encountered in, for example, conference question, and others related to it.
  • Addresssanitizer + Code Coverage Kostya Serebryany, Google Eurollvm 2014 New and Shiny -Fprofile-Instr-Generate

    Addresssanitizer + Code Coverage Kostya Serebryany, Google Eurollvm 2014 New and Shiny -Fprofile-Instr-Generate

    AddressSanitizer + Code Coverage Kostya Serebryany, Google EuroLLVM 2014 New and shiny -fprofile-instr-generate ● Coming this year ● Fast BB-level code coverage ● Increment a counter per every (*) BB ○ Possible contention on counters ● Creates special non-code sections ○ Counters ○ Function names, line numbers Meanwhile: ASanCoverage ● Tiny prototype-ish thing: ○ Part of AddressSanitizer ○ 30 lines in LLVM, 100 in run-time ● Function- or BB- level coverage ○ Booleans only, not counters ○ No contention ○ No extra sections in the binary At compile time: if (!*BB_Guard) { __sanitizer_cov(); *BB_Guard = 1; } At run time void __sanitizer_cov() { Record(GET_CALLER_PC()); } At exit time ● For every binary/DSO in the process: ○ Dump observed PCs in a separate file as 4-byte offsets At analysis time ● Compare/Merge using 20 lines of python ● Symbolize using regular DWARF % cat cov.c int main() { } % clang -g -fsanitize=address -mllvm -asan-coverage=1 cov. c % ASAN_OPTIONS=coverage=1 ./a.out % wc -c *sancov 4 a.out.15751.sancov % sancov.py print a.out.15751.sancov sancov.py: read 1 PCs from a.out.15751.sancov sancov.py: 1 files merged; 1 PCs total 0x4850b7 % sancov.py print *.sancov | llvm-symbolizer --obj=a.out main /tmp/cov.c:1:0 Fuzzing with coverage feedback ● Test corpus: N random tests ● Randomly mutate random test ○ If new BB is covered -- add this test to the corpus ● Many new bugs in well fuzzed projects! Feedback from our customers ● Speed is paramount ● Binary size is important ○ Permanent & temporary storage, tmps, I/O ○ Stripping non-code
  • Development of a Personal Diet Plan Database Application for Persons with Severe Food Allergies Heather Suzanne Ward Regis University

    Development of a Personal Diet Plan Database Application for Persons with Severe Food Allergies Heather Suzanne Ward Regis University

    Regis University ePublications at Regis University All Regis University Theses Summer 2005 Development Of A Personal Diet Plan Database Application For Persons With Severe Food Allergies Heather Suzanne Ward Regis University Follow this and additional works at: https://epublications.regis.edu/theses Part of the Computer Sciences Commons Recommended Citation Ward, Heather Suzanne, "Development Of A Personal Diet Plan Database Application For Persons With Severe Food Allergies" (2005). All Regis University Theses. 772. https://epublications.regis.edu/theses/772 This Thesis - Open Access is brought to you for free and open access by ePublications at Regis University. It has been accepted for inclusion in All Regis University Theses by an authorized administrator of ePublications at Regis University. For more information, please contact [email protected]. Regis University School for Professional Studies Graduate Programs Final Project/Thesis Disclaimer Use of the materials available in the Regis University Thesis Collection (“Collection”) is limited and restricted to those users who agree to comply with the following terms of use. Regis University reserves the right to deny access to the Collection to any person who violates these terms of use or who seeks to or does alter, avoid or supersede the functional conditions, restrictions and limitations of the Collection. The site may be used only for lawful purposes. The user is solely responsible for knowing and adhering to any and all applicable laws, rules, and regulations relating or pertaining to use of the Collection. All content in this Collection is owned by and subject to the exclusive control of Regis University and the authors of the materials.
  • Java Core Technologies (Part III)

    Java Core Technologies (Part III)

    Extreme Java G22.3033-007 Session 5 - Main Theme Java Core Technologies (Part III) Dr. Jean-Claude Franchitti New York University Computer Science Department Courant Institute of Mathematical Sciences 1 Agenda • Summary of Previous Session / Epilogue • Java Media APIs •Java 2D API • Java Media Framework • XBeans and Visual XBeans •JNI • Java Generative Programming Technology • Language Features •Threads • Readings 2 1 Summary of Previous Session • Summary of Previous Session • Java AWT and Swing components • Drag and Drop • Graphics and Multimedia Components (JavaMedia) • JavaBeans • Readings • Class Project & Assignment #2b 3 Java Technologies/Features in Scope • Technologies: • Collections Framework • Input Method Framework • Swing • Drag and Drop • JavaBeans •JNI • Security • Language Features: • Threads • Assertions 4 2 Part I Session 4 Epilogue 5 Drag and Drop (http://java.sun.com/j2se/1.4/docs/guide/swing/1.4/dnd.html) • See Session 4 Handout on Drag and Drop Data Transfer • Adding Drag and Drop to a JTree • In J2SE 1.4, JTree supports Drag but developers must implement and install a custom TransferHandler to fully support imports of data on Drop targets • http://www.javaworld.com/javaworld/javatips/jw-javatip97.html • http://www.javaworld.com/javaworld/javatips/jw-javatip114.html • http://sourceforge.net/projects/ijtree/ 6 3 JavaBeans and JAF (http://java.sun.com/j2se/1.4/docs/guide/beans/changes14.html) • See Session 4 Handout on Component Modeling with JavaBeans • Download the BeanBuilder • http://java.sun.com/products/javabeans/beanbuilder/index.html
  • Configuring the Codewarrior Tools to Debug a BSC9131RDB Board

    Configuring the Codewarrior Tools to Debug a BSC9131RDB Board

    Freescale Semiconductor Document Number: AN4384 Application Note Rev. 0, 07/2012 Configuring the CodeWarrior Tools to Debug a BSC9131RDB Board by Freescale Semiconductor, Inc. Austin, TX This application note describes how to configure Contents the Freescale CodeWarrior tools to support 1 BSC9131 overview .................................................... 2 developing and debugging of embedded 2 Debugger configuration strategies ............................. 4 3 BSC9131RDB board setup ........................................ 6 software on the BSC9131RDB board. 4 Two-TAP connection scheme .................................... 7 5 Revision history ........................................................ 33 Because the QorIQ Qonverge BSC9131 Appendix A How to Disable the SC3850 Caches ....... 34 processor contains two disparate processor Appendix B DSP Debugging with a BSP Present ....... 38 Appendix C Ethernet TAP Run Controller Options ..... 39 cores, writing software for the part requires two different sets of CodeWarrior tools: one to manage the DSP core, and another to manage the system core. Care must be taken to ensure that the two sets of tools do not interact with each other. Both the proper configuration sequences and the potential pitfalls are described here. This application note assumes the use of CodeWarrior for StarCore DSPs v10.2.10 or later, and CodeWarrior for Power Architecture® v10.1.2 or later. © 2012 Freescale Semiconductor, Inc. 1 BSC9131 overview The Freescale QorIQ Qonverge BSC9131 contains two processor cores, each of whose microarchitecture is optimized for a specific purpose: StarCore SC3850 DSP core implements high-throughput signal processing functions Power Architecture e500 core implements high-volume network functions These cores manage a number of powerful peripherals, all of which are interconnected through a low- latency switching fabric (Figure 1). However, for debugging setup purposes, only the cores are considered in this application note.
  • Codewarrior™ Build Tools Reference for Freescale™ 56800/E Hybrid Controllers

    Codewarrior™ Build Tools Reference for Freescale™ 56800/E Hybrid Controllers

    Freescale Semiconductor, Inc. CodeWarrior™ Build Tools Reference for Freescale™ 56800/E Hybrid Controllers Revised 28 October 2004 For More Information: www.freescale.com Freescale Semiconductor, Inc. Metrowerks and the Metrowerks logo are registered trademarks of Metrowerks Corporation in the United States and/ or other countries. CodeWarrior is a trademark or registered trademark of Metrowerks Corporation in the United States and/or other countries. All other trade names and trademarks are the property of their respective owners. Copyright © 2004 Metrowerks Corporation. ALL RIGHTS RESERVED. No portion of this document may be reproduced or transmitted in any form or by any means, electronic or me- chanical, without prior written permission from Metrowerks. Use of this document and related materials are governed by the license agreement that accompanied the product to which this manual pertains. This document may be printed for non-commercial personal use only in accordance with the aforementioned license agree- ment. If you do not have a copy of the license agreement, contact your Metrowerks representative or call 1-800- 377-5416 (if outside the U.S., call +1-512-996-5300). Metrowerks reserves the right to make changes to any product described or referred to in this document without further notice. Metrowerks makes no warranty, representation or guarantee regarding the merchantability or fitness of its prod- ucts for any particular purpose, nor does Metrowerks assume any liability arising out of the application or use of any product
  • A Zero Kernel Operating System: Rethinking Microkernel Design by Leveraging Tagged Architectures and Memory-Safe Languages

    A Zero Kernel Operating System: Rethinking Microkernel Design by Leveraging Tagged Architectures and Memory-Safe Languages

    A Zero Kernel Operating System: Rethinking Microkernel Design by Leveraging Tagged Architectures and Memory-Safe Languages by Justin Restivo B.S., Massachusetts Institute of Technology (2019) Submitted to the Department of Electrical Engineering and Computer Science in partial fulfillment of the requirements for the degree of Masters of Engineering in Computer Science and Engineering at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY February 2020 © Massachusetts Institute of Technology 2020. All rights reserved. Author............................................................................ Department of Electrical Engineering and Computer Science January 29, 2020 Certified by....................................................................... Dr. Howard Shrobe Principal Research Scientist, MIT CSAIL Thesis Supervisor Certified by....................................................................... Dr. Hamed Okhravi Senior Staff Member, MIT Lincoln Laboratory Thesis Supervisor Certified by....................................................................... Dr. Samuel Jero Technical Staff, MIT Lincoln Laboratory Thesis Supervisor Accepted by...................................................................... Katrina LaCurts Chair, Master of Engineering Thesis Committee DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited. This material is based upon work supported by the Assistant Secretary of Defense for Research and Engineering under Air Force Contract No. FA8702-15-D-0001. Any opinions, findings, conclusions
  • Guidelines on Minimum Standards for Developer Verification of Software

    Guidelines on Minimum Standards for Developer Verification of Software

    Guidelines on Minimum Standards for Developer Verification of Software Paul E. Black Barbara Guttman Vadim Okun Software and Systems Division Information Technology Laboratory July 2021 Abstract Executive Order (EO) 14028, Improving the Nation’s Cybersecurity, 12 May 2021, di- rects the National Institute of Standards and Technology (NIST) to recommend minimum standards for software testing within 60 days. This document describes eleven recommen- dations for software verification techniques as well as providing supplemental information about the techniques and references for further information. It recommends the following techniques: • Threat modeling to look for design-level security issues • Automated testing for consistency and to minimize human effort • Static code scanning to look for top bugs • Heuristic tools to look for possible hardcoded secrets • Use of built-in checks and protections • “Black box” test cases • Code-based structural test cases • Historical test cases • Fuzzing • Web app scanners, if applicable • Address included code (libraries, packages, services) The document does not address the totality of software verification, but instead recom- mends techniques that are broadly applicable and form the minimum standards. The document was developed by NIST in consultation with the National Security Agency. Additionally, we received input from numerous outside organizations through papers sub- mitted to a NIST workshop on the Executive Order held in early June, 2021 and discussion at the workshop as well as follow up with several of the submitters. Keywords software assurance; verification; testing; static analysis; fuzzing; code review; software security. Disclaimer Any mention of commercial products or reference to commercial organizations is for infor- mation only; it does not imply recommendation or endorsement by NIST, nor is it intended to imply that the products mentioned are necessarily the best available for the purpose.